Evaluation Approach

The Future Skills Centre supports a variety of pilot programs that are testing new frontiers in skills development for Canadian workers. We work with new and established non-profit organizations, post-secondary institutions, private sector councils, municipal agencies and others, on projects addressing a wide variety of skills development challenges. We are exploring the experiences of people who participate in these programs, and work towards documenting “what it takes” to deliver services and programs at the scale needed and with long-term sustainability in mind.

Learning and evaluation activities are key to this work. FSC encourages a comprehensive evaluation approach that explores how to build insights at all levels – individuals, institutions, and systems. To do so, we support partners working with a range of rigorous methods to systematically identify hypotheses, challenge assumptions, and assess outcomes.

We support partners in helping them understand the challenges that workers and employers are facing, explore potential new ways of meeting those challenges through programs, services and resources, and assess the impact of the intervention. We encourage partners to involve program participants, community-stakeholders, and front-line providers in the design, implementation and analysis of these activities. We also encourage partners to reflect not just on project successes, but also what didn’t work and why – all with the goal of fostering a culture around innovation and continuous improvement.

Our evaluations deploy a variety of methods and approaches, to provide a solid understanding of the following:

Professionals in a work meeting brainstorming for a project.

Core issues and problems, whether articulated as long-standing structural issues facing particular workers and employers, or the need for institutions to improve the delivery of key services.

Understanding the overall theory of change and what change(s) we think would help address or solve for the underlying problem(s). For many organizations delivering skills training for example, this involves thinking carefully about how a program works and how activities are intended to lead to certain measurable outputs and outcomes.

Outlining key questions that the project will be able to address. With a mandate to support and learn from innovation, program implementation is directly tied to learning questions and building our understanding of effective approaches. Each project and partner identifies learning questions the project will address.

Specifying the evidence to be gathered over the course of a project, and how this evidence will be used to help answer the key questions and build collective and ecosystem-level understanding.

To achieve these goals, FSC and its partners work with a range of learning and evaluation providers. Blueprint provides specialized approaches for projects exploring opportunities to achieve a larger scale. Toronto Metropolitan University’s Diversity Institute supports partners that are addressing issues related to equity and inclusion. In addition, FSC supports organizations to engage with trusted third party evaluation providers to capture learning from and about implementation, as well as broader implications for policy and practice.

FSC recognizes that an inclusive approach to data helps generate meaningful learning. For partners attempting to assess the impact of their work on individuals, FSC has developed a Common Outcomes Framework to help inform data gathering efforts. In addition to individual-level data, FSC encourages partners to gather data about implementation and performance, and supports work that allows for bench-marking cost and effectiveness and finding opportunities for continuous improvement.

Sharing our learning

We engage our partners and the broader skills ecosystem on an ongoing basis. We encourage partners to share insights and reflections from evaluation plans, needs assessments, mid-project and final evaluations in a variety of formats including through our Communities of Practice, stakeholder events and fora, and individual webinars. In addition, we make knowledge and synthesized insights available to policy makers, employers and employer groups, and training providers for briefings and discussions on our website and available for briefings as requested.

Evidence reports

We rely on thoughtful and rigorous evaluation to identify the solutions with the most promise and with the potential to be scaled across Canada to meet the needs of individuals and employers. For each project, we develop and execute a customized evaluation plan that is linked to a larger learning agenda that helps promising interventions improve their performance and impact over time.