FAST: Facilitating Access to Skilled Talent

Evaluation Approach

Our evaluation of the FAST pilot is classified as an Effectiveness Evaluation, focused on understanding to what extent the pilot produces intended outcomes, and the individual and implementation factors that support success. The results of this evaluation will be used to move the FAST model along the Future Skills Centre’s evidence pipeline, by building evidence of the model’s effectiveness to inform future evaluation and scaling decisions. Findings will also be linked to a broader learning agenda to ensure that we can leverage learnings to inform future innovations in the areas of digitally delivered interventions and supporting newcomer jobseekers, and to advance the Future Skills Centre’s mandate of mobilizing evidence to inform practice, programs, and policies.

To understand the degree to which the pilot produces intended outcomes, and the individual success factors associated with those outcomes, this evaluation will measure the full set of participant outcomes outlined in the Future Skills Centre’s Shared Outcomes Framework, including skills gains, enrollment in further education, and long-term employment outcomes.

To identify the implementation factors associated with success and support continuous improvement, the evaluation will focus on gathering the reflections of delivery partners at expansion sites in Manitoba, Ontario, and Nova Scotia to better understand what is required to scale the model to new contexts. The evaluation will also support continuous improvement by gathering participant reflections on strengths and gaps in the platform, and carrying out targeted process improvement activities to use evaluation findings to test adaptations to the model.

Finally, the evaluation will collect data on a number of pilot-specific outcomes to better understand how FAST is meeting the needs of its target population, and how implementation could be adapted to improve performance. These outcomes include completion of specific FAST modules, engagement in follow-up employment services, and the commensurability of employment to participant skill levels.

Over the two-year span of this project, we will carry out this evaluation by:

• Collecting detailed socio-demographic and outcome data for FAST participants, including platform completion, enrolment in further training, and employment.
• Interviewing participants, IEC-BC staff, staff at expansion service delivery partners, and employers, to elicit perspectives on key success factors in scaling the model and meeting employer hiring needs.
• Testing variations in platform delivery to identify opportunities to improve participant experience and outcomes, and generate preliminary findings on the model’s impact.
• Delivering interim reports on outcomes achieved, implementation findings, and recommendations to partners to inform continuous improvement of the model.
• Analyzing multiple sources of program data to assess pilot effectiveness and identify implementation findings to be communicated in a final pilot report.

Results of the FAST evaluation are expected to be published Fall 2021.