Evaluation is at the heart of the Future Skills Centre innovation projects

Generating evidence about how to prepare Canadians for the changing economy and future of work is central to the mission of the Future Skills Centre.

In partnership with diverse organizations across the Canada, the Centre is funding innovation projects to conduct real-world tests of promising skills development solutions and contribute to a growing evidence base on what works, why, and how. 

The Future Skills Centre is excited to be working with our first wave of innovation project partners to evaluate their initiatives. Each pilot evaluation combines methodological rigour with a focus on generating actionable results. This approach is based on the belief that evaluations should not only contribute to a broader evidence base on what works, but also inform the day-to-day practices and decision-making of our partners. 

This approach requires a deep commitment to collaboration, co-design and learning alongside our project partners. For our partners, working closely with the Centre’s evaluators has highlighted the appeal of this approach, and an appreciation of the value that evaluations that are meaningfully connected to their learning priorities and needs can add.

Collaborating to design rigorous evaluations

The Immigrant Employment Council of British Columbia (IEC-BC) has received funding from the Future Skills Centre for its Facilitating Access to Skills Talent (FAST) project, an innovative online skills platform that helps newcomers “find occupations that make full use of their skills,” says Patrick MacKenzie, CEO of IEC-BC, “and find those jobs quickly.” 

Like all of the Future Skills Centre projects, FAST is being rigorously evaluated, but in a way that’s different from what MacKenzie’s seen before: “It feels very much like a partnership, which is really helpful,” he says. “What they help to do is bring a real rigour… It’s a very collaborative approach.”

The Centre is committed to engaging with project partners throughout the evaluation design process. First, the Future Skills Centre evaluators meet with partners to understand the delivery model and learning goals. Together, they co-design parts of the evaluation plan.

The evaluators then further develop the evaluation plan, measurement framework, and analytical design. Every evaluation plan includes outcome measures that are common to all of the Future Skills Centre projects, in order to compare like-with-like and help the Centre rigorously assess the collective impact of innovation project. But, context is still important. Evaluators bring their final plan back to partner organizations to ensure it aligns with their needs and goals. 

A human touch

It’s key for a project with lots of moving parts like the Indigenous ICT (Information Communication and Technology) Centre to have the opportunity to work closely with evaluators and develop a shared understanding of the project goals. The Future Skills Centre is funding the ICT Centre to give Indigenous people in Winnipeg and The Pas skills training, coaching, and mentorship to work in the sector, as well as a nine-month work placement to cement new careers. 

Lawrence Daniels is the CEO of FireSpirit which has itself developed and implemented evaluations. For Daniels, what has set the Centre’s process apart has been the communication with evaluators. Even at an early stage, Daniels says, evaluators have been active with “setting up the data points, in-person meetings and collaboration.” This human touch is helpful, he says. “Typically we don’t get to meet the evaluator.”

Understanding community needs

The Future Skills Centre is funding and evaluating Actua’s Future Skills Program for Indigenous and Northern Youth, focused on STEM skills. The project is grounded in communities. Youth receive land-based learning from community Elders and Knowledge-Keepers. Youth don’t need to look outside their communities for applications of STEM, says Leslie Cuthbertson, the COO of Actua  “Because it’s already there.” 

“For example, looking at changes in weather – students can make a weather station. They can program that weather station to collect data, and review and analyze that data on a computer.  It starts with the land and advances to the applied STEM,” Cuthbertson says. 

Participants earn high school credit from the project, which includes mentorship from Indigenous undergraduate STEM students, in-class work, and industry collaboration.

The project is accountable to the communities where it operates. “We don’t just parachute into a community and start a program,” Cuthbertson says. “We must take the time to develop the trust relationship with the community. In this way, the project is a true collaborative process.” 

This has special implications for evaluation and data collection. 

“We are, as an organization, committed to following the OCAP principles – ownership, control, access and possession,” Cuthbertson says, referring to the standards for Indigenous data governance. The Future Skills Centre evaluators are working closely with Actua to consider how to align the evaluation with OCAP principles.

The evaluation must also align with the needs of the communities the project is engaging with. “It’s also critical that the evaluation process itself – as in, how we decide what information we are looking for, and how we obtain that information – must fit and work with community priorities,” says Cuthbertson.

The project partners have appreciated the Centre’s evaluators’ commitment to this approach.  “They have been respectful of our needs,” Cuthbertson says, “and community needs.” She is optimistic about how the evaluation will proceed. “Honestly, it has been a very very positive and constructive process.”