Mind the Gap: How changes in PIAAC data collection limit what we can learn about skills, and what we can do to fix the gap in the future

With the initial burst of interest in the OECD’s release of the second cycle of data from the Programme for the Internal Assessment of Adult Skill (PIAAC) waning, the time to take more careful and measured looks at what the data show has come. The data is certainly a source of deep insight that should help educators, policymakers and skills practitioners develop better strategies to engage workers, make use of their skills and pursue policies and programs to enhance those skills. At the same time, the second cycle of PIAAC data has limitations that analysts and others will need to navigate as they ask and try to answer key questions about adult Canadians’ skills. What are those limitations and how might they affect the kinds of questions we can ask and the answers we uncover?  

Demographic Limits

Canadian research using PIAAC Cycle 1 data uncovered valuable insights about skills inequities among Indigenous peoples, recent immigrants, and other groups in Canada. It appears that research using Cycle 2 data will face greater hurdles to unpacking skills differences among groups – and therefore offer less to Canada’s skills discourse and practice. Ppublicly available data from Statistics Canada for Cycle 2 are conspicuously lacking in demographic data beyond the categories of gender, age, and occupation. While additional demographic data will likely become available in the coming months, we should expect fewer demographic breakdowns in Cycle 2 relative to Cycle 1 because of one central deficiency: a much smaller sample size.

Smaller Sample

PIAAC Cycle 1 was conducted over three rounds between 2011 and 2018 (with Canada’s data collected in 2012) and surveyed roughly 245,000 individuals worldwide. Canada had a unique distinction of collecting the largest sample of any of the 39 participating countries with a total of 27,285 respondents. In fact, Canada oversampled many sub-populations and regions to enable detailed, disaggregated analysis of Indigenous peoples’ and immigrants’ skills, as well as differences by province and region. 

PIAAC Cycle 2 is much more limited in this regard. Canadian Cycle 2 data collection (which began in 2022 and was released in December 2024) generated only 11,697 responses – less than half of the number collected in Cycle 1. The most recent sample also lacks the targeted oversampling of regions and groups that was an extraordinarily valuable feature of the first cycle. To be sure, data collection was hindered by the COVID-19 pandemic. Fewer people were willing to allow surveyors into their homes to administer multiple hour assessments, and reliable data collection partners were hard to find. The upshot is that, while the Cycle 2 sample is large enough for a variety of core analyses, nuanced demographic and regional analysis will suffer. 

Longitudinal Limitations

Change from Cycle 1 to Cycle 2 will also affect prospects for longitudinal analysis. For many researchers, one of the major potential advantages of a second PIAAC cycle was the opportunity to conduct analyses of skills changes over time. And while some of this work is still possible, optimism has been weakened in the face of the data released.

The smaller sample makes some points of comparisons harder to engineer, particularly when it comes to regional and demographic breakdowns. Moreover, changes to the survey itself make certain longitudinal analyses harder, if not impossible, to execute. For example, whereas in Cycle 1, the survey assessed problem solving in a technology rich environment (PS-TRE), Cycle 2 moved to a new measure of problem solving: adaptive problem solving (APS). The rationale for the change is sound – i.e., ensuring that the skills we measure keep up with social and economic realities – but it does curtail opportunities to understand how the state of certain skills change over time. 

Bruised, but not Beaten

Despite dents in the Cycle 2 data relative to Cycle 1, PIAAC still offers a unique and robust opportunity for the skills community to better understand and improve Canada’s overall skills performance. There is much that can be done with the data and our forthcoming Canadian PIAAC Research Agenda will lay out some possibilities and priorities. But researchers pursuing insights using the latest data will need to be mindful of the gaps. 

Looking ahead to the next Cycle of PIAAC, Canada would be well-advised to start working now on strategies to improve data collection and ensure that the next iteration is designed to help answer questions that matter to Canadians. As our Canadian PIAAC Research Agenda will show, insights from PIAAC can help us understand and develop better strategies to improve productivity, growth, health and well-being. But the insights and strategies can only be as good as the data on which they rest. If Canada is serious about improving social and economic outcomes, it needs to go all in on PIAAC Cycle 3.

The views, thoughts and opinions expressed here are the author’s own and do not necessarily reflect the viewpoint, official policy or position of the Future Skills Centre or any of its staff members or consortium partners.