Expert article
Modifier 11 October 2022
by Ronald Lievens, UIA Expert

Passport for Work: first results of the impact study & invitation to final event

PfW impact study

Over the course of the last 12 months, an empirical study has been conducted towards the impact of Passport for Work on a large group of job seekers in the  Eindhoven region. Specifically, an experimental group of 92 participants was compared with 219 randomly assigned job seekers in the control group. 

Through a combination of quantitative analysis (based on administrative records on an individual's labour market status) and qualitative semi-structured interviews and focus groups, 15 result indicators were assessed. The outcomes of each of these result indicators are made available in the project's final report, which will be published during the final event on 31 October.

In this web article, two particular indicators are highlighted, pertaining to the employability theme. Within this theme, two result indicators were defined:

1. 200 jobseekers will be “sustainably matched with an employer (either for a period longer than 6 months or in a position fitting their educational attainment level)

2. PfW use increases jobseekers' self-awareness in terms of skills and opportunities

Partager

This was analyzed by comparing the labour market outcomes of the experimental group with the control group. Due to ethical considerations, randomization was not applied to the experiment group, there were no restrictions imposed on participating in the workshops. To establish whether the principle of randomization was adhered to, the mean age of the control group and the PfW group was compared by means of an independent samples T-test, as well as the time they were in the system. Furthermore, the gender distribution was statistically compared through a Chi Square test. As can be seen in Table 1, no statistically significant differences were found between the two groups in terms of gender and age. However, the PfW group appeared to be in the system significantly longer than the control group, arguably representing some extent of self-selection bias. It is therefore questionable whether the control group and the PfW group are completely comparable on all characteristics.

Table 1. Frequencies per group

 

PfW ( n = 91) *

Control ( n = 219)

Statistical Comparison

Age

M = 40.61 (SD = 10.40)

M = 39.26 (SD = 12.45)

t (199.87) = -.99, p = .325

Number of men

46 (50.5%)

97 (44.3%)

ꭓ 2 (1) = .69, p = .406

Time in system (in months)

M = 15.22 (SD = 22.30)

M = 6.09 (SD = .41)

t (93.37) = -3.87, p < .001

*n = 89 due to missing data

 

The job matching outcomes of these two groups were mapped into the following possible six categories:

(A) Found a job independently, (B) found a job through the municipality’s career counselors, (C) found a subsidized job, (D) active in basic job searching training, (E) active in advanced job searching training and F) other. Respondents could only be assigned to one of these categories. The “other” group was not taken into account for the following analyses, so 39 respondents (of which 10 from the PfW group) were not taken into account for the further analyses.

Table 2 shows the frequencies of the categories per group. To find out to what extent the relationships between the categories differ between the PfW group and the control group, a comparison was made by means of a Chi-square test. This analysis shows that there are no significant differences between the two groups, ꭓ2 (4) = 8.80, p = .066. However, there seems to be a descriptive difference between the groups.

Table 2. Outcomes per group

 

PfW ( n = 81)

Control ( n = 190)

 

A Found a job independently

12 (14.8%)

38 (20.0%)

ꭓ 2 (1) = .82, p = .364

B Found a job through career counseling

10 (12.3%)

17 (8.9%)

ꭓ 2 (1) = .84, p = .359

C Found a subsidized job

8 (9.9%)

43 (22.6%)

ꭓ 2 (1) = 5.50, p = .019

D In basic job searching training

32 (39.5%)

55 (28.9%)

ꭓ 2 (1) = 3.22, p = .073

E In advanced job searching training

19 (23.5%)

37 (19.5%)

ꭓ 2 (1) = .67, p = .406

 

To check whether the groups differ or whether respondents eventually found work, a variable was created in which the A, B and C categories scored as work found and the D and E categories were scored as no work found. In the PfW group, 30 (37.0%) of the 81 respondents eventually found work and 51 (63.0%) did not find work. In the control group of 190 respondents, 98 (51.6%) respondents found work and 92 (48.4%) respondents did not find work. A Chi-square test was used to compare the extent to which these ratios differ from each other. It appears that the control group found work significantly more often than the PfW group, ꭓ2 (1) = 4.82, p = .028.

There are two possible explanations for the lower job finding rates of the experiment group. First of all, the algorithm-based matching module was not in use for a large portion of the study due to the experienced difficulty of mobilizing employers (it started to see its first use in August, near the end of the experiment). Second, it appears that the self-selection bias present in the control group hinders the comparability of the two groups: the participants in the workshop are arguably facing a larger distance to the labour market. This is reflected by the larger proportion of candidates in the basic job searching category in the experiment group (although the difference is borderline significant). It is important to emphasize that these statistics are descriptive by nature, and that there is no empirical support for establishing a generalizable, “net effect” of the Passport for Work platform in terms of sustainable job matching. However, it can be argued that the platform aides the mobilization of relatively vulnerable job seekers, which facing the highest socio-economic risks.

To facilitate the development of job matching insights at a later stage, a job seeker monitoring system has been established. Through this system, the labour market outcomes of job seekers in the Passport for Work platform can be assessed periodically. The system serves two main functions: improving the algorithm-based job matching (in which a growing number of matches allows for more precise matching criteria) and gaining insight in actual job matches. 

This result indicator was researched through a user experience (UX) survey as well as semi-structured interviews with the job coaches that were involved in the organization of Passport for Work workshops, in which the 91 participants of the experiment group were guided in the use of the platform and the creation of their skills profile.

Table 1. UX survey (N=24)

In the UX survey, 59% agreed (strongly) with the statement that Passport for Work enhance their awareness levels of their own skills. 21% disagreed with the statement and 21% of respondents were neutral. In the workshops, it was observed that the participants’ self-efficacy levels were relatively low, most of them were either discouraged or unmotivated jobseekers who had been inactive for prolonged periods of time. Their experiences with the job market have been mostly negative, which they attributed to stigma’s based on (a combination of) their gender, age and/or (lack of) educational qualifications. The effects of this accumulated over time, with a vicious cycle of lacking work experience reinforcing their inability to obtain employment, and in turn discouraging their job searching motivation and behavior.

The job coaches reported that in the majority of the workshops, the participants felt increasingly motivated by using the Passport for Work platform. It provided them with a sense of pride and, in some cases, contributed to restoring their perceived self-worth. To illustrate, one workshop participant printed the outcome of the skills assessment (the “skillprint”) and hung it up on their fridge as a daily reminder of their professional profile and skillset. This positive sentiment was observed among many of the workshop participants.

In terms of opportunities, two observations were made. First of all, the ability to match one’s skills profile with several occupations across different sectors of industry broadens their perspective on suitable jobs. It helps them to think outside of the constraints of their previous work experience. Second, the skills profile provides structure and guidance in thinking about one’s skills. For many participants, who did not have a CV, this was new and found to be both motivating and valuable, especially due to the fact that at the end of the assessment the skillprint was a tangible outcome which could be used for both self-reflection and sharing with prospective employers. It was reported that participants felt that they became better aware of their skills, aspirations, interests and developmental areas. As a result, the perceived distance between them and prospective employers decreased.

To learn more about the above outcomes of the Passport for Work project you are cordially invited to the final conference in the online auditorium of the High Tech Campus Conference Center in Eindhoven during the afternoon of Monday the 31st of October 2022. An online platform has been set up for guests to be able to attend the final conference remotely. Secure your place and sign up today at https://lnkd.in/eUNUqVKm 

The programme consists of keynote speeches of labour market professor Ton Wilthagen and Kristine Langenbucher, the head of the Employment and Skills Unit within the OECD's Center for Entrepreneurship, SMEs, Regions and Cities. Furthermore, several workshops en panel discussions are held around the topic of the skill-based economy, focusing on aspects such as measuring and matching skills, mobilizing employers, and scaling regional to (inter)national impact.

Other news from this project

Passport for Work refugee workshop

Passport for Work supports refugee integration into the labour market

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
social innovation P4W

Passport for Work’s scientific impact study kicks off

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
Zoom-in video podcast

Passport for Work Zoom-in: The Skill Based Economy

The Dutch labour market is facing unprecedented shortages, calling for a skill based economy in which skill based matching, reskilling and upskilling ...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
Dutch Skills Survey

Major milestone for P4W: national skills survey complete

In October 2020, Passport4Work started an ambitious national research endeavour: replicating the survey of the leading skills taxonomy O*NET in the Du...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
P4W focus groups in HE

Passport4Work expands target group: focus groups with University students

Striving for an inclusive labour market instrument, Passport4Work organized 2 focus group sessions with University students to establish its utility b...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
P4W Journal 3

Passport4Work Journal 3

Passport4Work is nearing the product delivery phase. Learn more about the state of play of the project and its contributions to a skills-based economy...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
P4W at HR Live

Passport4Work launches employer prototype at HR Live

Passport4Work reached a major milestone in its project, by recently finishing the first prototype of the employer interface of the tool. This prototyp...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
P4W zoom in 2

Passport for Work Zoom-in: Skills based matching

In the second Passport for Work zoom-in (the first can be found here), the implications of skills based matching are explored among employers. We visi...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Jobs and skills in the local economy
P4W Journal 2: A phase of intensive technical development and prototyping!

P4W Journal 2: A phase of intensive technical development and prototyping!

In his 2nd journal, UIA Expert Ronald Lievens reflects on the current state of affairs in the Passport4Work (P4W) project, its developments, and the d...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Cover

Passport4Work completes series of tests of first prototype (video)

In his 1st zoom-in, P4W expert Ronald Lievens, explores why soft skills are a key part of the Passport4Work project. Especially in these times, with c...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
P4W Web Article 3

Passport4Work starts ambitious Dutch validation of US-based O*NET framework

Passport4Work has started a widespread, ambitious national study among Dutch employers, job seekers and intermediaries to contextualize and validate t...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Jobs and skills in the local economy
P4W_zoom in

How and why soft skills are important to the Passport4Work?

In his 1st zoom-in, P4W expert Ronald Lievens, explores why soft skills are a key part of the Passport4Work project. Especially in these times, with c...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

P4W Web Article 2

Passport4Work update: abstract skills in layman's terms

As described in the previous web article, P4W is developing tools for skills assessment, eLearnings and job matching....

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Jobs and skills in the local economy
Eindhoven

P4W Expert Journal 1: Get to know the project and what happened in the first 6 months!

In his 1st Journal, Ronald Lievens, UIA expert of the P4W project, discusses all key activities that you need to know about it, what the project is ab...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Emplois et compétences dans l'économie locale
P4W Web Article 1

Passport4Work: Towards a common understanding of skills

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

Jobs and skills in the local economy
An update on Passport4Work

An update on Passport4Work

2020 is already well underway; time for an update on Passport4Work. The project team has not been idle in recent months and a lot has happened. In Jan...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands

All partners of the P4W project

Kick-off of the project Passport4Work

To officially inform our local and national stakeholders, a kick-off for the project was held on Thursday June 13th. This coincided with a national da...

P4W - Passport4Work an intersectoral skills passport with gamified skills assessment and improvement

Jobs and skills in the local economy

Eindhoven - The Netherlands