Influencing Job Outcomes with Data
Background
The NJ Department of Labor wanted to improve job outcomes for its residents through training opportunities. Residents were vulnerable to investing time and money in training that wouldn’t yield the type of job outcomes that they hoped for.
The state received a grant through the Data for the American Dream (D4AD) initiative to build an online tool to help residents make better decisions about job-related education through labor data. The state had years of data on training providers, in-demand occupations, and student outcomes.
The previous tool had been built more than 15 years ago and was difficult for people to use and the department to update with fresh information.
Product Goals
Help the unemployed and underemployed in NJ make more informed decisions about which training opportunities to pursue
Make One-Stop Career Center counselors more efficient and effective at recommending training opportunities that qualify for state funding
Long Term North Star: Improve the job outcomes of the unemployed and underemployed in the state of NJ
Team
2 Product Managers
2 Developers
1 Product Designer (That’s me!)
Phase 1: Exploratory Research
We did several rounds of research with both “Explorers” (low income, unemployed or underemployed, living in New Jersey) and “Counselors” (those who worked in government centers to help Explorers find trainings and qualify for funding). These rounds of research fell into a few distinct phases:
Phase 1: Exploratory Research to understand the needs and habits of our core user base
Phase 2: Validation Research to get user feedback on an initial concept
Phase 3: Usability Research on our beta to get user feedback on live product
For Phase 1: Exploratory Research, our learning goals were:
Figure out our target demographic
Understand and prioritize the top pain points to solve for within our demographic
Methods:
Outreach: local non-profits targeting similar demographics, screened for unemployed and low-income through user testing service, NJ career counselors
Interviews: open-ended questions about their job search, past training experiences, understanding of state services
Through this first round of interviewing, we decided to prioritize low-income, long-term unemployed users and we formed a persona that included a combination of the pain points and behaviors we observed during our research.
Phase 2: Validation of EArly Concept
We brainstormed many different product concepts with ideas as far flung as creating a YouTube channel targeted to job hunters, but we ultimately decided to wireframe and test the concept of a basic search engine for finding local training opportunities. Here is the set of those wireframes:
Value Prop Test
Here we wanted to test the value prop of the whole product and find out if people wanted to know more about the funding process, search for classes, learn about student outcomes or something else.
Search Results Exploration
If they got to this sparse second page of search results, we wanted to know what details were missing for them. What did they want to know about each training opportunity?
Key Learnings
People went straight for the search
Many liked the tagline of “Get $4000 from the state”
Many wanted to understand what they would get out of a training in terms of skills or certifications, degrees
All found it hard to compare providers
Validated that we were going in the right direction with this general flow and format
Cost was by far the most important factor for everyone when assessing learning opportunities
Next Iteration
Basic search with keyword matches on description & title
Training details page with more information on logistics, outcomes, provider info
More logistics information exposed on search listing
Filters for searches
Launching Beta
Audience:
A subset of “Explorers” we had already interviewed
Learnings from Beta:
Are people finding the trainings they want to find?
Mostly, though they’re still getting too many irrelevant results
Missing data made the results look fake or untrustworthy
Can they filter to see only results that are relevant to them?
People generally understand the filters but often misuse the Employment Rate filter
Many are surprised by how few results they see after applying filters
Most requested filter: location
Once they look at a specific training, do they have enough information to assess the class?
Lots of confusion over the salary graph
Wanted more information on careers
Do they know what to do next in order to apply for state funding for the training?
Many were unsure of next steps
A Few More Iterations & Additions
DESIGN SYSTEM
Key considerations
Maximum accessibility
Easy maintainability (only 1 engineer, and eventually no designer)
Modularity for creating new page layouts (provider pages, etc.)
Current State
See the latest version of this product (my last involvement was in 2019)