Analysis of work

"Analysis of work" is the Principles' umbrella term for what has been called job analysis, work analysis, and competency modeling. It is the empirical foundation that links a selection procedure to the job. Most other validation strategies depend on it.

Two purposes — be clear which you're serving

1. Define worker requirements (predictors). Identify the KSAOs/competencies workers need to be

successful, and how similar requirements are across settings or job families.

1. Develop criteria. Assemble the information needed to understand the work, its setting, and

the organization's goals so you can build relevant, uncontaminated criterion measures.

A single analysis can serve both, but design it deliberately for the purpose(s) at hand.

Level of detail — match it to the use

The required detail depends on the intended use and the availability of existing information.

• Less detail may suffice when prior research lets you generalize requirements across a job

family, or when an indicator's relevance is self-evident (e.g., absenteeism, turnover are relevant to virtually all jobs).

• More detail is needed when little work information exists, or when you intend to develop

predictors of specific job knowledge (content-based work especially).

There is no single preferred method. Choose methods based on the nature of the work, the organizational setting, the workers, and the study's purpose, informed by the research literature.

Constructs vs. methods (frame the requirements correctly)

Distinguish what is measured (the predictor construct, e.g., general mental ability, conscientiousness, job knowledge) from how it is measured (the method, e.g., interview, work sample, test). Confounding the two (e.g., "the interview" as if it were a construct) produces uninterpretable comparisons later. Define requirements at the construct level where possible.

Process

1. Scope the work domain. Work complexity, environment, context, tasks, behaviors, activities,

and worker requirements. Decide which dimensions of work matter for your purpose.

1. Choose methods appropriate to the work and purpose (e.g., observation, interviews,

questionnaires, critical incidents, task inventories, existing documentation, ONET, competency modeling). Any method used should be understood by participants and have reasonable psychometric properties; note and investigate lack of SME consensus.

1. Plan the SME sample. Include incumbents/SMEs spanning relevant shifts, locations, equipment,

software, experience levels, and demographics. A broad, representative SME sample increases the representativeness of results. Don't assume same job title = same work — study multiple incumbents when complexity, context, or behaviors may differ.

1. Vet existing documentation. Existing job descriptions and prior analyses may or may not serve

the current purpose; evaluate relevance and currency before relying on them.

1. Account for changing work. When work is being transformed (or the job doesn't exist yet),

obtain reliable, relevant information about anticipated behaviors, activities, and KSAOs/competencies. Reanalyze when the nature of work has changed meaningfully since any prior analysis.

1. Derive worker requirements and/or criterion content. Translate work information into

KSAOs/competencies (for predictors) and into work behaviors/outcomes (for criteria).

1. Document methodology, data collection, analyses, results, and implications — including the

major work activities, important worker requirements, and their relationships to selection procedure content and scoring. Feeds the technical-validation-report.

Competency modeling — when it counts as work analysis

Competency models are widely used (for training, selection, compensation, signaling aspirational goals). A competency model can serve as the foundation for content-oriented or other validation only if it is detailed and rigorous — i.e., built like a rigorous work analysis, not a list of aspirational labels. You must judge whether the model is rigorous enough to support the intended inference. (See Campion et al., 2011, on best practices.)

Pitfalls

• Letting a glossy competency model substitute for a rigorous analysis when content evidence rests

on it.

• Grouping dissimilar jobs under one title and analyzing them as if homogeneous.
• Defining requirements as methods ("we need interviewing") instead of constructs.
• Using stale job descriptions for rapidly changing work.
• Insufficient/biased SME sampling.

Checklist

• ] Purpose (predictor requirements, criterion development, or both) stated
• ] Level of detail justified by the intended use
• ] Methods chosen and rationale recorded; psychometric quality considered
• ] SME sample spans relevant subgroups, shifts, locations, experience
• ] Requirements expressed as constructs (KSAOs/competencies), not methods
• ] Changing/future work accounted for if relevant
• ] Results documented and linked to selection procedure content and/or criteria

See also

validation-planning (upstream) · content-based-validation and criterion-related-validation (downstream consumers) · technical-validation-report

Source: Principles (5th ed., 2018), "Overview → Analysis of Work" and "Operational Considerations → Understanding Work and Worker Requirements."