---
name: openmatter-network/internal-structure-validation
source: https://app.decimal.ai/s/openmatter-network-internal-structure-validation@1/SKILL.md
source_sha256: e5dd6089e938
---

# Internal-structure validation

Evidence about how the **components of a procedure (items, tasks, subscales) relate to one another**
and conform to the conceptual framework that defines the procedure. Useful for **planning and
developing** a procedure and for supporting interpretation of total and subscores.

## The critical limitation — state it up front

Internal-structure evidence **does not, by itself, establish job relatedness.** It shows the
procedure measures the intended construct(s); it does **not** show those scores relate to work
behavior or outcomes. Job relatedness still requires linking scores to the work via
`criterion-related-validation`, `content-based-validation`, or `generalizing-validity-evidence`.
Treat internal structure as *support*, not a stand-alone case.

## Match the analysis to the conceptual framework

The relevant analyses depend on the procedure's intended structure:

- **Single dimension/construct intended** → offer evidence that covariances among components are
  accounted for by a **strong single factor**. Coefficient alpha / internal consistency can be
  appropriate evidence here.
- **Multidimensional structure intended** (hypothesized multifactor measure) → carefully examine
  **dimensionality** (e.g., confirmatory factor analysis fitting the proposed structural model).
  Overall internal consistency may be **inappropriate** as the index of choice.

Item inclusion should be driven **primarily by relevance to the construct/content domain** and only
secondarily by intercorrelations. Well-constructed components with **near-zero correlations** with
other components, scales, or the total score should **not** automatically be eliminated — if the
procedure deliberately spans different construct/content domains (e.g., a battery of a reading test,
an in-basket, and an interview), low inter-component correlations are expected and fine.

## Cautions on coefficient alpha

High internal consistency can be **misleading**:
- A long, multi-dimensional measure can show high alpha **simply because of the number of items**
  (Cortina, 1993), masking multidimensionality.
- A performance-rating form with theoretically unrelated scales can show high alpha because of
  **halo effect**, not true unidimensionality.

So a high alpha is **not** proof of a single construct. Generic internal-consistency indices do not
evaluate internal structure for multidimensional procedures.

## Subscores

When a multidimensional structure is proposed, evidence supporting inferences about **subcomponent**
scores may be needed if those subscores are interpreted or used. Don't report/interpret subscores you
haven't supported.

## Process

1. State the **conceptual framework**: how many dimensions/constructs, and how components map to
   them.
2. Choose analyses that fit that framework (single-factor evidence vs. CFA/dimensionality study).
3. Select components by **construct/content relevance** first; don't purge low-correlating items
   that belong conceptually.
4. Use alpha **only** where a single dimension is intended; otherwise examine dimensionality
   directly and beware item-count and halo artifacts.
5. Support **any** subscores you intend to interpret.
6. Remember to establish **job relatedness separately**.

## Pitfalls

- Treating internal-structure evidence as establishing **job relatedness** — it does not; that
  requires a separate source linking scores to work behavior/outcomes.
- Reporting a high coefficient alpha as proof of unidimensionality (item-count and halo artifacts).
- Purging well-constructed components that correlate near-zero with others when they belong
  **conceptually** to a deliberately multidimensional procedure.
- Using overall internal consistency as the index of choice for a hypothesized multifactor measure.
- Interpreting or reporting subscores without supporting evidence for those subcomponent inferences.

## Checklist

- [ ] Intended dimensionality (uni- vs. multi-) stated before analysis
- [ ] Analysis matched to that framework (single-factor evidence or CFA)
- [ ] Items/components chosen on construct/content relevance, not just correlations
- [ ] Alpha used only for intended-unidimensional measures; artifacts considered
- [ ] Subscores supported if interpreted
- [ ] Job-relatedness evidence obtained via a separate source

## See also

`criterion-related-validation` · `content-based-validation` · `generalizing-validity-evidence` ·
`fairness-and-bias-analysis` (DIF is an item-level structural concern) · `technical-validation-report`

*Source: Principles (5th ed., 2018), "Sources of Validity Evidence → Evidence of Validity Based on
Internal Structure."*