Computerized-Adaptive-Testing

Computerized adaptive testing (CAT) converts a test from a fixed sequence of items into a per-examinee measurement process: estimate the examinee’s latent trait level, choose the next item that is most informative at that estimate, update the estimate after the response, and stop when enough precision has been reached. The central bargain is powerful but unforgiving: CAT can often cut test length by roughly one-third to one-half for comparable precision, but only when supported by a calibrated, secure, sufficiently large item bank and a defensible measurement model.

Coverage note: verified through May 19, 2026.

Core claim

In the classical CAT framework associated with Frederic Lord’s item response theory work and Wainer’s operational CAT primer, the test is not a form but a sequential decision policy over a pre-calibrated Item Response Theory item bank. Lord’s 1980 book is explicitly an IRT treatment of “practical testing problems,” including item parameters, equating, item bias, and information functions; Wainer’s 2000 second edition frames CAT as a practical system involving item pools, calibration, proficiency estimation, testing algorithms, scaling, equating, precision, validity, exposure, and item-pool management (Lord 1980; Wainer 2000). ETS

The simplest CAT loop is:

initialize an ability or trait estimate, usually denoted θ^\hat\thetaθ^;
select an eligible item jjj that maximizes Fisher information Ij(θ^)I_j(\hat\theta)Ij(θ^);
administer the item and observe the response;
update θ^\hat\thetaθ^, usually by maximum likelihood, MAP, or EAP estimation;
recompute the standard error, approximately SE(θ^)≈1/IT(θ^)SE(\hat\theta) \approx 1/\sqrt{I_T(\hat\theta)}SE(θ^)≈1/IT(θ^);
stop when the standard error is below a target threshold, when a classification confidence rule is satisfied, or when a fixed maximum item count is reached.

That loop is a useful abstraction. Operational systems almost never implement a pure “maximum information, no constraints” policy because real tests must also satisfy content specifications, prevent overexposure of high-value items, obey timing and usability constraints, support score comparability, and maintain validity evidence across administrations.

The measurement object: θ\thetaθ, item parameters, and information

CAT assumes that each examinee has a latent proficiency, ability, symptom severity, or trait location θ\thetaθ. For cognitive exams, θ\thetaθ might represent quantitative reasoning ability or nursing competence; for personality measurement, it might represent a facet such as anxiety, assertiveness, dutifulness, or openness to ideas. The test is adaptive because it estimates θ\thetaθ online and uses that estimate to choose the next item.

For a dichotomously scored item under a two-parameter logistic model,

Pj(Y=1∣θ)=logit⁡−1(aj(θ−bj)),P_j(Y=1 \mid \theta) = \operatorname{logit}^{-1}(a_j(\theta-b_j)),Pj(Y=1∣θ)=logit−1(aj(θ−bj)), where aja_jaj is discrimination and bjb_jbj is difficulty or location. In the simplest case, the item is most informative near the point where the examinee has about a 50% chance of answering correctly. A high-discrimination item has a steeper item response curve and, therefore, more Fisher information around its location. For 3PL models used in some multiple-choice contexts, a lower asymptote cjc_jcj models pseudo-guessing; for personality and attitude items, ordered response models are more natural.

For Likert-style personality items, the standard model is often Samejima’s Graded Response Model (GRM), which handles ordered polytomous categories such as “strongly disagree” through “strongly agree.” Samejima’s model family is specifically described as dealing with ordered polytomous categories including attitude-survey response categories and partial-credit-like ordered outcomes (Samejima 1969/1997). Springer Link A GRM item has a discrimination parameter and multiple thresholds; its information is the sum, over response categories, of squared category-probability derivatives divided by category probabilities. In practical terms, a personality CAT chooses items whose response-category boundaries are most informative near the current trait estimate.

The canonical CAT algorithm

A CAT policy can be written compactly as:

j∗=arg⁡max⁡j∈B∖AIj(θ^t),j^* = \arg\max_{j \in \mathcal B \setminus \mathcal A} I_j(\hat\theta_t),j∗=argj∈B∖AmaxIj(θ^t), where B\mathcal BB is the bank, A\mathcal AA is the set of already administered items, and θ^t\hat\theta_tθ^t is the current estimate after ttt responses. In a real test, the argmax is constrained:

j∗=arg⁡max⁡j∈Eligible(B,A,Ct)Ij(θ^t),j^* = \arg\max_{j \in \mathrm{Eligible}(\mathcal B, \mathcal A, C_t)} I_j(\hat\theta_t),j∗=argj∈Eligible(B,A,Ct)maxIj(θ^t), where CtC_tCt encodes content, exposure, enemy-item, answer-key, timing, accessibility, and form-assembly constraints.

Initialization

A CAT needs a starting estimate. Common choices are:

Initialization strategy	Description	Failure mode
Population prior	Start at θ=0\theta = 0θ=0 or the population mean	Wastes early items for extreme examinees
Routing item	Give a medium-difficulty or broadly diagnostic first item	Can overexpose a few starting items
Background-informed prior	Use known covariates or previous scores	Raises fairness, privacy, and validity concerns
Bayesian prior	Use EAP/MAP estimation from the start	Prior can dominate short tests if not chosen carefully

Early in the test, maximum-likelihood estimates can be unstable or infinite if the examinee answers all early items correctly or incorrectly. Bayesian estimators such as EAP or MAP are often used early because they regularize the estimate. This is not merely a computational convenience; it changes early item selection because the posterior, not only the likelihood, drives the next estimate.

Item selection by Fisher information

Maximum Fisher information (MFI) selection is the canonical CAT rule. For achievement tests, this means choosing items near the current θ^\hat\thetaθ^ and with high discrimination. For personality tests, it means choosing items whose Likert thresholds and discrimination parameters are expected to reduce uncertainty about the person’s trait location. The reason CAT can shorten tests is that it avoids spending items far away from the examinee’s current estimated location: very easy items tell little about high-ability examinees, and very hard items tell little about low-ability examinees.

However, the item that maximizes local information at the current estimate is not always the globally best item. Early estimates are noisy, so a highly targeted item can be targeted to the wrong location. For classification tests, the best item may be one that is informative near the cut score rather than near the examinee’s current estimate. For multidimensional tests, an item may be valuable because it improves the determinant, trace, or Kullback–Leibler information of a multivariate posterior rather than maximizing scalar information on one θ\thetaθ.

Updating θ\thetaθ

After each response, the likelihood or posterior is updated. In a dichotomous IRT model,

L(θ)=∏i=1tPi(θ)yi(1−Pi(θ))1−yi.L(\theta) = \prod_{i=1}^t P_i(\theta)^{y_i}(1-P_i(\theta))^{1-y_i}.L(θ)=i=1∏tPi(θ)yi(1−Pi(θ))1−yi. For GRM or other polytomous models, the likelihood multiplies the probability of the observed category for each item. The updated θ^\hat\thetaθ^ can be the maximum likelihood estimate, the posterior mode, or the posterior mean.

The standard error is tied to test information:

IT(θ)=∑i=1tIi(θ),SE(θ^)≈1IT(θ^).I_T(\theta) = \sum_{i=1}^t I_i(\theta), \quad SE(\hat\theta) \approx \frac{1}{\sqrt{I_T(\hat\theta)}}.IT(θ)=i=1∑tIi(θ),SE(θ^)≈IT(θ^)1. This approximation is why information is the natural objective for item selection. Each item contributes information at some trait locations; CAT tries to spend items where the examinee currently needs precision.

Stopping

CAT stopping rules fall into three broad types:

Stopping rule	Typical use	Example
Precision threshold	Estimate θ\thetaθ until SE(θ^)SE(\hat\theta)SE(θ^) is below a target	Personality facets, ability reporting
Classification confidence	Stop when a confidence interval lies clearly above or below a cut score	Licensure/pass-fail exams
Fixed or bounded length	Stop after min/max items or fixed operational length	Admissions, military, research batteries

The NCLEX describes a classification version of this logic: after each item, the computer re-estimates the candidate’s ability from previous answers and item difficulties, selects the next item to obtain maximum information, and typically stops when it is 95% certain that the candidate is above or below the passing standard; it also has maximum-length and run-out-of-time rules (NCLEX CAT). NCLEX

The foundational tradeoff: efficiency for bank infrastructure

The recurring CAT bargain is:

Build and maintain a calibrated item bank, large enough to support selection, exposure control, content balancing, and score comparability; in exchange, administer fewer items per examinee for similar measurement precision.

This is why CAT is common in high-stakes operational testing but less common in small research projects. A fixed form can be built by writing a test and estimating reliability. A CAT requires calibrated item parameters, model fit evidence, item-level information functions, content metadata, exposure controls, software infrastructure, and continuous bank maintenance.

Evidence for the 30–50% reduction claim

The “30–50% fewer items” claim should be treated as a practical range, not a theorem. It depends on bank quality, trait distribution, target precision, dimensionality, exposure constraints, and stopping rules. Still, representative studies support the magnitude.

An early ASVAB-related study found that selected CAT subtests correlated as well with conventional ASVAB retesting even though the CAT subtests used half the number of items, concluding that CAT could achieve the same measurement precision with half the items (Moreno, Wetzel, McBride & Weiss 1984). Experts@Minnesota In personality assessment, Waller and Reise’s Absorption-scale simulations reported a 50% savings in administered items with little loss of precision, and a clinical-decision strategy that identified extreme scorers using about 25% of available items (Waller & Reise 1989). ResearchGate Hol, Vorst, and Mellenbergh simulated CAT with five-category Likert personality data under Samejima’s GRM and reported that, at one stopping rule, about 33% of the 36 dominance items were used while preserving qualitatively equivalent latent trait estimates (Hol, Vorst & Mellenbergh 2008). ResearchGate Nieto and colleagues calibrated a Big Five item pool and reported that administering four items per facet—120 items total from a 360-item final pool—provided accurate facet scores while preserving the FFM structure (Nieto et al. 2017). ResearchGate

The efficiency claim is strongest when the item bank contains many informative items across the full trait range. It weakens when items cluster around the mean, when many items have low discrimination, when the construct is multidimensional but modeled as unidimensional, when content balancing forces weak items into the test, or when exposure controls prevent using the best items.

CAT versus fixed forms and multistage testing

Design	Item sequence	Main advantage	Main cost
Fixed form	Same items for everyone	Simple administration, review, comparability, content control	Inefficient for examinees far from average difficulty
Item-level CAT	Next item chosen after each response	Highest adaptivity and precision per item	Complex exposure, content, review, and validity problems
Multistage testing	Modules/testlets selected adaptively	Operational compromise; easier review and content control	Less adaptive than item-level CAT
Section-adaptive testing	Whole sections adapt based on prior section	Examinees can review within sections; simpler operations	Coarser adaptation

Multistage Testing (MST) is a compromise between fixed forms and item-level CAT. Hendrickson describes multistage tests as administering sets of items adaptively and scoring them as units, giving many advantages of adaptive testing while avoiding some disadvantages of item-level adaptation (Hendrickson 2007). ResearchGate A review of computerized multistage testing notes that CAT adapts after each answered item, while MST selects modules or testlets; MST can help with practical problems such as content constraints, item exposure, and answer review, with relatively little loss of efficiency in some settings (Yan, von Davier & Lewis review). PMC

The GRE General Test is currently section-level adaptive in Verbal Reasoning and Quantitative Reasoning: the first section is of average difficulty and the second section’s difficulty depends on performance on the first section; ETS also emphasizes that this design allows preview, review, marking, and answer changes within a section (ETS GRE structure). ETS That is not Lord-style item-level CAT. It is better understood as a section-adaptive or MST-like operational design.

The item-exposure problem

Pure MFI item selection has an obvious failure mode: the same highly discriminating items near common θ\thetaθ regions get selected repeatedly. In high-stakes testing, that creates a security problem. In low-stakes research, it creates a bank-aging and content-narrowing problem. Wainer’s primer explicitly treats item pools, real item selection, usage patterns, the number of new items required, and item-pool management as central CAT topics rather than implementation details (Wainer 2000). Routledge

Exposure control exists because a CAT bank is not just a statistical object. It is a security asset. The best psychometric item may be the worst operational item if it has already been seen by many examinees.

Common exposure and randomization strategies

Strategy	Mechanism	Benefit	Cost
Randomesque selection	Randomly choose among the top kkk informative items	Simple reduction in overuse	Slight precision loss
Sympson–Hetter-style control	Probabilistically administer selected items based on exposure parameters	Direct cap on exposure rates	Requires calibration or simulation
Stratified selection	Use less-discriminating items early, higher-discriminating items later	Preserves strong items for later precise targeting	Can reduce early efficiency
Shadow testing	Assemble a full constrained “shadow form” at each step, administer one item	Handles content/exposure constraints coherently	Computationally complex
Rotating panels	Use different active pools over time	Operational security and refresh	Requires more items
Content balancing	Enforce content quotas or penalties	Protects construct coverage	May select less informative items

Van der Linden and Veldkamp’s shadow-test approach treats exposure control as constraints on the assembly process; they describe probabilistic item-ineligibility constraints and report negligible impact on estimator bias and mean squared error in an empirical LSAT item-pool study (van der Linden & Veldkamp 2004). Sage Journals The important lesson is that exposure control is not an afterthought added after item selection. It changes the selection policy.

In personality measurement, the exposure problem is usually less severe than in high-stakes licensure or admissions, but it still matters. If a public online personality CAT always selects the same central high-discrimination items, those items become culturally familiar, easier to game, and less representative of the intended construct. For an adaptive IPIP or SAPA-derived instrument, item freshness and content breadth matter even when formal test security is not the primary concern.

Operational assessments

ASVAB

The CAT-ASVAB is an operational example of adaptive testing in military selection. The official ASVAB site describes CAT-ASVAB as an automated system that displays questions, scores responses, computes AFQT and composite scores, and adaptively tailors questions to the individual examinee. All examinees start with a medium-difficulty question; correct responses lead to harder questions and incorrect responses to easier ones; the stated result is higher precision and shorter test length than the paper-and-pencil ASVAB (CAT-ASVAB). ASVAB

The ASVAB case illustrates the “industrial” nature of CAT. The value is not just fewer items. The operational system also supports immediate scoring, standardized instructions, flexible start times, self-paced responding, electronic record handling, and test-score precision. It also imposes constraints: because the test is adaptive, applicants cannot change an answer once they move to the next question. That constraint is not a psychometric theorem; it is a consequence of item-level adaptivity, where the next item depends on the previous response.

NCLEX

The NCLEX is the cleanest public example of the Lord/Wainer CAT loop expressed in operational language. NCSBN explains that after each item, the computer re-estimates ability from previous answers and item difficulties, selects the next item that is optimal for maximum information, and gains precision with each answered item. It then applies pass/fail rules, most commonly a 95% confidence interval rule around the passing standard, with maximum-length and run-out-of-time fallbacks (NCLEX CAT). NCLEX

This is a classification CAT. The goal is not merely to estimate θ\thetaθ as precisely as possible everywhere. The goal is to decide whether the candidate’s competence is above a legally and professionally meaningful threshold. That distinction matters: classification tests can stop early for examinees clearly above or below the cut, while examinees near the cut require more items.

GRE

The GRE is often casually grouped with CAT, but its current General Test design is section-level adaptive, not item-level adaptive. ETS states that the first Verbal and Quantitative sections are of average difficulty, the second section’s difficulty depends on performance on the first, and scoring accounts for both number correct and section difficulty (ETS GRE structure). ETS

This design shows why MST-like systems remain attractive. Section-level adaptivity gives up item-by-item precision gains, but it preserves features examinees value: previewing, marking, reviewing, and changing answers within a section. For high-stakes public testing, those interface and fairness properties may outweigh the marginal precision gains of item-level CAT.

Personality measurement: why CAT is tempting and why it is harder than it looks

Personality inventories are unusually attractive targets for CAT because they are long, redundant by design, and composed of ordinal self-report items that can be modeled using GRM or related polytomous IRT models. A 300-item personality inventory imposes fatigue, lowers completion rates, and can degrade data quality. A CAT could ask fewer items while retaining facet-level precision.

But personality CAT is not just “educational CAT with Likert items.” It faces distinctive problems:

personality traits are often multidimensional and hierarchical;
response styles such as acquiescence, extremity, and social desirability can contaminate trait estimates;
items are not “correct” or “incorrect,” so information comes from endorsement thresholds, not difficulty alone;
validity often depends on broad content coverage, not only statistical precision;
facets can be correlated, allowing multidimensional borrowing but also risking construct blur.

Early personality CAT

Waller and Reise’s 1989 Absorption-scale work is a foundational demonstration. They used real-data simulations from 1,000 participants who had taken a 34-item Absorption scale and compared fixed-length and clinical-decision adaptive strategies. The study reported a 50% savings in administered items with little loss of precision and identified extreme scorers with perfect accuracy using about 25% of the available items on average (Waller & Reise 1989). ResearchGate

Reise and Henson extended the logic to computerized and adaptive administration of the NEO PI-R; Makransky, Mortensen, and Glas later investigated multidimensional CAT for NEO PI-R facet scores. Makransky et al. argued that narrow personality facet scores are commonly used in clinical and organizational settings, but scoring facets separately can be inefficient when facets are correlated; their MCAT approach exploited facet correlations to improve precision and shorten the NEO PI-R without attenuating precision (Makransky, Mortensen & Glas 2013). Sage Journals

Hol, Vorst, and Mellenbergh modeled ordinal polytomous personality responses with Samejima’s GRM and simulated CAT on a Dutch dominance scale. Their study is important because it directly addresses Likert personality data rather than dichotomous achievement items; it found substantial item savings while preserving qualitative equivalence between CAT estimates and full-pool estimates (Hol, Vorst & Mellenbergh 2008). ResearchGate

Nieto and colleagues provide a direct Big Five CAT example. Their 2017 study built a 480-item pool for Five Factor Model facets, calibrated facets separately, removed weaker items, and retained a 360-item final pool. Their post-hoc simulations found that four items per facet—120 total items—could provide accurate facet scores while maintaining the FFM structure (Nieto et al. 2017). ResearchGate

A later Nieto, Abad, and Olea study compared unidimensional CAT, multidimensional CAT, and bifactor multidimensional CAT for Big Five measurement. It reported that with 12 items per domain, multidimensional CAT and bifactor CAT were more efficient for highly multidimensional constructs such as Agreeableness, while differences were smaller for more nearly unidimensional traits such as Extraversion. It also found better content balance across facets under MCAT and MCAT-B (Nieto, Abad & Olea 2018). ResearchGate

This is directly relevant to any Big Five Facets use case. If the goal is only broad-domain scoring, a domain-level CAT may be short. If the goal is facet-level scoring across all 30 facets, the test must sample every facet or use a multidimensional model that borrows information from correlated facets. Borrowing can improve efficiency, but it cannot eliminate the need for validation at the facet level.

Personality disorder and clinical CAT

The CAT-PD project shows CAT’s role in dimensional personality pathology. Simms, Goldberg, Roberts, Watson, Welte, and Rotterman introduced a project aimed at identifying a comprehensive set of personality-disorder-relevant traits, developing a large initial item pool, calibrating final item sets using IRT, and using CAT to measure the resulting traits efficiently (Simms et al. 2011). acsu.buffalo.edu

The CAT-PD lineage matters because it shifts the conversation from “shorter normal-range personality tests” to adaptive assessment of clinically relevant trait continua. That is closer to the broader movement toward dimensional taxonomies in psychopathology, including HiTOP. A source-integrity note is necessary here: the prompt’s “Forbes et al. 2017 — CAT for Big Five and HEXACO” appears to conflate sources. The verifiable 2017 paper with Forbes as a coauthor is the Kotov et al. HiTOP paper, a dimensional psychopathology taxonomy article, not a Big Five/HEXACO CAT paper; it is relevant to dimensional clinical measurement, but not direct evidence for CAT item selection (Kotov et al. 2017). Experts@Minnesota The directly relevant Big Five CAT literature is Waller & Reise, Reise & Henson, Makransky et al., Hol et al., and Nieto et al.

SAPA, IPIP, HEXACO, and public-domain item banks

CAT needs item banks. Public-domain personality infrastructure makes CAT more plausible outside proprietary assessment vendors.

The International Personality Item Pool (IPIP) official site states that it includes more than 3,000 items and over 250 scales, and that the items and scales are public domain (IPIP). IPIP The IPIP NEO facet key lists preliminary IPIP scales corresponding to the 30 NEO-PI-R facet scales, with keyed and reverse-keyed items for each facet (IPIP NEO facets). IPIP Johnson’s IPIP-NEO-120 paper states that the IPIP-NEO is a 300-item inventory measuring constructs similar to the NEO PI-R’s 30 facet scales, and that the 120-item version was designed to represent both the five broad domains and 30 facets with four items per facet (Johnson 2014). Bpb Us E1 Wpmucdn

SAPA—the Synthetic Aperture Personality Assessment project—adds a large-scale calibration logic: rather than every participant answering every item, participants answer overlapping random subsets from a large item pool, enabling large covariance and item-parameter estimation with planned missingness. Recent summaries describe the SAPA Personality Inventory as using 696 unique IPIP items and very large samples, with random item sampling from a larger pool (SAPA / Irwing et al. 2024). Sage Journals The SAPA tradition is not identical to CAT. SAPA is primarily a data-collection and bank-calibration method; CAT is an adaptive administration method. But SAPA-like planned missingness is highly relevant to building the calibrated banks that CAT requires.

HEXACO is also relevant because it introduces a six-domain alternative to the Big Five, including Honesty-Humility, Emotionality, Extraversion, Agreeableness, Conscientiousness, and Openness. The HEXACO-PI-R site describes the instrument as assessing six major personality dimensions (HEXACO-PI-R). HEXACO CAT for HEXACO-style measurement is plausible, but the evidence base is thinner than for Big Five/NEO-style CAT unless a specific calibrated HEXACO bank is available and validated.

For a system like Psyche that wants rich personality measurement without a 300-item burden, CAT is an obvious candidate. The IPIP-NEO-300 covers five domains and 30 facets; Johnson’s paper describes the original 300-item IPIP-NEO as measuring constructs similar to the NEO PI-R’s 30 facet scales, with six narrower facets in each broad domain (Johnson 2014). Bpb Us E1 Wpmucdn A naive fixed short form, such as four items per facet, reduces burden but treats item selection as one-size-fits-all. A CAT version would instead maintain a calibrated item pool for each facet and select the most informative next item for each person.

A practical design for an IPIP-NEO-300 CAT would look like this:

Component	CAT design choice
Measurement target	30 facet-level θ\thetaθ estimates, optionally nested in five domains
Item model	GRM for ordered Likert responses
Calibration data	Large sample with full IPIP-NEO-300, SAPA-style planned missingness, or both
Bank	More than the original 10 items per facet if possible; original items alone are a minimal bank
Selection	Per-facet MFI, MCAT, or bifactor MCAT depending on validated structure
Stopping	Facet-level SE threshold, minimum items per facet, maximum total length
Output	Facet θ\thetaθ, domain aggregates, posterior SEs, confidence bands
Validity checks	Correlation with full IPIP-NEO-300, NEO PI-R convergence, factor structure, response-style robustness

The strictest version—preserve 30 independent facet-level theta estimates—cannot shrink indefinitely. Each facet must receive enough items to estimate its location. Nieto et al.’s Big Five facet CAT result gives a realistic lower-bound intuition: four items per facet, or 120 items total, can work when the bank has been built and calibrated for that purpose (Nieto et al. 2017). ResearchGate Johnson’s IPIP-NEO-120 fixed form also lands at four items per facet and reports generally favorable psychometric properties, while warning that shorter facet scales are likely suitable for research but probably not for important individual decisions (Johnson 2014). Bpb Us E1 Wpmucdn

A more aggressive Psyche design could use multidimensional CAT. If facets within a domain are correlated, a response to one facet item gives partial information about neighboring facets. Makransky et al. show the rationale for MCAT in personality facets: exploiting correlations among facets can improve precision and shorten administration (Makransky et al. 2013). Sage Journals Nieto et al. 2018 adds that bifactor and multidimensional CAT can improve efficiency and content balance for highly multidimensional Big Five domains (Nieto, Abad & Olea 2018). ResearchGate

The hard boundary is validity. A CAT can preserve estimated precision while silently narrowing content if it repeatedly selects the same high-information items. For Psyche, the correct success metric is not only lower SE. It is whether the adaptive facet estimates preserve the interpretive meaning of the IPIP-NEO-300 facets, predict the same external criteria, maintain the same factor structure, and remain robust to response styles and adversarial self-presentation.

Limitations

Item-bank quality is the bottleneck

CAT cannot rescue a bad bank. It amplifies the bank’s properties. If the best items are narrow, biased, locally dependent, or written in a way that confounds reading level with the target trait, the adaptive algorithm will select them more often. The better the selector, the more it exploits the bank’s weaknesses.

A defensible CAT bank needs:

enough items at each relevant θ\thetaθ location;
high-discrimination items without excessive redundancy;
content metadata for balancing;
evidence of unidimensionality or a validated multidimensional model;
differential item functioning checks;
stable item parameters across populations and time;
exposure and retirement policies;
enough new-item seeding to replenish the bank.

CAT forms are harder to construct and explain

A fixed test has a form. A CAT has a policy. That makes test development harder to audit. Two examinees can receive different items, different content mixtures within allowed constraints, and different numbers of items. The reported score may be comparable, but the lived test experience differs.

That difference creates communication problems. Examinees may feel punished by harder items, even though being routed to harder items can indicate strong performance. In clinical or personality contexts, users may distrust a short adaptive profile unless the report explains both the estimate and its uncertainty.

Validity concerns when items differ across examinees

The psychometric argument for CAT is conditional score comparability: if the IRT model is correct and the bank is calibrated, different item sets can yield comparable θ\thetaθ estimates. The validity argument is broader. It must show that adaptive scores support the intended interpretation and use.

Threats include:

Threat	Why it matters
Content undercoverage	MFI may overselect narrow high-discrimination items
Local dependence	Similar items can inflate information
DIF	Items may behave differently across groups
Response style	Acquiescence/extremity can distort Likert trait estimates
Construct drift	Online populations or language norms can change
Coaching/gaming	Exposed items and transparent wording invite strategic responses
Algorithmic opacity	Users may not understand why different people receive different items

CAT’s promise is measurement efficiency, not automatic validity.

Active research directions

Multistage testing as an operational compromise

MST remains important because it reduces some of CAT’s operational friction. Module-level adaptation preserves some efficiency while allowing better content control, easier review within modules, and simpler form documentation. This is why section-adaptive and module-adaptive designs persist in high-stakes programs even when item-level CAT is psychometrically attractive. The GRE’s current section-level adaptive design is a live example of this tradeoff (ETS GRE structure). ETS

Personality structure is hierarchical: broad domains contain facets, facets correlate, and response styles can introduce method factors. Multidimensional CAT and bifactor CAT are natural tools for this setting. The key research question is not whether MCAT can reduce standard errors—it can—but whether the model’s borrowing of information preserves facet interpretation.

For personality inventories, MCAT can improve content balance and efficiency, especially for highly multidimensional domains; Nieto et al. 2018 is direct evidence for this in Big Five measurement (Nieto, Abad & Olea 2018). ResearchGate But multidimensional borrowing can also blur the line between “we measured this facet” and “we inferred this facet from nearby traits.” For high-stakes personality use, that distinction matters.

Automatic item generation and LLM-built banks

The biggest practical bottleneck in CAT is bank development. Recent work explores automatic item generation, including LLM-generated personality item banks. Gao and colleagues’ 2026 Assessment article reports using three LLMs to generate Simplified Chinese Big Five personality items and evaluate the resulting adaptive item bank through two empirical testing rounds, framing LLM-based item generation as a way to address CAT item-bank bottlenecks (Gao et al. 2026). Sage Journals

This is promising but not settled. LLM-generated items can be fluent, scalable, and semantically targeted, but fluency is not validity. Generated items still require human review, calibration, DIF checks, response-process evidence, external validity evidence, and monitoring for construct contamination. A large synthetic bank is not a calibrated bank until real responses have estimated its parameters.

Open question: does CAT transfer to AI-elicited responses?

The most Psyche-relevant open question is whether CAT’s gains carry to less-validated domains such as AI-Elicited Psychometrics: conversational responses, reflective essays, simulated interviews, or model-mediated personality probes. CAT works best when the response model is explicit, calibrated, and stable. AI-elicited responses violate or complicate each condition.

A Likert item has a fixed response space. A conversational prompt produces open-ended text. To use CAT, the system must convert that response into scored indicators—perhaps by human coding, an embedding model, a classifier, or an LLM judge. That adds a second measurement model on top of the item response model. The adaptive system may be precise with respect to the scorer while invalid with respect to the psychological construct.

The CAT framework can still be useful. An AI-elicited adaptive personality system could choose the next prompt based on posterior uncertainty over facets, ask discriminating follow-ups, and stop when uncertainty falls below a threshold. But the evidence base is thin. The field has stronger evidence for CAT over calibrated educational and self-report item banks than for adaptive measurement over AI-scored natural-language responses. Until AI-elicited response models demonstrate stability, invariance, and external validity, CAT-like precision should be treated as algorithmic confidence, not psychological truth.

Practical design checklist

A serious CAT implementation should answer these questions before deployment:

Layer	Required question
Construct	What exactly is θ\thetaθ, and what interpretations does it support?
Bank	Are there enough calibrated items across the trait range?
Model	Does IRT/GRM/MIRT fit well enough for the intended use?
Selection	Is the objective maximum information, classification information, content balance, or a hybrid?
Exposure	How are item overuse and test overlap controlled?
Stopping	Is stopping based on SE, classification confidence, fixed length, or a policy mixture?
Validity	Do adaptive scores reproduce the full-form structure and external relations?
Fairness	Is there DIF across demographic, linguistic, cultural, and accessibility groups?
UX	Can examinees understand why item difficulty changes and why review may be restricted?
Maintenance	How are new items seeded, calibrated, retired, and monitored?

The design should be evaluated against both psychometric and operational metrics: RMSE, bias, conditional SE, classification accuracy, item exposure, test overlap, content coverage, completion time, dropout, user comprehension, and external validity.

Summary

Computerized adaptive testing is best understood as information routing under a measurement model. The Lord/Wainer framework selects items per examinee by maximizing information at the current θ\thetaθ estimate, updates θ\thetaθ after each response, and stops when the estimate is precise enough or an operational item limit is reached. In exchange for the heavy infrastructure of calibrated item banks, model checking, exposure control, and validity work, CAT can substantially reduce item burden—often in the 30–50% range in favorable settings.

For operational tests such as NCLEX and CAT-ASVAB, CAT is already part of mature assessment infrastructure. For GRE-like settings, multistage or section-adaptive testing offers a compromise between adaptivity and operational usability. For personality measurement, CAT is especially attractive for long inventories such as the IPIP-NEO-300, Big Five facet batteries, HEXACO-style instruments, and clinical trait systems such as CAT-PD. But personality CAT must preserve facet meaning, not merely reduce standard errors. The frontier is not the CAT algorithm itself; it is whether item banks, multidimensional models, and AI-assisted item or response generation can support the same level of validity evidence that made CAT successful in more mature testing domains.

References

Lord 1980 — Frederic M. Lord, Applications of Item Response Theory to Practical Testing Problems.
Wainer 2000 — Howard Wainer et al., Computerized Adaptive Testing: A Primer, 2nd ed.
Samejima 1969/1997 — Fumiko Samejima, graded response model for ordered polytomous categories.
NCLEX CAT — NCSBN, official description of NCLEX computerized adaptive testing and stopping rules.
Moreno, Wetzel, McBride & Weiss 1984 — ASVAB/CAT subtest comparison showing half-length CAT precision.
Waller & Reise 1989 — computerized adaptive personality assessment with the Absorption scale.
Hol, Vorst & Mellenbergh 2008 — CAT simulation for ordinal polytomous personality data under the GRM.
Nieto et al. 2017 — Big Five item-pool calibration and adaptive facet assessment.
Hendrickson 2007 — NCME instructional module on multistage testing.
Yan, von Davier & Lewis review — review of computerized multistage testing.
ETS GRE structure — official GRE section-level adaptation description.
van der Linden & Veldkamp 2004 — shadow-test exposure control.
CAT-ASVAB — official CAT-ASVAB description.
Makransky, Mortensen & Glas 2013 — multidimensional CAT for NEO PI-R facets.
Nieto, Abad & Olea 2018 — bifactor computerized adaptive testing for Big Five domains/facets.
Simms et al. 2011 — CAT-PD project.
Kotov et al. 2017 — HiTOP dimensional psychopathology taxonomy, with Forbes as coauthor.
IPIP — International Personality Item Pool.
IPIP NEO facets — IPIP items corresponding to 30 NEO-PI-R facet scales.
Johnson 2014 — IPIP-NEO-120 development from the IPIP-NEO-300.
SAPA / Irwing et al. 2024 — SAPA Personality Inventory and large IPIP-based item-pool use.
HEXACO-PI-R — official HEXACO Personality Inventory-Revised site.
Gao et al. 2026 — LLM-generated Big Five CAT item bank.

Companion entries

Core theory: Item Response Theory, Fisher Information, Test Information Function, Theta Estimation, Graded Response Model, Bayesian Ability Estimation, Differential Item Functioning

Adaptive testing practice: Computerized Adaptive Testing, Exposure Control, Shadow Testing, Item Bank Calibration, Content Balancing, Multistage Testing, Section-Adaptive Testing, Operational Assessment Validity

Personality measurement: Big Five Facets, HEXACO, IPIP-NEO-300, IPIP-NEO-120, SAPA, CAT-PD, Multidimensional CAT, Bifactor Personality Models

Psyche applications: Psyche, Adaptive Personality Assessment, Facet-Level Measurement, Personality Item Banks, AI-Elicited Psychometrics, LLM-Generated Item Banks

Counterarguments and risks: Validity Under Adaptive Forms, Construct Drift, Response Style Bias, Local Item Dependence, Test Security, Algorithmic Confidence Versus Psychological Validity