RAND/UCLA Appropriateness Method

The RAND/UCLA Appropriateness Method (RAM) rates the appropriateness of clinical scenarios — typically whether a procedure, test, or treatment is appropriate, uncertain, or inappropriate for a defined patient presentation.

Delphi Studio implements RAM through the RAND/UCLA Appropriateness preset: 9-point scale, appropriateness dimension, and the Disagreement Index (DI).

When to use it

Appropriate	Less appropriate
Procedure indication panels	Competency agreement (use modified e-Delphi)
Diagnostic test ordering	Policy priority ranking
Treatment pathway review	Curriculum learning outcomes

RAM items are usually scenario-based: a clinical vignette plus a proposed action to rate.

Example item:

68-year-old woman with stable angina, no diabetes, normal renal function. Rate the appropriateness of immediate coronary angiography within 24 hours.

The 9-point scale

Score	Meaning
1	Extremely inappropriate
2
3
4
5	Equivocal
6
7
8
9	Extremely appropriate

Default consensus rules

Rule	Default threshold
Median for appropriate	≥ 7
Median for inappropriate	≤ 3
% in target zone	≥ 75%
IQR	≤ 2
Disagreement Index (DI)	≤ 1.0
Bimodality detection	Enabled

Shared guardrails

The RAND/UCLA preset applies the same platform-wide analysis guardrails as modified e-Delphi:

• Items with fewer than 3 valid ratings are classified insufficient_data rather than scored.
• Response rate below 70% adds a warning; below 50% round close is held until the researcher acknowledges the low response.
• Presets default to a minimum of two rounds; the first round always returns insufficient_history because no prior round exists for comparison.
• Stability requires round-over-round change in median ≤ 1 and IQR ≤ 1; if more than 20% of items are unstable, another round is recommended.
• Kendall's W is computed automatically each round as a concordance check.

Disagreement Index (DI)

The DI distinguishes true agreement from polarized panels that happen to have a median in the target zone.

DI = (P70 − P30) / 2.35

Where P70 and P30 are the 70th and 30th percentiles of ratings.

DI value	Interpretation
≤ 1.0	Low disagreement — consensus credible
Greater than 1.0	High disagreement — panel split despite median
null	Fewer than 3 valid ratings, or non-9-point scale

Delphi Studio computes DI automatically on 9-point appropriateness rounds. Items with acceptable median but high DI remain indeterminate until a further round or adjudication.

Panel requirements

RAND/UCLA manuals traditionally recommend two panels (e.g. internal medicine and surgery) rating the same scenarios. Delphi Studio supports:

• Subgroup analysis by expertise field captured at enrollment
• Single-panel studies when resources are limited (document the limitation in methods)

Panel size guidance	Notes
9–12 per subgroup	Common in published RAM studies
Minimum 3 for DI	DI is not computed below n = 3

Subgroup divergence downgrade

When cross-stakeholder consensus conditions are configured, panel-wide agreement can be downgraded to consensus_with_subgroup_divergence: if a prespecified stakeholder group of at least the configured minimum size shows opposition above the configured threshold, the item is not retained as consensus and instead carries forward for re-rating. This prevents a numerical majority from silently overriding a smaller stakeholder group's position.

Differences from modified e-Delphi preset

Aspect	Modified e-Delphi	RAND/UCLA
Scale	5-point Likert	9-point appropriateness
Primary dimension	Agreement	Appropriateness
DI	Not used	Required
New item suggestions	Allowed	Disabled by default
Typical items	Statements	Clinical scenarios

Classification flow

Reporting

RAM studies should report:

• Panel specialty mix and selection process
• Scenario development (literature review, focus groups)
• Per-scenario median, % appropriate/inappropriate, DI
• Round-over-round movement for contested scenarios

Delphi Studio's study report and CREDES export include appropriateness-specific tables when the RAND/UCLA preset is used.

Next steps

• Comparing Delphi types — choose the right preset
• Interpreting results — read heatmaps and dispositions
• Clinical guideline example — RAM-style scenario setup