Interpreting results

After a round closes, the Results view becomes your main decision-making surface. It brings together statistical output, flags, and system recommendations so you can determine which items have reached consensus, which need revision or another round of rating, and whether the study is ready to conclude.

Per-item results

The Results view lists every item in the round as a card carrying its statistics, classification, and any flags. The round view offers the same information as a compact table.

Column	What it shows	How to use it
Item text	Current wording (with revision indicator if changed)	Identify which version of the item was rated
Median (IQR)	Central tendency and spread	Lower IQR indicates greater agreement
% agreement	Share of ratings at or above the agree-zone floor, shown as "N% ≥ K"	Compare against the study's agreement threshold
Distribution	Rating counts across the scale's zones	Reveals whether agreement is broad or clustered
Disposition	Decision badge — Included, Re-rate, Excluded, or Awaiting — alongside the computed outcome (consensus_in, consensus_out, no_consensus, consensus_with_subgroup_divergence, or insufficient_data)	Starting point for your decision

Two flags appear inline on the item itself:

• Bimodality: at least the configured share of ratings sits in each of the agree and disagree zones, with few in the middle — strong opinions at both ends
• Unable to rate: a large share of abstentions, suggesting the item may not be evaluable by the full panel

Related signals appear elsewhere rather than as item flags: instability since the previous round is summarized in the stability panel and top-movers table; too few valid ratings produces the insufficient_data outcome instead of a consensus class; and stakeholder divergence surfaces as the consensus_with_subgroup_divergence outcome, carried into Plan next round and the final report.

The Results view combines per-item statistics, zone classification, disposition recommendations, and flags in a single decision-making surface.

Reading a single item

Example 1 — Clear consensus (retain)

Field	Value	Interpretation
Median	4.5	Strong central agreement
IQR	0.5	Very low dispersion
% Agree (4–5)	93%	Overwhelming support in the agree zone
Zone	Agree	Meets all consensus criteria
Disposition	retain	Ready to include in final set

Example 2 — False consensus risk

Field	Value	Interpretation
Median	4	Appears acceptable at first glance
IQR	2	High dispersion
Bimodal flag	Yes	Significant opinions at both ends
Zone	Indeterminate	Correctly held back despite median

In this case, the median looks acceptable, but high spread and bimodality correctly prevent a consensus classification.

Disposition decisions

Use the system's recommended disposition as a starting point, then apply your judgment:

Disposition	When to use	Typical outcome
retain	Clear consensus in the Agree zone	Item joins the final set
drop	Clear consensus in the Disagree zone or team decides to exclude	Item removed from study
revise	Indeterminate + wording issues identified	Item reworded and re-rated in next round
re_rate	Indeterminate but wording is acceptable	Item carried forward unchanged for another round

Overrides

You may override the system's recommended disposition before you confirm it. Overrides require a rationale of at least 20 characters and are recorded in the audit log with the previous and new decision. Once a disposition is confirmed it is frozen: further corrections go through a derived re-analysis rather than an edit, so the contemporaneous result stays intact.

Common scenarios for overrides include:

• Borderline IQR: Retaining an item with a note in the discussion section
• Clinical or safety imperative: Revising wording even when the panel reached agreement
• Panel fatigue: Dropping unresolved items after the maximum number of rounds

Overrides should be used thoughtfully and transparently.

Derived re-analysis

Use derived re-analysis when the frozen round result is still correct as history, but a correction scenario requires a recomputed view:

Scenario	Example
Withdrawal	A panelist withdraws after Round 2 closes; you need to show which items would change if that panelist were excluded
Data correction	A panelist was enrolled in the wrong stratum or a misclassification is discovered

How it works:

1. Choose a closed or analyzed round and the panelists to exclude.
2. Delphi Studio recomputes per-item statistics and classification against the same frozen consensus rules.
3. You receive a per-item diff (original status → new status) and a changed-item count.
4. The original dispositions and hash-chained audit log are never rewritten.
5. Optionally designate the re-analysis as the authoritative reference for the manuscript while keeping the original for the record.

Re-analyses appear in the Results view's re-analysis panel and travel in the trace bundle as analysis_versions.json, so reviewers can see both the contemporaneous result and the corrected view.

Sensitivity analysis (post hoc)

Threshold sensitivity compares how dispositions shift under alternative rules (for example 80% vs 75% agreement). Always present these as post hoc robustness checks, not as the pre-specified analysis path.

Stopping recommendation panel

After analysis completes, review the stopping recommendation. Common signals include:

Signal	Suggested action
All (or nearly all) items classified as retain or drop	Stop the study and move to reporting
Small number of stable indeterminate items	Adjudicate the remaining items or document as a limitation
Majority of items still moving	Plan another round
Maximum number of rounds reached	Conclude with a clear table of unresolved items

The recommendation is advisory. You should consider it alongside response rates, stability trends, and the overall goals of the study.

Round-over-round movement

The trajectory view helps you assess whether the panel is converging:

Pattern	Interpretation	Implication
Median drifting toward the agree zone	Convergence occurring	Positive sign — consider continuing
IQR shrinking	Opinions becoming more consolidated	Good progress toward stability
Frequent zone flips (Agree ↔ Indeterminate)	Unstable opinions	May need more rounds or item revision
Little change across two rounds	Stability achieved	Candidate for stopping

This view is particularly useful when deciding whether another round is likely to produce meaningful movement.

Subgroup divergence

When different subgroups (e.g., by specialty or experience level) show meaningfully different medians, you have several options:

1. Note the divergence in the results.
2. Decide on an approach: report overall consensus with a caveat, aim for stratified consensus, or revise the item.
3. Document your decision and rationale in the manuscript discussion.

Subgroup A median	Subgroup B median	Recommended action
4.5	4.3	Report overall consensus
4.5	2.8	Flag divergence; consider revision or stratified reporting

Publication readiness

The study overview includes a publication readiness checklist that pulls information from across the platform:

Criterion	Source
Protocol and consensus rules documented	Launch snapshot and methodology export
Panel size and response rates reported	Panel and round statistics
Item-level outcomes available	Results view
Qualitative synthesis reviewed	Qualitative workspace
Methods and results text generated	Manuscript drafting tool
Tables and figures exportable	Report and export surfaces

Completing this checklist helps ensure you have the necessary documentation before finalizing the study.

From results to next round

A typical workflow after reviewing results is:

1. Review the heatmap and flags for items needing attention.
2. Revise item wording in the item bank where needed.
3. Run qualitative analysis if open comments informed revisions.
4. Use Plan next round and document your inclusion rationale.
5. Lock feedback packets (for Round 2+).
6. Open the next round.

This sequence keeps the process structured and auditable.

Next steps

• Consensus analytics — statistics, cross-stakeholder rules, re-analysis mechanics
• Running rounds — plan and open the next round
• Multi-round worked example — end-to-end scenario
• Reporting & exports — publish findings