What is the Delphi method?

The Delphi method is a structured, iterative process for eliciting and synthesizing expert judgment to reach group consensus on complex questions. It is particularly valuable in health services research, clinical guideline development, medical education, and health policy, where empirical data alone are insufficient and expert consensus must be obtained systematically. Unlike face-to-face meetings, which can be influenced by dominant personalities or group dynamics, the Delphi method preserves anonymity, controls feedback, and applies explicit stopping rules.

Delphi Studio implements a modern, web-based modified electronic Delphi (e-Delphi) that embeds these methodological safeguards directly into the study workflow. The platform's study wizard, launch snapshot, and researcher-supervised pipelines capture declared protocol choices before data collection and preserve later changes for review.

Origins and purpose

The method was developed in the 1950s at the RAND Corporation by Norman Dalkey and Olaf Helmer, initially to forecast technological developments under conditions of uncertainty for military planning. Researchers rapidly recognized its broader utility for achieving consensus on multifaceted problems where no single expert possesses complete knowledge and where direct interaction might introduce bias.

The defining characteristic of Delphi is controlled iteration: experts provide private ratings and comments, receive structured, anonymized group feedback, and re-rate until responses stabilize or pre-specified stopping criteria are satisfied. This iterative, mediated exchange reduces noise while preserving the diversity of expert perspectives.

The iterative Delphi process

A typical modified e-Delphi study in health research follows these stages:

Preparation (pre-Round 1) — A review, prior study, framework, interview process, steering group, or exploratory round generates an initial item bank (statements, competencies, indicators, or scenarios). Expert panelists are identified according to explicit eligibility criteria relevant to the question. Delphi Studio supports CSV import or structured entry of the item bank and capture of panel criteria on the Panel rules step.
Round 1 — Panelists independently rate each item — commonly on a 5- or 9-point ordinal scale for agreement or appropriateness — and supply free-text comments or suggestions for new or revised items. An explicit unable to rate / insufficient expertise option is provided. All ratings and comments remain private to each panelist.
Analysis and controlled feedback — Researchers calculate descriptive statistics (median, interquartile range [IQR], and percentage of ratings falling within pre-defined agreement zones) and conduct qualitative thematic analysis of comments. Anonymized, aggregated feedback is returned to each panelist, showing their own prior rating alongside group statistics and representative de-identified comments.
Subsequent rounds (Round 2+) — Panelists review the feedback and re-rate items. Items that have not yet reached consensus or that generated substantive comments may be revised by the research team, with full version lineage preserved. Iteration continues while items remain "in motion."
Stopping and consensus classification — The study terminates according to pre-declared rules — maximum number of rounds, stability of group statistics (e.g., minimal change in median or IQR between rounds), or a target proportion of items reaching consensus. Each item is then classified (consensus in, consensus out, or equivocal / no consensus).

Delphi Studio supports steps 3–5 through configurable analysis pipelines, locked feedback, and tracked item versions. Researchers retain responsibility for reviewing feedback, revising items, documenting overrides, and deciding when to stop. The launch snapshot taken at study activation freezes the declared protocol, rating scales, consensus definitions, and stopping rules; later permitted changes remain versioned or audited without rewriting prior-round rule snapshots.

Why researchers use the Delphi method

Strength	Explanation	Delphi Studio implementation
Reduces dominance bias	Independent responding limits opportunities for senior or vocal participants to steer discussion.	Panelist identities are never visible to peers; only approved, de-identified group evidence is shared in controlled feedback.
Geographic and temporal reach	Experts participate asynchronously from any location.	Web-based interface with automated email invitations, reminders, and participation dashboards.
Explicit, pre-specified rules	Consensus thresholds and stopping criteria are declared before data collection.	Study wizard requires scales, agreement zones, and stopping rules before launch; rules are locked via launch snapshot.
Auditability and reproducibility	Every input, decision, and output can be documented and reconstructed.	Launch snapshot, complete rating histories, item version lineage, and exportable datasets support CREDES-aligned reporting.
Handles complexity	Suitable for large item banks spanning multiple domains (guidelines, curricula, quality indicators).	Scalable item bank with taxonomy, versioning, and qualitative pipelines for multi-domain studies.

Classical versus modified Delphi

Aspect	Classical Delphi	Modified Delphi	Delphi Studio support
Number of rounds	Typically 2–4	Usually 2–3	Wizard defaults to 2–3 rounds with a configurable maximum.
Feedback	Statistical summary ± qualitative comments	Same, often with tighter governance and item revision	Automated median, IQR, zone %, and thematic summaries; researcher approves revisions and feedback packets.
Anonymity	Panelists usually do not know peers' identities	Commonly implemented as response anonymity or anonymity to peers	Panelists never see peer identities or individual responses; researchers use pseudonyms in routine views, with reasoned audited identity reveal for authorized operational needs.
Item revision	Items may be reworded between rounds	Revision tracked with version lineage	Full version history via `parentItemId` chain; changes timestamped and linked to feedback themes.
Literature base	Sometimes omitted	Often includes systematic review or evidence summary	Item bank supports CSV import with domain taxonomy; evidence can be attached to item text.

Modified e-Delphi is the default form implemented in Delphi Studio: independent responding, controlled statistical and qualitative feedback, tracked item revision, and declarative consensus rules preserved in launch and per-round snapshots. The label alone is not a protocol; every study should state exactly what was modified and why.

What the Delphi method is — and is not

The Delphi method is:

A structured technique for achieving expert consensus on statements, scenarios, competencies, or indicators.
Iterative by design; convergence is measured quantitatively and qualitatively rather than assumed after a single survey.
Transparent when panel composition, rating scales, consensus thresholds, attrition, and stability metrics are reported in accordance with established guidance (e.g., CREDES).

The Delphi method is not:

A substitute for systematic review or meta-analysis (although a Delphi study frequently follows one).
A simple vote or majority-rule exercise; consensus is defined by pre-specified statistical rules, not plurality.
A focus group or live debate; there is no synchronous discussion — comments are mediated, de-identified, and presented in aggregated form only.

Delphi Studio enforces these distinctions by design. The platform contains no live-chat or synchronous discussion features. All interactions occur through structured, asynchronous rounds with researcher-controlled feedback content.

Key methodological commitments for peer-reviewed Delphi studies

When a Delphi study is intended for publication, reviewers expect explicit documentation of the following elements. Delphi Studio records many of the underlying fields and events, but investigators must still supply the methodological rationale and manuscript context.

Panel composition — Clear description of how experts were identified, the domains and perspectives represented, and response rates. Delphi Studio records configured panel criteria, stakeholder roles, qualifications, and participation data for reporting.
Rating scale — Defined anchors, numeric ranges, and handling of "unable to rate" responses. The wizard supports 5- and 7-point Likert, RAND/UCLA 9-point, binary include/exclude, and custom scales; unable-to-rate is a first-class response option with dedicated analytics.
Consensus definition — Pre-specified rules combining central tendency (median), dispersion (IQR), and agreement percentages within defined zones. Rules are declared in the wizard, applied automatically by the analytics engine, and snapshotted per round.
Stopping criteria — Maximum rounds, stability targets (e.g., change in median or IQR below a threshold), or a minimum proportion of items reaching consensus. The platform computes stability metrics and surfaces stopping recommendations after each round close.
Feedback content — Exactly what panelists receive between rounds (own rating, group statistics, and which comments). Feedback templates are configurable yet standardized within a study; every feedback instance is logged in the audit trail.
Attrition and missing data — Documentation of dropouts and whether non-responders were carried forward or excluded. Dashboards display cumulative and per-round response rates, with attrition caveats on stopping recommendations.
Reproducibility and auditability — An immutable record of the protocol, data, and analytic decisions. The launch snapshot creates a timestamped archive of study configuration at activation; subsequent changes are versioned in the hash-chained audit log.

Method	Best suited for	Preset in Delphi Studio
Modified e-Delphi	Agreement on statements, competencies, quality indicators, or policy options	Modified e-Delphi (default)
RAND/UCLA Appropriateness Method	Clinical appropriateness of procedures or interventions	RAND/UCLA Appropriateness
Custom	Full researcher control with explicit confirmation of all defaults	Custom

Read Comparing Delphi types for a decision guide on selecting the appropriate preset.

Ensuring reporting quality and auditability

For biomedical consensus research, ACCORD is the broad reporting guideline and CREDES provides complementary Delphi-specific guidance developed in palliative care. ACCORD explicitly guides reporting rather than prescribing how a consensus exercise must be conducted. Delphi Studio preserves fields, snapshots, and events that support both frameworks; it does not make a study compliant by itself. See the ACCORD and CREDES crosswalk.

Reporting topic	Delphi Studio evidence
Transparent panel selection	Panel rules, expertise fields, enrollment audit log
Defined consensus process	Frozen consensus rules + per-round snapshots
Documented feedback	Locked feedback packets per panelist
Attrition reporting	Response-rate dashboards and attrition caveats
Reproducible analysis	Trace bundle + ratings CSV

Analysis-ready datasets, feedback snapshots, and item-version histories can streamline transparent methods and results reporting. See Reporting & exports for the current artifact formats and review responsibilities.

Origins and purpose​

The iterative Delphi process​

Why researchers use the Delphi method​

Classical versus modified Delphi​

What the Delphi method is — and is not​

Key methodological commitments for peer-reviewed Delphi studies​

Related methods available in Delphi Studio​

Ensuring reporting quality and auditability​

Further reading​