Comparing Delphi types
A Delphi Studio study makes two separate choices. The workflow, chosen on New study, says what happens and in what order. The method framework, chosen on Setup's Method step, says which approach and decision rules govern the rounds. The two combine freely: a Modified Delphi can generate its items first or rate items you already have.
Should you use Delphi?
Choose a Delphi design when structured judgment is needed because evidence is incomplete, inconsistent, or difficult to apply—for example, to prioritize outcomes, refine definitions, or develop recommendations with relevant stakeholders.
Choose another method when the main objective is:
- a representative estimate of population opinion;
- an empirical question that adequate data can answer directly;
- real-time negotiation or joint problem-solving; or
- endorsement of a conclusion already selected by the research team.
The defensible design page provides a fuller fit assessment and protocol checklist.
Workflows on New study
| Workflow | What happens | Suits |
|---|---|---|
| Rate existing items | Panelists rate statements you already have across successive rounds, with group feedback between rounds | Guidelines, competencies, indicators |
| Generate items, then rate | Round 1 asks panelists to propose candidate items; the team consolidates them, and later rounds rate the list | Open topics without a starting item set |
| Define and refine, then rate | Panelists first agree on definitions, terminology or a framework's structure; the resulting items are then rated | Taxonomy development and framework validation |
| Build a custom workflow | One rating round to start; you add rounds, a questionnaire and rules | Designs the others do not cover |
The Suits column is a guide to fit. It is not the wording on the New study screen. The four workflow names are. A study can also start from a template, a past study or an imported template file. People and responses are never copied. See New study.
Method frameworks
| Modified Delphi | Classical Delphi | RAND/UCLA Appropriateness | Custom method | |
|---|---|---|---|---|
| Question type | Do experts agree? | Do experts agree, on items they proposed? | Is this scenario appropriate? | You define |
| How statements are classified | Percentage of ratings in a zone, with a spread limit | Same as Modified | Panel median's zone, with a disagreement test | You set the rules |
| Suggested scale | Likert 1–5 | Likert 1–5 | 1–9 | You choose |
| Usual workflow | Any | Generate items, then rate | Any | Any |
Modified Delphi carries a Recommended badge on the Method screen. That badge is not a scientific requirement. A new study stores no framework until someone chooses one.
The table suggests Likert 1–5 for Modified Delphi and Classical Delphi, and 1–9 for RAND/UCLA. Choosing a framework does not switch the scale. It records only which framework was chosen. A new study starts on a 1–9 scale unless the workspace default or the creator's own starting values say otherwise. The scale and the rules are set on Rating & consensus. A suggested scale is not the method's definition. If the study departs from that suggestion, say so in the Methods text. Choosing the framework does not write that sentence. See Study design and Thresholds.
Decision flowchart
The diagram states the same fit as the tables above. It does not choose a median boundary, a panel-size table, or whether a study that omits a discussion may use the RAND/UCLA name.
Scale options
Rating & consensus offers these scales under every framework, within the scales your organization allows:
| Scale | Use when |
|---|---|
| 1–9 scale | The built-in default; RAND/UCLA appropriateness; importance ratings |
| Likert 1–5 | Standard agreement |
| Likert 1–7 | Finer agreement steps |
| 0–10 scale | Numeric rating from zero |
| Custom | Your own range and zones |
Each scale has disagree, uncertain and agree zones that the rules count. A new study starts with neutral anchors, which do not name the decision a zone feeds. A study can save other wording. Decision words are warned about, not refused. The figures a panelist sees still do not include that decision. Setup can name up to five dimensions, such as importance and feasibility, and mark one primary. The stored answer is one rating. The primary dimension is the one the consensus rule uses.
Rules and stopping
The screen label is Modified Delphi: percentage and spread. A new study starts from these values: retain at 70 percent or more in the agree zone, exclude at 70 percent or more in the disagree zone, an interquartile-range ceiling of 2, and at least 7 responses before a statement is classified. Choosing Likert 1–5 or Likert 1–7 sets that ceiling to 1. These are starting values, not a universal rule. On a 1–9 scale the spread may use the disagreement index instead of the interquartile range. That index is the modified-Delphi spread option. It is not the RAND/UCLA disagreement test. See Consensus analytics.
Under RAND/UCLA, on the default zones, a statement is Appropriate when the median is 7 to 9 and Inappropriate when it is 1 to 3, in both cases without disagreement, and Uncertain otherwise. That is this product's integer rule. See RAND/UCLA.
The number of rounds is planned on Round design. Stopping is advice, not a rule the software uses to end the study. A study can record advice to stop once a stated share of statements (90% by default) is settled. Settled means retained or excluded. Under RAND/UCLA those results are labeled Appropriate and Inappropriate. The research team decides whether to continue. After the last planned round, the principal investigator can add one further rating round as a recorded amendment.
Anonymity
Panelists never see each other's identities or individual ratings. Staff screens show pseudonymous codes. People being recruited can be shown by name on the recruiting screens. Revealing an enrolled person's name takes a written reason and is audited. A coordinator can only request a reveal; a principal investigator or an administrator approves it. See Identity and pseudonyms.
Choosing Custom method
Choose Custom method when the presets do not fit, for example:
- policy or other designs the presets do not cover;
- a scale with zones of your own; or
- decision rules your protocol defines differently.
Real-time negotiation is a reason to choose a method other than Delphi, not a mode this application runs.
Custom method suggests no scale or rules. Rating & consensus keeps the values the study already has, which start from the workspace defaults and any starting values the creator set, until you change them. You write the Methods text yourself. Method and Rating & consensus each need their own confirmation before launch. A material change on one of those screens withdraws that screen's confirmation. A change to stability settings or to stopping advice does not withdraw the rating confirmation.
Hybrid studies
A study has one consensus method and one primary dimension. Teams that need agreement on indicator importance and appropriateness ratings on clinical scenarios usually run them as separate studies or sequential phases.