Glossary
Quick reference for terms used in Delphi Studio and in the consensus literature.
A–C
| Term | Definition |
|---|---|
| Attrition | The loss of panelists across rounds. Delphi Studio automatically tracks response rates per round and per item, compares the Round 1 rating distributions of completers versus non-completers, and includes attrition summaries in CREDES-aligned exports so that potential bias can be assessed and reported. |
| Bimodality | A polarized response distribution in which a substantial proportion of ratings cluster at both ends of the scale (e.g., strong agreement and strong disagreement). When enabled, Delphi Studio automatically flags items meeting the bimodality threshold even if the median and IQR appear acceptable, prompting qualitative review. |
| Classification | The application of pre-specified, frozen consensus rules to an item's statistical profile (median, IQR, zone percentages) after a round closes. Classification assigns each item to an Agree, Disagree, or Indeterminate zone and derives the corresponding consensus status (consensus_in, consensus_out, no_consensus, or consensus_with_subgroup_divergence). See Consensus analytics. |
| Codebook | The approved, researcher-curated set of qualitative themes and codes used to categorize open-text comments. Delphi Studio supports codebook application during qualitative review and exports coded comments in REFI-QDA format for use in NVivo, MAXQDA, or other qualitative software. |
| Comment approval mode | Study setting that controls whether comments enter controlled feedback only after human review (review_all) or may auto-approve when redaction finds them clean (auto_when_clean). See Qualitative analysis. |
| Consensus | The state reached when an item satisfies all pre-defined statistical thresholds (typically a combination of median position, IQR dispersion, and percentage of ratings in the target zone). Consensus is always evaluated against rules locked in the LaunchSnapshot. |
| Controlled feedback | The structured, anonymized information returned to panelists between rounds. It includes group statistics (median, IQR, histograms, zone bars) and approved de-identified comments. In Delphi Studio, feedback is locked before the next round opens, generating an immutable per-panelist snapshot for auditability. |
| Cross-stakeholder consensus | Optional rule that downgrades panel-wide agreement when a prespecified stakeholder group of sufficient size shows substantial opposition. See Consensus analytics. |
| CREDES | Guidance on Conducting and REporting DElphi Studies (Jünger et al., Palliat Med 2017), developed for Delphi studies in palliative care. It provides useful complementary conduct and reporting guidance. ACCORD is the broader consensus-method reporting guideline; Delphi Studio supports both through structured exports and frozen protocol records. |
| CREDES export | A manuscript-ready methods section (Markdown) organized around the CREDES reporting domains, automatically populated with panel characteristics, round details, consensus definitions, attrition data, and other required elements from the study record. The study's final report is a separate DOCX export. |
D–I
| Term | Definition |
|---|---|
| Dimension | A distinct rating axis on which panelists evaluate an item (e.g., Agreement, Importance, Clarity, Feasibility, Relevance). One dimension is designated primary for consensus classification; secondary dimensions enrich analysis and reporting without affecting disposition. |
| Derived re-analysis | A recomputed classification of a closed round after excluding selected panelists (e.g. late withdrawal), stored as a versioned diff without mutating frozen dispositions. See Interpreting results. |
| Disposition | The final recommended outcome for an item after all rounds: retain (consensus achieved), revise (researcher-initiated rewording with version lineage), drop, or re_rate in a subsequent round. Dispositions are derived from zone classification and researcher review. |
| Disagreement Index (DI) | A RAND/UCLA measure of panel polarization calculated as (P70 − P30) / 2.35, where P70 and P30 are the 70th and 30th percentiles of ratings. Delphi Studio computes and displays DI when the RAND/UCLA preset is used. |
| Feedback packet | The immutable, per-panelist snapshot of controlled feedback generated after a round closes. It contains the exact statistics, histograms, and approved comments each panelist sees and is stored for reproducibility and audit purposes. |
| Grace window | Optional minutes after closesAt during which late ratings are still accepted and labeled in-grace; the close sweep waits out the window when enabled. See Running rounds. |
| Indeterminate | The zone assigned to an item when it fails to meet either the Agree or Disagree consensus thresholds. Indeterminate items typically carry forward to the next round or are flagged for researcher review and possible revision. |
| IQR | Interquartile range — the primary measure of response dispersion used in most modified e-Delphi studies. Delphi Studio calculates IQR automatically and uses it (together with median and zone percentage) in consensus rules. |
| Item | A single statement, competency, indicator, scenario, or question that panelists rate. Items exist within an item bank, support versioning, and carry metadata such as taxonomy tags and source references. |
| Item class | A logical grouping of items that can share custom consensus rule predicates (e.g., different thresholds for "core" versus "optional" competencies). |
| Item suggestion | A panelist-proposed statement awaiting researcher approval or rejection in Plan next round when new-item suggestions are allowed. |
K–P
| Term | Definition |
|---|---|
| Kendall's W | Kendall's coefficient of concordance (0–1), a measure of overall agreement across the entire panel. Delphi Studio computes and reports Kendall's W as a supplementary stability or concordance metric. See Consensus analytics. |
| LaunchSnapshot | The checksummed, immutable record of the complete study methodology (scales, zones, consensus rules, stopping criteria, item bank version, and protocol settings) captured at the moment of study activation. All subsequent analysis and reporting are anchored to this snapshot. |
| Layer | An evaluation tier in multi-layer studies, allowing different panels or criteria to be applied sequentially or in parallel. |
| Magic link | A secure, time-limited signed URL that allows panelists to access their rating interface without creating an account. Delphi Studio uses magic links to simplify onboarding while maintaining security and audit trails. |
| MAD | Median absolute deviation — a robust, outlier-resistant measure of spread sometimes reported alongside IQR. |
| Panel | The expert group attached to a study. Delphi Studio supports panel management including stratified recruitment, expertise tagging, response-rate monitoring, and pseudonymized identification (e.g., Panelist 1). |
| Pseudonym | The opaque identifier (e.g., Panelist 1; organization-directory experts appear as Expert 1, Expert 2, …) shown to other panelists and in most researcher views instead of real names, preserving anonymity while enabling longitudinal tracking. |
| Protocol version | A tracked amendment history of the study protocol. Each version captures section-level snapshots so that changes to methodology or item wording can be audited and reported. |
| Primary stakeholder role | The analytic stratum assigned to each panelist (e.g. clinician, patient). Used for balance warnings and optional cross-stakeholder consensus; dual affiliations are not double-counted. |
Q–Z
| Term | Definition |
|---|---|
| Qual run | A durable background job that performs AI-assisted thematic analysis or coding of open-text comments. Results are reviewable and editable by researchers before being incorporated into controlled feedback or final reports. See Qualitative analysis. |
| REFI-QDA | The REFI Qualitative Data Analysis interchange format (XML-based) used to export coded comments and codebooks for import into NVivo, MAXQDA, ATLAS.ti, and similar tools. |
| Round | One complete iterative rating cycle. Rounds progress through defined states (draft → scheduled → open → closed → analyzed) and are the fundamental unit of iteration in Delphi studies. See the Round state-transition diagram. |
| Source citation | Typed evidence link on an item (literature, guideline, interview, prior study, optional URL) that survives revisions and appears in results and report tables. |
| Statement quality review | Advisory pre-launch pass over the item bank for multi-barreled wording, reading level, ambiguity, and near-duplicates. |
| Stopping recommendation | Algorithmic guidance generated by Delphi Studio suggesting whether to continue to another round, stop because consensus or stability criteria are met, or apply caveats (e.g., due to high attrition or persistent bimodality). |
| Stability | The degree of change in key statistics (median, IQR, zone membership) between consecutive rounds. Stability metrics help researchers decide whether further iteration is likely to produce meaningful movement. |
| Study event log | Researcher-recorded timeline of external evidence, outages, incidents, and protocol deviations — distinct from the system audit trail. |
| Subgroup analysis | Breakdown of results by demographic characteristics, expertise domains, or other strata. Delphi Studio supports filtered views and exports of subgroup statistics for sensitivity analyses. |
| Trace bundle | A checksummed, timestamped archive containing rating data, feedback packets, item version history, the audit log, and study events. Each file's SHA-256 hash is listed in integrity.json, and the bundle hash is recorded in the audit log. See Reporting & exports. |
| Unable to rate | A valid response option indicating that a panelist lacks sufficient expertise or information to rate an item. Unable-to-rate responses are excluded from median, IQR, and zone calculations but are reported separately. |
| Unavailable | Panelist state for incapacity or unreachability: reminders stop, prospective denominators exclude them, prior-round data remains valid, and the state is reversible — distinct from withdrawal. See Panel management. |
| Zone | One of three bands (Agree, Disagree, or Indeterminate) into which an item is classified after a round using the frozen consensus rules. Zone assignment directly determines consensus status. See Key concepts. |
Related
- Key concepts — how these terms connect in a study workflow
- Modified e-Delphi — default thresholds and preset configuration
- What is the Delphi method? — methodological background