Skip to main content

Architecture of Delphi studies in Delphi Studio

Delphi Studio is designed around the core principles of the Delphi method: structured iteration, anonymity, controlled feedback, and transparent, pre-specified rules. Rather than a traditional software architecture document, this section describes the methodological architecture: how Delphi Studio organizes studies, participants, statements and rounds to support rigorous and defensible consensus research.

Overview​

A Delphi study in Delphi Studio is built around a clear, logical structure that mirrors best practices in consensus methodology:

  • A Study defines the overall research question, method and rules, set in Setup.
  • A Panel of experts is recruited, screened, consented and enrolled.
  • An Item bank holds the statements to be rated, each with a permanent ID.
  • Rounds provide the iterative structure through which panelists answer and receive feedback.
  • Controlled feedback and decisions between rounds drive convergence toward consensus.
  • Decisions and data are captured in an auditable record.

Studies also draw on organization-level building blocks: an expert directory for recruiting panelists across studies, and reusable templates. A template carries profile fields, panel criteria and screening questions, and the analysis plan. It never carries statements, people or responses.

This structure keeps the study transparent and supports reporting against guidelines such as CREDES and ACCORD.

Core entities and their relationships​

The diagram below shows the main conceptual components of a Delphi study and how they connect:

In words: a study has method and rules, a panel of panelists, and an item bank of statements. Rounds collect ratings and comments. Those results feed a researcher decision and controlled feedback. The decision leads to the next round or to final results. The research team designs the study and confirms the decision.

A classification is not the decision. While another round remains, an unsettled class is not a reword. The labels are on Interpreting results.

See Key concepts for round states, ratings, statuses and study roles.

The round lifecycle​

Each round is one complete cycle of answering, results and decision. Delphi Studio moves every round through the same states:

  1. Planned: in the round plan. A round leaves Planned only when the one before it has closed.
  2. Building: the team prepares the questions and the feedback, and the principal investigator approves the round.
  3. Open: panelists answer independently and may add comments.
  4. Reviewing: collection has closed. Results are computed from submitted answers and reviewed.
  5. Decision: the team decides what happens to each statement next.
  6. Closed: the round is a read-only record.

Opening is manual by default. Setup can let an approved round open on its scheduled date and close collection at its deadline. Nothing is approved automatically. See Rounds.

Controlled feedback and iteration​

Before rating again, panelists see controlled feedback from the previous round. As Setup chooses, the live block can show the median, Middle half of ratings, the three zone shares, the person's own earlier rating, and comments the team chose. It does not print a row labelled IQR, it does not draw a histogram, and it does not show why the wording changed. Show a histogram on Round design, and the bars in the setup preview, are a different screen. A reworded statement does not carry the earlier group figures. What a class leads to, and what that reword withholds, is explained on Interpreting results.

Every panelist sees the same group information. Results based on fewer people than the study's smallest reportable group are withheld. Feedback shows the previous round's current verified results, and the page says so.

Researcher oversight and governance​

While Delphi Studio automates much of the technical work, the research team keeps oversight at key points:

  • Defining the method and decision rules before launch
  • Approving each round before it opens
  • Reviewing results after each round
  • Confirming the decision plan, with a recorded reason for any override
  • Deciding a clarity review, including whether the wording changes
  • Deciding when the study ends

Who may take each step is set by study role, and the server checks it on every action. Important methodological and interpretive decisions stay with the research team; the platform handles calculation, consistency and record-keeping.

Auditability and reproducibility​

Delphi Studio is built to support transparent and reproducible research:

  • What locks at launch, and which amendments remain possible, is on Launch and amendments. An amendment does not re-judge a round already rated.
  • Each round's results are stored as sealed results versions, checked on every read.
  • The decision log is append-only and hash-chained.
  • Statement revisions keep their permanent ID and cite the results they were based on.
  • The files a study can download are listed on Reporting.

See Audit trail.

Panelist experience​

From the panelist's side, the process is meant to be simple and low-burden:

  • They receive their own round link by email. They need no account or password.
  • They rate statements independently, and their answers save as they go.
  • Between rounds, they see structured feedback.
  • Their identity is protected from other panelists, and staff see a pseudonym from enrollment on.

See For panelists.