Audit trail
Delphi studies are frequently subject to scrutiny from journal reviewers, institutional review boards (IRBs), replication teams, and internal collaborators. Delphi Studio maintains a tamper-evident system audit trail for governed actions across the study lifecycle, plus a separate study event log for researcher-recorded narrative context. Together they provide structured evidence for transparency and reproducibility; investigators remain responsible for recording material off-platform events and explaining the study's conduct.
System audit log vs study event log
| Record | Who writes it | Purpose |
|---|---|---|
| System audit log | Platform (automatic) | Tamper-evident hash chain of invitations, ratings, rule edits, AI runs, exports, unmasks, role changes |
| Study event log | Research team (manual) | Dated narrative of things a reader needs to interpret results: external evidence between rounds, service outages, data incidents, protocol deviations |
Both appear in reports and the trace bundle. Do not use the audit log as a substitute for study events — reviewers need the human story as well as the system chronology.
Why it matters
A robust audit trail answers critical questions from different stakeholders:
| Stakeholder | Key question it answers | Value |
|---|---|---|
| Journal reviewer | Were consensus rules pre-specified and consistently applied? | Supports methodological transparency and reduces requests for raw data |
| IRB | Who accessed panelist identities and under what circumstances? | Demonstrates compliance with privacy and consent requirements |
| Replication team | Can the final classifications be reproduced from the raw ratings and rules? | Enables independent verification of results |
| Co-investigator | Who changed item wording, consensus thresholds, or dispositions — and why? | Maintains internal accountability and team trust |
Hash chain
The audit log is implemented as a hash chain. Each entry contains:
- A namespaced action (e.g.,
study.consensus_rules.updated) - The payload (before/after values, IDs, and relevant metadata)
- The SHA-256 hash of the previous entry (
prevHash) - Its own SHA-256 hash
An advisory database lock prevents concurrent writes from creating conflicting chains. Because each entry is cryptographically linked to the one before it, any attempt to alter or delete earlier records would break the chain and be immediately detectable.
This design provides strong protection against tampering while remaining efficient for day-to-day use.
What gets logged
Delphi Studio records events across the entire study lifecycle.
Design phase
| Action | Example information logged |
|---|---|
study.method.updated | Changes to planned number of rounds or Delphi type |
study.consensus_rules.updated | Before-and-after values for thresholds and zones |
study.item.revised | Link to parent item and new wording |
study.warning_acknowledged | Which warning was acknowledged and the researcher's rationale |
Operations
| Action | Example information logged |
|---|---|
round.opened / round.closed | Timestamps and the set of items included |
round.feedback_config.updated | Changes to feedback settings |
panelist.invite | Pseudonym (real identity kept separate) |
panelist.consent | Protocol hash at time of consent |
panelist.withdraw | Timestamp of withdrawal |
panelist.marked_unavailable | Reason and actor (incapacity, not disengagement) |
panelist.stakeholder_role_updated | Primary role changes that affect analytic strata |
Analysis & AI
| Action | Example information logged |
|---|---|
rating.submit | Pseudonym, item, round, and dimension values |
disposition.override | Researcher rationale and previous vs. new disposition |
ai_run.created / ai_run.completed | Provider, model, purpose, prompt or payload provenance, and state transition |
qual_adjudication.decided | Which gate was decided and by whom |
Study event log (researcher-recorded)
Record study events from the study’s integrity / protocol surfaces when something happens that methods or results readers should know about:
| Kind | Examples |
|---|---|
| External evidence / co-intervention | A national guideline published between rounds |
| Service outage | Platform or email delivery downtime during a round window |
| Data / confidentiality incident | Suspected disclosure; containment steps taken |
| Protocol deviation | Unplanned change in process that still requires transparent reporting |
| Other | Catch-all with free-text description |
Each event has a title, optional description, occurred-at timestamp, and optional link to a round. Events are append-only for the study record and feed the manuscript report and trace bundle (study_events.csv).
Frozen snapshots
In addition to the audit and study-event records, Delphi Studio captures immutable snapshots at key moments:
| Snapshot | When it is captured | Purpose |
|---|---|---|
| Launch snapshot | When the study is activated | Freezes the complete methodology for the entire study |
| Consensus rule snapshot | Each time a round closes | Records the exact rules used to classify items in that round |
| Feedback snapshot | When feedback packets are locked | Creates the immutable view each panelist saw |
| Protocol version | With every protocol amendment | Tracks section-level changes over time |
These snapshots allow researchers and external reviewers to know exactly what rules were in effect at any point in the study.
Trace bundle export
The Trace Bundle (/export/trace-bundle) is a comprehensive reproducibility package that includes:
- The complete audit log (in JSONL format)
- Researcher-recorded study events
- The Launch snapshot
- Per-round consensus rule snapshots
- Derived re-analysis versions (when present)
- A signed reproducibility manifest
A third party can use this bundle, together with the exported ratings CSV, to independently verify the hash chain and re-run the classification logic to confirm that the published dispositions match what the rules and data would produce.
Offline verification
To verify a study externally:
- Export the trace bundle and ratings CSV from Delphi Studio.
- Verify the hash chain using the provided manifest script or manual SHA-256 checks.
- Re-run the classification logic using the analytics engine against the raw ratings and frozen rules.
- Compare the resulting dispositions with those reported in the study.
If the rules were not changed after data collection, the results should match.
Identity access
Real panelist identities are stored in an encrypted table separate from ratings and audit events. Authorized identity reveal requires a reason and generates panelist.unmask before identity is returned. Routine analytic and directory views do not include real names or contact details.
IRB closeout package
For IRB closeout or regulatory submission, the following artifacts are recommended:
| Artifact | Format | Purpose |
|---|---|---|
| Audit log | JSONL | Complete record of all study events |
| Consent and protocol versions | In-app view + export | Evidence of informed consent over time |
| Panel state | CSV | Summary of participation and withdrawals |
| Methodology snapshot | Launch snapshot (JSON) | Frozen rules and settings at study start |
| Trace bundle | Signed archive | Comprehensive reproducibility package |
Next steps
- Reporting & exports — trace bundle and CSV exports
- Interpreting results — re-analysis without rewriting history
- Collaboration & roles — who can unmask identities or change roles
- Structure — methodological building blocks