# File formats and schemas

Every file a run produces, who owns it, and how to print the contract it validates against.

A run directory is the interface. Every file in it is schema-validated, and the
schema module in the package is the single source of truth for all of them — it
generates the JSON Schema, the contract documentation the agent reads, and the
error messages you get when validation fails.

```bash
research-scan schema --name EvidencePacket   # one model, as JSON Schema
research-scan schema --md                    # every model, as Markdown
```

## The run directory, in pipeline order

`research/scans/<YYYY-MM-DD>-<slug>/`

| File | Owner | What it is |
|---|---|---|
| `brief.md` | input | The brief, or the question. |
| `queries.json` | agent | The query plan: queries, sub-criteria, `must_not`, `anchors`. |
| `candidates.json` | CLI | The retrieved pool, deduped, filtered and capped. |
| `screen-batches/NN.json` | CLI | Candidates handed out for screening. |
| `screen.json` | agent | A 0–3 score per candidate, with reason and criteria hit. |
| `expanded.json` | CLI | Citation-graph additions. |
| `expanded-round2.json` | CLI | Gap-round additions. |
| `coverage.json` | CLI | Per sub-criterion coverage, one snapshot per round. |
| `shortlist.json` | CLI | Ordered and cut for the reranker, split in-window / outside-window. |
| `ranked.json` | agent | The rerank. |
| `evidence.json` | CLI | **The deliverable.** One `EvidencePacket` per paper. |
| `evidence.md` | CLI | The same list, rendered for a human. |
| `evidence.bib` | CLI | BibTeX. Suppress with `--no-bib`. |
| `manifest.json` | CLI | The audit trail: every drop, every source failure. |
| `retrieval.log.jsonl` | CLI | Per-call retrieval log. |
| `expansion.log.jsonl` | CLI | Per-call expansion log. |
| `verify.log.jsonl` | CLI | Per-call verification log. |

Gap-round variants carry a suffix: `retrieval-r2.log.jsonl`,
`expansion-r2.log.jsonl`.

**Neither half edits the other's files.**

### Screening batch names

| Prefix | Origin |
|---|---|
| `01`, `02`, … | Retrieved candidates |
| `x01`, … | Citation-graph expansion |
| `r01`, … | Gap round, retrieved |
| `xr01`, … | Gap round, expansion |

## EvidencePacket

The shippable unit: a candidate, plus its rerank, plus its verification, plus why
it was selected.

**From the candidate:** `cid`, `title`, `abstract`, `tldr`, `authors[]`, `year`,
`publication_date`, `venue`, `type`, `ids` (`doi`, `arxiv`, `pmid`, `openalex`,
`s2`), `citation_count`, `influential_citation_count`, `is_retracted`, `oa_url`,
`origins[]`, `outside_window`.

**From the rerank:** `criteria`, `overall`, `evidence_level`, `relation`, `flags`,
`key_finding`, `methodology`, `why_it_matters`, `limitations[]`,
`relevance_reason`.

**From emit:** `verification` (required — `emit` exits 2 without it), `rank`,
`selection_reason`, `url`.

`cid` is a stable content id: the first twelve hex characters of a hash of the
highest-priority identifier available, preferring DOI, then arXiv id, then PMID,
then a normalised title and year.

## Enumerations

**`relation`** — how a paper stands to the brief: `design-changing`,
`plan-influencing`, `closely-related`, `contradicting`, `foundational`.

**Discovery origin** — how it was found, recorded separately on each origin:
`query`, `references`, `citations`, `recommendations`, `anchor`.

**`selection_reason`** — why it made the list: `score`, `foundational`, `review`,
`contradicting`, `diversity`, `backfill`.

**`evidence_level`** — `systematic-review`, `meta-analysis`, `rct`, `prospective`,
`observational`, `experimental`, `computational`, `qualitative`, `other`.

**`query_type`** — `direct`, `terminology`, `mechanism`, `method`, `adjacent`,
`contradictory`, `review`, `emerging`, `gap`.

**`profile`** — `quick`, `standard`, `deep`.

**`domain`** — `behavioral`, `cs`, `biomed`, `general`.

**`source`** — `openalex`, `s2`, `arxiv`, `pubmed`.

**`work_type`** — `article`, `preprint`, `review`, `book-chapter`, `other`.

**`verified_by`** — `crossref`, `openalex`, `arxiv`, `s2`.

**`mismatch`** — `doi_unresolved`, `title`, `year`, `author`, `retracted`,
`no_record`.

## Validation rules

**Unknown keys are rejected everywhere.** A misspelled field is not a field that
defaulted; it is an error naming the path.

**Exit 2 lists every offending path**, not just the first.

**Constraints are not loosened.** `limitations` requires at least one entry; a
packet without one does not validate.

## Reading a run without the tool

Nothing here needs Research Scan to interpret.

```bash
# the shortlist, with verification status
jq '.[] | {rank, title, relation, verified: .verification.verified}' evidence.json

# anything that failed verification
jq '.[] | select(.verification.verified | not)
        | {title, mismatches: .verification.mismatches}' evidence.json

# what was dropped, and why
jq '.drops' manifest.json
```