File formats and schemas

A run directory is the interface. Every file in it is schema-validated, and the schema module in the package is the single source of truth for all of them — it generates the JSON Schema, the contract documentation the agent reads, and the error messages you get when validation fails.

research-scan schema --name EvidencePacket   # one model, as JSON Schema
research-scan schema --md                    # every model, as Markdown

The run directory, in pipeline order

research/scans/<YYYY-MM-DD>-<slug>/

Gap-round variants carry a suffix: retrieval-r2.log.jsonl, expansion-r2.log.jsonl.

Neither half edits the other’s files.

Screening batch names

EvidencePacket

The shippable unit: a candidate, plus its rerank, plus its verification, plus why it was selected.

From the candidate: cid, title, abstract, tldr, authors[], year, publication_date, venue, type, ids (doi, arxiv, pmid, openalex, s2), citation_count, influential_citation_count, is_retracted, oa_url, origins[], outside_window.

From the rerank: criteria, overall, evidence_level, relation, flags, key_finding, methodology, why_it_matters, limitations[], relevance_reason.

From emit: verification (required — emit exits 2 without it), rank, selection_reason, url.

cid is a stable content id: the first twelve hex characters of a hash of the highest-priority identifier available, preferring DOI, then arXiv id, then PMID, then a normalised title and year.

Enumerations

relation — how a paper stands to the brief: design-changing, plan-influencing, closely-related, contradicting, foundational.

Discovery origin — how it was found, recorded separately on each origin: query, references, citations, recommendations, anchor.

selection_reason — why it made the list: score, foundational, review, contradicting, diversity, backfill.

evidence_level — systematic-review, meta-analysis, rct, prospective, observational, experimental, computational, qualitative, other.

query_type — direct, terminology, mechanism, method, adjacent, contradictory, review, emerging, gap.

profile — quick, standard, deep.

domain — behavioral, cs, biomed, general.

source — openalex, s2, arxiv, pubmed.

work_type — article, preprint, review, book-chapter, other.

verified_by — crossref, openalex, arxiv, s2.

mismatch — doi_unresolved, title, year, author, retracted, no_record.

Validation rules

Unknown keys are rejected everywhere. A misspelled field is not a field that defaulted; it is an error naming the path.

Exit 2 lists every offending path, not just the first.

Constraints are not loosened. limitations requires at least one entry; a packet without one does not validate.

Reading a run without the tool

Nothing here needs Research Scan to interpret.

# the shortlist, with verification status
jq '.[] | {rank, title, relation, verified: .verification.verified}' evidence.json

# anything that failed verification
jq '.[] | select(.verification.verified | not)
        | {title, mismatches: .verification.mismatches}' evidence.json

# what was dropped, and why
jq '.drops' manifest.json