Contributing

Research Scan is Apache-2.0 and developed in the open at Synectic-Research/research-scan.

The project has one unusual rule — the eval gate — and it is the one worth reading before you write code.

Development setup

git clone https://github.com/Synectic-Research/research-scan
cd research-scan
uv sync                                   # environment, including the dev group
uv run pytest -q                          # the suite; nothing here touches the network
uv tool install --editable .              # optional: research-scan on PATH, tracking the tree

Running an actual scan needs credentials. Contributing to the code does not — the test suite mocks every source, and no test outside the live-marked set opens a socket.

The quality gate

Both must be clean before you open a pull request, and both run in seconds:

uv run ruff check
uv run pytest -q

uv run ruff format is the canonical formatter, at line length 100.

If you change schema.py, regenerate the contract documentation in the same commit — a test compares them:

uv run research-scan schema --md > skills/research-scan/references/schemas.md

Measured or reverted

Any change to retrieval, expansion, screening caps, or selection behaviour must arrive with golden-set eval results showing no recall regression.

A green test suite is not evidence for these changes. The tests prove the code does what it says, not that the scan finds better papers.

research-scan eval --topic <topic> --run <run-dir> --json

Run it on both golden topics in eval/golden/, at the cheapest stage that can see your change. --stage candidates is enough for anything in retrieval or expansion; a selection change needs an emit diff against a committed run. Put the before-and-after numbers in the pull request description.

A change that does not move a number, and does not buy something else you can name, gets reverted. That is not a judgement about the idea — docs/measurements.md records several good ideas that measured flat, and they are kept there precisely so nobody spends the afternoon again. If your change measures flat and you still think it is right, say so and argue it. The record of what was tried is more valuable than a tidy diff.

Two things make this cheap rather than painful:

• Scoring at the candidates stage costs no agent tokens. Use it first.
• If a golden topic moves, diagnose the miss before changing anything. The measurements file classifies misses as terminology gaps, query-plan variance or cap effects, and the three want different fixes.

One more rule, from the measurement history: a measured run is not quotable until every routed source reports zero failures. A run with a dead source can produce a number that looks good.

Architecture rules a change must hold

These are boundaries, not preferences. A patch that crosses one will be sent back regardless of what it does.

No LLM SDK in this package, ever. No model calls, no prompts. Every judgement belongs to the hosting agent. This is the property that makes the tool harness-agnostic.

Dependencies are the declared list. Adding one is an architectural decision, not an implementation detail.

No scraping. Documented APIs only.

Credentials only through the config module. Nothing else reads the environment for a key, which is what lets every secret be redacted before anything is logged or cached.

Only the HTTP module talks to the network. Unit tests mock at that layer.

Neither half edits the other’s files. The CLI owns candidates.json, shortlist.json, manifest.json and evidence.*; the agent owns queries.json, screen.json and ranked.json.

The MCP adapter shells out to the CLI. It never imports a stage’s internals, and it makes no judgement about papers.

A few anti-patterns are called out by name: hand-tuned weight vectors (the rubric scores, code selects), rewarding citation count or venue prestige, repairing metadata from model memory, rewriting the agent’s queries in code, loosening a schema constraint because a test failed, and letting a fallback hide a broken primary path.

Pull requests

• Branch from main, then open a pull request. main is never force-pushed, and neither is any branch someone else may have fetched.
• One concern per commit, imperative mood, subject under 72 characters.
• If your change is architectural, say what you considered and rejected. The repo documents reverted experiments on purpose.

Releases

Releases go out across four surfaces: PyPI, a GitHub Release, the plugin marketplace and the MCP registry. Version numbers follow SemVer, and every release is gated on measurement.

The full procedure — including which files move together in a single commit, and how a release is verified and rolled back — is in RELEASING.md. It is a maintainer document; contributors do not need it to land a change.

Where things live

Reporting a security issue

Please use GitHub private vulnerability reporting rather than a public issue. The policy is in SECURITY.md.

Also

CODE_OF_CONDUCT.md · CHANGELOG.md · CITATION.cff · LICENSE (Apache-2.0) · THIRD_PARTY_LICENSES.md