What current guidance asks of AI in screening.

On 9 November 2026 NICE published its AI methods for evidence framework. This page summarises what it and other references ask of anyone using AI in a systematic review, with the source of each point, and shows where Compendium stands.

No organisation cited here has assessed, approved or recommended the Compendium software; the acceptance of our poster at ISPOR Europe 2026 was the review of an abstract, not of the product.

Three levels of AI use in title and abstract screening

The NICE framework groups the role of AI in this step into three categories and takes a different position on each. [1]

SupportLow riskThe AI suggests or ranks; people review every record and make every decision. For NICE this is low risk, as long as the review shows clearly that people kept the screening decisions.
Semi-automationConditionalFor example, the AI and a human reviewer screen the same records independently. NICE allows it when an independent human adjudicator settles each disagreement between the reviewer and the AI and both the way disagreements are settled and their outcomes are clearly documented, and asks for robust validation of the AI system.
AutomationHigher riskThe AI makes the screening decisions. NICE sees much greater methodological risk here than at the other two levels. It should exclude only when validation has shown a high degree of assurance, and what it keeps should still be screened by a person.

Compendium works at the support level only. The app suggests a verdict and quotes the passage; each record waits for a person to press Include, Maybe or Exclude. By default the suggestions come from rules built into the app; the local language model is optional.

What the framework asks, and what Compendium does

Decisions stay with people

NICENICE treats support use as low risk only when it is clearly shown that the screening decisions stayed with the human reviewers.

CompendiumThe AI never includes or excludes a record. The PRISMA flow reports zero records excluded by automation tools, and the app refuses to report any other number.

A score removes nothing

NICEIn support use, when a tool scores records, the score should not be turned into a cut-off, fixed in advance, below which records are excluded.

CompendiumNo score removes a record. A suggestion is shown beside it and nothing leaves the queue because of it.

A failure never excludes

NICEIf the system fails, no record may be excluded because of it; otherwise relevant studies could be left out.

CompendiumIf the model fails, takes too long or cites no passage, the assistant abstains and the record stays with the reviewer.

A quoted passage must exist

NICEFor extracting passages to support screening, NICE does not regard generative AI as reliable, because it can change, drop or invent information.

CompendiumFor that reason the app does not trust the passage the model returns. Each quote is looked up in the source text and shown as it stands there; one that is not found is left out of the evidence. The reasoning the model writes is not checked this way.

Automation bias

NICEReviewers should keep in mind, while applying the eligibility criteria, that an AI score may be swaying them.

CompendiumThe app records when a suggestion was shown to each reviewer, and the report states whether the app showed any to the second reviewer.

Where suggestions are offered

NICEWhen a tool has neither been shown to generalise across domains and study types nor been tested for the ones at hand, NICE says not to use AI and to send those records to human review.

CompendiumAI suggestions in screening are offered only in the review profiles the assistant was prepared for, today the intervention profile. In the others, no AI suggestion is shown.

Data and copyright

NICEAnyone using AI has to meet the data requirements that apply: privacy, confidentiality and data protection, cyber security, and obligations on copyright, licensing and intellectual property.

CompendiumThe default suggestions come from rules built into the app. The language model is optional and runs in the Ollama you install on your computer, after consent per type of task. No abstract or PDF is sent to an AI service. The app does not check the copyright of the PDFs you attach: that is up to you.

Less variation

NICEFor title and abstract screening, prompts and settings should be chosen so that the output varies as little as possible.

CompendiumIn screening, the assistant asks the local model for temperature 0 and records its identifier. This reduces variation; it does not guarantee identical output.

Reporting

NICEThe report should give how many records the AI screened, how many people reviewed, how many went to adjudication and how many were included or excluded.

CompendiumThe Report drafts the AI-use statement for title and abstract screening from the review's records: suggestions shown and how often the reviewer agreed or decided otherwise.

What is not established

Performance in another review

NICEHow well a model performs can change with the review: one that does well in one review can do much worse in another.

CompendiumNo independent validation of Compendium's assistant is published, and this site gives no accuracy figures for it. An evaluation run by the developer was presented as a poster at ISPOR Europe 2026. Inside the app, the calibration report compares the suggestions with your own decisions in your own review, and only there.

Full automation

NICENICE does not recommend fully automating full-text screening or, for now, data extraction. In extraction it accepts support and semi-automation on conditions that include checking every item the AI extracts against the primary source.

CompendiumNeither step is automated. In extraction, each suggested value shows the passage it came from, and nothing is saved until a person accepts.

Stopping rules

NICEThere is little evidence on when it is best to stop screening where active learning or prioritisation is used.

CompendiumThere is no stopping rule: only a person can decide a record. If the reviewer asks, the app can order the pending records by likely inclusion; that changes the order and removes nothing.

Other references

Cochrane, Campbell, JBI and CEE

Joint position statement, 2025: Any use of AI or automation that makes or suggests judgments should be fully and transparently reported in the evidence synthesis report. The author remains accountable for the decision to use AI and for its effect on the review. [2]

RAISE

The RAISE recommendations (version 4, September 2026, still a draft for consultation) rate generative language models in study selection as Requires validation within the review, and ask developers to report publicly how a tool was evaluated and when it should not be used. [7]

PRISMA 2020

Item 8 asks for details of the automation tools used in study selection, and the flow diagram has a box for records marked as ineligible by automation tools. PRISMA 2020 has no guidance of its own for generative AI yet; a partial update is in preparation. [3]

World Health Organization

WHO guidance on AI is at the level of principles, among them transparency, human oversight and accountability; it sets no rule for screening or any other review step. A 2026 report on research ethics asks researchers to disclose which AI tools they used, their performance and how they were applied. [4] [5]

ISPOR Task Force

The good practices report on generative AI for systematic literature reviews (September 2026) supports its use in title and abstract screening aimed at high recall, inside workflows with human review at every decision, and does not support autonomous use for any task. [8]

European Union (JCA)

For joint clinical assessment dossiers, the principles adopted in July 2026 require human oversight in every AI-assisted step and a declaration of each step where AI was used, study screening included, with the tool, its version, date, developer and purpose. [9]

Brazil, Ministry of Health

The methodological guideline for systematic reviews (2021) asks for at least two independent reviewers in study selection and names software that can help with it. It does not address AI. [10]

GIN-McMaster

For guideline development, the 2026 checklist extension asks to Document all the processes in which AI tools were used. [6]

What to state in your manuscript

  1. Use and role

    Say that AI was used, at which step, and that it suggested while people decided.

  2. Tool and version

    Name the app, the model and its version or identifier.

  3. Checking and numbers

    Describe how people checked the output and report the counts: records with a suggestion and disagreements.

The statement Compendium drafts covers title and abstract screening. If you used suggestions at full text or in extraction, or the ordered queue, state that yourself. Check the instructions of your journal, funder or agency: they may ask for more.

Sources

  1. NICE. AI methods for evidence framework (ECD20). Published 9 November 2026. Chapters: Principles for using AI in evidence activities; AI methods for evidence synthesis. NICE presented it at ISPOR Europe 2026 as an initial version open to public feedback, so the text may change.
  2. Cochrane, Campbell Collaboration, JBI, Collaboration for Environmental Evidence. Position statement on artificial intelligence (AI) use in evidence synthesis. November 2025.
  3. PRISMA 2020. Item 8, selection process. EQUATOR Network.
  4. World Health Organization. Ethics and governance of artificial intelligence for health. 2021.
  5. World Health Organization. Artificial intelligence-related health research: ethics review and oversight. 2026.
  6. Guidelines International Network, McMaster University. Guideline Development Checklist extension for integrating artificial intelligence. 2026.
  7. RAISE. Responsible use of AI in evidence synthesis: recommendations and guidance. Version 4, draft for consultation, August and September 2026.
  8. Fleurence RL, et al. Generative artificial intelligence for systematic literature reviews: a good practices report of an ISPOR special task force. Value in Health, 2026. Read as abstract.
  9. Member State Coordination Group on Health Technology Assessment. General principles on the use of artificial intelligence in the preparation of dossiers for joint clinical assessments. Adopted 15 July 2026.
  10. Brasil, Ministério da Saúde. Diretrizes metodológicas: elaboração de revisão sistemática e meta-análise de ensaios clínicos randomizados. 2021.

Sources read on 14 November 2026. What NICE says is given in our own words; quotations from the other sources are in the original English. This summary is written by the developer of Compendium, who has an interest in the product; it is informative and does not replace reading the sources.