How to Critically Appraise a Research Paper: A Practical Guide for Doctors
"Ability to critically appraise evidence" appears on a great many NHS person specifications, yet many doctors were only taught it as a block of statistics before finals. Critical appraisal is not about memorising formulas. It is a repeatable way of asking three questions of any paper: can I trust it, what does it show, and does it apply to my patients?
Quick answer
To critically appraise a research paper, first define its question using PICO and check the study design suits that question. Then judge validity (randomisation, allocation concealment, blinding, losses to follow-up), results (effect size, confidence interval, absolute risk reduction, NNT) and applicability to your patients. A structured tool such as a CASP checklist keeps you consistent and stops the abstract steering you.
What you will learn in our free Research and Critical Appraisal: Evidence-Based Medicine for Doctors course
'Understanding of research' and 'ability to critically appraise evidence' are on almost every NHS person specification. Build the skills — and the evidence — that get you shortlisted. Our free online Research and Critical Appraisal: Evidence-Based Medicine for Doctors course is written for NHS doctors and international medical graduates. You study at your own pace in 6 short modules, with worked examples, interactive exercises and a short quiz at the end. There is no sign-up, and your progress is saved so you can stop and come back.
The 6 modules
Evidence-based medicine: the basics: Why evidence matters in everyday NHS practice, What is evidence-based medicine?, Asking a focused question — PICO, Research, audit or service evaluation?
Finding the evidence: Where to find evidence, Searching well
Study designs and bias: Study designs — the hierarchy of evidence, Choose the right design, Bias and confounding
Statistics in plain English: p-values and confidence intervals, Relative risk, absolute risk and NNT, Diagnostic tests, More terms you will meet
Critical appraisal step by step: Appraising a randomised controlled trial, Presenting at journal club, From evidence to the patient
Doing research and getting published: Research ethics and approvals, Good Clinical Practice and the 2026 UK trial rules, Realistic ways to get involved, Authorship, your CV and interviews, Download your appraisal
By the end of the course you will be able to
Explain evidence-based medicine and turn a clinical problem into a focused PICO question
Search effectively for evidence using the right sources
Choose the right study design for a question and recognise common biases
Interpret p-values, confidence intervals, relative and absolute risk, NNT, sensitivity, specificity and predictive values
Critically appraise a randomised controlled trial using a structured checklist
Present a paper at journal club and apply evidence to individual patients
Explain research ethics, HRA approval, Good Clinical Practice and the UK's 2026 clinical trial rules
Find realistic ways to get involved in research and publish, and describe it on your CV
Why it helps your NHS application
Statistics in plain English, with worked examples: confidence intervals, NNT, sensitivity and specificity
Appraise a trial step by step and present it at journal club — then download your appraisal
Realistic routes into research and publication, plus the UK's new 2026 clinical trial rules
How to write it on your NHS application
In your CV and on NHS Jobs or Trac: list it under "Research and publications" (or in the training and courses section of the form), for example:
Research and Critical Appraisal: Evidence-Based Medicine for Doctors, online CPD course, CareerPro UK. Certificate of completion, [month and year]; verifiable online, ID CP-XXXXXXXX.
In your supporting statement: show what you learned and how you use it. For example (adapt it so it is true for you):
"I critically appraise evidence using structured checklists such as CASP, a skill I developed through an online research and critical appraisal course, and I present papers at journal club."
At interview: the course also prepares you for questions such as "Tell us about a research paper you have read recently. How did you appraise it?"
Your certificate of completion
When you pass the quiz, you can download a certificate pack:
A verifiable PDF certificate of completion with your name, the course, your CPD time and a QR code that shortlisters and appraisers can scan
An appraisal-ready reflective note, pre-filled with the course learning outcomes
A ready-to-paste line for your NHS Jobs or Trac application and CV, and a one-click "Add to LinkedIn" button
Studying is free. The certificate pack is a one-off £5, and you pay only after you pass
Online course — not Good Clinical Practice (GCP) certification.
Step 1: Pin down the question with PICO
Before judging how well a study was done, be clear about what it asked. PICO gives you four parts:
P – Population: who was studied (for example, adults with type 2 diabetes)
I – Intervention or exposure: what was tested (a drug, test or risk factor)
C – Comparison: compared with what (placebo, usual care, another treatment)
O – Outcome: what was measured (ideally something patients care about, such as admissions or death)
If you cannot work out the PICO from the methods, that is itself a warning sign about how clearly the paper is reported. Writing your own PICO also makes it obvious later whether the trial population looks like your patients.
Step 2: Check the design matches the question
Different questions need different designs. Ask this first, before any detail:
Does a treatment work? A randomised controlled trial (RCT), or a systematic review of RCTs.
How accurate is a test? A study comparing the test against a reference standard in a relevant group of patients.
What happens to patients over time? A cohort study.
Does an exposure cause harm, or what causes a rare disease? A cohort or case-control study.
What is it like for patients? A qualitative study.
The "hierarchy of evidence" is a useful starting point, not a rule. A well-conducted cohort study can be more informative than a poorly run trial. Guideline developers, including NICE, often use the GRADE approach to rate how much confidence they have in a body of evidence.
Step 3: Use a CASP checklist to judge validity
The Critical Appraisal Skills Programme (CASP) publishes free checklists for most study types, including RCTs, systematic reviews, cohort, case-control, diagnostic, qualitative, cross-sectional and economic evaluation studies. Using the right checklist means you ask the same important questions every time.
The current CASP RCT checklist has four sections and 11 questions. In summary it asks:
Is the basic design valid? Was there a clearly defined question? Were participants randomly allocated, with the allocation concealed? Were all participants accounted for at the end?
Was the study methodologically sound? Were participants, investigators and outcome assessors blinded? Were the groups similar at the start? Apart from the intervention, were they treated the same?
What are the results? Were effects reported fully, and how precise are they?
Will the results help locally? Do they apply to your population, and would the intervention add value compared with existing options?
Where to look in the paper
Read the title and abstract for the question, then go straight to the methods. Look at:
Table 1, which shows whether the groups were similar at baseline.
The flow diagram, which shows how many people dropped out of each arm and why.
The primary outcome, and whether it matches what the trial registered (for example on ISRCTN or ClinicalTrials.gov).
The analysis type. Intention-to-treat analyses people in the groups they were randomised to, which preserves the benefit of randomisation. A per-protocol analysis alone can make a treatment look better than it is in practice.
The biases worth naming
Selection bias: the groups differ at the start. Randomisation and concealed allocation reduce it.
Performance bias: groups are treated differently apart from the intervention. Blinding participants and clinicians reduces it.
Detection bias: outcomes are measured differently. Blinding outcome assessors reduces it.
Attrition bias: people drop out unevenly between groups.
Reporting and publication bias: positive results are more likely to be published or highlighted.
Confounding: a third factor linked to both exposure and outcome, which mainly affects observational studies.
Reporting guidelines collected by the EQUATOR Network, such as CONSORT for trials, STROBE for observational studies and PRISMA for systematic reviews, tell you what a well-reported paper should include.
Step 4: Read the results, in plain English
You do not need to calculate statistics from raw data, but you do need to read them correctly.
p-values
A p-value tells you how likely a result at least this extreme would be if there were truly no difference. By convention, p < 0.05 is called "statistically significant". It does not tell you how large or important an effect is.
Confidence intervals
A 95% confidence interval (CI) is the range likely to contain the true effect. A narrow interval means a precise estimate. For a difference, a CI that crosses 0 is not statistically significant. For a ratio (relative risk, odds ratio, hazard ratio), a CI that crosses 1 is not. Look at both ends: if the true effect were at the smaller end, would it still matter to your patients?
Relative versus absolute risk, and NNT
This is where headlines mislead. Suppose 15 in 100 patients on placebo have a stroke, compared with 10 in 100 on a new drug.
Relative risk = 10 ÷ 15 = 0.67, so the relative risk reduction is about 33%.
Absolute risk reduction (ARR) = 15% − 10% = 5%.
Number needed to treat (NNT) = 1 ÷ 0.05 = 20. Twenty people need treatment for one extra person to avoid a stroke, over the trial's time period.
"Cuts stroke by a third" and "one in 20 benefits" describe the same result. Always ask: what is the absolute difference, and over what time? The same relative effect gives a bigger absolute benefit to a patient at higher baseline risk.
Statistical versus clinical significance
A very large trial can make a tiny difference statistically significant. Ask whether the size of the effect would matter to a patient, and weigh it against harms. The number needed to harm (NNH) works the same way as NNT for side effects.
Step 5: Decide whether it applies, then write a bottom line
Evidence-based medicine combines the best available evidence with clinical expertise and the patient's values. So finish by asking:
Are the trial patients like mine in age, severity and other conditions?
Were all outcomes that matter considered, including harms?
Do benefits outweigh harms, burdens and costs?
How does this fit with current NICE or Royal College guidance?
Then write a one-sentence bottom line, such as: "Well-conducted trial with a modest absolute benefit in younger patients than ours; not enough on its own to change our practice before guidance is updated."
Our free Research and Critical Appraisal course takes you through each of these steps with worked NNT and diagnostic test examples, and helps you build a one-page appraisal of a real paper.
How appraisal comes up in NHS applications and interviews
National specialty training person specifications commonly list research understanding and critical appraisal under essential criteria. The Core Psychiatry CT1 2026 specification, for example, includes the "capacity to critically appraise research in psychiatry". Trust-grade and specialty doctor posts often use similar wording. Our guide to the NHS person specification explains how to evidence each criterion.
Common interview prompts include:
"Tell us about a paper you have read recently."
"How would you decide whether a new study should change your practice?"
"What is the difference between relative and absolute risk?"
A strong answer to the first question takes about a minute and follows the appraisal structure: the PICO and design, the key result with absolute numbers, one strength and one limitation, how it fits with guidance, and whether it changed what you do. Presenting at journal club is the best rehearsal, and it gives you something concrete to write on your application under research and teaching.
On your application, be specific: "Presented a critical appraisal of [trial] at departmental journal club, [month year]" is stronger than "good appraisal skills". If you would like help with the wording, see our guide to writing an NHS supporting statement as an IMG.
Start the free course
Research and Critical Appraisal: Evidence-Based Medicine for Doctors covers PICO, searching, study designs and bias, statistics in plain English, step-by-step appraisal of a trial and journal club presentation, in short modules you can study free. You finish with a quiz, a downloadable appraisal template, and the option of a verifiable certificate and a reflective note for appraisal; the course is not accredited and is not Good Clinical Practice certification. Start the free course here.
If you would like your NHS application reviewed and rewritten (£35), email cvreviewexperts@gmail.com.
Frequently asked questions
What is the CASP checklist?
CASP checklists are free structured question lists from the Critical Appraisal Skills Programme, with a separate checklist for each main study type. The RCT checklist asks whether the design is valid, whether the study was methodologically sound, what the results are and whether they will help locally.
What is the difference between relative risk reduction and absolute risk reduction?
Relative risk reduction is the proportional fall in risk compared with the control group, while absolute risk reduction is the simple difference between the two event rates. If risk falls from 2% to 1%, the relative reduction is 50% but the absolute reduction is only 1%.
How do you calculate the number needed to treat?
Divide 1 by the absolute risk reduction expressed as a proportion, then round up to a whole number. An absolute risk reduction of 5% (0.05) gives an NNT of 20 over the study's time period.
What does it mean if a confidence interval crosses 1?
For a ratio such as a relative risk, odds ratio or hazard ratio, a value of 1 means no difference. If the 95% confidence interval includes 1, the result is not statistically significant at the conventional level.
Why is intention-to-treat analysis preferred in trials?
Intention-to-treat analysis keeps everyone in the group they were randomised to, whether or not they completed treatment. This preserves the balance created by randomisation and gives a more realistic estimate of how a treatment performs in practice.
Comments