Back

Validate imaging AI against radiologists, on real cases

See how an AI tool performs alone, how radiologists perform alone, and how they perform together.
Book a demo
Research-Pipeline-Monitor XDR
Research- Results table

A vendor report shows how an AI tool performed on the vendor's data. A reader study shows how it performs on your data and your population: the same cases, read with and without the AI, scored on one form. Collective Minds runs that study in one workflow, so your time goes into the design and not into moving files.
Research-Pipeline-on tablet

How it works


  • Bring the cases. Send the exams through Connect or upload them on the web.
  • Set the ground truth. Expert radiologists label each case, for example, by consensus of two.
  • Add the AI output. Attach results from one or more AI engines to the cases.
  • Read with and without the AI. Each reader works from a personal task list, in phases if the design needs it.
  • Export the results. Per examination, per image and per finding.

What changes

  • One form, every reader. Everyone scores the same cases the same way.
  • AI and reads, linked. See where the AI helped and where it did not.
  • A record of the study. Who read which case, and when.

Projects like this

Regulatory reader study for an AI company

An AI company needed a compliant, scalable platform for a reader and annotation study to support a regulatory submission.
What we did: a cloud workflow for reading, annotation and consensus on 352 examinations, with 25 project members.
Result: the study finished on schedule and delivered its results before the regulatory deadline.

Read the Success Story

Regulated reader study for a MedTech company

A healthcare company needed a cloud solution for a regulated reader study, with images from several centers in several countries and reviewers in different countries.
What we did: secure data transfer by Connect, quality control, reading events for the classifications, and a reviewer from our MSK radiology community.
Result: images were collected, assessed and delivered in a traceable, compliant way.

Read the Success Story

Fracture detection in children with a rare bone disease

A children's hospital and a university tested a commercial AI tool.
What we did: 48 patients, 336 images, 7 readers, read with and without the AI.
Result: per examination, 74.8% for the AI alone, 83.4% for radiologists and 90.7% for radiologists with the AI. Published open access in European Radiology, 2025.

Two AI engines, one dataset

A university hospital wanted to compare two commercial fracture-detection engines.
What we did: 1,006 radiographs, read by radiologists and radiologist trainees, so both engines were judged on identical cases.

A reader study for an AI vendor

A vendor needed reader evidence in Europe.
What we did: 340 X-ray subjects, 15 readers in three geographies, three experts creating the ground truth.

Questions we hear

What is a reader study for AI validation?

Radiologists read the same cases with and without an AI tool, and the results are compared with a reference standard. It shows the accuracy of the AI alone, the readers alone and both together.
Can we compare more than one AI engine?

Yes. Attach results from several engines to the same cases and compare them on identical data. One university hospital did this with two commercial engines.
How are patient details protected?

Exams are de-identified before they are used in the study, and each project is open only to invited people. Your data protection team should confirm the setup for your study.
Who does what?

Collective Minds provides the platform and the workflow. Who recruits the readers, sets the ground truth and runs the analysis depends on the study. We agree it with you at the start.

Planning an AI validation?

Tell us about your study and we will show you how it would run.
Book a demo
ipad