MedOmni¶
The open-source ecosystem for omni-modality medical intelligence. MedOmni is open infrastructure for medical AI on any data modality: tables, waveforms, images, volumes and notes.
A clinician reads lab tables, ECGs, sleep recordings, X-rays, retinal scans and notes side by side. MedOmni aims to be the shared, open infrastructure where agents and models learn to do the same: take in any medical data modality, be tested on it, and improve. A dataset or encoder added to MedOmni once can be used by every agent and model tested on it.
MedOmni is a shared Core with projects built on it. Each project depends on Core, and the projects exchange data only through Core's contracts, so each can be used and changed on its own.
This is v0.3.0, the first public release
MedOmni v0.3.0 (git tag v0.3.0) is the first public release and the first milestone of MedOmni. It has:
- the
medomnipackage and command - versioned contracts shared by all projects
- extension points for agents, datasets, tasks and encoders, filled by pull request or plug-in package without changing Core
Data use and patient privacy¶
MedOmni works in a sensitive medical domain, and we respect the data use agreement of every dataset it supports. No patient data ships with MedOmni: you build tasks from generated dummy data, open datasets such as VitalDB, or your own licensed copy of a dataset such as MIMIC-IV, and what is built from patient data stays on your machine.
What v0.3.0 includes¶
The projects and the core they share:
| Project | Does | Package | Command |
|---|---|---|---|
| Bench | Runs AI agents on built medical tasks and scores them, whoever built the tasks. A task gives an agent one patient's evidence and a clinical question; a fixed verifier scores the answer. It scores the agent as a whole (scaffold and model), as Terminal-Bench and SWE-bench do. | medomni.bench |
medomni bench |
| Perception | Serves medical encoders through an HTTP gateway. Each encoder reads one kind of input, such as a 12-lead ECG, and returns evidence for an agent or your own code. | medomni.perception |
medomni perception |
| Studio | Makes and builds tasks: dataset survey, task proposals from clinicians, review and sign-off, cohort previews, MedOmni's task builder, the audit and reports. | medomni.studio |
medomni studio |
| Core | Shared by every project: the contracts, the medomni command, plug-in loading, dataset loading, the family converter interface, the cache, the built-task contract and the testing kit. |
medomni.core |
medomni |
Not in v0.3.0: MedOmni Agent (a later project) and a PyPI release.
Who it is for¶
| Project | You are | You want to | Start here |
|---|---|---|---|
| Bench | Agent evaluator | Test your agent on MedOmni tasks and read the scores | Test your agent |
| Perception | Encoder user | Call medical encoders over HTTP from your own code | Use the encoders |
| Bench, Studio or Perception | Contributor | Add an agent to Bench, a dataset or a task to Studio, or an encoder to Perception | Contribute |
How it fits together¶
Bench and Perception each have their own package and content folder, and depend only on Core. Studio sits above both: its builder writes built tasks, which Bench runs and scores. Inside a task, the agent calls Perception's encoders through the perception service: Perception's gateway, which Bench starts beside the agent, with a call budget and a call log. Tasks, traces and results stay on your machine. Architecture shows the layout.
Where next¶
- Get started: install and score a first run on dummy data.
- Design principles: how MedOmni treats data, tasks and contributions, and where its design comes from.
- Concepts: the terms used across these docs.
- Architecture: the repository layout and how the projects fit.