Understanding CONSORT-AI: Why Clinical Trials Needed an Upgrade

# Understanding CONSORT-AI: Why Clinical Trials Needed an Upgrade Artificial intelligence is rapidly moving from tech laboratories into doctors’ offices. From tools that scan X-rays for early signs of cancer to algorithms that predict patient deterioration, AI holds incredible promise for modern healthcare. But this rapid integration brings up a critical question: *How do we know these AI tools are actually safe and effective in the real world?* Traditionally, the gold standard for testing new medical treatments has been the clinical trial, guided by strict reporting rules known as the CONSORT statement. However, when it comes to AI, traditional trials have a major blind spot. ## The AI Blind Spot Let’s look at a hypothetical example: **SkinCheck-AI**, a smartphone app designed to help doctors detect melanoma. If we tested SkinCheck-AI using standard clinical trial guidelines, the resulting report might tell us that the app was generally effective. But it would likely miss crucial, AI-specific details: * Which exact version of the algorithm was used? (Since software updates constantly, v1.0 might be safe, but v1.2 might have a bug). * What happens if a doctor inputs a blurry, poorly lit photo? * Did the AI perform equally well across all skin tones, or did it misdiagnose certain demographics? Standard clinical trials simply weren't built to ask these questions. ## Enter CONSORT-AI To fix this, an international group of experts developed **CONSORT-AI**. This is an official extension of the standard clinical trial guidelines, adding **14 new, AI-specific reporting items**. CONSORT-AI forces researchers and developers to show their work. It focuses on three core pillars: **1. Clinical Context:** It’s not enough to say an AI works in a vacuum. CONSORT-AI requires developers to explain exactly where the AI fits into the clinical workflow. For SkinCheck-AI, is it meant to replace a biopsy, or just flag high-risk patients for a specialist? **2. Data Quality:** AI is only as good as the data it’s fed. The new guidelines require strict definitions of input data. If SkinCheck-AI was only tested on high-resolution clinical cameras, researchers must explicitly state that smartphone photos were excluded, preventing doctors from misusing the tool later. **3. Human Teamwork:** Who has the final say? AI rarely operates entirely on its own. CONSORT-AI demands clarity on how the human and the AI interact. If SkinCheck-AI flags a mole as cancerous but the doctor disagrees, the trial must explain how that disagreement is resolved. ## Mandatory Error Analysis Perhaps the most important addition in CONSORT-AI is the requirement for mandatory error analysis. It is no longer acceptable to just report overall accuracy. Researchers must break down *when* and *why* the AI fails. If our hypothetical SkinCheck-AI struggles to identify melanomas on darker skin tones, CONSORT-AI ensures that this flaw is documented and published. This level of transparency is essential for preventing systemic harm and bias in healthcare. ## The Bottom Line AI has the potential to revolutionize medicine, but we cannot afford to evaluate it like a standard pill. CONSORT-AI provides the rigorous framework we need to ensure that when AI enters the clinic, it does so safely, transparently, and effectively. Ultimately, transparent AI means safer patients. ```eof

# Executive Report: Terminology Coverage Gaps in HIS Naming Conventions **Subject:** Assessment of ICD-11 and SNOMED CT Coverage Across Hospital Information System Domains, and Recommended Architecture for Billing Completeness **Date:** 5 November 2026 **Prepared for:** HIS Design and Clinical Informatics Leadership **Status:** Final --- ## 1. Executive Summary This report assesses whether ICD-11 and SNOMED CT, used together, can provide a complete naming convention for a Hospital Information System (HIS). The conclusion is unambiguous: **they cannot.** The two terminologies are designed for clinical documentation and statistical classification respectively, and neither is scoped for administrative, operational, financial, or workflow naming. Coverage varies sharply by domain—substantial in some (clinical investigations, imaging anatomy), superficial in others (physical therapy, medications), and essentially absent in the most critical gap area: **hospital billing**. The recommended solution is not a search for a single comprehensive terminology, but the adoption of a **layered coding architecture** in which each terminology system serves the domain it was designed for, with HIS-local master data filling residual gaps. --- ## 2. Scope and Method The assessment covered the following HIS naming domains: - Administrative and financial concepts - Operational and non-clinical entities - Laboratory tests and investigations - Medications - Medical imaging - Physical therapy - Clinical investigations (e.g., cardiac stress test) - Hospital billing and reimbursement For each domain, the coverage of ICD-11 and SNOMED CT was evaluated, and residual gaps requiring local or third-party standards were identified. --- ## 3. Domain-by-Domain Findings | Domain | SNOMED CT Coverage | ICD-11 Coverage | Residual Gap | |---|---|---|---| | **Administrative / financial** | None (explicitly out of scope) | None | Entire domain requires local definition | | **Operational / non-clinical** | None (excludes veterinary, local extensions, classification phrases) | None | Local extensions required | | **Emerging technology (e.g., AI tools)** | ~50% of health technology terms absent | ~50% absent | No formal representation exists | | **Laboratory tests** | Substantial via PaLM content set (~3,900 reportable items) | Effectively absent (only coarse "laboratory examination" code) | Genomics, imaging, detailed methodological attributes; LOINC needed as primary identifier | | **Medications** | Partial (substance-level concepts; medication names are the largest gap category at ~50% of missing concepts) | Absent (only classifies adverse effects and substance use disorders) | RxNorm or local medication master data required as product identifier | | **Medical imaging** | Strong (recommended standard for imaging anatomy; detailed procedure concepts) | Absent (only "radiological examination" administrative code) | Protocol parameters, contrast details, equipment attributes | | **Physical therapy** | Superficial (coarse concepts only: "physical therapy procedure," "physiotherapist") | Minimal (only "rehabilitation care" contact classification) | Treatment techniques, exercise prescriptions, frequencies, outcome measures | | **Clinical investigations (e.g., cardiac stress test)** | Substantial (granular concepts distinguishing modality and imaging companion) | Absent | Protocol details (e.g., Bruce protocol), contraindication logic | | **Hospital billing** | Not designed for billing | Not designed for billing; requires procedure classification and grouping layer | Entire billing architecture | --- ## 4. Critical Finding: Billing Cannot Be Constructed from ICD-11 and SNOMED CT This is the most consequential finding for HIS design. **Neither terminology is a billing system.** - ICD-11 is a statistical classification, not a reimbursement nomenclature. - SNOMED CT is a clinical terminology whose granularity far exceeds billing requirements. - The mapping between SNOMED CT and ICD-11 is **not reliable enough for automated billing generation**. Documented mapping errors include concepts mapped to overly broad categories, concepts mapped to clinically incorrect codes (e.g., "unspecified anemia" mapped to "constrictive pericarditis"), and incorrect ICD-11 code assignment. - International precedent confirms the gap: NHS England combines ICD-10 with **OPCS-4** (procedure classification) and **HRG** (Healthcare Resource Groups) to support reimbursement—ICD alone is insufficient. - Historical evidence shows vendor readiness is a binding constraint: ICD-10 transitions took far longer than planned because revenue cycle and EHR vendors did not deliver updates on schedule. --- ## 5. Recommended Solution: Layered Coding Architecture The report recommends a five-layer architecture. Each layer has a defined owner and purpose; no single terminology is expected to span multiple layers. **Layer 1 — Clinical Documentation (SNOMED CT).** Capture clinical detail at the point of care: diagnoses, procedures, symptoms, observations. Purpose: accurate recording, not bill generation. **Layer 2 — Classification Aggregation (ICD-11 + a procedure classification system).** Aggregate clinical diagnoses into statistical categories, paired with a procedure classification (the OPCS-4 analogue in the local jurisdiction). These two together form the input to reimbursement grouping. **Layer 3 — Billing Grouping (Local DRG / HRG).** Map diagnosis classification + procedure classification + patient attributes (age, complications, etc.) to local billing groups. This layer is governed entirely by local health insurance policy and is independent of ICD-11 and SNOMED CT, which serve only as inputs. **Layer 4 — Itemized Identification (Specialized Terminologies + Local Master Data).** Domain-specific identifiers: - Laboratory tests: **LOINC** as primary identifier - Medications: **local medication master data** or RxNorm as product identifier; SNOMED CT supplements ingredient and clinical associations - Medical consumables: **UDI** or local material codes - Imaging and procedures: SNOMED CT procedure concepts as clinical naming basis; billing items defined locally **Layer 5 — Local Extension (HIS Custom).** All billing administrative attributes: charge codes, prices, insurance coverage categories, billing units, package relationships. Maintained by HIS local master data; not available from either terminology. --- ## 6. Implementation Risks 1. **Mapping reliability.** SNOMED CT-to-ICD-11 mapping cannot be trusted for automated billing code generation. Human review and locally maintained mapping rules are mandatory. 2. **Vendor readiness.** Software updates from revenue cycle and EHR vendors have historically lagged terminology transitions. Vendor capability must be treated as a design constraint from the outset. 3. **Licensing.** SNOMED CT anatomical code sets are not fully free (limited free set and Global Patient Set only); implementation requires membership or licensing, which constrains barrier-free global adoption. 4. **Concept lifecycle.** SNOMED CT concepts are retired and replaced (e.g., certain exercise stress echocardiography codes). HIS implementations must enforce use of current active concept IDs. 5. **Local policy dependence.** Billing grouping and pricing are jurisdiction-specific; the architecture must be designed for local configuration, not hard-coded to any international standard. --- ## 7. Recommendations 1. **Adopt the layered architecture** as the HIS naming convention framework. Do not attempt to derive billing from ICD-11 or SNOMED CT directly. 2. **Designate domain-primary identifiers** explicitly: SNOMED CT for clinical procedures and findings; LOINC for laboratory tests; RxNorm or local master data for medications; UDI for consumables; ICD-11 plus a procedure classification for statistical aggregation. 3. **Invest in a maintained mapping layer** between SNOMED CT and ICD-11, with human review workflows and version control. 4. **Treat HIS local master data as a first-class asset**, not a residual—it carries all billing, pricing, and workflow attributes. 5. **Engage revenue cycle and EHR vendors early** on terminology transition timelines; contract for delivery milestones. 6. **Plan for local extension governance**, including processes for adding concepts in genomics, emerging technologies, physical therapy detail, and protocol-level investigation attributes. --- ## 8. Conclusion ICD-11 and SNOMED CT are necessary but insufficient components of an HIS naming convention. Their combined coverage is strong for clinical documentation and procedure naming, weak for medications and physical therapy, and absent for administrative, operational, and billing domains. A complete hospital billing capability requires a layered architecture in which specialized terminologies (LOINC, RxNorm, UDI), a procedure classification system, local billing groups, and HIS-local master data each perform the role they were designed for. **The completeness of the HIS naming convention depends not on the breadth of any single terminology, but on the rigor of the architecture that connects them.** --- *Prepared based on the terminology coverage assessment conducted in this engagement. Domain-specific findings reflect the evidence reviewed during the assessment; local jurisdictional requirements should be validated against applicable health insurance and reimbursement regulations before implementation.*

 

From Clinical Notes to Standardized Data: Open-Source AI and FHIR for Teaching Hospitals

 

 

 

For medical practitioners and lecturers in teaching hospitals, clinical documentation is a double-edged sword. On one hand, free-text clinical narratives capture the nuanced reality of patient care. On the other, this unstructured text is difficult to use for large-scale clinical research, student training, and interoperability.

The goal has always been standardizing this data into the HL7 FHIR (Fast Healthcare Interoperability Resources) format. Today, AI-driven Natural Language Processing (NLP) is making this transformation seamless, accurate, and—most importantly—accessible through open-source technology.

Here is a practical guide on how teaching hospitals can leverage open-source Large Language Models (LLMs) to transform unstructured clinical text into FHIR formats, and the minimal hardware required to do it.

How AI Automates Mapping and Extraction

Transforming a physician’s note into a structured JSON FHIR bundle is a complex task, but modern transformer-based models and Retrieval-Augmented Generation (RAG) are achieving high semantic accuracy.

• Entity Extraction: Open-source NLP engines can now read free text and accurately identify clinical concepts—like medications, lab results, and patient conditions. They map these to specific FHIR resources (e.g., Observation, Condition, and MedicationRequest) achieving impressive F1-scores of 0.89 to 0.94.

• LLM Transformation: Using structured prompting, open-weight models like Llama 3.2 can convert messy clinical narratives into valid FHIR R4 bundles. This vastly outperforms traditional, rigid rule-based systems.

• Semantic Standardization: The AI doesn't just extract words; it understands them. It normalizes extracted data against universal terminologies like SNOMED CT and LOINC, ensuring true interoperability and drastically reducing the manual coding overhead for medical records departments.

The Implementation Architecture

Integrating this into a teaching hospital's workflow doesn't require tearing down existing infrastructure.

1. Pipeline Integration: The system typically sits downstream. It parses raw HL7 v2 messages or clinical notes from your EMR, uses NLP to extract the entities, and generates FHIR resources via RESTful APIs.

2. Self-Correction Loops: Advanced AI agents act like meticulous medical coders. They use validation tools to check for FHIR compliance (ensuring mandatory fields are filled) and automatically rewrite mappings if errors are detected.

3. Efficiency: AI-driven conversion reduces integration configuration time by 30–60%. Remarkably, resource generation can hit sub-220 millisecond latency, meaning real-time processing is a reality.

The Open-Source Advantage

For academic and teaching hospitals, budget constraints and data privacy (HIPAA/PDPA compliance) make cloud-based, proprietary AI tools challenging to adopt. Fortunately, the open-source community is thriving.

• NL-FHIR & SMART Text2FHIR: These open-source projects provide ready-to-use NLP modules specifically designed to extract concepts from clinician notes and map them directly to FHIR.

• HL7 Standards Alignment: HL7 International is actively developing specifications to make FHIR more usable by AI, leveraging its machine-readable JSON/XML structure. By using open-source tools aligned with these standards, teaching hospitals ensure their research data remains future-proof.

Minimal Device Requirements for Local Deployment

To protect patient privacy, teaching hospitals should run these models locally (on-premises). You do not need a multi-million-dollar supercomputer to get started.

For a Departmental Server (Running models like Llama 3.2 8B):

• GPU: A single consumer-grade high-end GPU (e.g., NVIDIA RTX 3090 or 4090 with 24GB VRAM) or an entry-level workstation GPU (NVIDIA RTX A4000/A5000).

• RAM: 32GB to 64GB of system RAM.

• Storage: 500GB NVMe SSD for fast model loading and data processing.

• Note: This minimal setup is sufficient for a local REST API that students and clinicians can query for converting notes to FHIR in real-time.

Bringing it to the Wards: miniLLMs on Mobile Devices

Can a medical student or attending physician run these transformations directly on a tablet at the patient's bedside, completely offline? Yes.

The rise of miniLLMs (Small Language Models) and model quantization (compressing models with minimal quality loss) has made mobile deployment viable.

• The Models: Models like Llama 3.2 (1B and 3B parameter versions), Microsoft's Phi-3 Mini, or Google's Gemma 2B are small enough to run on mobile processors.

• The Hardware: A modern high-end tablet (like an iPad Pro with an M-series chip) or a flagship Android device (with 8GB+ of RAM) can run these models locally using frameworks like llama.cpp or MLC LLM.

• The Use Case: While a mobile device might not process bulk batches of hospital records, a miniLLM can easily process a single patient note dictated by a doctor on the ward, extracting SNOMED codes and generating a preliminary FHIR resource instantly, without ever sending sensitive patient data over the Wi-Fi.

Conclusion

The leap from unstructured clinical narratives to structured FHIR data is no longer gated by expensive, proprietary software. By embracing open-source models, teaching hospitals can build cost-effective, secure, and highly accurate pipelines that enhance clinical research, streamline billing, and provide an incredible training ground for the next generation of medical professionals.