Project case study
Multimodal AI for Early Disease Detection
A multimodal disease-detection prototype recognized with second prize at MIT Mysore's Project Expo.
Problem
A research prototype exploring how medical images, patient context, and clinical text can be combined for early disease-detection workflows.
Context
The project was developed from June 2025 as a research-heavy attempt to combine medical images, electronic health records, genetic information, and symptoms rather than treating diagnosis as a single-input prediction problem.
My Role
Engineer
Contribution
- Defined a multimodal disease-detection workflow spanning medical images, electronic health records, genetic information, and patient-reported symptoms.
- Structured deep-learning feature extraction and data fusion as separate stages so each modality can be evaluated before fusion.
- Prepared the research-heavy system as a demonstrable Project Expo submission.
Architecture
Modality inputs
Collects medical images, health records, genetic information, and symptom signals.
Feeds Feature encoders
Feature encoders
Extracts modality-specific representations with deep-learning models.
Feeds Fusion layer
Fusion layer
Combines the modality representations into one prediction context.
Feeds Detection output
Detection output
Presents the early-detection result for review in the project workflow.
Terminal stage
Engineering Decisions
Fuse multiple modalities instead of relying on one source
Reason: The project targets patterns that can be missed when images, records, genetics, or symptoms are evaluated alone.
Tradeoff: The system depends on aligned, representative data across modalities and is harder to validate than a single-input model.
Keep modality processing separable before fusion
Reason: Separate encoders make each input path easier to inspect and replace.
Tradeoff: The pipeline has more interfaces and synchronization work than one end-to-end input path.
Outcomes
Received second prize for Multimodal AI Disease Detection at the Department of CSE (Data Science), MIT Mysore Project Expo.
Sources: cert-14
Limitations
- The available evidence verifies Project Expo recognition, not clinical accuracy, safety, or deployment readiness.
- No public clinical benchmark, model card, or patient-facing deployment is linked.
Stack
- Python
- Deep Learning
- ResNet
- BERT
- Healthcare AI
- Data Fusion