Project case study

Real-Time Sign Language Interpreter

Live computer-vision interpretation wired into an interface that feels immediate instead of academic.

Repository not publicly availableLive demo not publicly available

Problem

A multi-modal interpretation system that streams sign-language analysis into a responsive browser UI with realtime visual feedback.

Built from June 2024 to February 2025 as a webcam-based communication prototype using a decoupled Python backend and a JavaScript presentation layer.

AI/ML Engineer, Data Systems Builder & Motion UI Developer

Built a decoupled FastAPI and WebSocket backend to process a live webcam feed without coupling inference to the browser rendering loop.
Integrated MediaPipe Holistic tracking for face, hand, and pose landmarks and a pre-trained Caffe DNN for the facial-analysis module.
Implemented a One-Euro Filter in Python to reduce temporal landmark jitter.
Streamed processed results to a JavaScript frontend with landmark and status overlays.

Camera stream
Captures the live webcam frames used by the perception pipeline.
Feeds FastAPI gateway
FastAPI gateway
Coordinates frame processing and WebSocket delivery.
Feeds Perception modules, Temporal filter
Perception modules
Runs MediaPipe landmark tracking and the Caffe facial-analysis model.
Feeds Temporal filter
Temporal filter
Applies One-Euro smoothing before the results reach the interface.
Feeds Browser UI
Browser UI
Renders the streamed feed, landmarks, and dynamic overlays.
Terminal stage

Reason: The browser can remain responsive while Python coordinates the computer-vision workload.

Tradeoff: WebSocket transport introduces connection lifecycle and synchronization work.

Reason: Raw frame-to-frame landmark movement produced visible jitter.

Tradeoff: Smoothing can add latency and requires tuning for fast gestures.

No measured outcome is documented in the current source archive.

No public latency, recognition-accuracy, or accessibility benchmark is linked.
The facial-analysis module uses a pre-trained classification model and is presented as a prototype component, not an identity or medical system.

No external evidence is linked to this project.