AdaptSense
Cross-platform accessibility app translating between 13 media and language combinations with AI.
- React Native
- Expo
- Gemini API
- Accessibility
The problem
Accessibility tools tend to solve one translation in one direction - speech-to-text, or text-to-speech, and that's it. Visually and hearing-impaired users live across many combinations of input and output, and switching apps for each one is friction they shouldn't have to carry.
AdaptSense treats modality translation as one pipeline instead of a dozen features.
What I built
A cross-platform mobile app built with React Native and Expo, designed around daily accessibility for visually and hearing-impaired users.
At its core is a flexible media processing pipeline: it accepts audio, image, text or braille input and dynamically routes it to the right visual or auditory output - including sign-language GIFs. Gemini APIs in the backend handle the translation work across 13 distinct language and media combinations.
Architecture
The Expo app captures or receives media and hands it to a backend pipeline that classifies the input type, selects the target output modality, and dispatches to the appropriate Gemini-powered translation path. Output rendering is modality-aware on the client - audio playback, visual display, or GIF-based sign language.
Results
- One pipeline handles 13 unique language and media combinations with high accuracy.
- Runs cross-platform from a single React Native codebase via Expo.
- Input-agnostic by design: audio, image, text and braille all enter the same routing pipeline.
Stack
- App
- React Native · Expo · TypeScript
- AI
- Gemini API