AI & LLM4 WeeksiOS • Android
Fluently AI: Interactive Language Learning with On-Device OCR & Voice AI
Developed an interactive language learning app combining Google ML Kit camera OCR for real-world vocabulary capture and Whisper voice analysis for pronunciation grading.
Client
Fluently Education Ltd.
Role
Lead Mobile Engineer
Timeline
4 Weeks
Platforms
iOS, Android
<100ms
OCR Scan Speed
Real-Time
Voice Scoring
15k+
Daily Active Users
+28%
Retention Boost
The Core Engineering Challenge
- Real-time camera text recognition had to operate offline on low-end smartphones without lagging the live camera preview feed.
- Audio recording and pronunciation analysis required low-latency audio capture and synchronization with speech models.
Technical Solution & Implementation
- Integrated Google ML Kit Text Recognition to scan vocabulary from physical books and signs in under 100 milliseconds.
- Engineered an interactive audio recording and waveform visualization widget with instant Whisper API scoring.
- Built a spaced-repetition memory engine (SM-2 algorithm) that surfaces review flashcards at scientifically optimal learning intervals.
- Created engaging gamified quiz interfaces with micro-haptics and animated progress feedback.
Technical Architecture Highlights
On-Device ML Kit Pipeline
Direct camera image stream piping to native OCR C++ libraries without bridge serialization bottlenecks.
Spaced-Repetition Algorithm (SM-2)
Local algorithm tracking recall accuracy scores and scheduling future flashcard repetitions.
Technologies & Tools Used
FlutterGoogle ML Kit OCROpenAI Whisper AudioGPT-4o-miniRiverpodRevenueCat SubscriptionsLottie Animations
Measurable Business & Tech Results
- Sub-100ms camera text recognition running completely on-device.
- 28% increase in 30-day user retention driven by the automated spaced-repetition card algorithm.
- Seamless in-app subscription flow converting freemium learners into paid members.
Developed an interactive language learning app combining Google ML Kit camera OCR for real-world vocabulary capture and Whisper voice analysis for pronunciation grading.
Need a similar mobile architecture?
Mobile AI & LLM Integration
Let's discuss how we can engineer a high-performance cross-platform application for your business.