Back to projects
In developmentVoice AI · Multi-Agent · Healthcare

Mama Desk

An autonomous AI front-desk employee. Natural voice dialogues answering in about a second on fully local models, dynamic patient intake and symptom triage, live doctor-calendar scheduling, and schema-validated EHR extraction. The first scenario is a clinic, Hope Medical Clinic.

PythonWhisperLlama 3.1Kokoro TTSMulti-AgentFastAPIWebSockets
~1sVoice response, one local GPU
24/7Always-on digital workforce
4 agentsReception · knowledge · scheduling · EHR
FHIR R4Schema-validated EHR output

What it does

Real-time voice dialogue

Streaming speech-to-text, a retrieval-grounded LLM brain, and sentence-streamed TTS keep each reply to about a second, with no cloud AI APIs. It feels like a conversation, not a phone tree, and the pipeline is built to go faster.

Dynamic intake & symptom triage

Conversational patient intake with live triage classification. Urgent cases are flagged and routed to clinical staff while routine requests are handled end to end by the AI.

Live calendar scheduling

A scheduler agent executes real doctor-calendar bookings mid-conversation: availability checks, confirmations, and rescheduling, with no human in the loop.

Automatic EHR extraction

Every call becomes a schema-validated medical record (chief complaint, vitals mentioned, history) in FHIR R4 format.

Product screens

Live call console

mamadesk.oaimama.com / live calls
Mama Desk live call console mid-booking
A booking call in progress: the agent state, the live transcript with the sources each answer is grounded in, retrieved knowledge with scores, and every action the agent takes.

When the AI should not answer

live calls / billing escalation
Billing question escalated to a human
A refund request with no policy behind it: retrieval comes back weak, confidence drops below the threshold, and the call goes to a person.
handoff desk
Handoff desk with the context the AI gathered
Reception picks up with the transcript, what the AI already verified, and the knowledge it checked, so the caller never repeats themselves.

What the caller sees

Talk to Sarah start screen on a phone
One tap to talk, in English or Bangla.
Voice call in progress on a phone
The voice ring follows the agent: listening, thinking, speaking.
Call ended with the booked appointment
The call ends with the booking confirmed on screen.

Around the call

patient portal
Patient portal with upcoming appointments
Patient portal: appointments the agent booked, reports, prescriptions and billing in one place.
agent configuration
Agent configuration: knowledge, tools, handoff rules
Agent configuration: what it may look up, which tools it may run, and the rules that send a caller to a person.

Evolved from my local-first voice RAG assistant: streaming Whisper STT, a retrieval-grounded Llama 3.1 brain, and sentence-streamed Kokoro TTS running on a single consumer GPU, with no cloud AI APIs.