The Voice-First Scheme Agent is a public, voice-first service that helps citizens in India discover the welfare schemes they qualify for. Citizens miss schemes because the rules are scattered across many programmes and written in complex language. This product replaces that search with a spoken conversation: a friendly girl voice asks a small set of questions, determines eligibility against structured scheme rules, and explains its reasoning in plain language.
The audience is the citizen caller — including people who read with difficulty or not at all — who needs the simplest possible path to an answer: no login, no forms, only name, age, gender and caste spoken aloud. A second audience is the team that gathers scheme rules from the government's myScheme portal and converts them into a structured form the agent can reason over. A third is the judge evaluator who role-plays callers with vague or contradictory answers and scores eligibility accuracy and the number of questions asked.
The agent speaks in at least two languages, and specifically Telugu, Hindi and English, using a girl voice that can talk in multiple voices across multiple languages. It supports regional speech recognition, adaptive questioning, contradiction detection, explainable reasoning, scheme recommendation, application guidance and offline support.
The product is a first-party React web application backed by a shared FastAPI service and a MariaDB store. It is deliberately anonymous: there is no login and no sign-up screen anywhere in the citizen journey. The citizen opens the app, picks a language, and speaks. The only personal facts the agent asks for are name, age, gender and caste — nothing else.
The voice agent is presented on the left side of the page as a persistent agent panel with a flat girl avatar, while a large 168px coral voice button sits in the middle of the main column so that anyone can find it. The agent speaks with a girl voice, animates while speaking, and shows a human avatar that nods and moves its mouth in time with the speech. Text is always paired with voice so a low-literacy user can follow, and font size can be increased or decreased. Dark, light and neutral screen modes are available.
Scheme rules are owned by the team, not by the citizen: a scheme rules curator gathers rules from the government myScheme portal and converts them into a structured form the agent reasons over. That curation surface is role-restricted and separate from the citizen surfaces. Open speech datasets such as Mozilla Common Voice and AI4Bharat's collections are used as speech data sources for the voice and recognition layers.
Offline support means the citizen surfaces remain usable when connectivity drops: the last loaded scheme rules and the conversation state are kept locally so the agent can continue asking and answering, and results reconcile when the connection returns. Model-provider calls for language understanding and speech run through the LiteLLM gateway; the application's own data lives in MariaDB, and PostgreSQL exists only to support LiteLLM.
Everything described here is current scope. No future-horizon features are defined in this document.
Government myScheme portal (content_source, authoritative). The portal is the authoritative source of scheme rules. Each scheme gathered from it is converted into a structured record the agent reasons over, carrying at minimum: scheme name, the categories it serves by age, gender and caste, its eligibility conditions, the plain-language reason it applies, and its application guidance. The portal also supplies the breadth requirement that every scheme present in India is covered, including job schemes for every category.
Mozilla Common Voice (content_source). An open speech dataset used as a source of speech audio for the voice and regional speech recognition layers, covering the supported languages.
AI4Bharat speech collections (content_source). An open speech dataset used alongside Common Voice as a source of speech audio for Telugu, Hindi and English voice and recognition work.
FR-1 — Voice agent asks, determines and explains. As a citizen caller, I should speak with a voice agent that asks me questions, determines my eligibility and explains its reasoning, so that I understand why a scheme does or does not apply. Provenance: explicit. Lifecycle: the caller presses the voice button (trigger); the agent asks adaptive questions and the caller answers by speech (input); the agent produces an eligibility determination with a spoken and written reason (observable result); if the agent cannot determine eligibility it says so plainly and offers to re-ask (failure/recovery); the caller continues to results or corrects an answer (continuation). Access: no login. Acceptance: every determination is accompanied by a plain-language reason.
FR-2 — At least two languages, including Telugu, Hindi and English. As a citizen caller, I should use the agent in Telugu, Hindi or English, so that I can speak and listen in a language I understand. Provenance: explicit. Lifecycle: the caller selects a language before the session (input); the agent asks, recognises and speaks in that language (result); if recognition fails in that language the agent re-asks simply (recovery); the caller may change language mid-session (continuation). Acceptance: all three languages are selectable and the agent speaks and recognises each.
FR-3 — Girl voice across multiple voices and languages. As a citizen caller, I should hear a girl voice that can talk in multiple voices and multiple languages, so that the agent feels friendly and natural. Provenance: explicit. Lifecycle: the agent speaks each turn in the selected language using a girl voice (result); the voice is consistent across languages (observable); if audio output is unavailable the text remains (recovery). Acceptance: the speaking voice is a girl voice in Telugu, Hindi and English.
FR-4 — Only name, age, gender and caste; no login. As a citizen caller, I should be asked only my name, age, gender and caste and never be asked to log in, so that using the app is as easy as possible. Provenance: explicit. Lifecycle: the session begins with no account (access state); the agent asks exactly these four facts (input); the agent proceeds to eligibility (result); if an answer is missing the agent re-asks only that fact (recovery). Acceptance: no login or sign-up appears anywhere in the citizen journey, and no fact beyond the four is requested.
FR-5 — Contradiction detection. As a citizen caller, I should have contradictions in my answers detected, so that my result is not based on conflicting facts. Provenance: explicit. Lifecycle: the caller gives answers that conflict (input); the agent detects the conflict and names it in plain language (result); the agent asks a clarifying question (recovery); the caller resolves it and the session continues (continuation). Acceptance: a conflicting answer pair produces a visible and spoken contradiction notice and a clarifying question.
FR-6 — Adaptive questioning. As a citizen caller, I should be asked only the questions needed for my situation, so that the conversation stays short. Provenance: explicit. Lifecycle: the agent uses prior answers to choose the next question (result); questions that cannot change the outcome are skipped (observable); if an answer is unclear the agent re-asks that question (recovery). Acceptance: the number of questions asked is tracked and reported for scoring.
FR-7 — Scheme recommendation and application guidance. As a citizen caller, I should receive recommended schemes and guidance on how to apply, so that I can act on my eligibility. Provenance: explicit. Lifecycle: eligibility is determined (input); eligible schemes are recommended with a reason line and application guidance (result); if no scheme matches the agent explains why (recovery); the caller opens guidance or starts a new conversation (continuation). Acceptance: each eligible scheme shows a plain-language reason and application guidance.
FR-8 — Offline support. As a citizen caller, I should keep using the agent when connectivity drops, so that a weak connection does not end my conversation. Provenance: explicit. Lifecycle: connectivity drops mid-session (trigger); the agent continues asking and answering from locally held rules and state, showing an offline badge (result); when the connection returns, results reconcile (continuation); if a needed rule is not cached the agent says so plainly (recovery). Acceptance: the session continues offline and the offline state is visible.
FR-9 — Structured scheme rules from myScheme. As a scheme rules curator, I should gather scheme rules from the government myScheme portal and convert them into a structured form the agent can reason over, so that eligibility is accurate. Provenance: explicit. Lifecycle: the curator gathers a scheme's rules (input); converts them into the structured form and saves (action); the record becomes available to the agent (result); validation failure names the incomplete field (recovery); the curator continues with the next scheme (continuation). Access: role-restricted curator surface. Acceptance: saved structured records drive eligibility determinations.
FR-10 — Open speech datasets. As a builder of the voice layer, I should use open speech datasets such as Mozilla Common Voice and AI4Bharat's collections, so that regional speech recognition and the girl voice work across Telugu, Hindi and English. Provenance: explicit. Acceptance: the speech layer is built on these datasets.
FR-11 — Categories by age, gender and caste. As a citizen caller, I should see categories based on age, gender and caste, so that I can browse schemes that fit me. Provenance: explicit. Lifecycle: the caller opens the scheme catalogue (trigger); selects age, gender or caste category chips (input); matching schemes are listed (result); an empty category shows a plain message and a prompt to clear the filter (recovery). Acceptance: age, gender and caste categories are present and filter the catalogue.
FR-12 — Every scheme in India, including job schemes. As a citizen caller, I should find every scheme present in India, including job schemes for every category, so that nothing I qualify for is missing. Provenance: explicit. Acceptance: the catalogue covers all schemes gathered from myScheme, including job schemes across every category.
FR-13 — Big voice button in the middle. As a citizen caller, I should see a big voice button in the middle of the screen, so that anyone can find it immediately. Provenance: explicit. Acceptance: the voice button is a 168px circle centred in the main column at every width.
FR-14 — Animation and human avatar while speaking. As a citizen caller, I should see an animation and a human version while the agent speaks, so that the voice-first experience feels alive and human. Provenance: explicit. Lifecycle: the agent begins speaking (trigger); the avatar nods and its mouth moves with the speech and the button pulses (result); with reduced motion a still avatar and static ring are shown (recovery). Acceptance: the avatar animates in time with speech and degrades to a still state under reduced motion.
FR-15 — Font size increase and decrease. As a citizen caller, I should increase or decrease the text size, so that I can read comfortably. Provenance: explicit. Lifecycle: the caller presses increase or decrease (input); text scales within 0.9x to 1.4x (result); the setting persists across pages (continuation). Acceptance: both controls work and text stays whole at every scale.
FR-16 — Dark, light and neutral screen modes. As a citizen caller, I should switch between dark, light and neutral screen modes, so that the screen suits my eyes and light. Provenance: explicit. Lifecycle: the caller selects a mode (input); the interface recolours (result); the choice persists (continuation). Acceptance: all three modes are selectable and legible.
FR-17 — Regional speech recognition. As a citizen caller, I should have my regional speech recognised, so that I can speak naturally in my language. Provenance: explicit. Lifecycle: the caller speaks (input); regional speech recognition transcribes in the selected language (result); unrecognised speech triggers a simple re-ask (recovery). Acceptance: Telugu, Hindi and English speech are recognised.
FR-18 — As simple as possible for an illiterate user. As a citizen caller who reads with difficulty, I should be able to use the app with voice and large simple controls alone, so that literacy is never a barrier. Provenance: explicit. Acceptance: every citizen action is reachable by voice or a large labelled control, and text is always paired with speech.
FR-19 — Voice agent on the left. As a citizen caller, I should see the AI voice agent on the left side of the page, so that the conversation has a stable home. Provenance: explicit. Acceptance: the agent panel sits on the left at 1280px and stacks above the content at 375px.
FR-20 — Language selection before the session. As a citizen caller, I should choose Telugu, Hindi or English before the voice session, so that the agent speaks my language from the first word. Provenance: required_inference. Acceptance: the session does not begin until a language is chosen.
FR-21 — Speak the four facts. As a citizen caller, I should give my name, age, gender and caste by speech, so that I never have to type. Provenance: required_inference. Acceptance: each of the four facts is captured from speech and shown as recognised text.
FR-22 — Curator loads structured rules before eligibility. As a scheme rules curator, I should load structured scheme rules before eligibility checks run, so that the agent reasons over current rules. Provenance: required_inference. Acceptance: eligibility uses the saved structured records.
FR-23 — Judge scores accuracy and question count. As a judge evaluator, I should role-play a caller with vague or contradictory answers and score eligibility accuracy and the number of questions asked, so that the agent's quality is measured. Provenance: explicit. Lifecycle: the judge opens the voice session and role-plays a caller (input); the agent asks, detects contradictions and determines eligibility (result); the judge reads the eligibility outcome and the question count (observable); the judge scores accuracy and question count (completion). Access: no login, same citizen surfaces. Acceptance: the question count is visible on the results and the eligibility outcome is inspectable.
The citizen caller is a person in India who may qualify for welfare schemes but cannot easily find or read the rules. They may read with difficulty or not at all, so the product must work by voice and large simple controls. Their primary goal is to learn, in their own language, which schemes they qualify for and why. Their distinct responsibilities are: choosing Telugu, Hindi or English; speaking their name, age, gender and caste; answering adaptive questions; resolving contradictions when the agent names one; and hearing and reading the eligibility reasoning and application guidance. Their inputs are spoken answers and their decisions are which language to use, whether to correct an answer, and whether to open application guidance. They interact with the agent (the girl voice and avatar) and, indirectly, with the scheme rules the curator has structured. Their observable success is a clear, spoken and written list of eligible schemes with plain-language reasons and application guidance, reached without any login.
The scheme rules curator is a team member responsible for the accuracy behind the agent. Their product context is the Scheme Rules surface, which is role-restricted and separate from the citizen journey. Their primary goal is to convert scheme rules gathered from the government myScheme portal into a structured form the agent can reason over. Their distinct responsibilities are gathering rules, mapping each scheme to age, gender and caste categories, writing eligibility conditions and plain-language reasons, adding application guidance, and saving complete records. Their inputs are the myScheme portal rules; their decisions are how to express a condition in structured form and when a record is complete. They interact with the agent indirectly — their saved records drive every eligibility determination the citizen caller hears — and with the coverage list to see which schemes are structured. Their observable success is a saved, validated structured record that the agent uses, and a coverage list showing every scheme present in India, including job schemes for every category.
The judge evaluator is a person running the live test. Their product context is the same anonymous citizen surfaces, used deliberately as a caller. Their primary goal is to score eligibility accuracy and the number of questions asked. Their distinct responsibilities are role-playing a caller who gives vague or contradictory answers, observing how the agent handles the ambiguity and detects contradictions, reading the eligibility outcome, and counting the questions asked. Their inputs are deliberately vague or contradictory spoken answers; their decisions are what to answer and how to score. They interact with the agent as a caller would, and their observable success is a recorded eligibility outcome and question count they can score against the correct answer.
The muse is Haraldur Thorleifsson: big-hearted boldness for a voice that speaks for people. The headline is "Speak. We find your scheme."
Colour tokens (light mode). Background #FFF6E9 (cream ground, ~70% of the page); surface #FFFFFF; text #1B1A17; primary #E94F37 (coral — voice button and key actions); accent #1F8A70 (teal — eligible schemes); muted #6B6459.
Colour tokens (dark mode). Ground #14130F; text #FFF6E9; primary and accent retain coral #E94F37 and teal #1F8A70 for continuity.
Neutral mode. A low-saturation variant of the light tokens: cream ground softened toward a neutral surface, coral and teal desaturated while keeping contrast for the voice button and eligible panels.
Typography. Headings: Sora, bold at large sizes with tight tracking. Telugu and Hindi text: Noto Sans Telugu and Noto Sans Devanagari at the same weight. Body: Outfit. Modular scale 1.25 — 44/36/28/22/18/16 — with a user font-size control from 0.9x to 1.4x.
Shape language. Large rounded blocks with 28px radii and soft offset shadows. The voice button is a 168px circle with a coral ring. Sticker-like tags mark age, gender and caste.
Layout. The agent panel sits on the left at 1280px and stacks above the content at 375px. The big voice button stays centred in the main column at every width. Category chips for age, gender and caste sit below it. The Sora headline runs at 44px on mobile and 72px on desktop, flush left over the coral block.
Imagery. One flat vector girl avatar with warm skin tones and a coral headset. Small spot illustrations show documents, a phone and a check mark. No stock photos.
Avoid. No gradient blobs or glass panels; no blue or indigo primary colour; no dense tables or small grey labels; no stock photography of people; no login or sign-up screen.
The public entry is a cream page with a left agent column holding the flat girl avatar over a coral block. The centre holds the 168px coral voice button with a soft ring, and a bold Sora line above it reads "Speak. We find your scheme." Teal sticker tags under the button show Telugu, Hindi and English. The gesture is a warm, oversized invitation: the biggest object on the page is the thing you press to speak, and the friendly face beside it tells you who will answer. The concept recomposes only accepted content — the headline, subline, language tags, voice button, avatar and the font-size and screen-mode controls — and adds no new behaviour.
Interaction Model: Animated Motion Tempo: expressive Hero Dimensionality: layered_2d
Landing Hero Motion Brief. Focal subject: the flat girl avatar over the coral block and the centred 168px coral voice button. Thesis: the caller presses the big voice button → the ring pulses slowly and the avatar nods with her mouth moving in time with the girl voice → the caller hears the first question and knows the conversation has begun. Motion vocabulary: slow pulse ring while the agent speaks, gentle avatar nod and mouth movement, soft offset shadows on sticker tags. Composed first frame: cream ground, coral block on the left with the avatar, Sora headline flush left, the coral button centred with its ring at rest, teal language tags beneath. Reduced-motion state: a still avatar and a static ring, with all text and controls fully readable and usable.
prefers-reduced-motion, a usable static arrangement is provided: still avatar, static ring, wrapped or scrollable rows. Provenance: explicit (creative direction).
No login · No forms · Just speak
Tell us your name, age, gender and caste. A girl voice asks and explains in Telugu, Hindi or English.
Tap the big button. It listens to your voice only.
Choose your language above.
Pick the size you find comfortable and the colours that suit your eyes.
Saved on this device

No login · No forms · Just speak
Tell us your name, age, gender and caste. A girl voice asks and explains in Telugu, Hindi or English.
Tap the big button. It listens to your voice only.
Choose your language above.
Pick the size you find comfortable and the colours that suit your eyes.
Saved on this device
No comments yet. Be the first!