KshemaGPT Part 2: How AI, Voice, and Multilingual Agents Empower Indian Farmers

KshemaGPT is transforming crop insurance and agricultural support in India through a modular, AI-powered system. This blog explores how its agents — from user data to climate — work together to deliver personalized, voice-first, and region-specific assistance to farmers.
Choose Kshema’s Reliable Protection for Your Crops
  • Buy in easy steps
  • Premium Starts at INR 499
  • Protect 100+ Crops
  • Quick & Easy Claims
By submitting this form, you agree to be contacted on your phone number or e-mail address by Kshema General Insurance and/or its representatives.
Download the App & Save 5%

User Data Agent: Personalizing the Farmer Experience

The User Data Agent plays a crucial role in personalising the KshemaGPT experience. It connects directly to user transaction databases, enabling the system to access detailed user profiles, policy purchases, premium payment history, and claims settlement status. It helps with support queries such as checking the status of an insurance claim or submitting a grievance which are logged in a separate database. The status of any of these previously raised issues can be asked later as well. This agent is itself a mini multi-agent system. One sub-agent is responsible for retrieving user details, while another handles query registration and tracking. Both agents interact with the databases via the Model Context Protocol (MCP), which allows LLMs to operate on external data sources securely and efficiently. To translate natural language queries into actionable database requests, we use a custom Natural Language-to-SQL (NL2SQL) model. This enables users to interact with complex backend systems using simple conversational language, ensuring seamless access to important personal data and support services.
  • User profiles
  • Policy purchases
  • Premium payment history
  • Claims settlement status
It supports queries like checking insurance claim status or submitting grievances. These are logged separately and can be tracked later.

Crop agent

When farmers have crop-related queries, they get routed to the Crop Agent. Initially, we explored several general-purpose agronomy models, but they lacked the specificity needed for region-specific crops and local farming practices.

We then integrated Dhenu2-In-Llama3.1-8B-Instruct, a specialised model trained by KissanAI on extensive crop data. This dramatically improved the relevance and depth of our responses enabling questions about disease management, pest control, and best practices, receiving tailored advice based on their specific crops and region.

The Climate Agent offers real-time weather updates, climate trend analysis, and predictive insights to help farmers make informed decisions about sowing, irrigation, and harvesting. Our initial approach used third-party APIs for weather data, but we encountered inconsistencies. By incorporating multiple sources and fine-tuning predictive models, we enhanced the accuracy of climate forecasts.

To further improve localisation, we trained the Climate Agent using in-house climate data from recent years, specifically focused on Indian agricultural regions. This data-driven approach allows KshemaGPT to offer highly localised weather-based recommendations. Farmers receive tailored insights on monsoon patterns, drought predictions, and optimal sowing windows based on actual historical trends, making it far more reliable than generic weather services.

India’s linguistic diversity demands local language support. KshemaGPT uses:

IndicTrans2 by AI4Bharat

Supports English, Hindi, Telugu, and Tamil

Delivers farmer-friendly responses in native languages

Typing isn’t always practical for farmers, our main category of consumers. Many aren’t familiar with keyboards or comfortable expressing themselves in written form. Speaking, on the other hand, comes naturally. To support this, we implemented a speech-to-text pipeline using the open-source whisper-large-v3 model from OpenAI.

Typing isn’t practical for many farmers. KshemaGPT uses:

Whisper-large-v3 by OpenAI

Detects spoken language

Transcribes speech to English

Responds in the user’s preferred language

This voice-first model makes KshemaGPT intuitive and accessible.

Building LLM-based applications comes with its own set of challenges—complex control flows, non-deterministic outputs, and diverse user intent. Debugging and evaluating these systems, especially at scale, can be extremely difficult without proper tools.

We use Langfuse to bring observability into the heart of KshemaGPT. Langfuse allows us to:

  • Trace and debug complex chains of model calls and tool invocations
  • Centrally manage and version prompts, allowing quick experimentation and tuning
  • Log and review user feedback for each interaction, helping us iteratively improve system behavior

Langfuse has become an essential part of our development and evaluation workflow, giving us the confidence to experiment boldly while staying grounded in real-world performance.

Building KshemaGPT has been a process filled with experimentation, iteration, constant learning, and evolution. Here are some key insights that emerged along the way:

  • Importance of routing logic: Our initial flat tool-calling setup didn’t scale. Hierarchical routing with a dedicated Router Agent drastically improved efficiency and accuracy.
  • Separation or sorting: We learned that lumping all documents into a single collection was a mistake. Sorting by policy type reduced retrieval errors and improved relevance.
  • Parsing is hard but OCR is harder: Handling PDFs with messy layouts and legibility of scanned pages required us to dig deep into OCR. GOT-OCR-2.0 was the breakthrough.
  • Chunking isn’t just chopping: Chunking strategies must align with the document structure and output format. Chunking LaTeX by section using GOT-OCR-2.0 brought structure and meaning to our retrieval pipeline.
  • Start simple before scaling up: Chroma DB was great for prototyping, but Milvus was  necessary for production. The same goes for embedding and reranking models.
  • Don’t ignore observability: Debugging multi-agent LLMs is hard. Langfuse helped us trace queries, manage prompts, and capture user feedback which played a huge role in iterative improvement.
  • Language and voice are key: Farmers prefer speaking over typing. Whisper’s speech-to-text and IndicTrans2’s multilingual translation let us serve users in their native language.
  • Real modularity enables iteration: Treating each capability as an agent lets us upgrade individual components like swapping parsers or models without breaking the system.

These learnings now inform every decision we make going forward with KshemaGPT.

Conclusion

KshemaGPT represents a meaningful step forward in applying AI to the crop insurance segment. From understanding complex policy documents to providing localised insights on weather and crops, every component of the system has been purpose-built to address the real needs of Indian farmers.
By leveraging open-source models and frameworks, we’ve built an extensible, multilingual, and modular platform that can evolve. The architecture has been carefully designed to ensure that each part——is independently robust yet works seamlessly as part of a larger whole.
What started as a simple experiment has matured into a full-scale production system. While there’s still a long way to go, the progress we have made so far affirms the potential of AI in transforming rural livelihoods. With continued learning, iteration, and collaboration, we hope to further bridge the gap between technological advancement and on-ground impact.
KshemaGPT is not the destination—it’s a solid foundation for what comes next.

Frequently Asked Questions About KshemaGPT

1. What is KshemaGPT and how does it help farmers?
KshemaGPT is an AI assistant that helps Indian farmers manage crop insurance, get weather updates, and receive farming advice in their local language using voice or text.
Yes. KshemaGPT supports Hindi, Telugu, Tamil, and English using IndicTrans2 for accurate translation.
Farmers can speak in their native language. Whisper transcribes it, and KshemaGPT responds in the same language.

Disclaimer:

“We do not assume any liability for any actions undertaken based on the information provided here. The information gathered from various sources and are displayed here for general guidance and does not constitute any professional advice or warranty of any kind.”