How Kshema Built Its Multi-Agent GPT System-Part 2

User Data agent

The User Data Agent plays a crucial role in personalising the KshemaGPT experience. It connects directly to user transaction databases, enabling the system to access detailed user profiles, policy purchases, premium payment history, and claims settlement status.

It helps with support queries such as checking the status of an insurance claim or submitting a grievance which are logged in a separate database. The status of any of these previously raised issues can be asked later as well.

This agent is itself a mini multi-agent system. One sub-agent is responsible for retrieving user details, while another handles query registration and tracking. Both agents interact with the databases via the Model Context Protocol (MCP), which allows LLMs to operate on external data sources securely and efficiently.

To translate natural language queries into actionable database requests, we use a custom Natural Language-to-SQL (NL2SQL) model. This enables users to interact with complex backend systems using simple conversational language, ensuring seamless access to important personal data and support services.

Crop agent

When farmers have crop-related queries, they get routed to the Crop Agent. Initially, we explored several general-purpose agronomy models, but they lacked the specificity needed for region-specific crops and local farming practices.

We then integrated Dhenu2-In-Llama3.1-8B-Instruct, a specialised model trained by KissanAI on extensive crop data. This dramatically improved the relevance and depth of our responses enabling questions about disease management, pest control, and best practices, receiving tailored advice based on their specific crops and region.

Climate agent

The Climate Agent offers real-time weather updates, climate trend analysis, and predictive insights to help farmers make informed decisions about sowing, irrigation, and harvesting. Our initial approach used third-party APIs for weather data, but we encountered inconsistencies. By incorporating multiple sources and fine-tuning predictive models, we enhanced the accuracy of climate forecasts.

To further improve localisation, we trained the Climate Agent using in-house climate data from recent years, specifically focused on Indian agricultural regions. This data-driven approach allows KshemaGPT to offer highly localised weather-based recommendations. Farmers receive tailored insights on monsoon patterns, drought predictions, and optimal sowing windows based on actual historical trends, making it far more reliable than generic weather services.

Translation

Providing answers in local languages is crucial, given the linguistic diversity of India. Currently, KshemaGPT supports English, Hindi, Telugu, and Tamil. We wanted to keep the entire system open-source, so we chose IndicTrans2 from AI4Bharat for translation. It performs well across supported languages and helps us deliver accessible, farmer-friendly responses.

Speech-to-text

Typing isn’t always practical for farmers, our main category of consumers. Many aren’t familiar with keyboards or comfortable expressing themselves in written form. Speaking, on the other hand, comes naturally. To support this, we implemented a speech-to-text pipeline using the open-source whisper-large-v3 model from OpenAI.

Whisper automatically detects the spoken language and transcribes it into English text. This feature allows farmers to ask their questions in any supported language without worrying about typing or switching keyboards. The transcription is then processed by KshemaGPT, which generates the appropriate response in the user’s preferred language. This voice-first interaction model brings us one step closer to creating an intuitive, accessible assistant for every farmer.

Tracing, Observability and Prompt Management

Building LLM-based applications comes with its own set of challenges—complex control flows, non-deterministic outputs, and diverse user intent. Debugging and evaluating these systems, especially at scale, can be extremely difficult without proper tools.

We use Langfuse to bring observability into the heart of KshemaGPT. Langfuse allows us to:

• Trace and debug complex chains of model calls and tool invocations

• Centrally manage and version prompts, allowing quick experimentation and tuning

• Log and review user feedback for each interaction, helping us iteratively improve system behavior

Langfuse has become an essential part of our development and evaluation workflow, giving us the confidence to experiment boldly while staying grounded in real-world performance.

Learnings

Building KshemaGPT has been a process filled with experimentation, iteration, constant learning, and evolution. Here are some key insights that emerged along the way:

• Importance of routing logic: Our initial flat tool-calling setup didn’t scale. Hierarchical routing with a dedicated Router Agent drastically improved efficiency and accuracy.

• Separation or sorting: We learned that lumping all documents into a single collection was a mistake. Sorting by policy type reduced retrieval errors and improved relevance.

• Parsing is hard but OCR is harder: Handling PDFs with messy layouts and legibility of scanned pages required us to dig deep into OCR. GOT-OCR-2.0 was the breakthrough.

• Chunking isn’t just chopping: Chunking strategies must align with the document structure and output format. Chunking LaTeX by section using GOT-OCR-2.0 brought structure and meaning to our retrieval pipeline.

• Start simple before scaling up: Chroma DB was great for prototyping, but Milvus was  necessary for production. The same goes for embedding and reranking models.

• Don’t ignore observability: Debugging multi-agent LLMs is hard. Langfuse helped us trace queries, manage prompts, and capture user feedback which played a huge role in iterative improvement.

• Language and voice are key: Farmers prefer speaking over typing. Whisper’s speech-to-text and IndicTrans2’s multilingual translation let us serve users in their native language.

• Real modularity enables iteration: Treating each capability as an agent lets us upgrade individual components like swapping parsers or models without breaking the system.

These learnings now inform every decision we make going forward with KshemaGPT.

Conclusion

KshemaGPT represents a meaningful step forward in applying AI to the crop insurance segment. From understanding complex policy documents to providing localised insights on weather and crops, every component of the system has been purpose-built to address the real needs of Indian farmers.

By leveraging open-source models and frameworks, we’ve built an extensible, multilingual, and modular platform that can evolve. The architecture has been carefully designed to ensure that each part——is independently robust yet works seamlessly as part of a larger whole.

What started as a simple experiment has matured into a full-scale production system. While there’s still a long way to go, the progress we have made so far affirms the potential of AI in transforming rural livelihoods. With continued learning, iteration, and collaboration, we hope to further bridge the gap between technological advancement and on-ground impact.

KshemaGPT is not the destination—it’s a solid foundation for what comes next.