arXiv Science⌕ Search

arXiv subjects

Tarun Jain

Publications and source records attributed to Tarun Jain.

3 recordsLinked to original sources

Orbit AI: Evaluating Tool-Augmented Agents For Enterprise Question Answering

Enterprise information is commonly stored across multiple platforms rather than in one centralized location. Answering questions about workplace activities may require information from several sources, including project documents, task management systems, and team conversations. Therefore, we evaluate Orbit AI, a tool-based retrieval-augmented generation (RAG) system that uses a ReAct agent implemented with LangGraph to retrieve and combine information from three different workspace sources. We compare three configurations: (1) a standalone LLM without retrieval, (2) document-based RAG, and (3) a multi-tool agent with access to all three sources. The held-out evaluation set contains 80 questions designed for Phase 3 workspace; Gemini 3.5 Flash Lite achieves an accuracy of 86.25% with the complete agent. Document-only RAG achieves 60.00% (26.25 pp higher), and the plain LLM achieves 23.13%. For multi-source questions, the full agent answers 84.38%, while document-only RAG answers 40.63%, a 43.75 pp increase. The evaluated Llama 3.2 configuration with local inference achieves 5.50% overall correctness under rubric v2.2. The Llama 3.2 full-agent configuration achieves a held-out accuracy of 52.50%, which is 1.88 pp higher than the accuracy of document-only RAG. The gain of Gemini is equivalent to 26.25 pp. The comparison must be taken with a pinch of salt, because Gemini was accessed via an API hosted by them, while Llama was run locally via Ollama and the models are of different size. It is therefore not possible to isolate the cause of the performance gaps in this experiment. The paper also addresses the constraints of the evaluation such as the small artificial workspace and the similarity of the Q53 problem to a problem in the system prompt.

cs.CR↗

News Reporter: A Multi-lingual LLM Framework for Broadcast T.V News

Large Language Models (LLMs) have fast become an essential tools to many conversational chatbots due to their ability to provide coherent answers for varied queries. Datasets used to train these LLMs are often a mix of generic and synthetic samples, thus lacking the verification needed to provide correct and verifiable answers for T.V. News. We collect and share a large collection of QA pairs extracted from transcripts of news recordings from various news-channels across the United States. Resultant QA pairs are then used to fine-tune an off-the-shelf LLM model. Our model surpasses base models of similar size on several open LLM benchmarks. We further integrate and propose a RAG method to improve contextualization of our answers and also point it to a verifiable news recording.

cs.CL↗

Accelerated infection testing at scale: a proposal for inference with single test on multiple patients

In pandemics or epidemics, public health authorities need to rapidly test a large number of individuals, both to determine the line of treatment as well as to know the spread of infection to plan containment, mitigation and future responses. However, the lack of adequate testing kits could be a bottleneck, especially in the case of unanticipated new diseases, such as COVID-19, where the testing technology, manufacturing capability, distribution, human skills and laboratories might be unavailable or in short supply. In addition, the cost of the standard PCR test is approximately USD 48, which is prohibitive for poorer patients and most governments. We address this bottleneck by proposing a test methodology that pools the sample from two (or more) patients in a single test. The key insight is that a single negative result from a pooled sample likely implies negative infection of all the individual patients. and It thereby rules out further tests for the patients. This protocol, therefore, requires significantly fewer tests. This may, however, result in somewhat increased false negatives. Our simulations show that combining samples from two patients with 7% underlying likelihood of infection implies that 36% fewer test kits are required, with 14% additional units of time for testing.

q-bio.OT↗