Adi Banerjee

Adi Banerjee

Applied Scientist
Amazon Web Services

Bio

Adi Banerjee is an Applied Scientist at Amazon Web Services, where he conducts GenAI research for AWS Marketing and Sales. His research focuses on multi-agent error attribution, efficient conversational planning, prompt and topology optimization, augmented retrieval systems, and LLM evaluation frameworks. His work has been published at conferences such as NeurIPS and KDD, addressing challenges in autonomous AI system improvement and robust RAG architectures.

Prior to AWS, Adi was a Senior Data Scientist at McKinsey & Company, developing recommendation engines and conducting competitive analytics for enterprise clients. He holds an MS in Operations Research from Georgia Institute of Technology and a BS in Chemical Engineering from Purdue University, where he also served as a teaching assistant for multiple courses.

Adi’s research synthesizes optimization theory and traditional modeling approaches, such as reinforcement learning, with practical GenAI applications to develop AI systems that are more scalable, reliable, and autonomous for business-critical environments.

Dynamic Automated Research Engine for Scalable AI Systems

Multi-agent AI systems face a critical scalability challenge: each new use-case requires manual architecture design, extensive prompt engineering, and continuous human monitoring to maintain performance. Traditional approaches struggle with two fundamental limitations – they cannot systematically identify which components are underperforming, and they lack mechanisms to autonomously adapt to changing requirements or failure modes.

DARE (Dynamic Agent Research and Evolution) addresses these limitations through a novel integration of proven optimization techniques. The framework combines hierarchical error attribution algorithms to pinpoint failure points in complex agent interactions, genetic algorithms for automated prompt evolution, and consensus-based evaluation mechanisms that eliminate the need for predefined numerical fitness functions. This optimization-driven approach enables agents to continuously self-improve without human intervention.

The system operates through an automated cycle: it performs query clustering to identify distinct task categories, automatically constructs specialized multi-agent architectures for each category, analyzes execution traces to attribute errors to specific agents and reasoning steps, triggers architecture modifications based on attribution results, and optimizes individual agent prompts through debate-driven evolutionary algorithms.

DARE’s methodology applies across diverse analytics domains. For business intelligence, it automates the creation of research agents that synthesize insights from multiple data sources. In predictive analytics, it optimizes agent workflows for feature engineering and model selection. For GenAI applications, it enables autonomous refinement of generation pipelines. In optimization contexts, it dynamically adjusts agent strategies based on solution quality feedback.

By transforming agent development from manual crafting to automated evolution, DARE fundamentally changes the economics of deploying AI systems at scale, enabling organizations to rapidly develop and maintain robust solutions across their entire use-case portfolio.

Essential / Professional