Work
AI systems, open-source integrations, courses, and earlier projects.
I have worked on internal coding agents, production RAG, LLM evaluation, open-source ML integrations, and technical education.
Current work
Agent engineering
Leading a team building an internal coding agent, harness benchmarks, memory, and methods for reusing successful prior runs.AgentsEvaluationLeadership
Leading a team building an internal coding agent, harness benchmarks, memory, and methods for reusing successful prior runs.AgentsEvaluationLeadership
NVIDIA NeMo Agent Toolkit × W&B Weave
Added fine-grained tracing and evaluation support for workflows built with NVIDIA NeMo Agent Toolkit.Technical report · Tracing PR · Evaluation PRAgentsObservability
Added fine-grained tracing and evaluation support for workflows built with NVIDIA NeMo Agent Toolkit.Technical report · Tracing PR · Evaluation PRAgentsObservability
LLM systems and evaluation
Wandbot: an evaluation-driven refactor
Reworked W&B’s open-source documentation assistant around systematic evaluation. Answer accuracy rose from 72% to 81%, latency fell by 84%, and a full evaluation run went from roughly two hours to under ten minutes. Nearly 50 experiments made the trade-offs between quality, speed, and cost visible.Read the case study · SourceRAGEvaluationProduction
Reworked W&B’s open-source documentation assistant around systematic evaluation. Answer accuracy rose from 72% to 81%, latency fell by 84%, and a full evaluation run went from roughly two hours to under ten minutes. Nearly 50 experiments made the trade-offs between quality, speed, and cost visible.Read the case study · SourceRAGEvaluationProduction
Evaluating, comparing, and optimizing LLM systems
Built a workflow for comparing system configurations with human labels, LLM judges, retrieval metrics, and parameter search.Technical reportLLMOpsEvals
Built a workflow for comparing system configurations with human labels, LLM judges, retrieval metrics, and parameter search.Technical reportLLMOpsEvals
LlamaIndex × W&B
Built W&B instrumentation for retrieval and LLM traces, debugging query pipelines, and versioning indexes and embeddings.WalkthroughLlamaIndexTracing
Built W&B instrumentation for retrieval and LLM traces, debugging query pipelines, and versioning indexes and embeddings.WalkthroughLlamaIndexTracing
Tools and integrations
My public GitHub history includes 200 merged pull requests. The groups below cover the largest integration projects.
A new generation of W&B callbacks for Keras
Added dedicated callbacks for metrics logging, model checkpointing, and rich model-prediction tables, alongside fixes for SavedModel and distributed-training workflows.Metrics logger · Model checkpointing · Evaluation tables · Distributed trainingKerasTensorFlow
Added dedicated callbacks for metrics logging, model checkpointing, and rich model-prediction tables, alongside fixes for SavedModel and distributed-training workflows.Metrics logger · Model checkpointing · Evaluation tables · Distributed trainingKerasTensorFlow
Native experiment-tracking hooks across the OpenMMLab stack
Built dedicated W&B integrations for object detection, image classification, and semantic segmentation, then followed them with maintenance fixes and runnable examples.MMDetection · MMClassification · MMSegmentationComputer visionOpenMMLab
Built dedicated W&B integrations for object detection, image classification, and semantic segmentation, then followed them with maintenance fixes and runnable examples.MMDetection · MMClassification · MMSegmentationComputer visionOpenMMLab
Instrumentation and evaluation for emerging LLM stacks
Added W&B instrumentation to LlamaIndex and integration support to EleutherAI’s LM Evaluation Harness; updated OpenAI fine-tuning, Cohere v2, and Wandbot evaluation workflows as their underlying APIs evolved.LlamaIndex · LM Evaluation Harness · OpenAI · Cohere · WandbotLLMOpsEvaluation
Added W&B instrumentation to LlamaIndex and integration support to EleutherAI’s LM Evaluation Harness; updated OpenAI fine-tuning, Cohere v2, and Wandbot evaluation workflows as their underlying APIs evolved.LlamaIndex · LM Evaluation Harness · OpenAI · Cohere · WandbotLLMOpsEvaluation
Tracing and evaluation across agent frameworks
Extended support for DSPy, MCP, LlamaIndex, CrewAI, Verifiers, and NVIDIA NeMo Agent Toolkit, including fine-grained tracing and evaluation inside external framework repositories.NeMo tracing · NeMo evaluation · MCP · DSPy · Verifiers · CrewAIAgentsTracingEvals
Extended support for DSPy, MCP, LlamaIndex, CrewAI, Verifiers, and NVIDIA NeMo Agent Toolkit, including fine-grained tracing and evaluation inside external framework repositories.NeMo tracing · NeMo evaluation · MCP · DSPy · Verifiers · CrewAIAgentsTracingEvals
I also helped build W&B’s Kaggle Ambassador program and became a Kaggle Notebooks Master. My notebooks span object detection, medical imaging, reproducibility, and interactive visualization.
Research publication
Feature extraction and classification of phonocardiograms using convolutional neural networks
With Devjyoti Chakraborty, Snehangshu Bhattacharya, Aritra Roy Gosthipaty, and Chira Datta. We studied an automated approach to classifying normal and abnormal heart sounds: transforming phonocardiogram recordings into spectrograms, addressing class imbalance, and using deep convolutional neural networks for feature extraction and classification.Read on IEEE Xplore · Google Scholar profileMedical MLSignal processingCNN
With Devjyoti Chakraborty, Snehangshu Bhattacharya, Aritra Roy Gosthipaty, and Chira Datta. We studied an automated approach to classifying normal and abnormal heart sounds: transforming phonocardiogram recordings into spectrograms, addressing class imbalance, and using deep convolutional neural networks for feature extraction and classification.Read on IEEE Xplore · Google Scholar profileMedical MLSignal processingCNN
Courses
LLM apps: Evaluation
Created with Anish Shah, with case studies from Paige Bailey and Graham Neubig. Covers programmatic evaluators, LLM judges, evaluator alignment, and real-world evaluation workflows.Take the course
Created with Anish Shah, with case studies from Paige Bailey and Graham Neubig. Covers programmatic evaluators, LLM judges, evaluator alignment, and real-world evaluation workflows.Take the course
RAG++: From POC to Production
Created with Bharat Ramanathan, with contributions from Meor Amer and Charles Pierse. Distills nearly two years of production RAG work into evaluation, data ingestion, query enhancement, hybrid retrieval, reranking, agentic RAG, and response synthesis.Take the course
Created with Bharat Ramanathan, with contributions from Meor Amer and Charles Pierse. Distills nearly two years of production RAG work into evaluation, data ingestion, query enhancement, hybrid retrieval, reranking, agentic RAG, and response synthesis.Take the course
Training and Fine-tuning LLMs
Co-taught with Darek Kłeczek and guest instructors Jonathan Frankle, Weiwei Yang, and Mark Saroufim. Covers model foundations, evaluation, data preparation, training, and fine-tuning.Take the course
Co-taught with Darek Kłeczek and guest instructors Jonathan Frankle, Weiwei Yang, and Mark Saroufim. Covers model foundations, evaluation, data preparation, training, and fine-tuning.Take the course
Earlier work
Faceless
Built a multitask face-analysis pipeline with TensorFlow and OpenCV for face detection and attribute prediction.TensorFlowOpenCV
Built a multitask face-analysis pipeline with TensorFlow and OpenCV for face detection and attribute prediction.TensorFlowOpenCV
American Sign Language classifier
Built and documented an end-to-end CNN project for recognizing ASL letters.Read the articleCNNWriting
Built and documented an end-to-end CNN project for recognizing ASL letters.Read the articleCNNWriting
Autonomous robots
Built line-following, maze-solving, object-detection, and grid-navigation robots. One system avoided obstacles, explored a grid, and retraced the shortest route.Embedded systemsNavigation
Built line-following, maze-solving, object-detection, and grid-navigation robots. One system avoided obstacles, explored a grid, and retraced the shortest route.Embedded systemsNavigation