LLM systems
LLM-powered internal tooling using Claude API and LangGraph — RAG over clinical documentation, multi-step agent workflows for automated reasoning by clinical and engineering teams.
SOFTWARE & AI ENGINEER
UMANGGARG.
ABOUT me
I BUILDAI SYSTEMSthat ship.
BUILD
Production systems end-to-end
LEARN
AI fundamentals from first principles
SHIP
Product, infra, and model pipelines
Software engineer with 8 years shipping production systems across healthcare AI, biotech, and enterprise. My work spans backend APIs, React frontends, LLM-integrated pipelines, and the cloud infrastructure underneath — taking complex systems end-to-end.
Most recently at PictorLabs.ai — a UCLA spinoff in digital pathology — fine-tuning Google's PathFoundation vision model for automated stain-type detection, building LLM tooling with Claude API and LangGraph, and leading delivery of ClearStain, an AI virtual H&E staining pipeline for whole slide images.
I'm at my best where AI engineering meets real production engineering — training pipelines, retrieval systems, API design, and the infrastructure that holds it all up. Engineering fundamentals first, frameworks second.
Education
MS Software Engineering
Rochester Institute of TechnologyNew York, USAGPA3.94
B.Tech Mechanical Engineering
ITM UniversityGurgaon, IndiaCareer
2025AI & FULL STACK ENGINEER
PictorLabs.ai — AI Digital Pathology (UCLA Spinoff)
AI & FULL STACK ENGINEER
PictorLabs.ai — AI Digital Pathology (UCLA Spinoff)
Core stack
- Claude API
- LangGraph
- PyTorch
- Kafka
- React
- Django
- AWS ECS/EKS
Led delivery of ClearStain — AI virtual staining pipeline for whole slide images, a major new undertaking integrated end-to-end into the existing VSH (Virtual Slide Hub) system. Owned Django API routing (organ / species / diagnosis-based predictor selection) and integrated ClearStain jobs with the platform's Kafka event flow and TorchServe model serving.
Model training
Within ClearStain, fine-tuned Google's PathFoundation vision model in PyTorch for automated stain-type detection (input validation before staining) — built full training pipeline including OpenSlide WSI patch extraction, tissue masking, and Macenko stain normalization.
Upload infrastructure
Built adaptive chunked multipart upload (React Web Workers + S3) with concurrent worker pool, per-chunk exponential backoff, and recovery from the last completed chunk after a network drop, for multi-GB slides on a multi-tenant SaaS.
2024SOFTWARE ENGINEER 2
Medhaz Corp — Workforce Platform
SOFTWARE ENGINEER 2
Medhaz Corp — Workforce Platform
Core stack
- Django REST
- React 18
- Python
Workforce platform
Built candidate data management portal using Python/Django REST Framework and React.js, processing structured workforce data across organizations.
React modernization
Led React 15→18 migration, optimizing performance via Concurrent Rendering and React Suspense.
2022SOFTWARE ENGINEER 2
Asurion — Insurance Tech
SOFTWARE ENGINEER 2
Asurion — Insurance Tech
Core stack
- Node.js
- React
- AWS Lambda
- SQS
- Micro-frontend
Entitlement Engine
Lead dev of the Entitlement Engine — insurance claim eligibility API serving 40,000+ users weekly at 99% uptime via optimized Node.js architecture.
Independent releases
Monolith→micro-frontend migration enabling independently deployable release cycles and faster iteration across teams.
Event-driven pipelines
Designed and maintained serverless event-driven Python data pipelines on AWS Lambda + API Gateway + S3 + SQS + CloudWatch — automated downstream workflows at scale.
2021REACT DEVELOPER
HYR Global Source — Client: The Kroger Co.
REACT DEVELOPER
HYR Global Source — Client: The Kroger Co.
Core stack
- React ES6
- Redux
- Jest
Frontend architecture
Built cross-functional frontend architecture and reusable React components deployed across multiple Kroger entities in an enterprise agile environment.
Testing & quality
Built test suites with Jest and React Testing Library, increasing coverage by 50%.
2019SOFTWARE DEVELOPER
GTR Event Technology
SOFTWARE DEVELOPER
GTR Event Technology
Core stack
- Node.js
- MongoDB
- OpenCV
- GraphQL
Event platforms
Built event registration and CEU platforms with automated CI/CD pipelines using React, Node.js, Express, MongoDB, and RESTful APIs.
Computer vision
Built a computer-vision system in Python + OpenCV for booth-layout detection from event floor plans — eliminated 50+ hours of manual work per event.
Selected work
ClearStain
Computer visionClosed-source · proprietary
Stack
- PyTorch
- Django
- Kafka
- TorchServe
- OpenSlide
- AWS ECS
AI virtual staining pipeline for whole slide images at PictorLabs.ai (UCLA spinoff, venture-backed).
Unstained brightfield to virtual H&E
Every stage is an event on Kafka, with prism-api as the hub. influx detects stain type; prism-api derives the image type, applies blur QC and routing rules, and schedules inference only for unstained brightfield. H&E goes to ReStain, non-RGB to DeepStain, and other stains are rejected. nexus skips blank crops, runs tissue patches through TorchServe, and writes the virtual H&E for the viewer.
Unstained BFVirtual H&EReStainDeepStainRejected
Led delivery end to end
Led ClearStain's integration into the existing Virtual Slide Hub and merged the epic across the backend and the frontend.
Stain-type detector
Fine-tuned a classifier on Google's PathFoundation: a frozen encoder with a trained head, fed by OpenSlide patch extraction, tissue masking, and Macenko stain normalization.
Fixed a silent failure
Colour thresholds had labelled other stains as unstained with full confidence, the one error that silently corrupts a result. I narrowed the detector to the two classes that gate staining, unstained and H&E, and validated it on a labelled slide set.
One field routes everything
The detected stain type gates all auto-staining. Image type is derived from channels and stain type on every read, never stored, so it cannot drift.
Clinical routing rules
Owned the Django routing: AutoStainerRules choose a predictor by organ, species, and diagnosis from a master data model, so science changes need no deploy.
Kafka integration
Wired ClearStain jobs into the platform's event flow, where each stage passes presigned URLs instead of image data.
Model serving
Integrated TorchServe serving for the new predictor. The inference engine skips blank crops before they reach the GPU.
Multi-GB uploads
Built chunked multipart upload to S3 with React Web Workers, a concurrent pool, and exponential backoff on failed parts.
Detect
influx validates the file and detects its stain type: unstained, H&E, or another stain. That result, sent to prism-api as a Kafka event, gates all auto-staining.
Cartographer
Codebase reasoningStack
- FastAPI
- React/Vite
- Qdrant Cloud
- tree-sitter
- FastMCP
- Voyage
- Cohere
- Docker
Codebase reasoning engine that turns any GitHub repository into an interactive, navigable knowledge surface — natural-language Q&A with source citations, on-demand Mermaid diagrams (architecture / class hierarchy / call graph), and three-phase agent-generated concept tours.
Repository to grounded answer
Indexing stores the repository before queries arrive. At query time, retrieve from the saved index, rerank candidates, then compose an answer. Agent mode may retrieve again while reasoning, before its final answer.
Agent mode onlySaved indexQuestionCited answer
Built from scratch
Every retrieval layer is implemented directly, with no LangChain or LlamaIndex.
AST chunks with context
tree-sitter splits code by structure, and each chunk gets an LLM-written description before embedding.
Hybrid search
Voyage code embeddings and BM25 in Qdrant, fused by RRF. HyDE or query expansion broadens recall.
Reranking
A Cohere cross-encoder with a local fallback reorders candidates; parent-class expansion restores surrounding code.
Agent mode
A ReAct loop over 12 MCP tools, including caller tracing and a notes memory. Tool calls run in parallel, and every step streams to the user live.
Diagrams on demand
Mermaid architecture, class hierarchy, and call-graph diagrams generated from the indexed code.
Concept tours
A multi-phase agent reads the README, maps the architecture, investigates in parallel, then writes a guided tour.
Graded answers
Technical answers are graded against their cited sources, and generation falls back across several LLM providers.
Index
At ingestion, AST chunks and optional contextual descriptions are embedded into Qdrant. Queries reuse this index.
A self-hosted autonomous agent, built from scratch — no agent framework, wired to the model over raw HTTP.
One agent, gated before and after every tool
Chat channels and the heartbeat start turns on one agent. Each tool call passes a permission policy that allows, refuses, or repairs it; tools run in a separate process. The reply is checked against the tool results, and a fabricated link is blocked so the agent retries. The run is recorded, and skill edits proposed from it apply only after approval, feeding back into the agent.
ChatHeartbeatTools, own processBlocked, retryApproved skills
No agent framework
Raw HTTP to DeepSeek V4 Flash via OpenRouter, behind a provider fallback chain, with a per-tick iteration budget and a daily cost ceiling.
One conversation everywhere
The same agent and history across the web console, Telegram, and Discord; a heartbeat runs each due task in its own isolated run.
Permission policy
Every tool call is allowed, refused with a reason the model reads, or run on repaired arguments. Identity arguments are pinned on unattended runs.
Guards on claims
Replies and deliveries are checked against what the tools returned: work with no evidence behind it, or a link no tool produced, is refused.
Tool suite over MCP
Tools run in a separate process over MCP, namespaced in every trace. Untrusted web content is wrapped before it reaches the model.
Human-gated skills
Reflection proposes skill edits from execution traces; none applies until approved on the web, Telegram, or Discord. Skills are versioned with rollback.
Measured self-improvement
Each approved edit is compared with the version before it on contract compliance, guard fires, and cost. The verdict is computed, never model-graded.
Audit and safety
Every tool call, cost, and guard decision is logged; outages are kept out of reflection, secrets are redacted, and a prompt-leak canary guards the loop.
Trigger
Chat on the web console, Telegram, or Discord and the heartbeat's due tasks all start turns on one agent: one conversation across channels, and each scheduled task in its own isolated run.
Turn a PDF into a guided lesson: a learning plan you approve, source-grounded questions, and a tutor alongside each exercise.
Document to guided learning
Extract the PDF, draft a plan and wait for learner approval, then run the quiz loop. Each question passes structural and faithfulness checks before use. Incorrect answers receive hints and retry the same question; correct answers advance to the next objective. Results appear only after the last objective.
ApprovedAll completeRetry / next question
Page-linked extraction
The PDF becomes per-page text, low-text pages are flagged, and chunks keep their page numbers.
Learner-approved plan
A LangGraph interrupt pauses after drafting objectives until the learner approves or edits them.
Two-gate question generation
Each question passes structural validation, then an LLM faithfulness judge. Failures are regenerated, never served.
Grounded questions
Every question cites the page and chunk it was written from.
Deterministic lesson engine
A state machine owns grading, hints, retries, and advancing. Answer keys stay on the server; lessons persist in PostgreSQL, with an in-memory fallback locally.
Guarded tutor
An in-lesson tutor gives hints and rejects any that contain the answer text. It is a literal check, not a guarantee against paraphrased leaks.
Learning report
First-try accuracy and topics to revisit, based on attempts and hints per objective.
Gate observability
A dashboard counts every rejected question and keeps each catch for inspection.
Source
A PDF becomes page-linked text. Its contents ground the learning objectives and generated questions.
Further explorations
More on GitHubSEED SCOUT
Research automation · 2026
A topic-to-memo research pipeline for seed-stage AI infrastructure and developer tooling. Discovers candidates, gathers bounded evidence, and produces ranked company memos. Model-generated assessments pass citation-ID checks; code computes weighted scores and confidence-aware recommendations. Stages are cached for replay, with input-based invalidation and recovery from incomplete runs. Citation checks verify source references, not whether every claim is true.
- TypeScript
- OpenRouter
- Zod
- Vitest
- Replayable Pipelines
TRANSFORMER FROM SCRATCH
Deep Learning · PyTorch · 2025
Full encoder-decoder transformer implemented in PyTorch — multi-head attention, positional encoding, layer norm, greedy decoding, label smoothing, LR warmup. Trained EN→ES on the Opus Books dataset, end-to-end from "Attention Is All You Need".
- PyTorch
- Attention
- Transformers
- NLP
RL EXPERIMENTS
Reinforcement Learning · 2025
Q-learning, DQN (Breakout), and PPO (LunarLander) trained on Gymnasium with Stable Baselines 3 — plus a custom multi-agent dual-taxi environment built from scratch (observation space, reward shaping, training loop). Domain breadth: not just LLMs.
- PyTorch
- Gymnasium
- SB3
- DQN · PPO
Tech stack
AI / LLMs
Intelligence
Retrieval
- RAG
- HyDE
- Hybrid Search
- Reranking
- Qdrant
Agents & orchestration
- ReAct
- Multi-agent
- Tool Use
- MCP
- LangChain
- LangGraph
- CrewAI
- OpenAI Agents SDK
- AutoGen
Evaluation & delivery
- Claude API
- Prompt Engineering
- Guardrails
- LLMOps
- Langfuse
- Evals
Applied ML
Models & vision
Training
- PyTorch
- scikit-learn
- Fine-tuning
- LoRA / QLoRA
- PEFT
Working with data
- OpenCV
- OpenSlide
- tree-sitter
Serving & exploration
- TorchServe
- HuggingFace
- Gradio
Full stack
Product
Interfaces
- React.js
- Redux
Services
- Django
- FastAPI
- Node.js
- Express
Contracts & data
- GraphQL
- Python
- TypeScript
- SQL
Cloud & systems
Infrastructure
Compute & deployment
- AWS S3 / ECS / EKS
- Docker
- CI/CD
- Vercel
Messaging
- Lambda + SQS
- Apache Kafka
Persistence
- PostgreSQL
- MongoDB
Engineering practices
- Event-driven architecture
- Microservices
- Micro-frontends
- REST API design
- Agile
Languages
- Python
- TypeScript
- JavaScript
- SQL
- Java
- Bash