Senior Forward Deployed Engineer.
GenAI Architect.

Building Scalable and Cost-Effective Voice Agents: A Platform-Based Blueprint

CVS Health Tech Blog

How to think about voice agents as a platform, separating telephony, LLM, TTS, and orchestration so a fleet of agents can ship without each one re-inventing the runtime.

TrueMem: Building a Model-Agnostic Memory Layer for AI

TrueFoundry engineering blog · byline

How I designed TrueMem, a two-tier (STM + LTM) persistent memory service for LLM applications, and why moving memory outside the model gives you vendor-portability for free.

Transforming Customer Interactions: Evolving IVR Systems for Enhanced Experiences

CVS Health Tech Blog

The migration path from rigid IVR menus to LLM-driven conversational flows, keeping integrations, observability, and compliance posture intact.

My Adventures with LLMs

Leanpub · book

A practitioner's walk-through of building LLMs from scratch in PyTorch, Transformers through DeepSeek, with a companion code repo for every chapter.

Jun 30, 2026

DPO and Agentic RL: Align Without a Reward Model, Then Step Into the World

Two moves close the series. First we simplify: Direct Preference Optimization reaches the same aligned model as RLHF while firing both the reward model and the RL loop, and SimPO and KTO trim it further. Then we expand: point the same group-relative gradient at a multi-step world and the model stops answering and starts acting, calling tools, reading observations, and earning a reward only at the end. You will see the DPO loss derived from the RLHF objective, every number of a worked preference, and a runnable multi-turn agent that learns to use a tool from reward alone.

Jun 23, 2026

GRPO: Teaching a Model to Reason by Comparing It to Itself

RLHF aligned a model with two heavy passengers: a learned reward model and a value-head critic. This post throws both away. You will see why the critic was only ever a baseline and how a group of sampled answers replaces it for free, why a verifiable check beats a hackable reward model for math and code, how GRPO wraps that group-relative advantage in PPO's clip, and how this exact recipe produced DeepSeek-R1, all grounded in a hands-on run that trains Llama-3.2-3B to reason on grade-school math.

Jun 16, 2026

RLHF: Teaching a Language Model What "Good" Means

A language model only ever predicts the next token. This post earns the sentence "PPO aligned ChatGPT": how generating text becomes an MDP, how human comparisons become a Bradley-Terry reward model, how a value head turns one end-of-answer score into a per-token signal, and how the KL leash keeps PPO from gaming a flawed reward, grounded in a hands-on build that aligns a tiny GPT-2 assistant.

Jun 9, 2026

TRPO and PPO: The Largest Safe Step a Policy Can Take

The gradient gives a direction, but who chooses the step size? This post explores the step-size crisis in RL, builds importance sampling from scratch, derives the surrogate objective, fences it with TRPO's KL constraint, then clips it with PPO, all grounded in a continuous-control example where PPO learns to swing a pendulum upright.

CARL: Cost-Optimized Online Container Placement on VMs Using Adversarial Reinforcement Learning

IEEE Transactions on Cloud Computing · 2025

Adversarial reinforcement learning formulation for online container-to-VM placement, framed as a multi-dimensional vector-bin-packing problem. A learner agent mimics an offline semi-optimal teacher solver while automatically learning a reward function for VM cost reduction, ending up out-performing the teacher it imitates. Evaluated on Google and Alibaba production cluster traces (5k–10k container requests across 2k–8k VMs): ~16% lower VM cost than classic heuristics and SOTA RL methods, ~1,900 placements per second onto ~8,900 candidate VMs, robust to inference-time workload distribution shift.