Prajwal Anagani

ML Engineer, Inference

Simplismart · Mar 2026 – Present

• Cut Gemma-4-31B p90 TTFT 381→178 ms and p90 latency 2.9→1.2 s on 8×H100 with custom FA4 hd512 kernels, fused norm+FP8-quant Triton kernels, FP8 GEMMs and MTP speculative decoding
• Built the Blackwell NVFP4 serving path (fused GELU/NVFP4 kernels, CUTLASS SM100 GEMM, FP8 KV): +39% requests/s within SLA vs stock
• Designed KV-cache reuse at scale with Mooncake prefix pinning and pool-aware Dynamo routing: p95 −31%, ~28% fewer workers
• Fixed KV-router prefix affinity in production: cache hit rate 0.34→0.98, TTFT p95 5.5 s→0.23 s

Deep Learning Engineer

RadiusAI · Jun 2024 – Feb 2026

• Built 10–15 production CV models for Fortune 10 clients handling 5K SKUs at 90% precision
• Served millions of images monthly via Triton with sub-1s latency
• Accelerated training 1.5x and integrated VLMs for +3% accuracy

ML Developer

CARAIO Technologies · Jun 2022 – Aug 2024

• Led team building chromosome analysis pipeline: 85% segmentation, 98% classification
• Built prototype to shipped product within 1 year
• Part of founding team at UC Berkeley SkyDeck incubated startup
• End-to-end: metaphase images to karyotype generation

Long-Prompt Serving Stack

Gemma-4-31B, TP8 on H100. Custom FlashAttention-4 head-dim-512 kernels, fused Triton norm+residual+FP8 quant, a 24× faster sampler and compute/all-reduce overlap. p90 TTFT 381→178 ms at 300 RPM.

NVFP4 on Blackwell

Gemma-4-31B on 8×B200: fused GELU+NVFP4 kernels, Triton paged-KV writes (118→26 µs), CUTLASS SM100 GEMM, FP8 KV and a prefill-aware router. 13.9 RPS in SLA vs ~10 stock.

KV-Cache Reuse at Scale

Prefix pinning in Mooncake, a hot-cache tier and pool-aware Dynamo routing for hybrid-attention models. Workload-switch p95 4.9 s→0.8 s; same SLA on ~28% fewer workers.

Mixed-Precision KV Cache

Split KV by head size: FP8 sliding-window layers on FA3, BF16 global layers on FA4. 1.83× KV capacity and −68% TTFT p50 at concurrency 64, throughput unchanged.

1T-Param MoE on 8×B200

Kimi K2.5/K2.6 parallelism study: TP8 beats DP8+EP by 3.4× on single-stream decode; SGLang + EAGLE3 reaches ~340 tok/s per user.

Production Traffic Replay

simplitrace rebuilds production traffic, including exact prefix-cache structure, as replayable benchmarks, backed by a ledger of 100+ experiments that keeps the losses too.

vLLM

vLLM-Omni: TeaCache for DiT inference and CPU offloading for diffusion models (merged). Added FLUX 2 model support.

Ground-up Series

Reproducing foundation model architectures from scratch with clear, runnable notebooks.

Chromosome Classification

Medical CV classification with focus on accuracy under data constraints. Part of CARAIO's diagnostic pipeline.

MWP-T5

Fine-tuned T5 for generating solvable math word problems. Published at IEEE conference.