I train language models from scratch on consumer hardware and build ▍. From 60M-parameter architecture ablations and 1 Mbp-context DNA models to safety layers for parallel coding agents and real-time computer vision in production.
Faculty-advised team project. Trained twenty-four ~60M-parameter GPT-style models on 1B FineWeb-Edu tokens (250+ GPU hours) under a single-axis protocol comparing MHA/MQA/GQA/MLA, RoPE, MoE, and Muon vs. AdamW — isolating why mechanisms designed for 7B–70B models break when compressed.
Faculty-advised team project. Linear-attention (KDA) models scale to long DNA sequences but lose exact recall — the same finite-state bound found in the 60M study. Mnemos adds a k-mer fast-weight memory to restore it.
Detects incidents, links them to recent deploys, recalls similar past incidents via Qdrant + Neo4j memory, and proposes fixes like rollbacks and restarts — with human approval required for anything destructive. Ships a terminal UI, web dashboard, and failure-injection demo; runs on Docker Compose or Kubernetes.
Memory-efficient PyTorch pipeline (gradient checkpointing, chunked cross-entropy, hybrid Muon + AdamW) fusing MLA and MoE (~1.5B total / ~400M active) with Multi-Token Prediction. Trains a 1.5B model on an 8GB RTX 5060 at 1,940 tok/s; full run on a $10 compute budget.
RAG API that answers questions over textbook- and legal-scale PDFs on the fly, without pre-computed embeddings. Combines AWS Titan and Gemini with token budgeting to keep latency low. Containerized and deployed on AWS App Runner with CI/CD through GitHub Actions.
Autonomous WhatsApp agent built on Google ADK and the Model Context Protocol: a Go messaging bridge plus a Python MCP server exposing tools for multi-step tasks.
Results from the 60M-parameter study above. Every model is identical except for one change (attention type, optimizer, etc.), so any difference in score comes from that change alone. Scores are mostly perplexity (PPL) — how surprised the model is by unseen text; lower is better. Pick a tab to see one experiment.
| AI / ML | PyTorch, Hugging Face, LLM training & fine-tuning, RAG, AI agents (LangGraph, MCP, Google ADK), MoE, MLA, RoPE, Muon, YOLOv8 |
| Backend & Cloud | FastAPI, Django, REST APIs, Docker, Kubernetes, AWS (App Runner, ECR, EC2), GitHub Actions |
| Data | PostgreSQL, SQLite, Qdrant, Neo4j, Pinecone, ChromaDB, PyArrow |
| Languages | Python, Go, SQL, JavaScript / TypeScript |
| Honors |
• Top 35 Finalist — Bajaj Finserv HackRx 6.0 (10,000+ teams) • 3-Star on CodeChef • NVIDIA DLI: Fundamentals of Deep Learning |