Skip to content
View vineetagarwal54's full-sized avatar

Highlights

  • Pro

Block or report vineetagarwal54

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
vineetagarwal54/README.md



About

I build backend and AI infrastructure systems, and I have been shipping production software for over three years, two of them full time before graduate school.

Most recently I worked on LLM inference: implementing speculative decoding from scratch, benchmarking vLLM, SGLang, and TensorRT-LLM against a 480B parameter model on Blackwell GPUs, and digging into why theoretical throughput gains disappear once a real serving loop is involved. Before that I spent two years on production web and mobile systems where the interesting problems were latency, query cost, and things breaking at 2am.

The through line is the same either way: understand the data path, find where the time actually goes, then build on top of it.

Full-Stack  ──▶  Cloud & Infra  ──▶  LLM Serving  ──▶  Applied AI Systems
     ▲                                                         │
     └──────────── measure it, then make it faster ◀───────────┘

Currently: Enterprise AI and Platform Intern at ServBeyond Solutions, building a retrieval pipeline over internal documentation that is now live for the team. Graduating: December 2026, MEng Computer Software Engineering, University of Maryland College Park, with a Graduate Certificate in Cloud Engineering.


LLM Inference and Serving

Machine Learning and Inference Engineering Intern, Runara, April to May 2026

Focus What I did
Speculative decoding Implemented it from scratch across two GPUs in PyTorch and HuggingFace Transformers, using the probabilistic rejection sampling acceptance criterion from the original paper rather than naive argmax matching. Characterized the draft to target cost ratio under which the technique actually pays for itself.
Acceptance rate debugging Traced a gap between 92 percent isolated draft agreement and 29 percent real in-loop acceptance down to KV cache and verification input handling, and showed argmax draft selection beats temperature sampling for acceptance.
Framework benchmarking Compared vLLM, SGLang, and TensorRT-LLM serving Qwen3 Coder 480B on RTX PRO 6000 Blackwell GPUs, profiling time to first token, tokens per second, VRAM footprint, KV cache growth, and multi-user scaling.
Quantization review Caught that the official FP8 checkpoint did not satisfy a 4-bit requirement and escalated it before GPU hours were spent on the wrong artifact.

PyTorch HuggingFace vLLM SGLang TensorRT-LLM KV cache FP8 / INT4 / AWQ RunPod Blackwell


Selected Projects

Privacy-first mobile AI assistant that performs text and image Q&A entirely on-device, without sending user content to a server.

Built with React Native and llama.rn, with local model downloads and verification, streaming responses, persistent conversations, cancellation, and device-aware failure handling.

React Native TypeScript llama.rn Qwen3-VL On-device AI

Desktop application for tracking job applications, built with React and Electron. One click capture from any job board through a browser bookmarklet, deadline tracking, drag and drop reordering, and JSON export and import.

Built it because spreadsheets kept losing my deadlines, and it has become the system I actually run my applications through.

React Electron JavaScript Local-first

Long running codebase analysis system. Specialized AutoGen agents (engineering, product, QA) work over a FAISS indexed repository to produce architecture insights, API documentation, and schema breakdowns.

Sessions are resumable, so analysis survives interruption on large repositories.

Python AutoGen LangChain FAISS Multi-agent

Real time collaborative whiteboard in a Turborepo monorepo: Next.js frontend, Express API, dedicated WebSocket server, shared UI package, PostgreSQL with Prisma.

Room based sessions with live multi-user drawing and conflict-free cursor sync.

TypeScript Next.js WebSocket PostgreSQL Prisma

Peer to peer video conferencing over WebRTC. Multi participant calls, screen sharing, in call chat, and recording with pause and resume.

React and TypeScript frontend, Node and Express signaling with Socket.IO and PeerJS.

WebRTC TypeScript Socket.IO PeerJS Node.js

Domain specific customer support assistant with a RAG pipeline over business documentation, Redis response caching, and a hybrid backend that falls back from OpenAI to local Hugging Face models when the API is unavailable.

Python FastAPI RAG Redis LangChain


Production Work

Xelpmoc Design and Tech, Software Developer, November 2022 to April 2024 Cut a critical query path from 75 seconds to under 10 seconds by restructuring joins, indexing high traffic tables, and layering Redis caching, roughly 4x throughput improvement. Built a WebRTC and Socket.IO interview platform sustaining 100 plus concurrent sessions under 200ms latency. Led architecture updates on a tourism platform that reduced production defects by 40 percent.

Svipes, Full Stack Developer, July 2024 to December 2024 Shipped 50 plus cross platform React Native screens with Redux Toolkit. Refactored the video module with asynchronous loading and caching, reducing playback stutter by up to 70 percent and load time by 1.3 seconds.


Stack

Languages

AI and Inference

Backend and Data

Cloud and Infra

Frontend


How I Work

I like systems that make sense, APIs that do not surprise you, and interfaces that guide instead of confuse. I would rather measure a bottleneck than guess at one, and I try to build things that survive past the first version.

If you are working on inference, backend infrastructure, or applied AI systems and want to build something together, reach out.

Build relentlessly. Ship quietly. Let the work speak.

Pinned Loading

  1. CollabDrawAI CollabDrawAI Public

    Real-time collaborative whiteboard built with a Turborepo monorepo: Next.js frontend, Express API, WebSocket server, shared UI package, and PostgreSQL database. Designed for low-latency sync, clean…

    TypeScript

  2. AIChatbot AIChatbot Public

    Domain-specific AI chatbot with RAG, vector search, Redis caching, and support for multiple LLM backends (OpenAI, Hugging Face, LangChain). Designed for real business use cases with contextual resp…

    Python

  3. RepoResearchAI RepoResearchAI Public

    RepoResearch is a long-running analysis system that uses AutoGen agents, vector search, and best-practice reasoning to break down any codebase. The flow is simple: upload → preprocess → agentic ana…

    Python

  4. carebridge-ai carebridge-ai Public

    Forked from heena5498/carebridge-ai

    CareBridge AI is a care-transition workspace for elderly discharge patients. The backend ingests discharge documents, extracts structured case data, runs validation checks, and supports nurse revie…

    JavaScript

  5. Job_Tracker Job_Tracker Public

    A desktop app for tracking your job applications — built with React + Electron. Features one-click capture from job boards via a browser bookmarklet, deadline tracking, drag-and-drop reordering, an…

    JavaScript 2 3

  6. Locra Locra Public

    Private, offline multimodal AI for Android using Qwen3-VL, llama.rn, and Whisper.

    TypeScript 1