Mouad

LLM Infrastructure Engineering Intern

Validated the performance of llm-d, an open-source KV-cache-aware router, replacing native Kubernetes routing to improve the scheduling of inference requests.. Leveraged llm-d’s custom metrics to enable more efficient autoscaling of model backends.. Benchmarked routing strategies to quantify gains in latency, throughput, and GPU utilization in a self-hosted LLM deployment context.