About the Role
We are looking for an exceptional Staff/Principal ML Engineer to join our client, a cutting-edge real-time AI research lab and inference provider backed by top-tier venture funds. Powering hundreds of millions of daily active users and serving over 10 trillion LLM tokens monthly, the platform operates at massive scale behind low-latency voice and conversational AI APIs. In this role, you will take true end-to-end technical ownership of inference optimization, containerizing foundational models and scaling sub-second, high-concurrency workloads on high-performance GPU infrastructure.
Responsibilities
•Taking complete end-to-end technical ownership of the model serving and inference optimization lifecycle.
•Taking cutting-edge speech, voice, and LLM architectures directly from the AI research team and containerizing them for production deployment.
•Optimizing serving runtimes and kernels for sub-second latency and ultra-high-concurrency workloads on distributed NVIDIA GPU clusters.
•Architecting and scaling distributed inference platforms using Ray, Kubernetes, and Docker to ensure 99.99% operational reliability under heavy global traffic.
•Diagnosing low-level system bottlenecks, memory allocations, and GPU compute performance across custom model serving pipelines.
•Driving architectural standards, mentoring senior team members, and collaborating cross-functionally between research and core platform engineering.
Requirements
•Staff/Principal-level software engineering and ML systems background (equivalent to Google L5/L6 or Meta E5/E6).
•Deep systems-level experience profiling, benchmarking, and optimizing high-throughput, low-latency machine learning pipelines in production.
•Proven track record scaling large distributed systems handling mission-critical, enterprise-grade traffic.
•Consultant mindset: exceptional technical leadership, high ownership, ability to make sound architectural trade-offs, and clear communication across engineering squads.
•Strong problem-solving abilities within low-level hardware-software interfaces and GPU architectures.
•Strong programming mastery in C++, Python, and/or Rust.
•Hands-on expertise with high-performance model serving frameworks: vLLM and TensorRT-LLM (TRT-LLM).
•Deep knowledge of CUDA programming, GPU kernel optimization, and NVIDIA GPU architecture.
•Proven production experience with containerization and distributed orchestration using Kubernetes, Ray, and Docker.
•Demonstrable track record deploying and running large-scale LLM/GenAI inference systems in production.
•Fluent English for daily international technical collaboration.
Conditions
•100% Remote working model (Serbia, Germany, UK, Switzerland).
