About the Role
You will own production reliability for a Kubernetes-native control plane that provisions and runs GPU inference infrastructure. You will operate host-lifecycle workflows, investigate networking and serving failures, extend automated remediation, manage SLOs and incident response, and turn production lessons into improvements in controllers, automation, observability, and platform design.
Requirements
•Experience operating production provisioning or host-lifecycle pipelines at fleet scale.
•Deep experience owning reliability for large-scale distributed systems.
•Hands-on InfiniBand and RDMA troubleshooting experience.
•Familiarity with multi-tenant model serving, LoRA or adapter serving, and differentiated SLOs.
•Experience eliminating operational toil through self-built automation and tooling.
•Hands-on experience with Kubernetes controllers, CRDs, reconciliation loops, and node-level remediation systems.
•Knowledge of event-driven architectures, backpressure, circuit breaking, metrics, logs, and traces.
•Ability to write and ship production code for GPU infrastructure.
Responsibilities
•Operate the production provisioning state machine and host-lifecycle pipeline.
•Diagnose failed state transitions, GPU driver and CUDA validation issues, health-check failures, and decommissioning issues.
•Investigate InfiniBand, RDMA, GPU, hero-model, and LoRA-serving failures.
•Extend automated self-healing with detection and remediation plugins.
•Own SLOs and error budgets for provisioning and fleet health and lead incident response.
•Feed production topology, interconnect, and scheduling failures into platform requirements.
•Improve inference-stack performance, latency, throughput, and efficiency.
•Build observability tools and implement solutions for bottlenecks and instability.
•Lead cross-functional reliability planning and stakeholder management.
Benefits
•Startup equity.
•Health insurance.
