Architect - DevOps
Role
We are looking for a DevOps engineer to own CI/CD for our ML and GenAI applications on Azure and Databricks. Infrastructure is already provisioned and managed by a separate platform team — your focus is the delivery pipeline, deployment automation, and the observability and guardrail layer around models and LLM applications in production.
Key Responsibilities
•Build and maintain CI/CD pipelines in GitHub Actions for Python applications, ML models, and GenAI/LLM services.
•Implement branching, versioning, and artifact promotion across DEV, SIT, and PROD environments.
•Automate deployment of Databricks jobs, notebooks, and workflows using Databricks Asset Bundles and the Databricks CLI.
•Automate model deployment to Databricks Model Serving and Azure ML online endpoints, including canary rollout and rollback.
•Integrate MLflow into pipelines for experiment tracking, model registry in Unity Catalog, and stage promotion.
•Add automated quality gates to pipelines: unit tests, linting, container scanning, model validation, and LLM evaluation checks.
•Implement LLM evaluation harnesses in CI — golden datasets, RAGAS-style retrieval metrics, LLM-as-judge, and regression tests that block bad prompt or model changes.
•Configure guardrails for GenAI applications: input/output filtering, PII redaction, prompt-injection and jailbreak checks, Azure AI Content Safety.
•Set up observability for models and LLM applications — MLflow Tracing, OpenTelemetry, Azure Monitor / Application Insights — covering latency, token usage, cost, tool calls, and failure modes.
•Configure monitoring and alerting for data drift, model performance degradation, and endpoint health.
•Manage secrets, service principals, and OIDC-based authentication between GitHub Actions and Azure/Databricks.
•Containerise applications with Docker and deploy to existing AKS or Azure Container Apps environments.
•Work with data scientists and GenAI engineers to move use cases from notebook to production reliably.
Required Skills & Experience
•6–10 years in DevOps, platform, or ML engineering roles.
•Strong hands-on experience building CI/CD with GitHub Actions — reusable workflows, matrix builds, environments, approvals, self-hosted runners.
•Solid Python skills, plus Bash, YAML, and Git.
•Hands-on experience with Databricks: Workflows, Unity Catalog, MLflow, Model Serving, Asset Bundles, Databricks CLI.
•Working knowledge of Azure: Azure ML, ADLS Gen2, Key Vault, ACR, AKS or Container Apps, Entra ID service principals, Azure Monitor.
•Practical MLOps experience: model packaging, registry, versioning, automated retraining, drift and performance monitoring.
•Practical LLMOps experience with production GenAI applications — RAG and/or agent frameworks such as LangChain or LangGraph.
•Hands-on experience implementing LLM guardrails and evaluation, not theoretical familiarity.
•Experience with Docker and container-based deployment.
•Experience with observability tooling and instrumenting applications for tracing and metrics.
Nice to Have
•Azure or Databricks certification.
•Experience in a regulated industry with model governance or audit requirements.
•Exposure to Langfuse or LangSmith, and to inference cost optimisation.
•Familiarity with Azure DevOps pipelines alongside GitHub Actions.
Qualifications
Bachelors - Computer Engineering, Bachelors - Computer Science, Bachelors - Information Technology, Masters - Computer Science
Certifications
Certified Kubernetes Administrator (CKA) - CNCF (Cloud Native Computing Foundation), HashiCorp Certified Terraform Associate - Udemy
Language
English (Required) — Advanced (C1)
Job Type
Regular, Hybrid work mode, Flex Time (India) work shift
