About the Role
We're looking for a Tech Lead who is more than a great data engineer - someone who brings equal depth in backend software engineering and broad fluency across data technologies, databases, and large-scale distributed systems. You'll own the architecture and hands-on development of our data platform while driving engineering excellence across the team.
Responsibilities
•Design, build, and maintain large-scale batch and streaming data pipelines (ETL/ELT) across multiple products
•Develop and optimize Spark workloads for performance, cost, and reliability on large data volumes
•Build and evolve our Data Lake / Lakehouse, including data modeling, partitioning, and storage formats
•Write production-grade backend services and REST APIs that expose data platform capabilities
•Build in data quality, observability, and monitoring from the start, and own your pipelines in production end to end
•Automate and improve the data platform, including AI-assisted development and LLM-powered data workflows
•Work closely with R&D and product teams, and contribute to design and code reviews
Requirements
•6+ years of hands-on Data Engineering experience building and operating production-scale pipelines
•Proven experience with building and maintaining a large scale Data Lake
•Deep, hands-on experience with Spark (tuning, partitioning, debugging large jobs)
•Strong Python and advanced SQL
•Backend development experience: REST APIs, microservices, service design patterns
•Broad database experience across relational, analytical/columnar, NoSQL, and streaming stores
•Experience with cloud-based data platforms
•Practical experience with AI tools and agents, such as integrating or orchestrating LLM-powered workflows in production
•Strong engineering fundamentals: testing, CI/CD, code quality, and performance profiling
•Self-directed: you take a problem from design to production with little guidance and suggest improvements without being asked
Nice to Have
•Hands-on Databricks experience is a major advantage
•Azure data services or EventHub
•Streaming frameworks and real-time processing (e.g., Spark Structured Streaming, Kafka, Flink)
•Table formats such as Delta Lake or Iceberg
•Background in cyber security, SaaS, or large-scale enterprise environments
•Experience with data governance, observability, or self-service platforms
