About the Role
You will build and operate large-scale pipelines for video, LiDAR, and sensor data. You will design data-transfer infrastructure, write Python and Terraform, improve reliability and observability, work on cloud networking, support partners, and participate in on-call support.
Requirements
•Experience building and operating large-scale data transfer, ingestion, or processing pipelines
•Software engineering and production systems experience
•Experience with workflow orchestration tools
•Experience building reliable and observable data systems
•Cloud platform experience
•Proficiency in Python and Terraform
•Experience with object storage, file movement, networking, or cloud data transfer
Responsibilities
•Build and operate large-scale data transfer pipelines for video, LiDAR, and sensor data
•Contribute to the technical design of the Data Transfer Hub
•Build distributed data transfer pipelines using Flyte, Kafka, and Azure
•Implement hub-and-spoke data transfer systems
•Write infrastructure as code in Terraform and pipeline logic in Python
•Build retry strategies, backfills, integrity checks, lifecycle tracking, dashboards, and alerting
•Develop cloud-to-cloud and cross-cloud networking solutions
•Participate in an on-call and support rotation
•Collaborate with product, commercial, and partner-facing teams
Benefits
•Hybrid working
