Role Overview
We are seeking an experienced Lead Data Engineer to join our billing transformation program. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing.
The role requires strong experience with AWS Glue, Snowflake, Python, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, and operational monitoring to ensure billing accuracy.
This is a critical role supporting a transformation from raw telemetry usage data to rated, validated, and invoice-ready billing data.
Key Responsibilities
•Data ingestion pipelines from telemetry systems.
•Curated usage datasets in Snowflake.
•Rating logic implementation and billing-ready datasets.
•SAP invoice feed generation.
•Data quality and reconciliation framework.
•Exception and error-handling process.
•CI/CD pipelines for data engineering deployments.
•Monitoring, alerting, and operational dashboards.
•Technical documentation and runbooks.
Required Skills
•Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.
•Experience with AWS Glue, PySpark, Python, SQL, and cloud-based data processing.
•Strong experience working with Snowflake as a data warehouse or enterprise data platform.
•Good understanding of usage-based billing, rating engines, invoice feeds, or financial data processing.
•Experience building data quality checks, reconciliation frameworks, audit controls, and exception handling processes.
•Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar.
•Strong SQL skills for data analysis, validation, troubleshooting, and performance optimization.
•Experience working with batch and event-driven data ingestion patterns.
•Understanding of data modelling, partitioning, performance tuning, and cost optimization.
Soft Skills
•Strong analytical skills and attention to detail—especially important for financial accuracy.
•Ability to collaborate with cross-functional teams including Finance, Engineering, and Analytics.
•Good communication skills to explain defects, logic, and test outcomes clearly.
•High ownership and adaptability, with willingness to take on both QA and minor development tasks.
Benefits
•We are an equal opportunity employer with hybrid working.
•Annual Leave and Parental Leave benefits.
•Upskilling opportunities with job-relevant training and certifications.
•Dedicated Employee Assistance and Wellbeing support.
What You'll Get in Return
•High-impact projects: We work on a variety of pioneering AI products and leverage extensive datasets to solve complex, high-value challenges.
•Competitive benefits: You will enjoy a strong compensation package, comprehensive benefits, and ongoing investment in career growth and skill development.
•Industry leadership: This is an opportunity to be a founding member of the organization that's delivering brand-new products that democratize modeling and analytics solutions.
•Collaborative environment: We provide opportunities for continuous learning and professional development in a work environment of dedicated, highly experienced teams.
This role is considered 'Blended', which requires attending the office at least three days per week, while some teams and colleagues choose to collaborate in the office more frequently.
Career Stage: Senior Associate
About LSEG
LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth. Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries.
