About the Role
The Data Engineer builds and operates the data pipelines of our cloud data platform, from ingestion to the curated layers that serve analytics, BI, ML and AI. The role sits in the Data Tribe, reports to the Data Engineering Team Lead, and works on a modern lakehouse built on Databricks and AWS, serving multiple brands and markets in a regulated industry. Alongside delivery, the Data Engineer mentors junior engineers and helps the team adopt the platform's standards.
Responsibilities
•Design, build and maintain batch and streaming pipelines (Kafka, Spark Structured Streaming, batch jobs) on Databricks and AWS.
•Implement the medallion layers (Bronze/Silver/Gold): ingestion, cleansing, conformance, business rules and curated data models.
•Develop in Python/PySpark and SQL following the team's coding, testing and deployment standards (version control, CI/CD, Databricks Asset Bundles).
•Build CDC and event-based ingestion from relational and streaming sources, including handling of schema evolution.
•Participate in migrating existing data workloads to the new platform, from a cloud data warehouse and from on-premises sources.
•Work within the market isolation and data sharing model defined by the architecture (brand/market split, access control in Unity Catalog).
•Contribute to operational readiness: monitoring, alerting, runbooks and on-call support for pipelines in production.
•Implement data quality checks, reconciliation and data contracts between producers and consumers, as defined by the Data Architect.
•Maintain lineage, documentation and metadata for the datasets the team owns.
•Apply security and compliance requirements in pipelines: access control, data classification, retention and auditability.
•Mentor junior data engineers: code reviews, pairing, onboarding to the platform and standards.
•Work with Data Admins, DevOps, BI and ML teams to deliver Gold-layer models and features that meet consumer needs.
•Participate in design discussions and architecture reviews; propose improvements to pipelines and tooling.
Requirements
•3+ years of hands-on data engineering experience building production pipelines.
•Experience with Apache Spark (PySpark) and SQL in production; experience with Databricks (Delta Lake, Workflows/DLT, Unity Catalog) is a strong advantage.
•Experience with streaming data: Kafka (or Amazon MSK) and Spark Structured Streaming, including CDC patterns.
•Experience with at least one major cloud, preferably AWS (S3, Glue, MSK, Redshift, Lambda, IAM).
•Experience with data modelling for analytics (dimensional or similar) and with the medallion/lakehouse approach.
•Experience with version control, CI/CD and automated testing for data pipelines.
•Strong Python and SQL skills; solid understanding of distributed data processing and performance tuning.
•Good knowledge of relational databases (PostgreSQL, SQL Server) and of data warehouse concepts.
•Understanding of data quality, lineage and governance practices.
•Ability to translate business requirements into reliable, well-documented data pipelines.
•Ability to guide and review the work of less experienced engineers.
•Good communication skills in English, written and spoken.
•Ownership mindset: takes responsibility for pipelines from design to production support.
•Analytical mindset with strong problem-solving skills.
•Attention to detail and care for data correctness.
•Team player, open to feedback and willing to share knowledge.
•Mentoring skills.
Nice to have
•Databricks or AWS certifications (e.g. Databricks Data Engineer Associate/Professional, AWS Data Engineer).
•Experience with Apache Iceberg and multi-engine lakehouse setups.
•Familiarity with the existing on-premises stack (Apache NiFi, ClickHouse, Citus/distributed PostgreSQL) is a plus for the migration period, but not a requirement.
•Experience migrating workloads from Redshift or on-premises systems to a cloud lakehouse.
•Infrastructure as Code (Terraform) and orchestration tools (Airflow, dbt).
•Experience in regulated or high-volume domains (gambling, fintech, e-commerce) with multi-tenant or multi-market data.
Conditions
•Generous annual leave
•Hybrid work
•Individual health insurance
•Paid sick leave
•Paid maternity & paternity leave
•Family reward
•Performance & referral bonuses
