Описание
The company builds enterprise software products, open source solutions, and accelerators.
Задачи
•Develop and maintain reusable Python SDK modules for data pipeline operations, including compaction, quality checking, schema evolution, and table lifecycle management
•Design and build scalable and reliable framework libraries for 100+ pipelines and diverse data domains
•Collaborate with Data Engineers to gather feedback, understand pain points, and iterate on the SDK
•Contribute to platform architecture discussions and performance tuning strategies
•Write high-quality documentation and enablement materials for framework consumers
•Help define and infuse data engineering best practices through enablement, SDKs, and templates
•Ensure data quality, consistency, and governance across Lakehouse environments
Требования
•Strong experience as a Data Engineer with proficiency in Databricks
•3+ Years of experience building reusable libraries, SDKs, or internal developer tooling
•Knowledge of Data Mesh and Data Product concepts, including data ownership, domain-oriented design, and self-serve data platforms
•Deep expertise in Delta Lake, Delta tables, and compaction optimized for high-performance workloads
•Proven experience designing and maintaining complex data pipelines on cloud object stores such as ADLS
•Strong Python programming skills for data engineering workloads
•Solid understanding of Lakehouse architecture and best practices for large-scale data platforms
•Hands-on experience with data pipeline monitoring, troubleshooting, and performance tuning
•Familiarity with CI/CD and workflow orchestration, including Databricks Jobs
•Experience working in agile teams with ownership, autonomy, and best practices
•Excellent problem-solving skills and ability to handle high-scale, complex data challenges
•English proficiency at B2 level or higher
Nice to have: Experience with agile methodologies such as Scrum and SAFe, experience developing enablement materials and technical documentation.
Условия
No conditions specified
