Задачи:
•Design, develop, and maintain scalable data pipelines using Apache Spark, with Databricks preferred
•Write efficient and optimized PySpark code for data transformation and processing
•Develop and execute complex SQL queries for data extraction, validation, and reporting
•Implement unit tests using pytest to ensure code reliability and maintainability
•Collaborate with data scientists, analysts, and other engineers to deliver high-quality data solutions
•Monitor and troubleshoot data workflows and performance issues
•Document technical designs, processes, and best practices
Требования:
•3+ Years of experience in Data Software Engineering
•Proven experience with Apache Spark, ideally in a Databricks environment
•Proficiency in PySpark and SQL
•Background in unit testing frameworks, especially pytest
•Understanding of data engineering principles and ETL processes
•Familiarity with version control systems such as Git
•Ability to work independently and in a collaborative team setting
•Excellent problem-solving and communication skills
•English proficiency at Upper-Intermediate level (B2) or higher
Nice to have:
•Experience with cloud platforms such as Azure, AWS, or GCP
•Knowledge of CI/CD pipelines and DevOps practices
•Familiarity with Delta Lake, MLflow, or other Databricks-native tools
Условия: не указаны
