Описание
The client is a reinsurance and risk management company that provides solutions across insurance, underwriting, and data-driven risk assessment. It helps clients address complex risks and create sustainable value.
Задачи
•Create and manage ETL processes using Azure Data Factory to automate data flows between systems and services
•Implement data storage solutions using Azure Data Lake and Cosmos DB, ensuring high availability and scalability
•Design and optimize data processes in Databricks, including Spark jobs using Delta Lake for data handling and versioning
•Develop distributed data processing workflows using PySpark
•Build and maintain analytics models using Delta Lake to improve data quality and performance
•Develop Python and PowerShell scripts to automate operational tasks, deployments, and data workflow monitoring
•Collaborate with development teams to integrate CI/CD practices using Azure DevOps and Git for version control
•Manage and optimize MS SQL Server databases and write efficient T-SQL queries for data manipulation and reporting
•Ensure data integrity and security through database management practices
•Implement logging and monitoring solutions to proactively address issues affecting data applications and workflows
•Conduct root cause analysis of data-related problems and develop solutions to prevent recurrence
•Collaborate with business intelligence teams to create and share reports using Power BI
•Provide insights and recommendations based on data analysis to guide business decisions
Требования
•Bachelor's degree in Computer Science, Information Technology, or a related field is preferred
•At least 5 years of experience in a similar role
•Proficiency in Python and PySpark for data processing, optimization, and automation
•Experience with PowerShell scripting and Azure Data Factory
•Strong understanding of MS SQL Server and T-SQL for database scripting
•Familiarity with Databricks, Delta Lake, Azure Data Lake, and Cosmos DB
•Experience with Git version control and CI/CD frameworks in Azure DevOps
•Solid problem-solving skills and ability to troubleshoot complex data issues
•Familiarity with DevOps methodologies, including Scrum and Agile software development practices
•Experience creating dashboards and reports using Power BI, and knowledge of data governance and data security best practices
•Excellent spoken and written English
Условия
No conditions specified
