Description
Oracle Database Migration Project to a GCP platform based on BigQuery / Pub/Sub / Cloud Storage at PKO BP bank.
Responsibilities
•Designing, implementing and developing Data Engineering solutions on Google Cloud Platform, primarily using BigQuery, Cloud Storage and Pub/Sub
•Migrating and rebuilding data processing and ingestion processes from Oracle/on-premise to GCP
•Designing and developing batch and streaming processes, together with mechanisms for quality control, retries and error handling
•Designing data structures and models in BigQuery, taking into account the performance, scalability and cost of data processing and storage
•Analyzing data and preparing logical and physical data models for Oracle, PostgreSQL and data layers in GCP
•Creating, analyzing and tuning SQL queries, including analyzing execution plans and identifying bottlenecks for large data volumes
•Analyzing existing Informatica PowerCenter processes and migrating them to target solutions in GCP
•Implementing data transformation, validation and integration logic in SQL and Python, where required by the designed solution
•Performing technical and impact analyses, analyzing dependencies and participating in agreeing on the solution specification
•Preparing unit and integration tests and supporting system, performance, regression and UAT testing
•Preparing changes for deployment, deployment instructions, and supporting deployment and post-deployment verification
•Diagnosing and resolving incidents and participating in root cause analysis of problems related to data processing and quality
•Creating and updating technical documentation and working with a code repository and CI/CD processes in accordance with the DevOps model
Requirements
•At least 2 years of commercial experience in designing and implementing data integration or processing processes, with the predominant part of practical experience involving Google Cloud Platform and working with BigQuery, Cloud Storage and Pub/Sub
•Practical experience with Oracle, including creating and optimizing SQL queries and working with large data volumes
•Development experience with Informatica PowerCenter enabling independent analysis and modification of ETL processes; however, this technology is not the main area of the role
Conditions
No conditions specified in the posting.
