Описание
Block Labs is a technology studio operating across Web3, Artificial Intelligence, and iGaming. It develops high-scale, production-grade platforms, including autonomous multi-agent AI systems, decentralized financial infrastructure, and high-frequency iGaming platforms.
Задачи
•Design, build, and maintain scalable data pipelines using AWS Glue (PySpark) or equivalent orchestration and transformation tools
•Engineer and optimise the ClickHouse warehouse for sub-second query performance across all back-offices
•Implement data contracts between back-office systems and the platform
•Build a feature-serving layer providing pre-computed features to AI agents at millisecond latency
•Integrate third-party databases, back-office APIs, and external systems including CRM, affiliates, and acquisition platforms
•Establish monitoring, alerting, and maintenance procedures, including pipeline health checks, freshness monitoring, anomaly detection, and data contract SLA enforcement
•Own CI/CD and infrastructure-as-code for data workloads
•Collaborate with data scientists, agent engineers, BI developers, and infrastructure teams to translate data requirements into reliable, production-grade pipelines
•Participate in design reviews and own domain decisions
Требования
•3+ Years building and operating production data pipelines at scale across streaming and batch paradigms
•Expertise in Apache Kafka or Amazon MSK, including topic design, consumer group management, offset handling, schema registry operations, and production troubleshooting of lag, rebalancing, and throughput issues
•Strong SQL and warehouse engineering skills
•Experience with columnar analytical databases such as ClickHouse, Druid, BigQuery, or Redshift
•Proficiency in PySpark or Spark Streaming for transformation jobs that normalise, enrich, and enforce business rules on event streams
•Data modelling skills, including normalised multi-tenant schemas, data contracts, and schema governance
•CI/CD and infrastructure-as-code experience, including automated pipeline testing and version-controlled deployments with CloudFormation, Terraform, or CDK
•Familiarity with containerised workloads such as ECS Fargate or Kubernetes
•Experience implementing pipeline health monitoring, automated data validation, freshness checks, and anomaly detection
Nice to have: experience with AWS Glue, Apache Airflow, or Apache NiFi; experience in iGaming, online casino, poker, or sportsbook platforms; exposure to blockchain or crypto-native transaction flows; AWS-native environment experience; feature store experience with SageMaker, Feast, or Tecton; work in regulated industries; experience migrating legacy query engines to modern analytical warehouses.
Условия
•Fully remote with asynchronous-first communication
•EU time zone overlap is preferred
•Small, high-autonomy team within the Data function
•Reports to the Head of Data and coordinates with the AI, BI, and Infrastructure Teams
•Architecture decisions are documented and debated
•On-call rotation will be established in the run phase; the build phase focuses on velocity with quality
