proSapient

Data engineer for knowledge platforms

In short

Senior data engineer role building and operating batch and real-time data pipelines that feed matching and search systems, working with Kafka, Elasticsearch/OpenSearch, Python, SQL and PostgreSQL. The company, proSapient, supports consulting and private equity firms in knowledge discovery for decision-making. Requires 5+ years building production data pipelines, strong Python and SQL skills, and hands-on Elasticsearch/OpenSearch experience. Based onsite in Lisbon with limited remote days allowed.

О компании
Every day, somewhere in the world, important decisions are made. Whether it is a private equity company deciding to invest millions into a business or a large corporation implementing a new strategic direction, these decisions impact employees, customers and other stakeholders. Consulting and private equity firms come to proSapient when they need to discover knowledge to help them make great decisions and succeed in their goals. It is our mission to support them in their discovery of knowledge.
Задачи
•Build and operate batch and real-time pipelines feeding the matching and search systems
•Work closely with real-time data streams and search infrastructure using tools like Kafka and Elasticsearch
•Model the datasets the product layer consumes — candidate features, interaction events, ranking signals — and keep the contracts stable
•Own the pipelines you build in production: monitoring, alerting, backfills, and the on-call reality of a broken 3am job
•Instrument recruiter and client behaviour so matching quality can be measured and improved
•Build the event pipelines behind our CSAT and workflow metrics
•Conduct and analyse experiments to rapidly iterate on the quality of data processing pipelines and make experiment data queryable without a bespoke analysis every time
•Add data quality checks and tests at the boundaries that matter and make failures loud instead of silent
•Reduce cost and latency where it counts, leaving the rest alone
•Partner with QA to make data correctness a testable property, rather than a hope
•Identify and provide useful data sources, internal or external, for our structured search and GenAI tools
Требования
•5+ years building production data pipelines
•Strong Python programming skills, with practical experience writing clean, production-ready code
•Strong SQL and proficiency with PostgreSQL or other relational databases
•Strong hands-on experience with Elasticsearch / OpenSearch, including performance optimisation and query tuning
•Deep understanding of both batch and event/streaming patterns, and a clear sense of when each is appropriate
•Practical data modelling skills for analytics and for serving application features
•Experience operating your own pipelines in production (and learned from it), alongside comfort working in a codebase with tests, code review and CI
Будет плюсом
•Message brokers like Kafka or RabbitMQ
•Cloud data warehouse — BigQuery preferred, equivalents fine
•Docker, Kubernetes, and cloud environments (AWS and/or GCP)
•Experience feeding search, ranking or ML systems specifically
•Building LLM-based extraction or enrichment pipelines
•Experience with dbt or a comparable transformation framework
•Observability tools (e.g. Prometheus, Datadog) and background job processing frameworks
•Exposure to Elixir services (part of our platform)
Условия
•Tenure gifts, including vouchers, extra holiday and sabbaticals for each year of employment
•Health insurance through Vitality
•Remote working for up to 20 days each year, giving you flexibility and a change of scenery
•Employee Assistance Programme with personalised health and wellbeing advice from specialist teams
•Enhanced maternity and paternity pay
•25 days' annual leave plus bank holidays, including a week's closure over Christmas
•MyMindPal app for online mental fitness support
•Corporate events, from quarterly gatherings to annual winter and summer parties

Stop applying.
Get referred.

$10 in free credits

In just 60 seconds, fully on autopilot — Eraldt's AI agents read your CV, map the roles you fit, find live vacancies and the people who can refer you, write personal emails, and get you referred.

This isn't an application: your CV doesn't go to the company, and we don't promise a referral for this exact role.

Discover who else lands interviews on autopilot