Requirements
•7+ years of experience building and shipping production software, ideally including LLM-powered agents that take real actions in a product. You've worked on complex, tool-using systems (multiple tools, planning or orchestration, sub-agents) not just simple, single-turn assistants, and you can walk us through something you shipped and how you knew it was actually working
•Comfortable in Ruby on Rails / Python or ready to pick them up quickly. Ruby on Rails is our production system and the foundation that Circle's AI Agents are built on, and proficiency in Python is a strong plus, especially for the data and evaluation side of the work
•Experience building evaluation or observability infrastructure for ML/AI systems. You've built eval pipelines, scorers, dashboards, or CI/CD for evals before and have experience with evaluation frameworks like Braintrust, LangSmith, or similar
•Experience designing datasets, annotation workflows, or labeling pipelines for ML/AI evaluation. You know how to turn raw examples into a dataset you can trust through human annotation, AI-generated data, or simulation
•Learn fast, experiment aggressively, and thrive in a highly dynamic environment. You'll need to be comfortable running a lot of experiments and letting the results settle the argument, including the ones that don't work (which will be many of them in the beginning)
•Comfortable in a fast-paced environment with ambiguity. You'll need to learn and pick up new technologies when projects require it
•You are proficient in English (spoken, written, and reading) at a CEFR Level C2 / ILR Level 5
