• Writing evals for output that precision and recall do not capture: is the diagnosis correct, is the cited source real and does it say what the agent claims, does a generated brief hold brand voice in Greek and Dutch as well as in English

• Checking whether the tracking data an agent depends on is sound before building on it, and quantifying the error when it is not

• Wiring an agent to a webhook or a scheduled trigger, and making the handler idempotent so a retry does not double-post a recommendation or apply the same keyword exclusion twice

• Deciding which steps in a flow warrant a frontier model and which run on something cheap, then proving that routing decision with numbers, because these flows run daily across many brands and the bill compounds

• Sitting in a vendor demo asking what their API actually exposes, what the rate limits and quotas are, what their data model looks like, and what integration really costs us

What you will bring

• 7+ years building data and ML systems in industry, spanning both sides of the LLM shift. We want the judgment that comes from having debugged systems before you could ask a model what was wrong.

• Somewhere in that history: you have been the only person who did a job end to end. First or only data hire, the single ML person in a small company, or a one-person function inside a large one. We are less interested in company stage than in the condition, because it is what forces someone to map the process, build it, deploy it and answer for it rather than hand each part to a specialist.

• Somewhere in that history: you have shipped something with permission to act on live systems affecting real customers, and you can tell us what you did to sleep at night.

• Expert in Python and ML.

• You ship end to end. Python someone else can still read in six months, a current toolchain (uv, Docker or an equivalent, we care that you re-examine your tooling, not which tool you landed on), your own container, your own instrumentation.

• Production experience with multi-step, tool-calling LLM workflows: orchestration, retries, idempotency, timeouts, partial-failure recovery. State-machine design, not only train/serve pipelines.

• Integration against third-party APIs you do not control. Auth flows, rate limits, pagination, sandbox behaviour that differs from production, and schema changes shipped without notice.

• Cost and latency engineering as a first-class concern: model routing, caching, batching, and the instinct to know what a flow costs per run before Finance asks.

• A safety instinct for systems that take actions: staging modes, approval gates, least-privilege scoping, a way back.

• Evaluation design for generative and agentic output: LLM-as-judge, golden-transcript regression suites, red-teaming. Including calibration: an agent that reports high confidence needs to be right at that rate, and you can show whether it is.

• Process mapping and quantification. You can sit with a domain expert, capture what actually happens rather than what the policy says, and attach hours to it.

• Technical vendor evaluation. Judging a martech vendor on API surface, data model, extensibility and true integration cost, not on the sales deck.

Nice to have

• Master’s or PhD in Computer Science, AI, Machine Learning or a related field

• PromptOps at scale: versioning, testing and rollback of prompts and models as production artefacts

• Prior exposure to martech, ad-tech or SEO tooling and their APIs, or to automation in any domain where output is customer-facing

• Experience evaluating generated output across multiple languages

Right to work

At any stage, please be prepared to provide proof of eligibility to work in the country you are applying for. Unfortunately, we are unable to support relocation packages or sponsor visas.

Come as you are

Everyone is welcome here. Diversity and inclusion are at our core. Far above any technical competence, we value respect, openness, and trusted collaboration. We do not tolerate intolerance.

ESG

At team.blue, our commitment to caring for the environment and each other is at the heart of everything we do. Our latest impact report showcases our ongoing ESG efforts and ambitious sustainability goals. Interested in learning more about our dedication to making a positive impact? Check it out here.

#LI-CC1