Member of Technical Staff (Agent RL & Post-Training)

About Pavo

Pavo is building the Applied Science Factory: a system that develops an understanding of production systems, investigates performance, constructs reliable evaluations, and helps engineers improve business outcomes.

We work with the underlying product, code, data, models, and experiment history. Our research spans training environments, scientific agent capabilities, and models for structured and behavioral data.

We're a small team with experience building large-scale ML systems at Spotify, ShareChat, and Sourcegraph. Members of Technical Staff own research and implementation end to end.

The role

Pavo is building the Applied Science Factory around a recurring workflow: reconstruct how a production system works, investigate its behavior, formulate hypotheses, build evaluations, and guide improvements through human checkpoints.

You'll own post-training methods for the models performing this work. Example tasks include selecting an appropriate offline evaluation for a ranking policy, checking counterfactual assumptions, diagnosing discrepancies between offline and online results, and designing follow-up experiments.

You'll work across training data, supervised fine-tuning, reinforcement learning, reward design, agent execution, and held-out evaluation. The goal is capability improvements that generalize across different systems and datasets.

What you'll build

• Create tasks and expert trajectories for scientific investigation, counterfactual evaluation, cohort analysis, experiment design, and interpreting model results.
• Develop SFT and RL training recipes, data mixtures, curricula, verifiable rewards, and controlled ablations.
• Run multi-step agents using data queries, code execution, retrieval, and other tools; inspect rollouts and diagnose failures.
• Build held-out evaluations of estimator correctness, assumptions, evidence quality, generalization, and reward hacking.
• Integrate improved model behavior into Pavo's applied-science workflows and measure performance on new problems.

What success looks like

• A held-out benchmark for a defined applied-science capability with audited baselines and clear scoring criteria.
• A reproducible post-training gain on previously unseen tasks, supported by ablations and rollout analysis.

What we look for

• Have trained or post-trained language models or tool-using agents and can discuss concrete experiments, improvements, and failure modes.
• Strong Python and PyTorch or JAX; hands-on experience with SFT, RLVR, policy optimization, or reward modeling.
• Experience with rollout generation, training jobs, experiment tracking, checkpoints, and reproducible evaluation.
• Strong experimental design and statistics; causal inference, offline policy evaluation, or recsys evaluation is valuable.

Relevant experience

• Long-horizon agents, RL with verifiable rewards, coding/scientific agents, offline evaluation, and production experimentation.

What we offer

• Own a substantial area of research and engineering, working directly with the founders.
• Work on production ML problems and develop reusable methods and systems. Salary and equity.

Pavo is an equal opportunity employer.