- Home
- ...
- Open Positions
- Job Detail
Description and Requirements
Description and Requirements
- Design evaluations that catch the failure modes of enterprise agents: hallucinated tool calls, policy violations, context collapse, regression under distribution shift, etc.
- Build the Agent Gym — task definitions, graders, reward signals, and trajectory capture — for multi-step agentic workflows.
- Run experimentation sweeps across prompts, models, and scaffolds; quantify trade-offs between accuracy, cost, and latency.
- Turn eval results into promotion gates and readiness reports that product teams can act on.
- Contribute to our Responsible AI tooling — grounding checks, policy enforcement, and human-in-the-loop escalation paths.
- Are currently pursuing a PhD in Computer Science, Machine Learning, AI, or a closely related field, with active research in LLMs, agents, reinforcement learning, AI safety, or evaluation methodology.
- Have produced non-trivial research or systems that work on modern LLM and agent stacks — multi-step tool-using agents, RAG pipelines, evaluation harnesses, and post-training.
- Can turn an open research question into testable hypotheses, choose strong baselines and ablations, interpret learning curves or reward trajectories honestly, and communicate findings clearly.
- Treat evaluation as a first-class AI research and engineering problem, not just a reporting layer.
- Published, submitted, or in-progress PhD research on LLM evaluation, agent benchmarks, alignment, RL environments, or related systems.
- Hands-on research experience with RLHF / RLVR, reward modeling, synthetic data generation, red-teaming, or scalable evaluation design.
- Contributions to open-source eval harnesses, agent scaffolds, observability tooling, or reproducible research infrastructure.
- Clear thinking about AI safety, deployment risk, benchmark validity, and the gap between academic results and enterprise production use.
Our commitment to you!
BMC’s culture is built around its people. We have 6000+ brilliant minds working together across the globe. You won’t be known just by your employee number, but for your true authentic self. BMC lets you be YOU!
If after reading the above, You’re unsure if you meet the qualifications of this role but are deeply excited about BMC and this team, we still encourage you to apply! We want to attract talents from diverse backgrounds and experience to ensure we face the world together with the best ideas!
BMC is committed to equal opportunity employment regardless of race, age, sex, creed, color, religion, citizenship status, sexual orientation, gender, gender expression, gender identity, national origin, disability, marital status, pregnancy, disabled veteran or status as a protected veteran. If you need a reasonable accommodation for any part of the application and hiring process, visit the accommodation request page.