10a Labs is looking to solve challenges related to AI safety and security by developing robust evaluation frameworks, designing and automating red-teaming strategies, and running adversarial testing initiatives.
Requirements
- Strong analytical toolkit (Python, SQL, Jupyter, scikit-learn, Pandas, etc.) and familiarity with modern ML tooling (e.g., PyTorch, Hugging Face, LangChain).
- Experience working with LLMs and embedding-based classification systems.
- Safety evaluation, red teaming, or adversarial content testing in LLMs.
- Trust & safety or risk-focused classification systems.
- Annotation ops, feedback loops, or evaluation pipeline design.
- Experience with open-source model evaluation tools (Promptfoo, DeepEval, etc.).
- Background in data science, applied ML, or ML engineering, with proven experience in production-grade systems.
Responsibilities
- Design the technical implementation of a robust red teaming project.
- Lead adversarial testing efforts (e.g., red teaming, evasion probes, jailbreak simulation) and analysis efforts.
- Work with researchers and domain experts to define labeling schemas and edge-case tests.
- Partner with ML and infrastructure engineers to ensure production readiness, observability, and performance targets.
- Automate red teaming, including developing automated workflows for prompt generation, model evaluation, and execution of AI experiments; fine-tune LLMs or classification systems.
- Brainstorm novel research approaches to both known and emerging problems involving AI, data, and the internet.
- Develop evaluation frameworks, design and automate red-teaming strategies, own quality metrics, and run adversarial testing initiatives.
Other
- 3-5 years of experience in applied data science, ML product work, or security-focused AI, including technical leadership or staff-level ownership.
- Has designed and evaluated real-world ML systems with a focus on model behavior, error analysis, and continuous improvement.
- Can design red teaming workflows to surface model blind spots and failure modes.
- Operates effectively across ML, infra, and policy / strategy contexts.
- Thinks like a builder, analyst, and communicator all in one.
