WitnessAI is building a product that provides security and governance guardrails for public and private LLMs, and they are looking for a Machine Learning Engineer to design, build, and evaluate language models that power these AI security products.
Requirements
- Strong software engineering background (Python, testing frameworks like pytest/unittest, CI/CD tools).
- Proficiency in ML frameworks such as PyTorch.
- Experience with data engineering tools (e.g., Spark, Kafka, Airflow).
- Familiarity with deploying models on cloud platforms (AWS, GCP, or Azure) and containerized environments (Docker, Kubernetes).
- Strong knowledge of ML fundamentals (supervised/unsupervised learning, deep learning, NLP).
- Research or industry experience in adversarial ML, model robustness, or explainable AI.
- Experience building interactive dashboards for model monitoring and visualization.
Responsibilities
- Build scalable pipelines to collect, preprocess, and manage datasets for training and evaluation of LLMs.
- Design and run experiments to evaluate LLMs on accuracy, robustness, fairness, and safety.
- Create dashboards, reports, and visualizations to communicate evaluation results, trends, and failure cases.
- Develop and leverage knowledge graphs to structure data, enrich evaluation, and improve context-driven model performance.
- Work with researchers to translate new ideas into engineering workflows, and with data scientists to automate QA checks and guardrails.
- Fine-tune, optimize, and integrate models into production systems with a focus on reliability, scalability, and monitoring and CI/CD best practices.
- Contribute to ML tooling and experimentation frameworks to accelerate iteration.
Other
- 2–5+ years working in machine learning or data science, ideally in a security or infrastructure-heavy environment.
- Interest or background in cybersecurity, adversarial ML, anomaly detection, or related fields.
- Startup Mindset: Comfortable working in fast-moving, ambiguous environments with a focus on shipping and iterating quickly.
- Hybrid work environment
- Competitive salary, health, dental, and vision insurance
