HyperFi is building a fast, flexible, and complex platform with a robust, event-driven architecture and is seeking an AI Systems & Data Engineer to join their team to build data pipelines within the Databricks environment and leverage that data to build AI agents
Requirements
- Python (primary language for all LLM + orchestration work)
- LangChain + LangGraph + LangSmith
- Databricks + PySpark for processing, labeling, and training context
- Gemini + model routing logic
- Postgres, and custom orchestration via MCP
- GitHub Actions, GCP
- Familiarity with Databricks Data Intelligence Platform which unifies data warehousing and AI use cases on a single platform
Responsibilities
- Design and build data pipelines in Databricks for ingesting unstructured data
- Construct retrieval-augmented generation (RAG) systems from scratch using ingested data
- Build agentic LLM pipelines utilizing frameworks like LangChain, LangGraph, and LangSmith
- Own orchestration of PySpark and Databricks workflows to prepare inputs and track outputs for AI models
- Instrument evaluation metrics and telemetry to guide the evolution of prompt strategies
- Work alongside product, frontend, and backend engineers to tightly integrate AI into user-facing flows
- Leverage Databricks features such as Auto Loader for automatic detection of new files on cloud storage and schema changes
Other
- 5-7 years of experience building production-grade ML, data, or AI systems
- Must be based in San Francisco, Las Vegas, or Tel Aviv
- Full-time role with competitive comp
- Flexible hours, async-friendly culture, engineering-led environment
- Strong grasp of prompt engineering, context construction, and retrieval design
