Nasdaq’s Data Platform turns messy, multi‑modal content into trusted, governed data that powers business decisions and AI experiences across the company. We’re looking for an Engineering Manager to lead a small, high‑impact team building multi‑agent workflows and “digital workers” that automate data onboarding, processing, transformation, and quality checks—so the right data is accurate, explainable, and on time.
Requirements
- Python, SQL, Bash; distributed data processing (e.g., Spark/Databricks).
- Proficiency with containerization (Docker), Git CICD pipelines, and AWS services (S3, EC2, ECS, EKS).
- Eventing/streaming and batch orchestration
- LLM/agentic patterns (prompting, RAG, embeddings/vector stores) with evaluation and guardrails.
- Experience building multi‑agent systems (tool routing, skill specialization, safety checks) for real‑world workflows.
- Databricks platform design, development administration; Delta/Unity Catalog; Mosaic AI.
Responsibilities
- Build, coach, and retain a diverse team of engineers; set clear expectations, growth plans, and feedback rituals.
- Establish healthy engineering practices: design reviews, coding standards, on‑call rotations, and learning time.
- Co‑own quarterly roadmaps/OKRs with Product; translate strategy into staffed, sequenced work with measurable outcomes.
- Run predictable execution (backlog, sprint/PI planning, risk/dep management) and communicate progress and trade‑offs crisply to stakeholders.
- Set the technical bar for AI‑enabled pipelines: multi‑agent orchestration, tool selection, prompt interfaces, evaluation, and guardrails.
- Guide designs for Acquire Extract Match Map Master flows: idempotent ingestion/CDC, OCR + layout‑aware parsing, entity resolution with explainability, LLM‑assisted mapping, and survivorship‑based mastering (with lineage/SCD2).
- Champion platform standards (e.g., schema/ontology, transformations, lineage/metadata, observability) and re‑usability across teams.
Other
- 7+ years in software/data engineering with 3+ years directly leading engineers (people manager or tech‑lead with formal management).
- Proven delivery of data platforms or large‑scale data/AI systems in production with measurable reliability and cost goals.
- Strong judgment on trade‑offs (speed vs. safety vs. cost), and clear, concise communication to exec and partner teams.
- Hiring/interviewing experience and partnering with university programs to build early‑career pipelines
- This position offers the opportunity for a hybrid work environment (at least 4 days a week in office, subject to change)
