Nasdaq's Data Platform is transforming how business decisions are made by powering a global organization to discover, acquire, and operationalize high-value datasets. The job is to design and scale the next generation of workflows for this platform.
Requirements
- Strong Python, SQL, bash; solid CS fundamentals (data structures, algorithms, networking/OS basics).
- Experience with agentic workflows and multi-agent orchestration enabling natural language workflows, efficient tool selection, and specialized AI capabilities to manage complex, real-world tasks.
- Familiarity with LLM‑powered enrichment (prompting, RAG, embeddings), vector stores, and evaluation of extraction/matching quality.
- Expose to distributed data systems and orchestration (e.g., Spark/Databricks, Kafka) and real‑time/batch pipelines
- Familiarity with embeddings/vector stores and evaluation techniques.
- Experience from internships, research, OSS, or substantial class projects building AI and distributed data‑centric software.
Responsibilities
- design and scale the next generation of workflows that acquire content from heterogeneous sources, extract structured knowledge, and assemble golden records with full lineage and governance
- design and implement components for our data platform with a strong emphasis on data onboarding and automation for ingestion, processing, transformation, and quality checks by building multi-agent workflows and digital workers
- Acquire — Plug in new sources (APIs, S3/Blob, SFTP, HTTP, buses); auth, throttling, CDC, idempotent loaders, and safe backfills.
- Extract — Parse PDFs/HTML/DOCX/Images with OCR and layout‑aware techniques; segment content for RAG and entity matching; enrich topics/language/license.
- Match — Help train and evaluate entity‑resolution models (Siamese/bi‑encoders + rules), define blocking keys, and add explainability.
- Map — Align schemas/ontologies; contribute to an LLM‑assisted “suggest approve apply” mapping flow with change‑impact checks.
- Master — Assemble golden records with survivorship rules; support SCD2 history, quality constraints, anomaly detection, and freshness SLOs.
Other
- BS/MS in CS, CE, EE, Math, or related field (or equivalent project/research experience).
- This position offers the opportunity for a hybrid work environment (at least 4 days a week in office, subject to change)
- Nasdaq is an equal opportunity employer.
- We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment.
- In addition to base salary, Nasdaq offers significant other compensation (annual bonus/commissions and equity), benefits, and opportunity for growth.
