CoreAI is Microsoft’s central organization building the foundational platforms and tools to power AI experiences across products including GitHub Copilot, Azure AI Foundry, and Copilot Studio. The organization is expanding rapidly and is establishing a world-class data engineering platform to deliver analytical insights to the company leadership and product owners.
Requirements
- 3+ years of experience implementing and optimizing data pipelines and distributed data systems (e.g., Azure Data Lake, Spark, Synapse, SQL databases)
- 3+ years of experience with modern data engineering languages and frameworks (e.g., PySpark, SQL, dbt)
- 5+ year(s) experience developing and deploying live production systems, as part of a product team
- 5+ year(s) experience developing and deploying products or systems at multiple points in the product cycle from ideation to shipping
- Demonstrated ability to architect solutions that support both real-time and batch analytics workloads
- Understanding of data governance, privacy, lineage, and security best practices, especially within highly regulated or enterprise environments
- Functional understanding of Machine Learning and 2+ year(s) experience in data mining
Responsibilities
- Data Architecture & Engineering: design, build, and optimize large-scale, robust data pipelines and architectures.
- Data Governance & Trust: follow best practices for data quality, lineage, security, and compliance.
- Analytics Enablement: partner with data scientists, analysts, and business leaders to translate business needs into technical solutions.
- Cross-Functional Collaboration: work closely with product managers, software engineers, AI researchers, and business stakeholders to align data solutions with business goals.
Other
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
- Experience collaborating in cross-functional teams and building consensus among stakeholders with diverse backgrounds (engineering, analytics, product, business)
- Communication skills to convey complex technical concepts to both technical and non-technical audiences
- Track record of driving innovation in data engineering, including experience leading technical projects and mentoring others
