CoreAI is establishing a world-class data engineering platform to deliver analytical insights to company leadership and product owners, enabling product teams to turn complex datasets into breakthrough business value and fueling AI-driven innovation.
Requirements
- 3+ years of experience implementing and optimizing data pipelines and distributed data systems (e.g., Azure Data Lake, Spark, Synapse, SQL databases)
- 3+ years of experience with modern data engineering languages and frameworks (e.g., PySpark, SQL, dbt)
- Demonstrated ability to architect solutions that support both real-time and batch analytics workloads
- Understanding of data governance, privacy, lineage, and security best practices, especially within highly regulated or enterprise environments
- Functional understanding of Machine Learning and 2+ year(s) experience in data mining
- Familiarity with Fabric, Geneva, SQL/Cosmos, Kusto, Power BI, and other data tools and technologies at Microsoft
- Track record of driving innovation in data engineering, including experience leading technical projects and mentoring others
Responsibilities
- design, build, and optimize large-scale, robust data pipelines and architectures.
- follow best practices for data quality, lineage, security, and compliance.
- partner with data scientists, analysts, and business leaders to translate business needs into technical solutions.
- work closely with product managers, software engineers, AI researchers, and business stakeholders to align data solutions with business goals.
- implementing and optimizing data pipelines and distributed data systems (e.g., Azure Data Lake, Spark, Synapse, SQL databases)
- architect solutions that support both real-time and batch analytics workloads
- driving innovation in data engineering, including experience leading technical projects and mentoring others
Other
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
- 5+ year(s) experience developing and deploying live production systems, as part of a product team
- 5+ year(s) experience developing and deploying products or systems at multiple points in the product cycle from ideation to shipping
- Experience collaborating in cross-functional teams and building consensus among stakeholders with diverse backgrounds (engineering, analytics, product, business)
