Microsoft's Commerce + Ecosystems (C+E) organization needs to produce accurate, reliable, and efficient records of charge, empower customers and partners through insights and analytics, and elevate critical commerce functions through code, data, models, insights, and email platforms. The Programmability Insights & Engineering (PIE) team specifically needs to deliver critical commerce experiences through deep integration across charges, billing, and pricing platforms by innovating in distributed data storage, intelligent routing, and scalable data modeling.
Requirements
- 2+ years of experience in software/data engineering.
- 1+ authoring Big Data ETL processing on cloud service using Spark Scala or other big data technologies.
- Proficiency in Apache Spark (PySpark or Scala) and distributed data processing.
- Experience with schema design and dimensional data modeling
- coding in languages including, but not limited to, C, C++, C-Sharp, Java, JavaScript, or Python
- Demonstrated experience leveraging AI tools and technologies to enhance engineering effectiveness
Responsibilities
- Design, develop, and maintain Spark-based data pipelines on Azure Synapse for large-scale anomaly detection and reporting.
- Implement distributed data processing solutions leveraging Spark for batch and streaming workloads.
- Coordinating data domain teams to understand datasets and onboard them to Anomaly detection platform.
- Ensure data quality, integrity, and compliance across multiple sources.
- Optimize Spark jobs for performance, scalability, and cost efficiency in cloud environments.
- Contribute to code reviews, design discussions, and architecture improvements.
- Design, build, and operate scalable data pipelines that run Anomaly detection on Commerce Datasets such as Usage, Charges, Pricing, Invoices, Credits and Balances.
Other
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
- Collaborate with product managers, data scientists, and engineering teams to deliver end-to-end solutions.
- seeking feedback, sharing ideas, and learning from diverse perspectives, living Microsoft’s values of respect, integrity, and accountability so everyone can thrive.
- Microsoft is an equal opportunity employer.
