Microsoft Security aspires to make the world a safer place for all by reshaping security and empowering users, customers, and developers with a security cloud. The Microsoft Identity security team is on a mission to protect users from account compromise and fraud, using data science and machine learning to detect and block malicious actors.
Requirements
- 3+ years of experience in developing tools and data pipelines processing very large volume data by using Apache Spark and Python or similar tools.
- 3+ years of experience with data governance, data compliance and/or data security.
- 4+ years of experience in data engineering languages like Python, PySpark, SQL and others.
- 3+ years of experience in data modeling.
- 5+ years of hands-on experience with modern data storage and processing technologies, including relational and NoSQL databases, key-value stores, Spark compute engines, distributed file systems such as HDFS and ADLS Gen2, as well as messaging systems like Event Hub.
- Proven expertise in handling and processing large-scale data volumes.
- Familiarity Azure DevOps, CICD pipeline and release automation.
Responsibilities
- Design, build, and maintain scalable, reliable, and observable data and ML infrastructure.
- Collaborate closely with data scientists, data source owners, and partner engineering teams to deliver seamless and secure data workflows.
- Create data observability to ensure data health, including data quality and an up-to-date data catalog.
- Optimize performance of big data jobs.
- Participate in technical design and code reviews and contribute to maintaining a clean, secure, and well-documented codebase.
- Proactively identify and resolve bottlenecks and inefficiencies in data pipelines and infrastructure.
- Maintain compliance with all applicable laws and policies across your assigned workstreams.
Other
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
- Embody our culture and values.
- Microsoft is an equal opportunity employer.
- We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.
