American Express is transforming its technology and product roadmap to deliver innovative payment products and services at a rapid pace, serving customers seamlessly across physical, digital, mobile, and social media. The company is moving from monolithic, tightly coupled, batch-based legacy platforms to a loosely coupled, event-driven, microservices-based architecture to meet long-term business goals.
Requirements
- Deep understanding of Hadoop and Spark Architecture and its working principle.
- Deep understanding of Data warehousing concepts.
- Ability to design and develop optimized Data pipelines for batch and real time data processing.
- Experience in data analytics and cleansing
- 5+ years’ experience on Python or Java Hands-on experience on writing and understanding complex SQL (Hive/PySpark- data frames), optimizing joins while processing huge amount of data.
- 3+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL and PySpark).
- Hands on Experience on Google Cloud Platform (BigQuery, DataProc, Cloud Composer)
Responsibilities
- Responsible for designing Data Engineering solutions on bigdata ecosystem, developing custom applications, and modifying existing applications to meet distinct and changing business requirements.
- Hands on coding, debugging, and documentation, as well as working closely with SRE team.
- Provide post implementation and ongoing support.
- Develop and design software applications, translating user needs into system architecture.
- Assess and validate application performance and integration of component systems and provide process flow diagrams.
- Test the engineering resilience of software and automation tools.
- Be part of an enthusiastic, high performing technology team developing solutions to drive engagement and loyalty within our existing cardmember base and attract new customers to the Amex brand.
Other
- Masters in computer applications or equivalent OR bachelor’s degree in engineering or computer science or equivalent.
- Ability to effectively communicate with internal and external business partners.
- Experience in leading teams of engineers and scrum teams
- Ability to learn and apply new programming concepts.
- Knowledge of Financial reporting ecosystem will be a plus.
