American Express is looking to build data pipelines to integrate, consolidate, and cleanse data from different source systems, structuring it for use in individual analytics applications.
Requirements
- Must have experience in identity and access management, infrastructure technology, big data requirement definition, security event monitoring.
- 8-10+ years of experience with proficiency in programming languages as Scala, Python and SQL.
- Proficient with big data technologies such as Spark, Iceberg, Iceberg, Hbase, Kafka, Nifi, Hadoop, HDFS, AVRO, Trino, StarBurst.
- Experience with Spark Streaming or other stream processing technologies.
- Proficient with No SQL technologies such as Solr, Elastic.
Responsibilities
- Should have a good understanding of extract, transform and load (ETL) tools and REST-oriented APIs for creating and managing data integration jobs, and providing data analysts and business users with simplified access to prepared data sets.
- Develop and debug complex code, conduct code reviews, and mentor other developers on coding in alignment with best practices.
- Must be capable of building solutions from a framework perspective thus ensuring reusability vs. building siloed solutions which have low utility value.
- Provide technical or analytical guidance as needed for issue management, project assessments, and reporting.
- Engineer and develop standard data analytic service offerings to drive risk reduction opportunities for the enterprise.
Other
- Experience in planning, documenting scope and estimating work effort, producing quality deliverables on time and budget using agile engineering practices.
- Bachelor's Degree in Software Engineering, Computer Science, Mathematics, Information Systems
- Critical thinker, ability to convince and persuade based on logical presentation of well-researched facts. Highly organized, detail-orientated with the ability to synthesize large amounts of data.
- Utilizes collaborative working practices.
- Strong verbal and written communications skills including, strong presentation skills, and ability to elicit and translate information from business and technology SMEs in to written requirements.
