Design, develop, automate, and support complex applications to extract, transform, and load data; Implement and automate complex processes; Monitor results and take corrective actions; Investigate, analyze, and resolve complex technical problems related to system functions, environment, and procedures; Manage service levels, standards, and configurations; Participate in Agile teams to meet goals; Development and support of Kafka integrations, including topics, producers, consumers and streaming applications; Design, develop and maintain Enterprise micro services - security, logging, common APIs and enforcing coding standards.
Requirements
- Design, build and maintain batch or real-time data pipelines.
- Automate data workflows such as data ingestion, aggregation, and ETL processing.
- Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
- Monitor data systems performance and implement optimization strategies.
- Must have experience with analyzing, tuning and optimizing queries in the RDBMS.
- Strong computer skills, including proficiency in at least one programming language (C/C++, Java, Python)
- hands-on experience in designing, developing and maintaining software solutions on ETL process utilizing IBM Infosphere DataStage/Quality Stage.
Responsibilities
- Design, develop, automate, and support complex applications to extract, transform, and load data
- Implement and automate complex processes
- Plan and conduct ETL unit and development tests
- Monitor results and take corrective actions
- Translate data access, transformation, and movement requirements into functional requirements and mapping designs
- Use SQL to design database schemas, optimize data retrieval, and implement ETL processes
- Development and support of Kafka integrations, including topics, producers, consumers and streaming applications
Other
- Participate in Agile teams to meet goals
- Must be able to produce thorough technical design documents, test plans and documented results.
- Experience with Agile methodologies and fast pace environment preferred.
- Willingness to learn new technologies.
- Excellent verbal and written communication skill to clearly and simply communicate complex information.
