The Senior Data Engineer is responsible for designing, building, and maintaining the systems and infrastructure that enable organizations to collect, store, process, and analyze large volumes of data. This role involves collaborating closely with data scientists, analysts, QA engineers, business process / functional SMEs, and other stakeholders to ensure that data is accessible, reliable, and secure. They are responsible for tasks such as data ingestion, data transformation, data integration, and data pipeline development. Senior Data Engineers also play a crucial part in data governance, ensuring that data is compliant with regulations and policies. This role also includes all aspects of software development and deployment of IT Operations to shorten delivery and time to market for data products and data pipelines and data warehousing. Overall, the incumbent helps organizations leverage their data to gain meaningful insights and make informed decisions.
Requirements
- Experience with data engineering tools
- Experience with ETL/ELT tools like IDMC
- Programming languages like Python and SQL
- Working knowledge of Jira for task tracking
- Deep understanding of data architecture principles
- Experience with cloud technologies like Snowflake and Azure
- 4-6+ years of data engineering experience.
Responsibilities
- Design, develop, and maintain efficient and robust data pipelines, ETL processes, and data workflows
- Develop and maintain data models and schema designs to support data storage and retrieval needs
- Optimize data storage, processing, and retrieval mechanisms for performance and scalability
- Provide technical leadership and guidance in the selection and implementation of data engineering tools and technologies
- Identify and implement best practices and standards for data engineering processes and tools
- Stay up-to-date with emerging technologies and trends in data engineering, evaluating their potential impact on our data infrastructure
- Implement and maintain data governance and security measures to protect sensitive information and to ensure data quality and integrity by implementing effective data validation and cleansing techniques
Other
- Partner with a team of data engineers, providing technical guidance, mentorship, and support
- Act as a technical point of contact for clients and stakeholders, providing updates and addressing any data related technical inquiries or concerns
- Collaborate with cross-functional teams to integrate data sources and enable seamless data flow across systems and with data scientists, analysts, and stakeholders to understand data requirements and develop scalable data solutions
- This position is not eligible for sponsorship for work authorization now or in the future, including conversion to H1-B visa.
- This position is 100% remote
