Wavicle Data Solutions, LLC seeks a Senior Data Engineer to design and build optimized data pipelines for driving analytic insights.
Requirements
- 4 years of experience in the following: Web-based mobile application development using at least one of the following technologies: React, Vue, NodeJS, REST based Web services, Python, and Java; Programming experience using Scala, Python, or Java.
- 3 years of experience in the following: Working with at least one of the following databases: Snowflake, AWS Redshift, Oracle, or SQL; Utilization of SQL to write complex, highly-optimized queries across large volumes of data; Implementing ETL pipelines using at least one of the following AWS services: Lambda, S3, API Gateway, SNS, or SQS.
- Develop pipeline objects using Apache Spark / PySpark/ Python or Scala.
- Develop and maintain RESTful APIs and backend services to support data processing workflows and integration between data pipelines and user-facing systems.
- Implement and manage CI/CD pipelines using Jenkins or GitHub Actions to automate the development, testing, and deployment of data engineering solutions.
- Guide the adoption of development best practices including version control (GIT), CI/CD pipelines, automated testing, and performance tuning.
Responsibilities
- Responsible for designing and building optimized data pipelines, in an on-prem or cloud environment for the purpose of driving analytic insights.
- Create the conceptual, logical, and physical data models.
- Develop and implement end-to-end data pipelines and data integration solutions using cloud platforms such as AWS or GCP.
- Architect end-to-end data solutions ensuring integration across cloud platforms (AWS, GCP) with optimized ETL pipelines.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of sources like Hadoop, Spark, or AWS Lambda.
- Lead solution design and development of web-based data applications, providing technical direction on the front-end frameworks such as React.js, Vue.js, and SmartGWT to build interactive interfaces that support analytics and data exploration.
- Design, develop, test, deploy, maintain and improve data integration pipeline.
Other
- Bachelor's Degree in Computer Science, Software Engineering, Data Analytics, Information Technology or related field plus 4 years of experience as a Data Engineer, Applications Developer, Programming, or Technical Lead or related occupations.
- Lead and/or mentor a small team of Data Engineers.
- Communicate effectively with client leadership and business stakeholders.
- Participate in proposal and/or SOW development.
- Telecommuting is permitted. Must also have authority to work permanently in the U.S.
