As a data engineer/developer, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure that are used for collecting, storing, processing, and analyzing large volumes of data.
Requirements
- Hands-on experience with building data-focused processes in languages such as Laravel/PHP, Python, or other languages.
- Knowledge and experience working in cloud platforms like AWS and/or Azure.
- Experience working with complex data objects in various formats like JSON, XML and across different DB designs like MySQL, MS SQL, and Salesforce DB’s.
- Proven experience in data engineering, ETL (Extract, Transform, Load) processes, and data pipeline development.
- Knowledge of or experience with Salesforce data models, API’s and high-level architecture.
- Experience with Business Intelligence tools such as Domo, Microsoft PowerBI, or other BI tools.
- Agile development experience working with tools such as Atlassian Jira, Confluence, and Bitbucket.
Responsibilities
- Implement data processing and transformation using state-of-the-art tools and processes.
- Build on top of existing capabilities and develop both custom features (e.g., Laravel/PHP, Python, or other frameworks) or enterprise software (e.g., Informatica, Mulesoft, or other tools).
- Design, develop, and maintain robust, scalable data pipelines for ingesting, transforming, and loading structured and unstructured data from various sources into data storage systems.
- Manage and optimize data storage solutions, including databases, operational data stores, and data warehouses, to ensure high performance, reliability, and security.
- Implement data quality checks, monitoring, and governance processes to ensure data accuracy, consistency, and compliance with regulatory requirements.
- Identify and address performance bottlenecks in data pipelines and infrastructure through optimization, tuning, and capacity planning.
- Work closely with data scientists, analysts, software engineers, and business stakeholders to understand data requirements, develop data solutions, and support analytical use cases.
Other
- Your role involves collaborating with business stakeholders, outside partners, other software developers, analysts, and other stakeholders to ensure data availability, reliability, and quality for decision-making and operational purposes.
- Documentation: Document data pipelines, infrastructure configurations, and best practices for data engineering standards, ensuring knowledge sharing and team collaboration.
- Excellent problem-solving skills, analytical thinking, and attention to detail.
- Effective communication and teamwork abilities to collaborate across multidisciplinary teams.
- Proven success working in both a team environment as well as independently.
