Karsun is seeking a Sr. Data Engineer to drive the next generation of data and AI capabilities using cutting-edge open-source technologies and Databricks ecosystem tools.
Requirements
- Proficiency in incorporating Python data tools, utilities and libraries for a superior product outcome
- Experience with AI/ML model deployment and integration.
- Experience of working in an AWS containerized environment with limited container level access.
- Superior skills and a high level of proficiency in Python scripting and SQL scripting
- Proven experience with ETL/ELT tools and frameworks (e.g., Apache Airflow, Prefect, etc.).
- Hands-on experience with data warehousing solutions (e.g., Redshift, AWS DataLake, Delta Lake).
- Experience with reporting and visualization tools (e.g., Tableau, Quicksight).
Responsibilities
- Develop, test, and maintain data pipelines, data APIs and other data services to support a data mesh architecture for application modernization into microservices frameworks.
- Collaborate with data analysts and business stakeholders to understand reporting requirements and deliver effective solutions.
- Develop and operationalize data testing automation using tools like Great Expectations in collaboration with the test automation teams.
- Automate data workflows to improve efficiency and accuracy.
- Automate the monitoring and alerting for data pipeline performance and troubleshoot issues promptly.
- Ensure compliance with data governance and security standards.
- Drive adoption of emerging data engineering, reporting and AI/ML technologies to deliver superior and quantifiable performance gains and new capabilities induction.
Other
- Bachelor’s or Master’s degree in computer science, Information Technology, or related field and 8-10+ years of experience
- Proven strong hands-n experience as a Sr, Data Engineer with a strong focus on data-pipeline and reporting solutions working independently
- Ability to quickly ramp up into new teams and ecosystems with a proactive approach
- Strong collaboration and communication skills to work with data analysts, business stakeholders, and infrastructure and cloud engineering teams
- Travel and location requirements not specified
