Nearsure is looking for a Senior Data Engineer to work with modern data technologies such as Apache Spark (PySpark or Spark), Apache Iceberg, and AWS services (S3, EMR, Athena, Glue), ensuring data quality, scalability, and operational excellence.
Requirements
- 5+ Years of experience working in data engineering, designing and developing large-scale data solutions.
- 5+ Years of experience working with Python for data manipulation, scripting, and integration tasks.
- 5+ Years of experience working with SQL & DBMS (e.g., PostgreSQL, MySQL, SQL Server), data modeling (Star Schema, Snowflake Schema) and query tuning and optimization.
- 3+ Years of experience working with Apache Spark (PySpark preferred).
- 3+ Years of experience working building batch ETL/ELT pipelines for large‑scale datasets.
- 3+ Years of experience working with AWS data services (S3, Athena/Presto, Glue, Lambda, CloudWatch) in production.
- 2+ Years of experience working with Apache Iceberg (table design, partition strategies, schema evolution, maintenance, ingestion pipelines into Apache Iceberg on S3).
Responsibilities
- Design, develop, and maintain batch ETL/ELT pipelines and data workflows for large-scale datasets in AWS.
- Implement and optimize data lakehouse architectures using Apache Iceberg on S3, ensuring schema evolution, partitioning strategies, and table maintenance.
- Build and tune distributed data processing jobs with Apache Spark (PySpark or Spark) for performance and cost efficiency.
- Orchestrate workflows using Apache Airflow, including DAG design, scheduling, and SLA monitoring.
- Apply best practices in code quality, version control (Git/GitHub), and CI/CD for data engineering projects.
- Ensure data quality, security, and compliance through validation, monitoring, and governance frameworks (Glue Catalog, IAM, encryption).
- Continuously evaluate and adopt emerging technologies and AI-powered tools to improve productivity and maintain technical excellence.
Other
- Join our close-knit LATAM remote team
- Say goodbye to micromanagement!
- Our People Care team is here from day one to support you
- Competitive USD salary
- Advanced English Level is required for this role, as you will work with US clients.
