Lucid is looking to build and optimize highly scalable data pipelines, storage layers, and compute systems across various business functions to support the next generation of their data platform.
Requirements
- Strong experience with modern data technologies such as Spark, Kafka, Airflow, Kubernetes, Docker, Trino or Presto, AWS Glue, and S3-based data lake patterns.
- Proven ability to design and implement scalable ETL and ELT data pipelines using batch and real-time frameworks.
- Deep knowledge of data lake and lakehouse architectures using Parquet, Iceberg, and metadata management concepts.
- Strong experience optimizing performance, reliability, and cost for large-scale data storage and compute systems.
- 8+ years of hands-on industry experience in data engineering or backend systems development.
- Demonstrated ability to drive technical decisions, influence architectural direction, and mentor other engineers.
- Ability to work across multiple teams and communicate complex technical concepts to diverse audiences.
Responsibilities
- Design, build, and optimize large-scale batch and streaming data pipelines that support vehicle data, manufacturing systems, and enterprise analytics use cases.
- Develop distributed data processing solutions using technologies such as Spark, Kafka, Trino, Airflow, Kubernetes, and AWS data services.
- Design efficient data lake and warehouse models using Parquet, Iceberg, and S3-based storage patterns that ensure performance, reliability, and cost efficiency.
- Serve as a technical leader for the data engineering team, providing deep expertise, unblocking complex issues, and guiding best practices for performance and reliability.
- Work closely with software, analytics, telemetry, manufacturing IT, and infrastructure teams to deliver robust and scalable data solutions.
- Implement data quality checks, monitoring, lineage, and automation to increase platform reliability and reduce operational overhead.
- Evaluate new frameworks, tools, and architectural approaches that elevate the data platform and support future scalability.
Other
- This role is required to be onsite at our headquarters in Newark, CA
- Track record of delivering high-impact data engineering solutions in a collaborative and fast-paced environment.
- Community for innovators who want to make an immediate and significant impact.
- If you are driven to create a better, more sustainable future, then this is the right place for you.
- Contribute to technical roadmaps and long-term vision for the evolution of Lucid’s data systems and data platform capabilities.
