DataDirect Networks (DDN) is seeking to solve the problem of building and optimizing high-performance data ingestion, transformation, and streaming frameworks that enable the world’s most demanding AI and analytics workloads.
Requirements
- 10+ years of engineering experience in data infrastructure, distributed systems, or large-scale analytics platforms.
- Proven track record of building and leading high-performance engineering teams.
- Deep technical expertise in open-source data frameworks such as Trino, Delta Lake, Apache Iceberg, Ray Data, or Mosaic Streaming.
- Strong understanding of data ingestion, transformation, streaming, and orchestration for AI and real-time workloads.
- Hands-on experience with cloud platforms (AWS, Azure, GCP) and hybrid deployment architectures.
- Proficiency in distributed computing, parallel data processing, and system-level performance optimization.
- Experience contributing to or leading open-source software initiatives.
Responsibilities
- Build, scale, and lead a high-performing team of engineers focused on data ingestion, transformation, and streaming components within the Infinia platform.
- Drive the design and implementation of high-performance, cloud-optimized connectors that integrate with industry-standard open-source tools.
- Oversee the development of end-to-end data workflows that power AI/ML model training, inferencing, and real-time analytics at scale.
- Architect and refine a robust, low-latency data engine that supports diverse formats, compute engines, and workloads across on-prem and multi-cloud deployments.
- Benchmark and optimize data pipelines to meet the extreme throughput and concurrency demands of modern AI applications.
- Champion platform-wide initiatives around data interoperability, metadata management, partitioning strategies, and workload-aware scheduling.
- Partner with Product Management and Architecture teams to define technical roadmaps and prioritize high-impact work aligned with customer use cases.
Other
- Occasional in-person meetings or team events may be required.
- Strong cross-functional collaboration and technical communication skills.
- Ability to work in a remote environment with a team located in different metropolitan areas (Boston, MA; Raleigh, NC; Denver, CO; or Tucson, AZ).
- Strong prioritization skills and ability to work in a fast-paced environment.
- Bachelor's degree in Computer Science, Engineering, or a related field (not explicitly mentioned but implied)
