Clear Street is looking to build high-performance real-time data pipelines to process financial market data and support its continued expansion into global markets.
Requirements
- 5+ years of professional experience implementing low-latency, high-throughput data pipelines
- Solid understanding of distributed systems and the challenges involved in real-time data pipelines (e.g., data consistency, fault tolerance, scalability).
- Familiarity with financial market data, including security prices, and asset classes like equities, options, futures, etc.
- Strong familiarity with Linux/BSD
- Familiarity with TCP/IP and UDP (Unicast/Multicast) networking
- Experience with system profiling with eBPF
- Experience with columnar storage and database systems
- Kernel bypass networking (DPDK or XDP)
- Provisioning and configuration of routing in the cloud or on-premise, including BGP, VPCs, network peering
- Experience with Aeron
Responsibilities
- Design, develop, and maintain real-time data pipelines to handle financial market data with low latency and high throughput in a resilient manner.
- Work with various asset classes such as equities, options, futures, and other financial instruments to ensure timely and accurate data processing.
- Develop efficient mechanisms for integrating market data feeds from exchanges and other sources into our systems.
- Troubleshoot and resolve performance issues, data discrepancies, and ensure data integrity across the pipeline.
- Continuously monitor the performance and health of data pipelines, identifying and mitigating potential issues before they impact system performance.
Other
- You communicate technical ideas with ease and always look to collaborate to deliver high quality products.
- You are a team player, with experience working effectively with other engineers toward common goals
- 4 days per week in-office requirement
- Competitive compensation, benefits, and perks
- 401k matching, gender neutral parental leave, and full medical, dental and vision insurance
