Bitsight is looking for an experienced Big Data Engineer to join their engineering team to help architect and evolve the data pipeline to support even larger volumes via cloud data warehouses, bring actionable security insights to the customer, and build trust with customers in the quality and integrity of their data.
Requirements
- Prior experience building and operating large-scale data pipelines
- Strong proficiency in modern Java language programming
- Strong computer science fundamentals in design, data structures, and algorithms
- Experience with Python 2 & 3
- Experience with Hadoop, Spark ecosystems
- Experience with Data Lakehouse tech stack, e.g. Apache Iceberg
- Experience with CI/CD and Kubernetes
Responsibilities
- architect and evolve the pipeline to support even larger volumes via cloud data warehouses
- integrating new data sources
- surfacing new features from existing data
- refining the rating algorithm
- collect, aggregate, and analyze data at scale using continuous, incremental development methodologies
- build trust with BitSight customers in the quality and integrity of our data
- ingest security-related intelligence data from many sources
Other
- Experience working with a team of engineers, designers, and product owners in delivering a product to end-users
- Excellent (English) written and verbal communication skills.
- Experience working in the IT security domain
- strong generalists: those with the critical thinking skills to use a variety of technologies and languages and pick up new concepts when needed.
- Strong collaboration skills are a must.
