Affinity stitches together billions of data points from massive datasets to create a powerful, accurate representation of the world's professional relationship graph. Based on this data, we offer our users the insights and visibility they need to nurture and tap into their team's network of opportunities. This role is part of the AI Insights team, which owns the services that power Affinity's industry-leading relationship intelligence platform. Our team extracts and retrieves information from billions of structured and unstructured data points to deliver insights to our customers.
Requirements
- 5+ years of experience as a Data Engineer or Data Platform Engineer, working on complex, sometimes ambiguous engineering projects across team boundaries.
- Proficiency in data modeling, data warehousing, and ETL pipeline development is essential.
- Proven hands-on experience building scalable data platforms and reliable data pipelines using Spark and Databricks, and familiarity with Hadoop, AWS SQS, AWS Kinesis, Kafka, or similar technologies.
- Comfortable working with large datasets and high-scale data ingestion, transformation, and distributed processing tools such as Apache Spark (Scala or Python).
- Strong proficiency in SQL.
- Familiar with industry-standard databases and analytics technologies, including Data Warehousing and Data Lakes.
- Experience with cloud platforms such as AWS, Databricks, GCP, Azure or related technologies.
Responsibilities
- Design scalable and reliable data pipelines to consume, integrate and analyze large volumes of complex data from different sources, supporting the evolving needs of our business.
- Help define our data roadmap. You'll collaborate with our team of machine learning engineers, software engineers, product, and business leaders to use data to shape product development.
- Build and maintain frameworks for measuring and monitoring data quality and integrity.
- Establish and optimize CI/CD processes, test frameworks, and infrastructure-as-code tooling.
- Build and implement robust data solutions using Spark, Python, Databricks, Kafka, and the AWS ecosystem (including S3, Redshift, EMR, Athena, Glue).
- Identify skill and process gaps within the team, and develop processes to drive team effectiveness and success.
- Articulate the trade-offs of different approaches to building ETL pipelines and storage solutions, providing clear recommendations aligned with product and business requirements.
Other
- Collaborate with machine learning engineers, software engineers, and product managers to shape the future of private capital's leading CRM platform.
- We iterate quickly. As such, you must be comfortable embracing ambiguity, be able to cut through it, and deliver value to our customers.
- We are candid, transparent, and speak our minds while simultaneously caring personally with each person we interact with.
- We make data-driven decisions and make the best decision for the moment based on the information available.
- Please include the word 'AI-Insights' in your answer to the first application question.
