The Hartford is seeking an AI Data Engineer to develop advanced AI systems leveraging generative AI technologies, implement AI pipelines, and integrate with their data infrastructure, specifically focusing on insurance industry use cases.
Requirements
- Experience with ETL tools (Informatica, IDMC, Talend etc.) & awareness of Big data tech stack - Hadoop, EMR & Pyspark
- Advanced knowledge of SQL as it pertains to data & analytics on any relational database Oracle, SQL Server, Snowflake etc.
- Awareness of data engineering, with at least some hands on with generative AI technologies.
- Ability to showcase implementation of production-ready enterprise-grade GenAI pipelines.
- Experience & awareness of prompt engineering techniques for large language models.
- Experience & awareness in implementing Retrieval-Augmented Generation (RAG) pipelines, integrating retrieval mechanisms with language models.
- Knowledge of vector databases and graph databases, including implementation and optimization.
Responsibilities
- Design, develop, and implement complex data pipelines for AI/ML, including those supporting RAG architectures, using technologies such as Python, Snowflake, AWS, GCP, and Vertex AI.
- Implement on end-to-end generative AI pipelines, from data ingestion to pipeline deployment and monitoring.
- Build and maintain data pipelines that ingest, transform, and load data from various sources (structured, unstructured, and semi-structured) into data warehouses, data lakes, vector databases (e.g., Pinecone, Weaviate, Faiss), and graph databases (e.g., Neo4j, Amazon Neptune).
- Develop and implement data quality checks, validation processes, and monitoring solutions to ensure data accuracy, consistency, and reliability.
- Develop complex AI systems, adhering to best practices in software engineering and AI development.
- Implement and optimize RAG architectures and pipelines.
- Develop solutions for handling unstructured data in AI pipelines.
Other
- Candidates must be authorized to work in the US without company sponsorship.
- The company will not support the STEM OPT I-983 Training Plan endorsement for this position.
- Bachelor's in Computer Science, Artificial Intelligence, or a related field.
- 2+ years of experience in data engineering
- This role will have a Hybrid work schedule, with the expectation of working in an office location (Hartford, CT) 3 days a week (Tuesday through Thursday).
