Quisitive is looking for a Sr. Data Engineer with strong AI experience to build the backbone of their data-driven solutions, playing a key role in designing and maintaining systems that power intelligent applications and knowledge management platforms.
Requirements
- Strong proficiency in Python for data processing and pipeline development
- Hands-on experience with Spark, Iceberg, and large-scale data frameworks
- Familiarity with Neo4j, LangChain, and LLM integration for AI-driven solutions
- Experience with Oracle DB, PostgreSQL, and PGVector for embedding strategies
- Comfortable working with S3 buckets, Parquet, and CSV formats
- Exposure to embedding models like BGEM 3 and Nomic
- Knowledge of Kerberos authentication and secrets management with HashiCorp Vault
Responsibilities
- Building and maintaining backend data ingestion and embedding pipelines
- Setting up environments, clone repositories, and running pipelines in JupyterHub
- Working on large-scale ETL processes, including converting Iceberg tables to Parquet and exporting data to S3 buckets
- Designing and optimizing schemas for Neo4j-based graph solutions
- Integrating knowledge workflows and KB articles into graph structures for advanced retrieval
- Troubleshooting data quality issues and optimizing Spark jobs for efficiency
- Implementing retry mechanisms and debugging full-stack issues related to large file operations
Other
- This role can be located anywhere in the United States.
- Please, no third-party agency inquiries, and we are unable to offer visa sponsorships at this time.
