Scrunch needs to acquire and process a lot of data using LLMs and traditional analytics methods to deliver roadmap features and provide customers with tools for the AI era.
Requirements
- Have extensive experience building & using web APIs, including handling authentication, authorization, caching, rate limiting, etc.
- Experience with relational (transactional) databases, particularly PostgreSQL.
- Analytical SQL, maybe with a modern analytical data platform like Snowflake, Google BigQuery, DuckDB, Spark SQL, etc.
- Experience with analytics packages like Pandas or Polars is a plus.
- Experience with streaming analytics is a plus.
- Big data tools like Flink, Beam (GCP Dataflow), Spark, etc.
- Experience using LLMs and/or traditional ML techniques to build classifiers, data extractors, summarizers, etc.
Responsibilities
- building features related to our data pipelines, usage of LLMs, analytics APIs, etc.
- building features related to React
- creating notifications
- creating alert emails
- creating scheduled reports
- creating connectors into third party systems
- writing batch and streaming/interactive data processing pipelines
Other
- This role is open to candidates located in and legally authorized to work in the US.
- Scrunch is a remote organization, but in-office or hybrid work is possible for candidates based in the NYC or Salt Lake City metro areas
- The ability to quickly understand source data and validate output data quality
- Not required but a big plus: experience building evals for LLM-based tools to assess and improve accuracy at these kinds of tasks
- Scrunch AI is an equal opportunity employer.
