Epic Games needs a Data Engineer to design, build, and maintain their data infrastructure to ensure the reliability and efficiency of data and systems used by the Ecosystem Security Core team, combating bad actors and keeping the platform safe for all users.
Requirements
- Experience of ETL technical design, automated data quality testing, QA and documentation, data warehousing, and data modeling
- Experience with Python for interaction with Web Services (e.g. Rest and Postman)
- Experience with using AWS, Databricks, Snowflake, Elastic or other comparable large scale analytics platforms
- Experience monitoring and managing databases (we use Elasticsearch/MongoDB/PostgreSQL)
- Experience with SQL
- Experience with data versioning tools
- Experience developing and maintaining data automation infrastructure for streaming & ETL pipelines, such as Apache Airflow
- Experience with GraphDB would be a plus
Responsibilities
- Design and implement automated end-to-end streaming & ETL process to prepare data for machine learning and ad-hoc analysis, including data anonymization
- Manage and scale the tools and technologies that we use to label data, which run on AWS
- Devise database structure and technology for storing and efficiently accessing large data sets (millions of records) of different types (text, images, videos, etc.)
- Use and implement data extraction APIs
- Write and invoke custom SQL procedures
- Support data versioning strategies using automated tools, such as DVC
- Support devising strategies for labeling of new data by humans
Other
- Strong analytical background: BSc or MSc in Computer Science/Software Engineering or related subject - candidates without a degree are welcome as long as they have proven extensive hands-on experience
- Demonstrated experience collaborating with product teams to understand how safety systems integrate with broader data systems
- This role is open to multiple locations across North America and Europe (including CA & WA).
