Maintaining the SC Enterprise Information System (SCEIS) and empowering decision makers to improve government for the citizens
Requirements
- Proven experience with data lakehouse architectures
- Expertise in designing and optimizing ELTETL workflows in distributed computing environments
- Proficient in writing modular, reusable, and testable code for data ingestion and transformation
- Python and SQL. Familiarity with Java is a plus
- Hands-on experience with Apache Spark
- Understanding of batch and real-time data processing paradigms
- Strong grasp of data modeling concepts (dimensional, normalized/denormalized, star/snowflake schemas) and data warehousing tools and concepts
Responsibilities
- Design, build, and test ELTETL pipelines responsible for data ingress, curating, and warehousing of data based on functional and non-functional requirements
- Develop and maintain enterprise data cataloging processes to ensure datasets are properly documented, discoverable, and accessible
- Develop and manage the deployment of data pipelines and applications through CICD pipelines andor GitOps based automation
- Work with architecture, productservice owners, and the development team to clarify work item functional and non-functional requirements
- Evaluate new and emerging technologies to determine which ones can help innovate andor implement new feature requests
Other
- A bachelor's degree in computer science, information technology, or a related technical field and 2 years of experience in cloud-based data platforms
- Applicants indicating college credit or degree(s) on the application will be required to bring a copy of college transcript to the interview
- Participate in peer code reviews to ensure code quality and consistent use of patterns and styles
