Schwarz Partners is looking to hire a Data Engineer to build data pipelines that transform raw, unstructured data into formats that can be used for analysis, and to create and maintain the analytics infrastructure that enables almost every other data function. This includes architectures such as databases, servers, and large-scale processing systems. The Data Engineer will use different technologies to collect and map the organization’s data landscape to help decision-makers find cost savings and optimization opportunities, and to display trends in collected analytics information, encouraging transparency with stakeholders.
Requirements
- Demonstrated experience with SQL in a large database environment.
- Direct experience utilizing SQL to develop queries or profile data.
- Experience in quantitative and qualitative analysis of data.
- Experienced level skills in Systems Analysis.
- Experienced level skills in Systems Engineering.
- Strong grasp of OneLake concepts: lakehouses vs. warehouses, shortcuts, mirroring, item/workspace structure.
- Hands-on with Delta Lake (Parquet, Delta tables, partitioning, V-ordering, Z-ordering, Vacuum retention).
Responsibilities
- Build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and cloud technologies.
- Assemble large, complex sets of data that meet non-functional and functional business requirements.
- Identify, design, and implement internal data-related process improvements.
- Conduct configuration & design of application to better leverage the enterprise.
- Prepare data for prescriptive and predictive modeling.
- Ability to advise and implement on improvements to data warehousing and data workflow architecture.
- Document high-level business workflows and transform into low-level technical requirements.
Other
- Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues.
- Use effective communication to work with application vendors.
- Assist in the creation and quality assurance review of design documents and test results to ensure all project requirements are satisfied.
- Think outside the box and come up with improvement and efficiency opportunities to streamline business and operational workflows.
- Ability to analyze complex information sets and communicate that information in a clear well thought out and well laid out manner.
