The Metropolitan Transportation Authority (MTA) needs to design, build, test, and deliver end-to-end, automated data pipelines over complex on-premises and off-premises platforms to extract data from multiple source systems, making it consistent, reliable, available, and usable for internal stakeholders and external parties, supporting the agency's and New York State's Open Data goals.
Requirements
- Experience developing front end data apps, from scratch, not just working on existing ones.
- Strong design skills, who can make calls about how things should be visually communicated.
- Experience actually hosting and deploying apps like this, in production, ideally where they have owned a significant part of the deployment/hosting.
- Strong skills in programming, database design, and data lake architectures for data engineering.
- Ability to work with data of different types — structured, semi-structured, unstructured — as well as from distinct disciplines — transportation, finance, HR, asset maintenance.
- Experience with data engineering orchestration tools (e.g., Airflow, Dagster, etc.) and DevOps tools (e.g., ADO, Git, Jenkins, Docker, etc.) and familiarity with related concepts (e.g., CI/CD).
- Extensive experience and ability to design and implement quality assurance and automated testing systems.
Responsibilities
- Lead projects to develop data pipelines, data warehouses, data marts, multi-dimensional cubes, and data lakes to collect, structure, and integrate a wide range of data sources; create data system assets that are efficient, timely, reliable, and accurate, robust, and scalable; and make information available to staff and decision-makers for analysis and consumption.
- Identify, propose, support, and carry out the development of new data sets, data access, extraction methodologies, algorithms, and other tools that support the agency's strategic goals, service quality, and efficiency, while advancing data-driven decision-making across the company through stakeholder partnering, tool design, and effective communications.
- Test data sets and pipelines, conducting root cause analyses to resolve issues, incorporating quality assurance functionality in work products, and finding enterprise-class solutions to project challenges where possible.
- Evaluate existing legacy data, algorithmic, and process solutions; redesign and implement modern data infrastructure to improve efficiency, timeliness, quality, robustness, and scalability.
- Play a lead role in developing standards for data architecture diagrams, system documentation, data models, business decision-making maps, and other design related artifacts to ensure consistency and improve understanding of both technical and non-technical users of Data & Analytics products.
- Provide input into data governance initiatives and influence the development of sustainable data management and governance practices across the agency while ensuring data governance and sustainability needs are built into project designs.
- Research and recommend the best pipeline technologies, toolsets, and applications to support the data science and reporting teams, keeping skills current and maintaining data systems in the latest technologies and cloud platforms.
Other
- Position at MTA Headquarters
- This position is eligible for teleworking, which is currently one day per week.
- Ability to play a lead role on the Data & Analytics team to set priorities and overall program planning in support of business goals.
- Knowledge of transit/ transportation systems and excellent judgment on how to manage or overcome technical, organizational, or governance constraints to meet business goals.
- Ability to collaborate and provide support to all levels of MTA, both technical and non-technical.
