Leidos needs an ETL Developer SME to shape, lead, and implement cutting-edge data flow solutions centered around Apache NiFi for Extracting, Transforming, and Loading (ETL) data into an enterprise Data Lake, driving critical customer needs in National Security.
Requirements
- Extensive experience designing, developing, and managing complex NiFi data flow solutions in large-scale enterprise environments.
- Extensive knowledge of NiFi architecture, processors, and configurations, along with hands-on experience with NiFi Registry and clustering for high availability and scalability.
- Proficiency in programming languages like Java and Python for custom NiFi processor development and scripting for automation.
- Proficiency writing and optimizing complex queries, along with experience in managing relational and NoSQL databases (e.g., Postgres, Elasticsearch, DynamoDB).
- In-depth experience with real-time streaming, and API integration (REST) for seamless data connectivity.
- In-depth experience with cloud platforms like AWS, Azure, or OCI and related data services
- Strong ability to analyze complex data challenges, identify root causes, and implement effective solutions.
Responsibilities
- Architecting and leading complex NiFi data pipeline design: Design and develop enterprise-level ETL architectures and implement NiFi data pipelines for large-scale data ingestion, transformation, and processing from diverse sources.
- Performance optimization and tuning: Lead the optimization of NiFi data flows, including processor tuning, memory management, and load balancing, ensuring optimal performance for batch and real-time processing.
- Advanced troubleshooting and problem resolution: Identify, diagnose, and resolve complex NiFi data flow issues, including performance bottlenecks, data discrepancies, and integration failures.
- Integrating with big data and cloud technologies: Seamlessly integrate NiFi with various databases, big data ecosystems, and cloud platforms (e.g., AWS, OCI, Azure), demonstrating expertise in relevant services (e.g., Kafka, Elasticsearch, S3, SQS/SNS).
- Defining best practices and standards: Lead the establishment of best practices for NiFi development, deployment, security, and governance, ensuring adherence to enterprise-wide data management policies.
- Documentation and knowledge sharing: Create and maintain comprehensive documentation for NiFi data flows, mappings, architectures, and standard operating procedures, ensuring knowledge transfer and promoting efficient team operations.
- Mentorship and team leadership: Mentor junior developers, provide technical guidance, conduct code reviews, and foster a collaborative learning environment.
Other
- To be considered must have an active TS/SCI with polygraph security clearance
- Master’s degree with 15 or more years of prior relevant experience or Doctorate with 13 or more years of relevant experience. Will consider experience in lieu of a degree.
- Strong ability to collaborate effectively with cross-functional teams, articulate technical concepts clearly, and provide effective mentorship.
- Collaboration and communication: Collaborate effectively with data architects, data engineers, application/service developers, and other stakeholders to translate business requirements into robust technical solutions and effectively communicate complex technical concepts to both technical and non-technical audiences.
