Filtr uses AI to scan 1000+ jobs and finds postings that perfectly matches your resume

Leidos is seeking a Databricks Data Engineer to develop/support new/existing data pipelines, and data analytics environments in Azure cloud-based data lake to support an enterprise scale Microsoft Azure based data analytics platform.

Requirements

• Databricks Data Engineer certification and 2+ years of experience maintaining Databricks platform and development in Spark
• 3+ years of experience developing and designing Ingestion flows (structured, streaming, and unstructured data) using cloud platform services with data quality
• Proficient at Python, Spark and R are essential.
• Knowledge and experience with data governance, including metadata management, enterprise data catalog, design standards, data quality governance, and data security.
• Experience with Agile process methodology, CI/CD automation, and cloud-based developments (Azure, AWS).
• Ability to work directly with clients and act as front line support for requests coming in from clients.
• .NET based development is a plus.

Responsibilities

• Design, build, and optimize scalable data solutions using Databricks and Medallion Architecture.
• Manage ingestion routines for processing multi-terabyte datasets efficiently for multiple projects simultaneously, where each project may have multiple Databricks workspaces.
• Integrate data from various structured and unstructured sources to enable high-quality business insights.
• Implement effective data management strategies to ensure data integrity, availability, and accessibility.
• Montor and support user requests, addressing platform or performance issues, cluster stability, Spark optimization, and configuration management.
• Collaborate with the team to enable advanced AI-driven analytics and data science workflows.
• Integrate with various Azure services including Azure Functions, Storage Services, Data Factory, Log Analytics, and User Management for seamless data workflows.

Other

• Candidate must be a U.S. citizen.
• BS degree in Computer Science or related field and 3+ years or Master’s degree with 2+ years of experience
• Ability to work directly with clients and act as front line support for requests coming in from clients.
• Clearly document and express the solution in form of architecture and interface diagrams.
• The ideal candidate will be based in or willing to relocate to the Washington, DC or Indianapolis, IN area to support periodic on-site activities.