SunnyData is looking to solve customer data engineering and data architecture needs with a focus on Databricks solutions.
Requirements
- Expertise in Data Engineering technologies (e.g., Spark, Hadoop, Kafka), Databricks platform, and data science/machine learning technologies (e.g., pandas, scikit-learn).
- Proficiency in Java, Python, and/or Scala.
- Cloud Platforms: AWS, Azure, and/or GCP.
- SQL Expertise: Ability to write, debug, and optimize SQL queries.
- Familiarity with cloud and distributed data storage systems such as S3, ADLS, HDFS, GCS, Kudu, ElasticSearch/Solr, Cassandra, or other NoSQL storage systems.
- Experience with data integration technologies like Spark, Kafka, Streamsets, Matillion, Fivetran, NiFi, AWS Data Migration Services, Azure DataFactory, and Informatica Intelligent Cloud Services (IICS).
- Expertise in automated data transformation and curation using tools like dbt, Spark, Spark streaming, and automated pipelines.
Responsibilities
- Design, build, and deploy comprehensive data solutions that capture, transform, and leverage data to support AI, ML, and business intelligence initiatives.
- Architect data solutions, incorporating best practices in data governance, security, and quality.
- Evaluate data sources for their value, recommending data inclusion strategies to enhance analytical processes.
- Perform system analysis, assess and resolve data and system defects, and apply appropriate corrections.
- Test data movement, transformation code, and data components to ensure accuracy and reliability.
- Lead internal teams, provide direction and mentorship to project teams to deliver solutions and educate end users on data products and analytic environments.
- Manage multiple customer accounts, ensuring timely delivery of solutions and tracking progress to report outcomes.
Other
- 4-year Bachelor's degree in Computer Science or a related field.
- 7+ years as a hands-on Solutions Architect and/or Sr. Data Engineer, designing and more recent experience implementing data solutions with a focus on DataBricks.
- Strong written and verbal communication skills with experience in client-facing roles.
- Ability to create and deliver detailed presentations to clients and stakeholders.
- Experience leading teams and mentoring other engineers.
