The company is looking for a Kafka Messaging Troubleshooter and Kafka Developer to identify and resolve issues within their Kafka messaging systems and to develop and implement Kafka-based data solutions.
Requirements
- Deep understanding of Kafka architecture: Thorough knowledge of how Kafka components like brokers, topics, partitions, consumer groups, and replication factors work together.
- Log analysis: Ability to interpret Kafka logs from producers, consumers, and brokers to identify error messages, warnings, and potential issues.
- Monitoring and metrics: Familiarity with monitoring tools to track key Kafka metrics like consumer lag, message throughput, broker CPU usage, and network latency.
- Distributed systems knowledge: Understanding of concepts like fault tolerance, data replication, leader election, and distributed consensus to troubleshoot issues related to cluster failures.
- Programming language proficiency: Strong coding skills in Java or Scala, as many Kafka applications are written in these languages, allowing you to debug custom producers and consumers.
- Network troubleshooting: Ability to diagnose network connectivity issues between brokers and clients, including checking network configurations and firewall rules.
- Kafka configuration management: Knowledge of Kafka configuration parameters, including topic creation, partition settings, replication factors, and consumer group settings.
Responsibilities
- Deep understanding of Kafka architecture: Thorough knowledge of how Kafka components like brokers, topics, partitions, consumer groups, and replication factors work together.
- Log analysis: Ability to interpret Kafka logs from producers, consumers, and brokers to identify error messages, warnings, and potential issues.
- Monitoring and metrics: Familiarity with monitoring tools to track key Kafka metrics like consumer lag, message throughput, broker CPU usage, and network latency.
- Distributed systems knowledge: Understanding of concepts like fault tolerance, data replication, leader election, and distributed consensus to troubleshoot issues related to cluster failures.
- Programming language proficiency: Strong coding skills in Java or Scala, as many Kafka applications are written in these languages, allowing you to debug custom producers and consumers.
- Network troubleshooting: Ability to diagnose network connectivity issues between brokers and clients, including checking network configurations and firewall rules.
- Kafka configuration management: Knowledge of Kafka configuration parameters, including topic creation, partition settings, replication factors, and consumer group settings.
Other
- Work with the business and IT team to understand business problems, and to design implement, and deliver an appropriate solution using Agile methodology across the larger program.
- Ability to work in fast-paced and dynamically changing environment.
- Ability to lead the effort; and work with minimum supervision.
- Strong problem-solving skills and a passion for debugging complex issues and mature code.
- Experience with using agile methodologies for software development.
