DataArt Armenia

Описание работы

We are seeking a Senior DevOps Engineer to join our team working on a high-load gambling platform, a large-scale system involving several hundred developers in development and maintenance.

/ru/senior-devops-engineer-242

Обязанности

• Deploy and configure Kafka clusters following best practices for scalability, security, and performance
• Collaborate with cross-functional teams to gather requirements and design Kafka infrastructure to meet data streaming needs
• Manage topics, partitions, replication factors, and broker configurations to ensure efficient data distribution and fault tolerance
• Implement monitoring and alerting solutions to proactively identify and address performance bottlenecks, resource constraints, and anomalies
• Conduct regular performance testing, load testing, and capacity planning to ensure clusters can handle anticipated workloads
• Design and implement high-availability strategies, including failover mechanisms and data replication across multiple data centers or cloud regions
• Develop and maintain disaster recovery plans and procedures to minimize data loss and downtime in the event of failures
• Implement and manage security measures such as encryption, authentication, and authorization to ensure data privacy and compliance with industry standards
• Stay updated on security vulnerabilities and patches, applying necessary updates to maintain a secure Kafka environment
• Diagnose and resolve Kafka-related issues, including performance degradation, data loss, and connectivity problems
• Collaborate with development and data engineering teams to troubleshoot consumer/producer application integration with Kafka
• Maintain thorough documentation of Kafka configurations, deployment processes, and troubleshooting procedures
• Provide training and knowledge sharing sessions to junior team members and other stakeholders
• Collaborate with cross-functional teams, including data engineers, developers, and system administrators, to ensure smooth integration of Kafka into our data ecosystem
• Communicate effectively with stakeholders to provide updates on Kafka performance, maintenance, and improvements
• Provide on-call support for critical issues outside of normal business hours

Требования

• Minimum of 5+ years of experience as a Kafka Administrator in production environment
• At least 2+ years of experience with AWS MSK in production environment
• Strong understanding of Kafka architecture, internals, and deployment
• Proficiency in automation tools and system administration
• Experience with Terraform
• Experience with CI/CD pipelines and containerization technologies like Docker and Kubernetes
• Familiarity with AWS services
• Excellent problem-solving and analytical skills
• Strong communication and collaboration abilities

Требуемый уровень кандидата: Старший

Дополнительная информация