OMD LLC

Описание работы

OneMarketData is continuously searching for bright talent with the skills to make an impact. From developers to data scientists, at OneTick you will have the opportunity to develop and enhance your problem-solving skills using a combination of analytics, imagination, and talent.

Overview
Our DevOps engineers team develops the infrastructure behind the hosted solutions and our software and data delivery lifecycle. Inside the company, we have two main directions - Cloud and Solutions.

Prior to advancing with your application, we kindly request that you review the CONSENT NOTICE FOR HR AND RECRUITING provided by OneMarketData. Your attention to this matter is greatly appreciated.

Our stack:

• AWS (some of the services we use are: EKS, EC2, S3, SGW, ASG, ALB, Lambda etc.);
• Terraform and Ansible as an IaC approach;
• Gitlab and Gitlab CI/CD;
• Python for programming and scripting purposes;
• Docker based (docker-compose, docker airflow plugin) containerization;
• Kubernetes (mostly EKS, but GKE and other Kubernetes engines are also being used) for Orchestration and Helm for its management.
• Prometheus/Victoria Metrics, Grafana, Loki, AWS CloudWatch and CloudTrailfor monitoring, logging and some statistics collection;
• Airflow for product data processing and MLFlow for Machine Learning;
• Jupyter and JupyterHub for data analysis;
• OneTick (time-series DBMS for market data);

Some other tools for different purposes - i.e Packer, HashiCorp Vault, Buildah, OpenVPN, Slack, Confluence and other popular and well known instruments :)

More information about the projects
In the Cloud Project, we have multi-account AWS infrastructure managed by AWS organization. Separate AWS accounts are necessary to host customer-facing environments. We have been providing for our customers different kinds of setups for our application. In general we use most of all common AWS resources like EC2, EKS, S3, VPC, ELB, etc but also the stack of AWS resources is pretty wide. Most of our AWS infrastructure is covered by IaC. CI/CD is running on GitLab.
We have more than 2 petabytes of data in S3 and EFS. We expose part of the data in s3 to the file system using Storage Gateways. And currently we are in the process of migration from setup on virtual machines to kubernetes, integration centralized logging and monitoring solution, migration of data loading process to airflow and optimizing infrastructure costs planning to improve performance at the same time.

Solutions project can be defined as a kind of company inside a company. We are creating highly demanded Fintech products used by the largest financial institutions in the world (banks, exchanges, investment funds, etc). The product contains ETL backend which is processing large amounts of data generating market alerts and analytics, and web browser frontend which shows all this data in user-friendly format and gives possibilities to customize and configure it. For customers, we offer different types of implementation - Hosted Solution, which is being hosted on our side; PaaS, which is being hosted in the customer's cloud and Deployed Solution, which can be deployed on the customer's on-prem infrastructure. We are automating everything to make the installation and configuring process for the customer as easy as possible and reduce the need for support to a minimum. As a DevOps engineer, you will work with a lot of popular tools - AWS (and possibly with other cloud providers for PaaS customers), Packer, Terraform, Ansible, GitLab CI/CD, Docker, Kubernetes; also we are using Apache Airflow, Tomcat, Grafana, Prometheus, Loki. The main programming language both for infrastructure and application is Python. Also, we're using bash for some kinds of tasks. You will work with a very strong team, where you won't need to explain to developers what Kubernetes is or how Docker works, and top management also has a really strong technical background.

Currently we are working on Kubernetes migration for most of our services including Airflow, MLFlow, Jupyter Hub and our OneTick product itself while also improving our internal release processes, CI/CD, monitoring and logging and so on. We are always trying to implement the best solutions for our product and internal environment.

/ru/devops-engineer-899

Обязанности

• Improving and extending our AWS infrastructure;
• Implementing Kubernetes, create and upgrade Helm charts for different services;
• Implementing and improving CI/CD pipelines for our environments, expanding the capabilities of our release mechanism;
• Improving logging and monitoring systems for the existing infrastructure and services;
• Upgrade the current infrastructure with new ideas to reduce costs and increase performance;
• Create architecture from scratch for large PaaS customers minimizing the support needance;
• Helping the developers to simplify and customize the development process;
• Helping with troubleshooting in case of AWS related issues.

Требования

• Experience more than 3 years.
• Experience with Linux system administration and web services Tomcat, NGINX, Apache Web server, etc.).
• Experience with general resources of AWS.
• CI/CD (Gitlab CI/CD is preferable).
• Experience with networks (AWS, routing and troubleshooting)
• Docker, Kubernetes.
• Experience with Bash and Python
• Experience with logging and monitoring tools is a plus.
• Experience with Terraform and Ansible

Требуемый уровень кандидата: Средний уровень

Дополнительная информация