Recent Posts

Atlas of global Apache CloudStack configuration variables, defined at the cluster level

The article describes not all the parameters, but only those that you usually need to change when configuring new CloudStack clouds. The settings for the cluster are applied without restarting the CloudStack management servers and only affect the cluster for which it is installed.

CloudStack Version: 4.11.1

The parameters descibed in the article can be configured at: Infrastructure / Clusters / Cluster Name / Settings Tab

Read more ...

DC/OS: Universal environment for development, testing and execution of applications in the world CI/CD and DevOps

If you face the requirements to run a big bunch of environments for a single app, if you work with microservice architectures, have diverse of apps or an environment where an app is deployed for every tennant, if you develop a high performance scalable application, it means that DC/OS is what you should pay attention to.

Read more ...

Apache CloudStack extension for hybrid event bus

logo
Today we would like to share a new plug-in event bus module – a hybrid event bus that combines two buses: inMemory Event Bus – to handle events inside CloudStack, and Kafka or RabbitMQ Event Bus – to interact with external handlers.

We use Kafka Event Bus in our CloudStack installations. Recently, after testing a new plug-in in the product environment, we found that this bus does not have event subscription functionality: internal event receivers can not subscribe to events, as there is simply no implementation.

This is a nasty thing, because it limits the CloudStack API. It is almost impossible to predict who would like to subscribe to events even inside the CloudStack core, for example, to implement deferred processing. Using the standard Kafka Event Bus, some of the event bus capabilities are no longer available, and troubleshooting this regression is very difficult. To overcome this, we implemented a “hybrid” bus that uses the inMemory bus for internal purposes and the Kafka or RabbitMQ bus for external communications.

You can find the extension on GitHub published under Apache 2 License.

Successful completion of the project to create a personalized ad platform

After six months of development, the project of ONELYA, in which our team acted as a key subcontractor, was successfully completed. Purpose of the project – creation of an advertising platform designed to generate personalized advertising offers based on automatic analysis of user behavior on sites and mobile applications. In this project we managed to demonstrate not only our well-known abilities to develop high-performance, scalable advertising delivery platforms, but also to apply modern approaches based on machine learning and real-time data analysis.

The project from our side was developed by a team of sixteen professionals:

  • two project managers;
  • three backend engineers;
  • two frontend engineers;
  • three QA engineers;
  • three ML engineers;
  • systems analyst;
  • technical writer;
  • DevOps expert.

Delivery time: more than six man-years.

Key technologies:

  • Languages: Scala/Akka, Python3, TypeScript;
  • Repositories: Apache Cassandra, Elasticsearch, HDFS, RocksDB;
  • Message Queuing: Apache Kafka;
  • Frameworks for big data: Apache Spark, PySpark;
  • DBMS: PostgreSQL, Liquibase;
  • UI: Angular 6, Fingerprint2, Swagger;
  • Infrastructure: Ansible, DC / OS, Docker, HAProxy, NGINX;
  • Automation E2E QA: Postman, Protractor;
  • Management: Gitlab, Gitlab CI, Atlassian JIRA, Atlassian Confluence.

Practices:

  • Agile/Scrum;
  • Continuous integration, continuous delivery;
  • Git-flow;
  • automated testing of backend, frontend, ML;
  • infrastructure as a code.


Automated testing for young and daring

— We will code tests later...
— Sorry, I’m not going to buy it.

Probably you are a programmer, a chief or a customer. The most important thing is that you are connected with software development, and just today you said it yourself or heard from someone something like that:

  • Now, we are developing the code, but we will develop tests later since currently, everything is changing. But as soon as changes stop, we will implement them immediately.
  • Let’s skip test design because it will slow us down. In this case, we will be able to move forward much faster.
  • Unfortunately, we don’t have any time for implementing tests, so we will develop only code… If we have time, we will do it.

There are many variations, but a result is almost always the same – later or never. When we are at the beginning of our career path, we are young, full of strength, self-confident, our judgment isn’t clouded, we are steeled by struggling with university laboratory researches, on which we worked all last days and maybe even weeks. It seems like the sky’s the limit for us. And we code so fast, that nothing is necessary but paper and that’s it. What tests, I beg you. It’s much faster to do it on your own rather than explain how to do it. Neglect of testing is growing inside: “It’s for losers, I don’t need it as I already develop it perfectly without it.“

Read more ...