We often develop highly available, scalable systems for different markets like fintech, adtech and internet services. Those systems are developed with industry proven, solid open source products and architectures which efficiently solve common demands. Specifically, there are a lot of well-known players here, like lambda-architecture for streaming data processing or microservice architecture for horizontally scalable high performance services, noSQL solutions like Redis, Aerospike, MongoDB, message brokers (Kafka, RabbitMQ), distributed coordination and discovery servers (Apache Zookeeper, Consul) and others. Those building blocks guarantee successfull application delivery in most of the cases and project team doesn’t spend development efforts for delivery of low level components, spending most of the time for the requirements development.
Sometimes, however the tasks occur which hardly can be solved with generic, well known building blocks mentioned before, and the need takes place in the development of the middleware or the search of appropriate very specialized means. The story is about our experience solving such kind of tasks.