Articles

CI/CD server utilization pattern differs from typical server patterns. The reason for that lays in the nature of the workload that such servers usually execute. There are two major types of such workloads which commonly present:

• build, test and deploy artifacts;
• execute deployed test environments for git branches, commits, configurations, etc.

Usually, it is better to split the workloads listed above, so that they would run on different servers. The main argument for doing that is based on the expectation that the build workload strives to utilize as much compute resources (CPU, IO) as possible to complete the job as soon as possible. When the workload of the second type is also run on the same server, engineers must enforce limiting policies to protect deployed environments from the starvation caused by excessive server resources utilization induced by the build workload.

The problem of a partial project build in CI/CD system can be pretty complex. It covers several areas, but most important are the build time spent to build the project when the changes detected, and the artifacts deployment impact on production and staging systems during their deployment.

The problem is easily addressed when such build systems as GNU make, Maven, NPM or others are in use. They manage dependencies automatically for certain types of projects. But when the build system does not support an automated dependency management for your project it is a big deal to implement a build in an efficient and performant way. Otherwise, the build will be slow, the artifacts will be duplicated in artifact repositories, connected services may be restarted even when there are no changed modules to deploy.

There are several approaches to decrease the impact of such excessive builds and all of them are based on the implementation of dependencies management:

• manual dependencies tracking;
• Dockerfile-based build and deploy;
• automated checksum-based dependency management.

All these approaches have pros and cons and must be carefully analyzed before the implementation. Let’s observe each of them briefly.

QCOW2 is a very popular format for virtual machine volumes which is used widely by service providers, corporates and individuals. It’s very convenient because implements thin provisioning, supports internal snapshots. Volumes are just regular files which can be kept in local filesystems or remote NFS storages. These features have made it the de-facto standard for use with KVM hypervisor which is also used very widely.

QCOW2 is a pretty reliable format, but from time to time disks in QCOW2 format may become corrupted. From our experience, the problem happens frequently when a user tries to snapshot a virtual machine disk which is under high IO load or when whole underlying storage is under high IO load. Sometimes it leads to broken volumes which cannot be attached to VM and VM cannot start with them.

So, it simply happens. Maybe it will be fixed in future Qemu versions and the article will be unuseful, but there are a lot of different versions of Qemu are deployed around the world and the problem mentioned may happen with VMs under their management. In this article, we demonstrate how to recover the problem.

GitLab CI provides a convenient mechanism for storing build artifacts in GitLab. An artifact is any file that a developer wants to store for a long time after a build completes successfully. This mechanism can be used to store distributions and various build logs. Artifacts are easy-to-use. In this article, using the example of publishing a built DEB-package that is downloadable from GitLab we will show how artifacts storing is implemented.

GitLab CI artifacts are not always convenient to use. Sometimes you may need to enable FTP, SFTP, publishing to Nexus or other artifacts management systems. However, often it is just enough to have a storage that enables the access to build results.

Not all parameters are described in the article, but only those that you usually need to change when configuring new CloudStack clouds. The parameters for the system are applied after reload of each CloudStack Management server.

CloudStack version: 4.11.1

The parameters described in the article can be configured at: Global Settings