We have been using CI/CD in all of our projects for the last four years, and selectively started to use it eight years ago. We have achieved significant features delivery pace with mastering CI/CD practices. It also helped us to increase the quality of delivered software products. If you are looking for the implementation of CI/CD tools and processes in the engineering teams, take a look at the following factors, which influence on its success and benefits. Ignoring those factors likely leads to failures and inefficiency during CI/CD implementation. We have discovered those success factors after several years of trials, but you will decrease the efforts and the time of adoption greatly, will avoid typical mistakes following them.
Project Participants Interest
Implementing CI/CD for the first time, ensure all project participants are interested in, otherwise it will not work. Before the beginning, find proper literature, articles, success stories and kindle the participants with the idea of CI/CD. If you can invite CI/CD enthusiast with broad experience who can give a talk about his/her success with those practices and answer the team’s questions or even join the team, it will increase success chances of the implementation greatly.
The team must clearly understand the expected positive results from the CI/CD approach usage before it will start using it. Ignoring this bootstrap may lead to the perception of CI/CD practices as an externally enforced process without real value.
Proper CI/CD Platform Selection
Right CI/CD platform choice is a key factor for successful implementation. A platform selected must be integrated. This means that it binds CI/CD tools with other necessary development products, like:
- version control systems like Git, SVN, Mercurial;
- task management system – Redmine, Gitlab issues, Github issues, Atlassian Jira;
- artifact management – Sonatype Nexus, Docker Registry, FTP or HTTP catalog;
- code testing tools;
- code coverage tools;
- documenting and knowledge base – Wikimedia, Atlassian Confluence, GitHub Wiki, Gitlab Wiki.
Developer tools vendors provide such integrated platforms. Most popular platforms used by more than 100000 developers around the globe are Gitlab CE/EE, Atlassian family (Confluence, Jira, BitBucket, Bamboo).
Atlassian sells the most integrated product set, but it’s pretty complex, requires trained product administrator and ownership cost depends on the number of seats. Other vendors don’t provide such integrated products as Atlassian does, but next obvious leader is Gitlab. Even Community Edition (CE) version provides a solid feature set: a version control system, task management, artifact storage for Docker, CI/CD, code testing, and coverage. All tools are integrated into a single product which is opposite from Atlassian’s approach – a single product for a single purpose. Gitlab CE is a preferred choice for small and medium teams. It is available as a cloud service or self-hosted solution.
Historically, a lot of developers use an open source tools Jenkins for CI/CD purposes. Currently, we in Bitworks think it’s time is over and it should not be used in new projects, especially if your team is new to CI/CD. Despite its flexibility and integration capabilities the fact it is a very standalone product and has very weak integrity with other software developer tools. We mean that it is very capable of integration but it doesn’t provide the best practices and straightforward approaches for CI/CD pipeline implementation. For the team, which strives to get the best from CI/CD in a short time, without running excessive activities it’s a disadvantage.
During the last two years, we moved all our CI/CD pipelines from Jenkins to Gitlab CE which greatly simplified our administrative routines and IT infrastructure.
Implementing Of Required Processes and Rules
Development processes determine the value which CI/CD can give a team. If a temp doesn’t have specific development rules, CI/CD implementation could be inefficient. We define a set of rules, which are influential for a successful CI/CD implementation.
Automate Everything
If the applications which you deliver cannot be deployed and set up without the requirement for special knowledge which only specially trained person has then CI/CD will not be successful. You should put efforts to automate deployments for every delivered application for every environment in either production or testing and integrate those deployment scenarios into CI/CD pipelines. You should have absolutely proven scripts for every deployment and every team member must be able to use them.
Use TDD, Integration and E2E Testing
CI/CD have zero value without unit and integration testing. To achieve even better results, implement E2E tests and embed them into your build pipelines. Unit-tests and integration tests must be created by the software developers together with the code for the delivered features. E2E tests are designed by developers in test and are integrated into the build scenarios of software branches which should be tested against regressions and where use case test scenarios must be checked.
Gitlab provides developers with the functionality to monitor a code test coverage. This mechanism helps managers monitoring the overall state of a test plan. We use it in many of our projects. Code Coverage is a very useful instrument for visual quality analysis.
Take a look at Coveralls service, which is free for open source projects and is very affordable for commercial use.
Establish a Source Code Branching Process
Use a process for source code development based on formal tasks, it normalizes team activities and supports frequent changes integrations into the main code branch. We often use Git-flow in Bitworks. Other teams can also take a look at Github Flow, Trunk-based Development or develop own process. The most important requirement is that the process must be prescriptive, definitive, answer all the controversial questions and all team members should understand its rules the same way.
Begin Doing a Code Review
Code review is a very important instrument for task quality and completeness estimation. This process ensures that only properly developed code which adopted best practices and satisfies business and technological requirements is integrated into the main code branch. Without reviews, CI/CD is almost a waste of time. Every task must pass a review. In a small team, a review is handled by a technical leader, in a medium one – by the technical leader and another experienced engineer.
When reviewing, try to guide team members through improving their skills toward best practices. Proper reviewing process enables faster education for engineers and provides a means to estimate the contribution efforts of every team member.
Use a Code Style
Every programming language provides developers with code style analysis tools. Choose the one for your programming language, configure, discuss the rationales behind with team members and add code style checking into the building pipelines. It helps to prevent of integration of bad style source code into the main branch.
Code style checking is an instrument for automatic quality check and control, which offloads reviewing people from doing extra work, blocking bad style code on the previous step.
Investigate Unsuccessful Builds
Every unsuccessful build event is a reason to start an investigation. If a build fails is stable, that is good from the point of view, that you can observe and improve until the fix of the problem, but if it fails occasionally it’s a very bad symptom, we recommend push extra efforts discovering and improving what’s wrong. Such strange fails influence very badly in the long term and often are evidence of race conditions, memory leaks, weak error handling and other serious quality problems. Ignoring failing build investigations leads to decreasing of CI/CD value. It’s a sign for the team that even code, which causes errors can be integrated into the project.
CI/CD practices require the highest development discipline to be useful. If you are not able to keep that, CI/CD process is a waste of time.
Decompose a Project
Split a project code into loosely coupled components which can be built separately, implement CI/CD for each of them. Implement an isolated deployment for every such component. Use microservice architecture when applicable and team skills are strong enough for managing that design.
High Performance CI/CD Compute Equipment
Dedicate enough server compute and storage resources for running CI/CD pipelines. The better test suite developed the more time CI/CD pipeline will be executed. If a build requires a lot of time then when it fails, the debugging is quite annoying and ineffective. It will lead to a lack of team’s motivation for CI/CD using.
Workstations also must be modern and high performing, so the developer must be able to run CI/CD locally before submitting their code for integration.
The faster CI/CD builds go, the more chances that developers will love it.
Fast Network Connection
Modern libraries and other software products rely on dozens or even hundreds external dependencies, so CI/CD operations can generate a significant load on the network line. Slow network connection leads to very long build times. Use local caching to decrease artifact download time and acquire top class network connectivity to decrease build times. If the link has lack of connectivity or poor access to certain network segments, it will lead to unpredictable build failures because of dependencies download problems. As a result, the team will be disappointed in CI/CD.
Artifact Management Implementation
The purpose of CI/CD is producing of build artifacts. Artifacts are everything – compiled code, transformed and generated files, different kinds of reports like build report, coverage report, and other build outcomes. Artifacts must be placed into artifact storages where they are accessible in a convenient way for stakeholders. Being poorly organized, artifacts are not accessible easily for the persons and machines and this places CI/CD practices under attack. E.g. test coverage report must be available to the reviewer in the form of web-pages, accessible by a hyperlink which is located in a build summary, otherwise, the person will not use it and coverage analysis just will be ignored.
If you develop code in Java, provide Sonatype Nexus, for Docker-based delivery – create Docker Registry, provide a proper repository for every artifact. Think about artifact access ways, develop removal policy for outdated files, describe artifact storages in the project’s knowledge base.
Implement Deployment Into Testing and Production Environments
Achieve fully automated deployment for all required branches without human assistance. Your QA engineers, product team, developers and other stakeholders must get access to new versions of the product as fast as possible. This approach speeds up the development, decreases a time required to elaborate a solution, delivers new features to the users immediately after code integration and regressions testing.
Design an CI/CD model
An example of the CI/CD model, used in one of our projects CI/CD are displayed below:
Explain the diagram to all project participants. Schedule the training on the use of the model for all stakeholders. Ensure, developed CI/CD model reflects project needs and all stakeholders are agree with the way it works. Review the model as often as it is required by the changing project demands.
Deepen Your Knowledge in Used CI/CD Tools And Create New Ones
CI/CD problems, you tackle with, are not unique – other people met them as well. Tools vendors try to address all typical cases and include demanding features in new versions. If an instrument has lack of required functionality, please be collaborative and put extra efforts into promoting required function to developers, assisting in the creation, publishing it if you created the function yourselves. By doing that you not only bring the value to the developer community but also expand your borders in understanding how other teams solve similar problems.
Try to analyze your CI/CD processes continuously, improve them where possible and look for new possibilities for productivity growth.
Read more about CI/CD and DevOps:
Please, share the post with friends if you like it.