In recent years, requirements for the software delivery pace have significantly gone up. At the same time, customers expect the same highest quality and the most frequent integration of changes into a product.
Suppliers have to improve development approaches to meet clients’ expectations on an efficient supplier and meet personnel expectations on an innovative and modern organization as well. Under the influence of these trends, traditional development approaches, where a development team consists of separated departments with highly specialized engineers working like an assembly-line (develop - test - deploy - run), start transforming into more universal units of mixed specialization.
A phalanx of approaches and practices appeared that solve problems related to new clients’ demands CI/CD, DevOps, agile methodologies etc. However, we often observe how efforts of new practices implementation frequently fail or turn into profanation, into a compliance certification (that’s it ISO9001, CMMI) which is waved right before a customer’s face but, nevertheless, an organization works without any changes. Do you remember this anecdote where 2 buddies have a conversation:
Person A: From now on I’m a professional photographer!
Person B: Really? How have you achieved it?
A: I’ve bought a professional camera.
Everybody remembers that 5-7 years ago every other person before 30 was a “professional” photographer. Now, it is the same, administrators suddenly became DevOps engineers, companies started using DevOps approaches. Although by virtue of last two years of communication, participation in conferences, giving speeches at reading forums, I’m falling under the impression that, most people just have heard about some feature and started to apply it for their self-presentation.
When I came up with an idea of this material, I was planning to write a detailed article how to implement CI/CD and other practices of fast development for improving team activity results. But after 8 pages of an introduction I realized that there was too much material and it’s better to start from something easy. In this article I would like to tell about an idea “You build it, you run it”, that you can generalize “If you do it, you are responsible for it”.
This statement was articulated by companies that are developing really sophisticated software and are highly competitive; in particular, such companies as Amazon and Netflix are promoting this. First time I heard this statement at a conference JAX in august of 2017 at a masterclass in Chaos Engineering and microservice delivery. That moment I realized that this is the very capacious statement which is necessary for rethinking of a delivery approach.
So, let’s start with an example. We will consider a typical delivery cycle of another software version. As a rule, it consists of several large stages.
- Code writing;
- code testing (testing, regression testing),
- bug fixing;
- repeating of stages 2-3 to achieve a required quality level;
- developing of migration routines;
- doing test deployment on clients data;
- delivering this code to a client.
Delivery pace depends on the how much time this cycle takes. Its duration is defined by the time every step requires. Steps 2-4 have some specific influence. Experience has shown that in case of a month development cycle, regression code quality testing can take 1-2 weeks for a team of 10-20 people, but steps 2-4 can easily take just 2-3 weeks.
By the way, expenses on the performing process increase in a linear fashion keeping developing process in the long term, causing a client fall under the impression of a team working slowly. In a such delivery process, collisions of activities occurs, so developers do both developing new functions and fixing bugs in a current functionality that were found within a code testing by a QA department.
This problem can be solved by transforming a responsibility role “QA engineer” into “Developer in Test”, so QA engineers join a development team, create integration and E2E tests, train developers to write the best tests, perform code analysis and audit, identify scenarios that require manual testing and automate them. Every found bug is covered with an automated test. A QA role merges together with a software developer role, as a result a new delivery approach appears that fully meets the paradigm “You build it, you run it” (You build it, you test it, you guarantee the quality of it).
Just compare it with what we had before: developers weren’t responsible for the quality, but for developing (build), QA engineers were in charge of the quality, so developers had such a thinking pattern “QA engineers will find bugs and we will fix them”. Attitude change has led to the fact that now each participant is responsible for both software creation and quality level.
There can be some more successful patterns of QA and developers cooperation that allow decreasing delivery time in reference to an example above. For instance, CI/CD implementation with an capability of each function testing cuts time of a testing step and QA department performs only regression testing beyond iteration development. However, an idea that responsibility lies on QA, but not on developers leads to shifting responsibility and leads to development slowdown due to repetitive task elaboration where errors were found.
Likewise, we can take a look at cooperation of research and development department and IT service management (ITSM). A common approach is that a developer creates software in his own sandbox, while IT service management is in charge of deployment, performance testing, operation and handling of users’ requests. As a rule, developers don’t think at all about the fact how their product will work in real environment till the moment of a test run where all flaws are found and the product is sent to be reworked.
Similar to a previous example, a split “We-They” appears. For example, “Our DBMS works well, but your code lags” or “Configure your DBMS right so that our code is able to work. Locally everything works perfectly”. It causes conflicts and absence of consensus in terms of responsibility for an ultimate outcome between these departments. DevOps practices were invented to eliminate this problem by means of deep involvement of developers into operating and taking responsibility for workplace production environment development (Infrastructure as a Code, Continuous Configuration Automation). ITSM turns into a collaborating team for R&D and it’s not in charge of a result anymore that a handed product magically will work in workplace production environment. Instead it helps developers to create a properly working in workplace environment product.
We can consider a following case. Let’s imagine that Jenny Michaels is an experienced administrator and Nick Snow is a developer who creates software working with database. Snow doesn’t fully have an analysis experience in DBMS in some exact operations. And he asks Michaels to give a consultation on this matter: “Jenny, I wrote a query that uses MySQL json-field and it doesn’t give required search efficiency with real data. I’d like you to help me to figure it out, so that I add right configuration directives into an application deployment script”.
This is an example of a changed collaboration. Firstly, Michaels isn’t responsible only for production service management anymore, but she is in charge of solving a quite exact problem (now he takes part in development). And Snow is responsible now not only for product development, but also for efficient work of the implementation with real data. As a result, Michaels starts understanding how a part of software works, what can be problematic at a stage of operation and Snow acquires knowledge of system administration which will help him to solve tasks more efficiently in the future.
If we generalize these two examples, we can emphasize that a critical change is that they moved away from non-overlapping responsibility assignment to a single point of responsibility. Both examples show how all participants became more developers, more quality engineers, and more IT service management than before. All of them now participate in software development and responsibility for a result now resides on one exact team and one exact person. Division of responsibility is consciously eliminated and instead of specialization-based activity division colleagues perform tasks together, learning from each other. This works because each one is fully in charge of quality of what he/she is doing.
It is very important so that a whole team feels it, so that there is a transition from responsibility separation to personal responsibility for a final result, mutual help and a common creation process. In a perfect process each participant starts to manage all the chain completely: “development - testing - documentation - deployment - user support”. Everybody in a team begins developing extra specializations and perceiving a product more comprehensive.
During this transformation not all participants want and are able to adapt to new circumstances and develop new skills. What to do with it is a subject of another discussion.
Implementation of an approach “You build it, you run it” can’t be brought into life without other significant practices implementation such as CI/CD, TDD, Integration Testing, Continuous Configuration Automation and IaC that allow making a development process really efficient and profitable.
Besides Dev & QA and Dev & Ops it’s possible to extend the approach to other team roles, for instance, to technical writer and customer support service.
P.S. This transformation can become something more than it is intended. For example, you can take things a step further and turn QA from a function performing quality control for a product into a function performing quality control over employee’s activities, a team and a company as well. Move from “What people do” to “How they do”.
If you like the article, please share it with other people.