Being good at any job requires continuous learning to become a habitual process of your professional life. Given the significance of DevOps in today’s day and age, it becomes mandatory for a software tester to have an understanding of it. So if you’re looking to find out what a software tester should know about DevOps, this blog is for you. Though there are several new terms revolving around DevOps like AIOps & TestOps, they are just the subsets of DevOps. Before jumping straight into the DevOps testing-related sections, you must first understand what DevOps is, the need for DevOps, and its principles. So let’s get started.
Definition of DevOps
“DevOps is about humans. DevOps is a set of practices and patterns that turn human capital into high-performance organizational capital” – John Willis. Another quote that clearly sums up everything about DevOps is from Gene Kim and it is as follows.
“DevOps is the emerging professional movement that advocates a collaborative working relationship between Development and IT Operations, resulting in the fast flow of planned work (i.e., high deploy rates), while simultaneously increasing the reliability, stability, resilience, and security of the production environment.” – Gene Kim.
The above statements strongly emphasize a collaborative working relationship between the Development and IT operations. The implication here is that Development and Operations shouldn’t be isolated at any cost.
Why do we need to merge Dev and Ops?
In the traditional software development approach, the development process would be commenced only if the requirements were captured fully. Post the completion of the development process, the software would be released to the QA team for quality check. One small mistake in the requirement phase will lead to massive reworks that could’ve been easily avoided.
Agile methodology advocates that one team should share the common goal instead of working on isolated goals. The reason is that it enables effective collaboration between businesses, developers, and testers to avoid miscommunication & misunderstanding. So the purpose here would be to keep everyone in the team on the same page so that they will be well aware of what needs to be delivered and how the delivery adds value to the customer.
But there is a catch when it comes to Agile as we are thinking only till the point where the code is deployed to production. Whereas, the remaining aspects like releasing the product in production machines, ensuring the product’s availability & stability are taken care of by the Operations team.
So let’s take a look at the kind of problems a team would face when the IT operations are isolated,
1. New Feature
Let’s say a new feature needs multiple configuration files for different environments. Then the dev team’s support is required until the feature is released to production without any errors. However, the dev team will say that their job is done as the code was staged and tested in pre-prod. It now becomes the Ops team’s responsibility to take care of the issue.
2. Patch Release
Another likely possibility is there might be a need for a patch release to fix a sudden or unexpected performance issue in the production environment. Since the Ops team is focused on the product’s stability, they will be keen to obtain proof that the patch will not impact the software’s stability. So they would raise a request to mimic the patch on lower environments. But in the meanwhile, end users will still be facing the performance issue until the proof is shown to the Ops team. It is a well-known fact that any performance issue that lasts for more than a day will most probably lead to financial losses for the business.
These are just 2 likely scenarios that could happen. There are many more issues that could arise when Dev and Ops are isolated. So we hope that you have understood the need to merge Dev and Ops together. In short, Agile teams develops and release the software frequently in lower environments. Since deploying in production is infrequent, their collaboration with Ops will not be effective to address key production issues.
When Dev + Ops = DevOps, new testing activities and tools will also be introduced.
DevOps Principles
We hope you’ve understood the need for DevOps by now. So let’s take a look at the principles based on which DevOps operate. After which we shall proceed to explore DevOps testing.
Eliminate Waste
Anything that increases the lead time without a reason is considered a waste. Waiting for additional information and developing features that are not required are perfect examples of this.
Build Quality In
Ensuring quality is not a job made only for the testers. Quality is everyone’s responsibility and should be built into the product and process from the very first step.
Create Knowledge
When software is released at frequent intervals, we will be able to get frequent feedback. So DevOps strongly encourages learning from feedback loops and improve the process.
Defer Commitment
If you have enough information about a task, proceed further without any delay. If not, postpone the decision until you get the vital information as revisiting any critical decision will lead to rework.
Deliver Fast
Continuous Integration allows you to push the local code changes into the master. It also lets us perform quality checks in testing environments. But when the development team pushes a bunch of new features and bug fixes into production on the day of release, it becomes very hard to manage the release. So the DevOps process encourages us to push smaller batches as we will be able to handle and rectify production issues quickly. As a result, your team will be able to deliver faster by pushing smaller batches at faster rates.
Respect People
A highly motivated team is essential for a product’s success. So when a process tries to blame the people for a failure, it is a clear sign that you are not in the right direction. DevOps lends itself to focus on the problem instead of the people during root cause analysis.
Optimise the whole
Let’s say you are writing automated tests. Your focus should be on the entire system and not just on the automated testing task. As a software testing company, our testers work by primarily focusing on the product and not on the testing tasks alone.
What is DevOps Testing?
As soon as Ops is brought into the picture, the team has to carry out additional testing activities & techniques. So in this section, you will learn the various testing techniques which are required in the DevOps process.
In DevOps, it is very common for you to see frequent delivery of any feature in small batches. The reason behind it is that if developers hand over a whole lot of changes for QA feedback, the testers will only be able to respond with their feedback in a day or two. Meanwhile, the developers would have to shift their focus towards developing other features.
So if any feedback is making a developer revisit the code that they had committed two or three days ago, then the developer has to pause the current work and recollect the committed code to make the changes as per the feedback. Since this process would significantly impact the productivity, the deployment is done frequently with small batches as it enables testers to provide quick feedback that makes it easy to revoke the release when it doesn’t go as expected.
A/B Testing
This type of testing involves presenting the same feature in two different ways to random end-users. Let’s say you are developing a signup form. You can submit two Signup forms with different field orders to different end-users. You can present the Signup Form A to one user group and the Signup Form B to another user group. Data-backed decisions are always good for your product. The reason why A/B testing is critical in DevOps is that it is instrumental in getting you quick feedback from end-users. It ultimately helps you to make better decisions.
Automated Acceptance Tests
In DevOps, every commit should trigger appropriate automated unit & acceptance tests. Automated regression testing frees people to perform exploratory testing. Though contractors are highly discouraged in DevOps, they are suitable to automate & manage acceptance tests. Codoid, as an automation testing company, has highly skilled automation testers, and our clients usually engage our test automation engineers to automate repetitive testing activities.
Canary Testing
Releasing a feature to a small group of users in production to get feedback before launching it to a large group is called Canary Testing. In the traditional development approach, the testing happens only in test environments. However, in DevOps, testing activities can happen before (Shift-Left) and after (Shift-Right) the release in production.
Exploratory Testing
Exploratory Testing is considered a problem-solving activity instead of a testing activity in DevOps. If automated regression tests are in place, testers can focus on Exploratory Testing to unearth new bugs, possible features and cover edge cases.
Chaos Engineering
Chaos Engineering is an experiment that can be used to check how your team is responding to a failure and verify if the system will be able to withstand the turbulent conditions in production. Chaos Engineering was introduced by Netflix in the year 2008.
Security Testing
Incorporate security tests early in the deployment pipeline to avoid late feedback.
CX Analytics
In classic performance testing, we would focus only on simulating traffic. However, we never try to concentrate on the client side’s performance and see how well the app is performing in low network bandwidth. As a software tester, you need to work closely with IT Ops teams to get various analytics reports such as Service Analytics, Log Analytics, Perf Analytics, and User Interaction Data. When you analyze the production monitoring data, you can understand how the new features are being used by the end-users and improve the continuous testing process.
Conclusion
So to sum things up, you have to be a continuous learner who focuses on methods to improve the product and deliver value. It is also crucial for everyone on the team to use the right tools and follow the DevOps culture. DevOps emphasizes automating the processes as much as possible. So to incorporate automated tests in the pipeline, you would need to know how to develop robust automated test suites to avoid false positives & negatives. If your scripts are useless, there is no way to achieve continuous testing as you would be continuously fixing the scripts instead. We hope that this has been an equally informative and enjoyable read. In the upcoming blog articles, we will be covering the various DevOps testing-related tools that one must know.
Comments(0)