by admin | Feb 21, 2020 | Software Testing, Fixed, Blog |
Continuous Testing is the method of providing feedback to the project stakeholders on quality of build through automated test execution as soon as the build is deployed with the help of QA tool stacks.
Since in the Traditional Software world, there is a waiting period of the development team to receive the feedback from the testing team post-deployment of code, which consecutively delayed the delivery. Continuous Testing came up with the approach of Test Quickly, Test Often and Test everything which helped to improve the code quality and quality of delivery by shortening the delivery cycle through the automated way.
By building the delivery pipeline where automated builds, test, and deployment are Orchestrated in one release workflow, there is seamless feedback on the each of the pipelines and based on the success of each of the pipeline process continues which helps to assess the quality of the release.
Why Continuous Testing (CT)?
It helps tightly connect Testing with Development and Operations team to ensure quicker time to market with better quality and saves cost
Instant feedback to developers and stakeholders on code quality and functional coverage to course-correct
Releasing with speed without quality is compromising Customer objective and hence CT should be plugged into CI and CD to deliver speed with quality to meet Client objective
Agile is shortening the release cycle with maximizing delivery throughput. 90% of delivery happens through Agile practices across Organization and hence DevOps (CI/CD/CT) is the key to meet this objective.
Key of Digital Transformation (which is swiftly happening in later 2018) for any customer is quality and the way to achieve with speed is through Continuous Testing
Popular Tool Stacks
Steps to achieve Continuous Testing:
Assess the number of Applications in-scope
Assess the Application category-UI (Web / Windows), API, Database, and The infrastructure needed for Application to be executed.
Criticality of Application
Number of Test cases / Application
Tooling Strategy
Validate and Assess CI / CD methodology
Framework Requirement Assessment for CI / CD Integration
Build Framework
Validate CT pipeline
Build sanity automation scripts
Integrate with CI / CD pipeline
Fine-tune framework and Test Automation approach
Start building Automation suites to achieve CI/CD/CT
Plan for N and N-1 Automation based on Organization goal
Assess the number of Applications in scope:
1) Before going to Continuous Testing, we need to assess the total number of Applications which need to be moved to CT pipeline needs to be assessed
2) This can be decided based on understanding the lifespan of Application, number of releases (major / minor / Fix release or Trivial) / year, stakeholder understanding and operational users type of this application
Assess the Application Category
1) Based on the number of Applications and its lifespan, understand the technology in which it is built with. Classify the Application type (UI, API, DB or any of 2 or 3 combinations)
2) Classify the Application based on the infrastructure as well. Some Application will work in Linux, some may support Windows as well as Linux etc. Also classify based on Mobile or other technology
Criticality of Application:
Below are the key factors that determine the criticality of Application:
Reputation damage if implemented wrongly or of any quality issue
Financial loss to company
Operational risk
Sensitive information leakage
Legal violation
Number of Test cases per Application
This is as good as understanding the size of Application
Regression Test cases created till date
We also need to categorize this based on UI, DB, API, etc
Tooling Strategy
Below are the pointers that will help to decide the tooling strategy
Organization policy
Organization support to Commercial tools / Open source tool
Organization willingness to invest in tools
Organization willingness to invest on tool training
Feasibility approach outcome / Application
Application classification and the tools support
Level of 3rd party customization involved from the development aspect
Tools support for CT (Integration support with other tools like Test Management, Reporting, Build Management, configuration management, etc)
Validate and assess CI / CD methodology
In order to achieve success in CT, Automation tool and CT requirement should meet the CI / CD requirement as well so that all 3 can communicate to each other successfully and deliver the desired result.
Example If the development team is working on Orchestration on a particular tool like Jenkins and if a Testing team is working on a tool which does not support Jenkins or which require additional effort to bring in this integration, then it is not advisable to use that tool for CT requirement. We should opt for a different tool which has ready-built support for Jenkins. Hence understanding CI / CD requirement beforehand is essential to reduce the effort from CT perspective in the later stage.
Framework Requirement Assessment for CI / CD Integration
Once tooling strategy is complete and CI / CD methodology also assessed, we can understand the Framework requirement to meet CT requirement.
Below are the additional feature expected in Framework for CT:
Re-usable libraries to support for multiple technologies and different object classification
Incorporation of BDD. BDD is a process where Business Analyst, Development team and Automation Testers come closer to work together to extend the functionality based on each release. Features are added by Business Analyst, The development team shares the wireframe + Object information to the Testing team and Automation team adds the code required to add this additional feature.
Connectivity with Test Management tool for reporting purposes. Good Automation framework should have automated Test set creation based on the build configuration, pull the scripts required for execution, execute them and update the result back in Test Management tool based on execution outcome.
Ability to report outcome based on where the trigger has been generated. Adhoc / Standalone execution or execution through CI or trigger from the Test Management tool
Define Dashboard requirement for Management and Key stakeholders
Build Framework
Based on the framework assessment and technologies, build a robust framework to meet the project requirements. It takes approximately 3-6 months to customize and build framework.
Validate CT pipeline
Post building the framework, validate CT pipeline is meeting the project requirement. Here we validate various touchpoints like how automated build definition works, whether Framework is able to integrate with Test Management tool and CI tools and dashboard result is working as expected.
Build Sanity Automation Scripts
Focus on automating sanity Test suites for each application which helps to detect the code stability to proceed for further Testing.
Integrate with CI / CD pipeline
Once the framework is built, Sanity scripts are developed and the CT pipeline is validated. It is time to integrate with the actual CI/ CD pipeline to evaluate CT process integration with CI/CD. Here again, validation is done to check whether Integrated portions provide detailed information to Management and key stake holders of the project.
Fine-tune framework and Test Automation approach
There might be a specific requirements from the project to project certain additional information to Management and that needs to be enhanced either in the framework level or in the CI/CD/CT definition and that needs to be worked out.
Start building Automation suites to achieve CI/CD/CT
Once the current sanity integration is done and tested, now is time to develop Automation scripts for Regression Test cases and develop different Automation suites based on need and integrate with CI/CD.
After adopting all these steps we have achieved CT goal for an Organization.
Plan for N and N-1 Automation based on Organization goal
Once base Regression suite is prepared, next activity would be to increase the coverage through N and N-1 Automation based on Organization requirement which would really bring in CT benefits.
Key Automation Metrics for Continuous Testing
Test Pass Percentage
Quality of Build
Automation Coverage for Sprint / Release
Requirement Coverage for Sprint / Release
Defect Resolution Time
Defect Rejection Ratio
Build success Rate
Application Crash Rate
Since in CT everything is Automated. The above metrics are also captured in an automated way in the dashboard to Management and Key stakeholders.
Schematic View
Key Factors that decide the success rate of Continuous Testing
Willingness of Management to undertake this digital strategy
Willingness of Management to invest time and money towards this initiative
Hiring right resource to perform this Transformation
Proper Analysis on Tool Strategy
Effort spent in Robust Framework creation
Training to key teams on Framework (Key to success is to have separate team for building framework and its customization with the delivery team focusing on building Automation suites)
Framework customization and Maintenance
Dedicated effort to track all these activity
Team should revisit their metrics for every quarter to see if that meets the project requirement and consider revising it to meet project goal
by admin | Jan 4, 2020 | Software Testing, Fixed, Blog |
The business of the telecom operators is an evolving landscape with data and non-voice services having surpassed voice services. Telecom companies therefore have the responsibility and must deliver top quality, innovative, and reliable services if they are to retain existing clients and attract new ones for the long term. A wide range of top quality mobile applications is part of the expectations that customers have from telecom companies as part of the evolving landscape. As a leading Software Testing Company our work within the telecom sector enables us to understand the challenges this sector faces with regard to the new trends and technologies. Augmented reality, the Internet of Things (IoT), virtual reality (VR), and many more are all technologies that customers expect, which forces telecom companies to restructure and readjust their traditional business strategies – posing a significant challenge for many operators.
The telecom industry is comprised of sections such as internet service providing, satellite and wireless communication, and more, but to gain and sustain a competitive edge each company must deliver customized and value added solutions to their customers. Innovative solutions for this industry would include software based updates, customer segmentation, customized marketing campaigns, and top quality customer service.
1. The significant and consistent elevation in new technologies, rise of customer expectations, and the increasing number of internet service providers and other competing telecom companies have led to a sharp decline in the profit margins for telecom companies overall. Such companies need to turn their attention toward ICT innovation, and ensure that their organizations remain flexible enough to adapt to the ever evolving digital transformation. As experts in the realm of QA services, we know that QA testing is essential to ensure optimal working of software and the systems, such that they deliver the expected output.
2. As the number of subscribers and a range of products increase, there is a need for tailored solutions. All this in turn makes customer care, operations, configuration of service, billing cycles and other such aspects, a lot more complicated. Telecom companies therefore, have increased spends on getting more resources and customized tools to ensure that customers get what they want and to gain an edge over competitors. As a result the financial overheads have increased significantly. With so much at stake, these companies must ensure that their products not only function as per customer requirements, but that the products are efficient, convenient, and secure. QA testing is therefore essential – not just to ensure proper functioning of the products, but also to mitigate the challenges and risks that telecom companies could face.
3. Meticulous and regular QA testing would ensure the proper implementation of the solutions, while addressing all the existing and possible issues. Telecom companies must ensure that their end to end solutions are properly supported such that their business remains profitable.
4. As mentioned, the challenges that telecom companies face, is the constant restructuring of infrastructure, the IT connectivity, and maintaining focus on offering reliable and affordable services. The most important is network security given that threats to technology are becoming more potent and frequent. The QA testing experts would be able set up a proper process and provide solutions customized to fit the business and customer base of the companies.
5. No business today can succeed without retaining existing customers and attracting a larger customer base. This can happen only when consistently high quality service is provided, along with effective back-end software, and a user-friendly and highly responsive customer use application. QA testing will ensure that telecom services are elevated and there is a significant reduction in the time to market and customer complaints.
As leaders in the realm of QA services for the telecom industry (and more), we have consistently helped our clients with all the features mentioned above. QA testing analyses all the major operating systems, while optimizing their testing along with hardware devices. Our in-depth expertise enables us to ensure the security, functionality, reliability, and compatibility of the applications used by the telecom sector through wide and rigorous software testing. Connect with us for specialized software testing services and a personalized software testing strategy for your business within the telecom sector (and many more).
by admin | Jan 2, 2020 | Software Testing, Fixed, Blog |
Localization testing falls within the realm of software testing services. This software testing technique ensures that the software product ‘behaves’ as per the local settings and culture. As a leading software testing company we understand the importance of customizing software / applications to suit the targeted country, language, and the cultural intricacies. Localization testing is focused mainly on content and user interface, ensuring that the application can be used easily by the target audience in a particular country.
To match and engage a particular demographic and locale, it is important that the software is appropriate from the linguistic and cultural aspects – this is why Localization testing is required. This form of testing is necessary to make changes in the user interface, as also the initial settings as per the requirements. Localization testing requires expert testers to repeat the same functions in order to verify the cultural appropriateness of UI, linguistic and typographical errors and more. The features of this form of testing include:
- A localized user interface (UI) – to ensure usability and comfort of the persons of a particular country and with specific cultural aspects
- Localized content and information – the content in the application would be prepared to be suitable to the local culture and language, and localization testing ensures that all these aspects actually match the comfort level of the locals
- Particular and Identifiable Language – This form of testing checks the compatibility of the language of the software with the language of a specified region
Compatibility of Hardware – The test ascertains that both the hardware and the software are compatible with the specified region
- The method for localization testing includes setting up build verification testing, functional testing, regression testing, and the completion sign-off.
Prior to detailed QA, build verification testing is undertaken – this is a subset of functional testing
- In order to run normal test cases and uncover log defects during execution, normal/functional testing must be conducted
- Regression testing is next to ensure that defects are resolved and that the defects did not affect other components of the application
Before delivering to the customer, there must be one final check on the build – the final and completion sign-off is a necessary step
The process of localization is a critical one, and we as the experts understand that localization testing must ensure that all the effort you put in will take your product to the widest possible range of audience, and within budget. Given our experience, we at Codoid are both international and local experts and have the ability to ensure scalability of apps, platforms and programs. This in turn ensures that no additional code will be required when foraying into new markets. Localization testing is an essential component and tool in spreading business to an increasing number of consumers. By paying attention to the minute details a business would be better equipped to impact the expectations, needs, wants, and behaviors of a larger consumer base and at a much bigger scale. Being able to reach out and engage a larger audience goes beyond the written and spoken word – connect with us to gain all this and more.
by admin | Dec 29, 2019 | Software Testing, Fixed, Blog |
Even though the concept of a self-organizing team has been propagated as part of the Agile drive, there is really no clear definition in the documentation of the best or even such a team. However, the underlying premise of such a team is that it would produce optimum results. As a leading Software Testing Services Company we understand the importance and benefit of such a company, and have made an attempt to put together an understanding of the critical self-organizing team.
From the name it seems simple to understand that such a team would work cohesively and towards meeting a predetermined mission and future vision. Such a team, while having a certain level of a decision making authority, would simultaneously have the onus of their working style and evolving a collaborative system of working. In the realm of software testing services, a self-organizing team would plan and strategize from the bottom up level, rather than the typical top down method. Such a team would have a system to determine and balance workload, ensure a proactive approach to work and problems, and ensure that all measures are geared towards achieving the objectives.
A self-organizing team operates differently to other teams – continuous improvement is the objective. As experts in the realm of QA, we understand the importance of teams that can function autonomously and yet produce the desired results. Our team is constantly organizing and evolving to manage their work, with the least amount of control and supervision. This in turn drives efficiency and optimum management of work since each member remains open to clear communication. A self-organizing team consistently enhances its skills, asks the right questions, and collaborates to get an increasing amount of work from existing and new clients. Additionally, members of such a team assign work amongst itself rather than waiting for the manager / supervisor to tell them what to do.
Each Agile team would have a different approach to organizing their work and would use the experience of all those in the team to move towards success, rather than depending on the knowledge of their manager. Such teams take complete ownership of their work and the possible problems and challenges. It seems a bit far-fetched that unrelated and disconnected individuals put together would be able to work cohesively and form a self-organizing team. However, that is the underlying premise of such a team – that the individuals comprising it would be carefully selected and would have a common goal. This selection process would be a critical responsibility of the management – proper deliberation, analysis of personalities and skills – to ensure proper group dynamics. We at Codoid use the following criteria to ensure that our teams are self-organizing in essence:
A team that consists of members from cross-disciplines and skills would be larger than wanted, but every skill required to take an idea to reality would be necessary. With time, however, some team members should and must develop a wider range of skills, allowing for other team members to move on to other teams / projects. This is a great way to utilize existing skills and grow the skills of each member.
A single team must never have members with the same level of technical expertise. It is important to balance the skill levels such that all the workload is properly divided. The skills of a highly experienced programmer for example, would be wasted on coding features with low impact, while a less skilled member would not be able to deal with the higher level complexities. Such a balanced mix would also be required when looking at those with domain knowledge.
While ensuring diversity would mean considering race, culture, gender, and skills, it also means putting together individuals who have varied yet innovative ways of thinking about and approaching problems and challenges. Team diversity would mean looking at individuals depending upon how much data they would require before arriving at effective and practical decisions.
Random members would take a lot of time before they start working collaboratively. When putting together an Agile team, it would make sense to put team members who would have worked together in previous projects. Additionally, matching communication styles and personalities would also help to quickly ensure cohesiveness and common purpose.
There is no doubt that putting together a self-organizing team in Agile is an extremely challenging task. It requires patience and time, and the expertise of a senior person to coach and facilitate all the factors mentioned to ensure a ‘well-oiled’ self-organizing team. The process of building and maintaining such a team therefore is ongoing – hence it would be better to align your business with an experienced vendor with a self-organizing team. Connect with us to gain all these advantages and more.
by admin | Jan 22, 2020 | Software Testing, Fixed, Blog |
Software testing companies should employ the fail-fast principle in their Software Development Lifecycle (SDLC) process because when your program fails, you can quickly fix the errors and promptly move to the next. Ideally, systems shouldn’t fail, and your application shouldn’t crash, but when it does, you should prevent frequent recurrences.
You are bound to err on the project, and the software will end up with a couple of bugs. System failures and software crashes are frequent, and deadlocks, data loss, privacy corruption, and inconsistency is particularly bad. The fail-fast principle prompts us to quickly fail early on in the process because if an error occurs, then it is easier to fix. It is better if the software fails immediately instead of later where you need to work around the failure. Since fail-safe makes errors appear earlier in the SDLC and bugs found are simpler to reproduce and cheaper to fix, it is an ideal situation to be in as it helps stabilize your software. Ensure there are fewer bugs and defects when it goes into production, therefore resulting in superior quality and launch-ready products. Fail-fast is a cost-effective method of application testing and a principle behind many agile practices.
Here are cases in software development when you need to employ fail-fast testing:
You write tests and cover all types of test case scenarios in which they may fail and highlight all the requirements that need to be met before implementation in a test-driven development method.
Developers are required to employ a method of continuous integration for current works with a shared repository, and it needs to be verified by automated builds to help the team to find and fix bugs early.
Instead of typically starting up the system or using the fall-back strategy if the system fails or stops due to missing environment variables and start-up parameters, you should be notified immediately to fix the issue.
Software theory usually prescribes one of two extremes to ensure quality. First is granting sole responsibility to an individual on the team; this removes accountability from the rest of the team. Second is the need to collaborate on everything that translates to follow the pack mentality. At Codoid, we suggest an alternate option, train, and educate all team members to become decision-makers on the project. Make sure this combined theory is put to the test by ensuring everyone is equally responsible for the outcome of the product quality. Let customers be empowered to make the right decision. Enunciate to your team the long-term consequences of their choices and explain how it affects user experience.
Providing software testing services can be hard for many companies. Yet, we are confident in our capability of predicting what may happen, and we take time to explain the possible outcome. We ensure that we get things done without having to give up the power of decision making. It is a risky gamble, but our work speaks for itself, take a look at our testimonials, and you’ll see we’re right.
