The Challenge

One major challenge was testing the application as quickly as possible because our client was eager on deploying their service with the newly developed subscription models. But testing required streaming videos for long hours. They also had two dedicated variants of their OTT platform for B2B and B2C users respectively.

As the availability of the content on the OTT platform varies in different regions, we also had to ensure that the right content is being displayed in the correct geolocation.

Our Solutions

• Performed End-to-End OTT Automation testing covering all scenarios.
• Parallel Testing with fully Independent scenarios without any flakiness.
• Created Automation Test Suites for their different Subscription models.
• Region-based testing using network manipulators.
• Seamless reporting with instant execution status provision.

Parallel Testing to Save Time

We wanted to ensure we completed our testing as quickly as possible without compromising on the quality. So we made use of our in-house device farm to run the automation tests across numerous devices at the same time. We also developed our automation scripts that are based on scenarios completely independent from each other and created automation test suites for every subscription model. This enabled us to even reduce the time required to complete the testing.

End-to-End OTT Test Automation

Be it a movie, series, or live program, we automated all content-specific controls such as skip intro, watch next episode, watch the previous episode, and so on. We also covered all other functionalities such as picture-in-picture, offline, swimlane navigation, playback, and so on.

We used VPNs to test if the OTT platform displayed the right content in the defined geo regions. We also performed role-based testing for both the B2B and B2C variants of their OTT platform.

A/B Testing

From the data we got from OTT analytics tools such as Youbora, and Mix Panel, we were able to perform A/B testing with different content recommendations in various swimlanes based on a user’s usage trends.

Collaboration

Our client followed Agile methodology for their project. So we had 14-day sprints during which we tested the OTT app without any delays. Our client wanted instantaneous updates on the status of the completed automation tests. So we completely automated our reporting flow so that the test execution status of every test gets updated instantaneously in their bug management tool.