by admin | Apr 6, 2021 | Automation Testing, Fixed, Blog, Latest Post |
Automation testing using JavaScript has been gaining popularity in recent days making a blog about the best JavaScript Testing Frameworks available the need of the hour. A conversation between business analysts, developers, and testers about a feature will become specifications. After that, the developers & testers convert the specs into automated acceptance tests. Automated tests help the team to drive the feature development as passing acceptance tests prove the fact that the agreed feature has been developed as expected.
To convert specifications into automated acceptance tests, devs & QAs should use the same programming language. Let’s say your team is using JavaScript for development and Java for automation testing. In this case, you won’t be able to enable effective collaboration between the developers & testers which is a big concern. Analysts & developers can capture acceptance criteria effectively. However, testers are good at adding edge cases making them invaluable as well. That is why everyone’s involvement right from the beginning is vital with common tools, frameworks, and language.
In this blog article, we have listed the best JavaScript Testing Frameworks which are widely used for writing unit tests and acceptance tests.
Jasmine
When developers focus more on components than features, they don’t add value to the business. Writing tests in plain English helps your team focus on the business goals and Jasmine is one of the best testing frameworks to make it possible. Using Jasmine you can write meaningful tests which can be understood by everyone in the team and help the developers to write higher quality code more efficiently.
Jasmine has support for Node.JS, RUBY, and Python. If you want to run the tests on a Web Browser instead of running them in interpreter/programming engine, then you can go for Jasmine Standalone. Now let’s find out how to use Jasmine in Node.JS beginning right from its installation.
Installation:
To install Jasmine, run the below npm command.
npm install jasmine —save-dev
Initialization
To set up Jasmine in a Node.JS project, you need spec files and jasmine.json (Configuration file). You can add these files manually. However, we have an npx command to initialize Jasmine in your project and it has been listed below
After running the above command, you can find the ‘spec’ folder and ‘jasmine.json’ file under the spec->support folder.
Add Jasmine in package.json
Now, you need to inform your node project that you are using Jasmine for testing. Open package.json and update the ‘test’ value as ‘jasmine’ as shown below.
{
"name": "jasmine-sample",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "jasmine"
},
"author": "",
"license": "ISC",
"devDependencies": {
"jasmine": "^3.7.0"
}
}
Describe
In order to group related test cases like how we group scenarios in a feature file in Cucumber, you need to create a JavaScript file and write a ‘describe’ function as shown below.
describe("User Management", ()=>{
})
In the above function, it is a placeholder to add ‘User Management’ related test cases.
Specs
Specs are your test cases. You need to write specs inside the ‘describe’ function as shown below.
describe("User Management", ()=>{
it("should signup",()=>{
})
it("should login", ()=>{
})
})
As we mentioned earlier, everyone in the team should understand what is the purpose of each test. When naming a test case for unit & acceptance tests, you need to describe what you are expecting. However, it is a cumbersome task for a developer to focus on names. If there are a rule and a standard which come from the testing framework, then the team members will be able to stick to the business goals and focus on achieving them instead of concentrating only on the code components.
That is why the Jasmine framework has the ‘it’ function and the first argument starts with ‘should’. When a test case says ‘It should do something’, the developer will focus on that something throughout the development which is exactly what we want.
How is a spec pass or fail determined? You can have multiple assertion lines inside a spec and even if one of those assertions fails, the spec will be marked as ‘Fail’.
Assertions
Jasmine has an in-built assertion function which is called ‘expect’. Expect function has one argument which takes the actual value. You can also chain the assertion using Matcher functions.
expect(“my string”).toMatch(/string$/) – Expect the actual value to match a regular expression.
expect(bigObject).toEqual( “foo”: ‘bar’, ‘baz’ ) – Expect the actual value to be equal to the expected value using deep equality comparison.
expect(thing).toBe(realThing) – Expect the actual value to be === to the expected value.
Configurations
Jasmine has a total of five configurations in the jasmine.json file. Let’s take a look at the purpose of each of these configurations one by one.
“specdir” – A directory path where you have the specs in the current project directory.
“specfiles” – You can include & exclude spec file paths with regular expression.
“helpers” – Before the test execution commences, you may need some supporting files that should be executed. The helpers config is the placeholder to mention the supporting JS files.
“stopSpecOnExpectationFailure” – This is a boolean config. If the value is true, then the spec execution will be stopped when the first assertion is failed.
“random” – Run the specs in random order.
Advantages of Jasmine
- You can customize the Jasmine report.
- It has a very strong community that supports the users.
- You can even create your own Matchers.
- It is extremely simple and easy to use making it the choice of many developers and testers mainly for its simplicity.
Jest
Jest is also a popular JavaScript Testing framework that works with Node.JS, React, Angular, Vue, Babel, TypeScript, and more. Jest was developed by Facebook and so it runs thousands of tests for Facebook on a daily basis. Apart from Facebook, there are many other prominent companies that also use Jest. So let’s see what makes Jest one of the best JavaScript Testing Frameworks and how you can implement it in your project. Let’s begin with its installation process.
Installation
You can easily install Jest by making use of the below code,
npm install --save-dev jest
Add Jasmine in package.json
Update scripts>test node as ‘jest’ in Package.json as shown below.
{
"name": "jest-sample",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "jest"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"jest": "^26.6.3"
}
}
Configuration
No configuration is required to write or run Jest tests. However, every testing framework has its own configuration file. To create a config file, you can run the below command.
Once you run the above command, Jest will ask you a few questions to create the config file.
Migrating from Jasmine to Jest
If you are planning to migrate from Jasmine to Jest, then we’ve got some good news for you as Jest is compatible with Jasmine tests. This makes the migration far less complicated. However, if you want to do a code-level migration, you can use third-party jest-codemods for the best results.
Assertions
In order to write assertions elegantly in a test, you would definitely need sophisticated Matchers. Jest has a lot of compressive matchers that you could use. However, we have listed some of the commonly used Matchers below.
test('two plus two is four', () => {
expect(2 + 2).toBe(4);
});
test('object assignment', () => {
const data = {one: 1};
data['two'] = 2;
expect(data).toEqual({one: 1, two: 2});
});
test('null', () => {
const n = null;
expect(n).toBeNull();
expect(n).toBeDefined();
expect(n).not.toBeUndefined();
expect(n).not.toBeTruthy();
expect(n).toBeFalsy();
});
Snapshot Testing
Your Acceptance Tests test the application under test by performing actions on the UI. However, it never validates whether the UI is rendered as same as before and if the CSS styles are applied correctly or not. Jest has a feature to address this issue (i.e. Visual Regression Testing). Using Jest, you can capture snapshots and compare them with the expected snapshot thereby creating a huge advantage if that is your primary need.
Advantages
If you want to run your tests in parallel, then Jest is your go-to testing framework. Jest has numerous assertion functions. In terms of ease of adoption, simplicity, support, and documentation, Jest is hands down the clear winner.
Mocha
Mocha is a JavaScript Testing framework only for Nod.JS. Mocha doesn’t have an in-built assertion library. However, it allows you to use the following libraries such as should.js, chai, expect.js, better-assert, and unexpected for assertions. You can execute tests in two modes (Serial & Parallel).
Conclusion
Jest and Jasmine are the best JavaScript Testing frameworks. However, most of the engineers are comfortable with Jasmine over Jest. As an automation testing company, we use Jasmine for Protractor automation testing. Jasmine is the default testing framework for Protractor. Our suggestion is not final and it shouldn’t influence your framework decision.
Jest has many sophisticated features such as Configuring Visual Testing (i.e. Snapshot Testing) which is not as easy to do in Jasmine. However, in Jest, it is pretty straightforward. Apart from that, test creation and assertions are similar in both these frameworks. If you feel Jest is suitable for your project, please go ahead.
by admin | Mar 29, 2021 | Automation Testing, Fixed, Blog, Latest Post |
Even the smallest hiccups like package installation errors can be a bane to your productivity. In this blog article, we have listed some of the basic npm commands which are used by developers and software testers regularly to resolve package installation errors and to even upgrade existing npm packages. As a software testing company, we are experts in automation testing services using Protractor & Jasmine frameworks.
For one to manage npm packages effectively, one should be aware of the most common npm commands. So let’s get the ball rolling with a few basic commands that have been categorized by their purpose.
Package Installation
To install a package in your current project using npm commands, you can make use of either of the codes given below:
npm install package-name -s
or
Use the code given below to install a package under Development dependencies in your current project by using npm commands:
npm install package-name —save-dev
You can install a package globally using npm commands if a package dependency is not for the project by using the below code:
npm install package-name -g
Install a specific package version
Sometimes you might find yourself to be in a situation where you need to install a specific package version rather than the most latest version. But when you run the npm install command, it will only install the latest version of a package. The solution is easy, as you can install a specific package version by simply using the version number in the installation command as shown below.
npm install package-name@version-number
Finding Outdated Packages
When the dependency list grows, it is a cumbersome job to find the outdated packages in a project. However, npm can make your job easy as it has a command to find the packages which can be upgraded.
When you run the above command, it will list the outdated packages from which you can find the version that you are looking for.
Clear npm Cache using npm commands
When a package installation doesn’t go as expected, it may be because of an npm cache issue. One best option would be to clear the cache and then execute the installation command again.
Before clearing the npm cache, make sure to check how many cache entries are available by using the below code.
Now you can clear the npm cache by using the below command.
Audit Packages
Using vulnerable npm packages can make your project less secure. So it is crucial to perform a security check for all the available project packages. You can do so by running the below command.
Once the audit has run, it will list the vulnerable packages which are supposed to be removed to make your packages more secure.
Installing Minor & Patch Releases
When you run the npm install command, it will install the packages which are mentioned in the package.json file. The npm install command can be used to install only the missing packages. But if you are looking to install recent minor versions using the npm install command, you have to prefix the caret (^) symbol before the package version in package.json. To install patch releases, you have to prefix the Tilde (~) symbol.
Conclusion
Auditing packages & removing unused packages periodically will help avoid catastrophic failures when deploying code in a production environment. Node.js is a memory-efficient environment. When an automation test suite runs for a longer duration, you should optimize it by improving the script and using good packages.
QA automation testers who are starting automation test scripting using JavaScript & Node.js will find these basic npm commands very helpful. Being an automation testing company, we use Protractor, Selenium WebDriver, and Jasmine frameworks for web apps automation testing, and therefore in the subsequent blog articles, we will be publishing more on Protractor and automation test scripting using JavaScript & Node.js. So make sure to keep an eye on our website so that you don’t miss out on the upcoming blogs.
by admin | Jan 13, 2022 | OTT Testing, Blog, Latest Post |
Even before the pandemic hit, OTT platforms have been on a rampage in terms of growth and popularity. Netflix has multiplied its user base 10 times over the last 10 years. There is heavy competition amidst the existing big players and the number of OTT platforms entering the market is also on the rise. Even if you have the best content in the market and fail to provide a good user interface and user experience, users will move away. So no OTT platform can afford to have mediocre testing that lets bugs reach the user. As a leading QA company, we deliver uncompromised quality to all our clients and have created this conclusive list of test cases for OTT Platform Testing that will help you achieve maximum coverage.
OTT is an abbreviation for “over-the-top,” which refers to any streaming media service that delivers content via the internet. The term “over-the-top” refers to the fact that the service is delivered on top of another platform. In other words, it is any service that allows you to deliver video and/or live stream feeds to any internet-connected device.
OTT Platform Testing
Robust and exhaustive OTT platform testing is required to successfully deliver video content. There are numerous things to test when it comes to OTT platform testing, but as stated earlier, they boil down to these two major aspects.
- The user interface (UI) allows users to interact with the OTT platform and view the video content.
- The user experience (UX) the person takes away after interacting with the UI.
Let’s dig deeper to get a better understanding of why these two factors are important. Let’s take Netflix as an example as we’ve already discussed how it has grown over the years. In an age that had episodes airing every week, Netflix introduced the binge-watching culture by dropping all of the episodes in one go. They did not want you to leave their platform once you entered.
So instead of just changing the release pattern, they built an intuitive UI that keeps recommending content based on your preferences. But if in case Netflix kept suggesting shows we weren’t interested in, we would just leave. At the same time, even if Netflix was able to provide good recommendations after making the user wait for 2 minutes, nobody is going to wait. That is why it is vital to test how people interact with the product and analyze their experience as well.
As one of the best OTT testing companies, we have created a list of test cases under each category to obtain maximum coverage and quality. We have also mentioned a list of the best testing tools that can also be used as OTT Testing tools. But before we proceed to that, let’s start with the test cases for User Interface and functionality.
UI and Functionality
Playback: Check to see if the video plays/pauses when you click on the video play icon. Also, see if the volume controls, and mute functionality works. Check for audio-video sync as well.
Video Quality: Make sure that the user is able to choose the video playback quality based on their network conditions.
Replay: Check if that the user can replay the video if they wish to see it again once it is finished.
Search: Verify that the video can be found when searched by the movie title, artist names, or relevant keywords that can be found in the description.
Search Results: Check the video’s thumbnail, title, description, length, age rating, etc. and see if the search has relevant autosuggestions. Also, make sure to see if clicking on the video plays the video or opens the content to show additional information.
Voice Commands: Since a major share of the OTT content is consumed via smart televisions, make sure that the voice commands by various assistants also work correctly.
Browse Features: You must be able to filter your search results and content in general so that you can browse through the content without any hassle. Check if the suggested content is based on the user’s viewing history.
History: Once a user starts watching a show or a movie, they must be able to pick up from where they left off and not have to start over again. If it is an OTT platform like YouTube, make sure the history function works well and see if the user is able to remove a video from the history or clear the history on the whole.
Offline Mode: Verify if the user is able to download the content they wish to keep offline and watch it without any internet connection. Also, make sure the downloaded files are not available locally to the user outside the platform.
Bandwidth Capacity: Check how well the content is streamed under different network conditions. If there is limited connectivity, see how well the video is played with lower bitrates.
Direct Access Link: Make sure that the video page can be accessed via a direct link to the video. See if the link can be shared easily using a third-party app.
Browser Compatibility: Verify that both the video and audio play in all flash-enabled browsers. And if it doesn’t, ensure that a meaningful message is shown to the users using a browser without flash or the necessary codec.
Watch list: Add content to a watch later or watch list to see if it gets updated properly. If there are any new additions like a new season to a show in the watch list, make sure it is notified to the user.
Notifications: Other notifications should include the addition of popular and relevant content, release of any content that had a countdown timer or reminder.
Language & Captions: Check if the mentioned captions are available in the content and also see if it in sync with the audio and visuals. If there are other language captions or audios available, then it must be possible to switch easily.
Audio output: Users might be connected to a different audio device via wires or Bluetooth when watching content on a television or mobile. Make sure that the audio is routed to that device properly without any sync issues.
Multitasking
Most of these test cases will be the ones you’d have to cover when testing on mobile platforms. Since OTT platforms employ mobile-only plans, it is an important part of OTT platform testing that can’t be neglected.
Call: Make sure that the video pauses when the user is watching using a mobile device. Also, check if the video resumes from the exact same point once the call is disconnected.
Notifications: Make sure that pop-up notifications don’t pause the playback or disturb it in any way unless the user clicks on it.
Mini Player: If the user does click on the notification, check if the player gets minimized and continues to play. Make sure to drag the mini-player around to see if it works properly. If the mobile or tablet supports split-screen viewing, then make sure the mini player is compatible with that too.
Account
Subscription: Ensure that only users who have paid the subscription fee are able to access the content and stream the video. If there are any ‘Pay per View’ or ‘Video on Demand’ content, test to see if it can be accessed by only those who have the extra payment.
User Profiles: The user must be able to create multiple user profiles and even have separate user histories. Verify if the user is able to select the profile through which he/she wants to watch as soon as the app is launched. There must also be kids or age-restricted profiles which can be accessed by the account with parental rights.
User Count: Make sure the platform is able to detect an account being used in more than the number of allotted screens for the package. Few plans allow different user profiles to access the platform at the same time and not allow the same content to be seen across multiple devices. The same applies to the number of devices that can have offline content. So keep an eye on such factors as well.
Login: Proper account recovery options must be available. But make sure that nobody else is able to access the account without the owner’s knowledge. If the account is being accessed from a new device, make sure that an OTP is sent for verification and enable the option to mark the device as a trusted device. Check if 2-factor authentication works if enabled.
Password: Make sure to suggest the user set a strong password. Ensure that older passwords cannot be reused. If the user wants to change the password, ensure that it is done with an OTP verification. There should also be an option to log out of all the devices.
Information: All the relevant account information like the plan name and so on should be easily accessible in one place for the user to review.
Payment and Security
Since there are both annual and monthly billing options, security features in any OTT platform can’t be taken lightly. User data has to be kept safe and a seamless payment process is very important in providing the best user experience. A user shouldn’t feel burdened or afraid while making the payment.
Successful Payment: Once the payment has been made, ensure that the changes reflect immediately in the account to prevent the user from having any doubts. Also, make sure the user is redirected back to the platform after a successful transaction.
Failure Scenarios: Make sure to think of various failure scenarios and test them out.
- Check what happens if the session ends during the payment process.
- Determine what happens if the payment gateway stops responding during the payment process.
- Make sure the account doesn’t get activated despite having a failed transaction.
- Observe what happens if the payment or transaction goes on hold.
- Check if the session expires within the time limit.
Payment Options: Try making a payment using all the various options like a credit card, debit card, and so on.
Localization: Make sure the amount is displayed in the current currency and the plans for that region are only being purchased. Also, try to change the payment gateway language during the payment process.
Pop-up Blocker: Examine the pop-up blocker settings and observe what happens when the pop-up blocker is turned on and off. At times few payment pages open only in a new window and the blocker might affect that.
Backend: Go beyond the front-end OTT platform testing and analyze what happens in the backend during the payment process.
Buffer Pages: Examine the buffer pages that exist between the payment gateway and the application.
Saved Payment options: If the user wishes to save the payment option, examine the database entries to see if they store credit card information.
Double Checks:
- During the payment process, double-check the error and security pages.
- If you are sending any email for the transaction, ensure that the content is encrypted.
- Verify if all the sensitive information is masked.
- Check that the payment is being made through a secure channel. In other words, the link must begin with HTTPS and not HTTP.
Gateway: Check if the payment is processed with an integrated page or if it is navigated to the bank’s payment page. If it is a page within the platform, make sure the transaction is processed only with the right OTP and card information.
OTT Testing Tools
Though there are no dedicated OTT Testing tools that we use for OTT automation testing, we can still use standard automation testing tools such as Appium and BrowserStack. Now let’s see how these tools will be useful when performing OTT testing.
Appium
Since a major share of OTT devices is powered by Android, Appium can be used to perform OTT Automation Testing. We are not just talking about the wide variety of Android Smart TVs in the market, even devices such as Amazon Firestick run on their own Custom OS (FireOS) that is built with Android Nougat as its base. Since Appium is an already popular testing tool, it can serve as the go-to OTT Testing Tool when it comes to automation.
BrowserStack
We have already seen how Appium can be used to achieve OTT Automation Testing on select devices. Another point of concern when it comes to OTT testing is numerous variations in the OTT devices in terms of an operating system, form factor, and so on. So a cloud testing platform such as BrowserStack is essential in assisting us in testing across a wide range of devices on the market. As an OTT Testing tool, BrowserStack currently supports both Android TVs and FireStick. But according to their website, other popular options like Roku and Apple Tv are on their way to the platform.
Conclusion:
We hope you have found these test cases for OTT Platform Testing to be informative. Though there are many more avenues to explore in OTT platform testing, these are the fundamentals you have to follow. The OTT platform is constantly being updated so that it doesn’t get outdated. Seeing the rise of popularity for TikTok, YouTube has introduced the Shorts feature. Understanding the content for short content, Netflix has created the ‘Fast Laughs’ feature. Netflix is also working on integrating games along with their subscription. So the landscape will always be changing and you should be on a journey of constant learning to keep up with all the new challenges.
Frequently Asked Questions
-
What is OTT testing?
OTT testing can be termed as the process of determining the capability of the platform to stream video and audio content over the internet by testing the platform’s UI, functionality, performance, and security.
-
What is OTT Automation?
OTT Automation is similar to any type of automation testing as it is the process of using automation tools to maintain test data, execute the tests, and analyze the results to increase the OTT platform’s overall quality.
-
What is an OTT Device?
Any device other than a desktop, laptop, tablet, or smartphone that is used to consume OTT content is called an OTT device. Some familiar examples are Smart TVs, Amazon FireStick, Roku TV, Apple TV, etc.
-
What is Streaming Testing?
Streaming Testing is used to test if the videos stream with the expected quality & functionality over the internet under different network bandwidths & loads to deliver a great user experience.
-
Are OTT and Connected TV the same?
No. OTT is the process of streaming content directly on a computer or mobile device and Connected TVs require the TV to be connected to a device with an active internet connection such as an Xbox, Apple TV, or Roku TV for streaming content.
