The Challenge

Cengage, a US-based educational content, technology, and services company that operates in more than 20 countries across the world was in need of an offshore software testing team to perform manual ETL testing. Since they provided services such as online classes, assignments, exams, and so on, there were individual reports for every book, student, instructor, course, platform, etc. So our primary goal was to identify issues such as source duplication, null values, and target duplication that were impacting the quality of their data.

Our Solutions

• Introduced automation testing to save countless man-hours.
• Set up a Robust Reporting framework using Lemon Cheesecake & Snowflake.
• Achieved CI/CD Integration by using Jenkins.
• Improved Data Quality by introducing Data Profiling with the help of Looker.
• Followed Agile methodology & supported every release without any delay.

Introducing Automation

Though we were tasked with performing only manual testing, we identified a scope for automation testing that could significantly streamline the entire process and save a lot of time and cost. So we wrote our automation script in LemonCheesecake framework using Python and demoed it to our clients. They were immediately impressed with the results and engaged us for automation testing as well.

As firm believers in continuous improvement, we never stopped improving our testing process. With the rise of DBT’s popularity, we identified its true potential and successfully implemented it in our project.

CI/CD Integration in an Agile environment

As our client followed Agile methodology, we had a release every sprint that would last 2 weeks. Over the course of our 7-year partnership, we are happy to claim that we have never been late on our deliverables. We never missed a deadline even if when we had to test a single database that was more than 3 TB in size. We achieved CI/CD Integration in this project by using tools such as Jenkins and Apache Airflow.

Our Reporting Pattern

Tracking the history of failures was an important necessity. So we opted to use the Lemon Cheesecake tool to achieve full integration with Snowflake as it helped us report the number of test cases passed and failed on a daily basis. In addition to that, we also ensured the test results were directly ingested into the database to keep track of the results with ease.

Data Profiling with Looker

Data was flowing to their databases from different sources such as the web, mobile, and so on. So with the implementation of data profiling, we were able to identify quality issues that would usually go undetected if we tested only the source and target databases. So we used Looker to create dedicated dashboards that would display the automation testing results for aspects such as level of ebook usage, number of subscriptions, activations, and so on.