Table of contents

Introduction

Scope and overview

Test Approach

Types of tests

Environment requirements

Use cases

Automation

Introduction

A test plan outlines the approach for testing a specific feature before it is delivered to the customer. In my experience I have seen folks take a different approach to creating test plans  depending on factors like business needs, based on automation frameworks available and the metrics to be measured. I wanted to consider an application that everyone can relate to and the Netflix subscription and cancellation model seemed very intriguing to me!

This test plan document addresses all the test cases related to the Netflix subscription and cancellation model. A first time user gets a 30 day trial period, after which they will be charged based on the credit card information on file.

Scope and overview

The Netflix subscription model has 3 different types of streaming plans:

• Basic
• Standard
• Premium

And 2 different DVD plans:

• Standard
• Premier

All these features need to be tested for 25 major countries across the world and in their respective locales. There could be slight variants in each feature according to the geographical location.

Each profile in an account can be managed based on various maturity ratings:

• TV-Y
• TV-Y7
• TV-G
• G
• TV-PG
• PG
• PG-13
• TV-14
• R
• TV-MA
• NC-17

Kids specific experience can be derived from creating a profile that curates only shows for kids. This profile removes direct access to account settings, thereby enforcing parental controls.

Test Approach

In this section, we define the following:

• Types of tests
• Environment requirements

Types of tests

Developers will be responsible for writing unit tests. Quality engineers will be responsible for writing:

• Integration tests
• Contract tests
• UI tests
• Load/Performance tests

Environment requirements

For delivering a high quality product, we want to test the feature in:

• Staging
• Pre-production
• Production

We need to be able to whitelist users in the above environments to ensure seamless testing.

Use cases

The high level outline of major scenarios are covered in this section.

Scenario 1: Testing User sign up and cancel

1. New users can sign up and are given a 30 day free trial. Users can choose to cancel subscription anytime within the 30 days without being charged.
2. Validate credit card information – if credit card expires after 30 day trial, user access should be suspended.
3. Existing users can cancel anytime and will have access to account till the end of the billing period.
4. Validate existing users can reactivate their accounts within 10 months and retrieve account settings and preferences. What if the user tries to reactivate 1 day after the 10 month period – will they be able to retrieve their settings and preferences?

Scenario 2: Testing Streaming plans

Users can sign up for one of the 3 streaming plans:

1. Basic plan:
   1. Validate available only on 1 screen
   2. Validate Standard definition video quality
   3. Validate Can be downloaded only on 1 device
2. Standard plan:
   1. Validate Can be used on 2 screens simultaneously
   2. Validate Full HD (1080p) video quality
   3. Validate Can be downloaded on 2 devices
3. Premium plan:
   1. Validate Availablity on 4 screens at a time
   2. Validate Full HD(1080p) & Ultra HD (4K)
   3. Validate Can be downloaded on 4 devices
4. If users try to switch between plans before the end of their billing period, how is the monthly fee pro-rated?
5. If users switch between 3 different plans in a given month, how is their monthly bill pro-rated?
6. Sign out of all devices and validate the availability of each plan

Scenario 3: Testing Auto cancel subscription

This recently added feature ensures that if a user hasn’t used Netflix in the past 1 year, then it sends an email to confirm subscription continuation. If the user doesn’t respond, then it auto cancels the account.

1. Users can always reactivate their account within 10 months of cancelling the account and retrieve their previous settings and preferences.
2. If credit card on file expires before the first year mark, then the subscription stands cancelled.
3. Freeloaders:
   1. Based on the plan, there can be ‘n’ number of secondary users added to an account. If a secondary user tries to cancel the subscription, then it sends a confirmation email to the account owner to validate.
   2. Also freeloaders could reset the clock even if the account owner doesn’t use the application.

Scenario 4: Testing DVD plan

Netflix still supports the DVD rental feature which offers 2 plans:

1. Standard plan:
   1. 7.99/ month
   2. 1 disc can be borrowed at a time
   3. Unlimited discs per month
   4. Cancel anytime
2. Premier plan
   1. 11.99/ month
   2. 2 discs can be borrowed at a time
   3. Unlimited discs per month
   4. Cancel anytime
3. Free trial
   1. Upto 30 days
   2. Cancel anytime
4. Borrow DVD and cancel subscription without returning
5. Borrow DVD and try to place another order before returning
6. Borrow DVD and if credit card expires before return, account gets suspended. User should be penalized for not returning DVD

Scenario 5: a.Profile management based on maturity ratings

There are 11 maturity ratings as defined here:

Account -> Viewing restrictions -> Profile maturity rating

Verify that the home feed corresponds to the maturity rating selections.

Scenario 5.b: Kids profiles

Verify that the home feed and suggestions are relevant to the maturity ratings selected for kids.

Validate that users do not have access to account settings.

Scenario 6: Locales

Ensure all the above scenarios work for 25 different countries and their locales across the world.

Automation

The above scenarios can be automated on the following platforms:

• Web
• iOS
• Android
• Mobile web
• Roku
• Apple TV
• Fire TV

By changing the country configuration in the backend to validate that the above scenarios work for different locations.

Integration tests are also critical to ensure that all components work well together. Contract tests to ensure API contracts are in place and these tests should be run in CI against PR commits apart from unit tests.

Hope you find this approach useful. I am curios to know about the items you consider in your test plan for the same application. Share your thoughts and let’s learn together!

• LinkedIn
• Twitter

---

---