Playwright with Test Runners

Playwright with Jest and Playwright native Test Runner

Wanted to share this as I gave a try and experienced a bit with Playwright on both test runners which are Jest and also with Playwright's native test runner Playwright Test. I have jotted down a few points on both of these test runners with JavaScript.

Let's get started ..!!!

Few differences between Playwright-Jest Vs Playwright Test

Vs

Authentication to skip Login - Usually we use microsoft authentication and on this part both the test runners are pretty much the same. We can actually add cookies or save the storage state to skip the login for multiple tests in one shot. To do that first open the Playwright's Codegen.

On your terminal on VS code, run this command npx playwright codegen --save-storage=cookie.json and you will see two windows opening up one is Incognito browser and other is Playwright Inspector like below.

Type the command and hit enter. You will see something like below

Now type in your website name on this incognito browser opened for you to store the cookies with name 'cookie.json' in your project folder

I'm using the website Automation Practice for the demo purpose and just as an example. 

Basically you navigate through the website manually and on the right you see your playwright inspector adding code for you. You can use this code or you may not depending on your priority. I've just navigated manually on this website using Playwright’s codegen here and saved the necessary cookies for my login.

Once you are done on navigating through different options, you can close the playwright inspector and the incognito browser.

Once you close the two windows you will see automatically 'cookie.json' file is saved to your project folder on VS code.

This file is the required cookie file for you to use to skip the login for your tests. After this run the following command below

npx playwright open --load-storage=cookie.json http://automationpractice.com/index.php(Your Website name) 

on your VS Code Terminal

Now, run the command 

npx playwright codegen --load-storage=cookie.json http://automationpractice.com/index.php  

on your VS Code Terminal to see how playwright writes your test code for you to use as shown below. 

You run these last two --load-storage commands mentioned above to restore all your cookies and local storage for your web apps to authenticated state.

In my case I validated the last two commands but mostly I've used only 'cookie.json' generated from the very first command mentioned above here which is 

npx playwright codegen --save-storage=cookie.json

For reference : Command Line tools

Now in your code you can add this like in two ways: -

For Authentication reference: Playwright-Authentication

With Playwright-Jest:

• Add Cookies : 

                 const fs = require('fs')

               const { chromium } = require('playwright')

               let browser, context, page

                    browser = await chromium.launch( { headless : false} )

context = await browser.newContext({

viewport: {

width: 1920,

height: 1080,

},

})

const cookies = JSON.parse(fs.readFileSync('./cookie.json', 'utf8'))

await context.addCookies(cookies)

page = await context.newPage()

await page.goto('Your Url')

                                 ( OR )

• Storage State : 

               let browser, context, page

                    browser = await chromium.launch()

page = await browser.newPage()

await page.goto('Your Url')

                  await page.context().storageState({ path: './cookie.json' });

With Playwright Test Runner: 

Playwright.config.(js or ts) file

projects: [

{

name: `Chrome`,

use: {

storageState: 'cookie.json',

browserName: `chromium`,

channel: `chrome`,

headless: false,

viewport: { width: 1720, height: 850 },

screenshot: `only-on-failure`,

video: `retain-on-failure`,

trace: `retain-on-failure`

}

},

I preferred the second one -----> "With Playwright Test Runner" First capture the cookies with playwright codegen command "npx playwright codegen --save-storage=cookie.json" and a new 'cookie.json' file is saved into your root directory with all the required cookies in it. Now, add this line storageState: 'cookie.json' like shown above in "playwright.config.js" or "playwright.config.ts" file and you are pretty much good to run your tests which will skip login.

Assertions - We get a little more assertions with Playwright Test compared to Jest.

For reference: expect-playwright and assertions Playwright version 1.14

Retries -  Jest when using with jest-circus/runner and annotation with 'test' only not with 'it' the retries work for tests with Jest. In the test we can use like jest.retryTimes(1) for example to run the test one more time incase of failure. 

However, you have to do a global setup and teardown in jest and this jest retries did not show me any console message that it retried for 1 time.

Here, Playwright test runner is absolutely brilliant where it provides this retries inbuilt when using the Playwright Test. It retries the failed test and gives you console message as well like below.

In your playwright.config.js this is how it looks like:

Screenshot & Videos for failed tests - You can get screenshots of failed tests in both Playwright-Jest and Playwright Test. 

Screenshots are also pretty much easy with Playwright Test all you need is to just add a simple and single line of code in your config file. With Jest you have to do a set up by adding a long line of code using CustomEnvironment.js file.

Video of the tests are one of the options which jest did not have where in Playwright Test have this feature inbuilt defining in its playwright.config.js file.

Screenshot set up with Jest-Playwright:

Screenshot and Video set up with Playwright Test:

Element Screenshot - You have this feature of capturing a screenshot at element level only in Playwright Test and this feature is not available in Playwright-Jest.

For reference:https://playwright.dev/docs/screenshots/#element-screenshot

Test Execution Time - Both Playwright-Jest and Playwright Test execute tests in parallel by default which is a fantastic feature when compared to any other tools like Cypress.The test execution time is almost the same in both but Playwright Test is a bit faster than Playwright-Jest with what I've experienced.

Reporter options on your console - With Playwright Test runner you have various reporting options to display your test results on the console like list, line, json, dot, junit which you don’t have with Playwright-Jest.

 

Test Reporting options - you have different options like Jest-stare, jest-html-reporter and Allure with Playwright-Jest. Playwright Test is launched recently in the month of June 2021 with playwright version 1.12 Playwright Test I think, and it's very new in the market. 

Playwright supports Allure reporting with npm package to install “experimental-allure-playwright“

We can get a test report something like this below:

With Playwright Version 1.16 playwright team introduced their own new html reporting 

option with the command “npx playwright show-report” and you can see the screenshot, trace and video of the failed test attached to the html report as shown below. cool isn't :) !

Playwright team is doing a fantastic job already coming with various new features with playwright.

Playwright Test runner supports both Allure reporting and Playwright's own Html Reporting now as mentioned above.

Locator keyword - With Playwright version 1.14 we have the keyword 'locator' in place which is just replacement of '$' to 'locator' for example:

From

const element = page.$('selector')

To

const element = page.locator('selector')

API Testing -  With Playwright version 1.16 we can do API testing just like supertest for example. Release link : playwright v1.16

Official documentation : Playwright API Testing

Conclusion:

In my case I felt both are brilliant and not much a difference but only some features like retries, video capture, console test result display, assertions with expect, tracing options are pushing me towards Playwright Test runner. 

Working with playwright test runner definitely blown away my mind with it's concept of Fixtures and Folio on TypeScript and other cool features.

Note: Folio is deprecated and archived by the owner now and replaced with Playwright test runner v1.12.0 Playwright test runner

Folio deprecated, GitHub repo: Folio

It is definitely faster when working with JavaScript as well, giving you options and features which are specific in meeting your requirements. There's much more for you to explore on it in the store.

For sample boiler plate code on GitHub:Playwright-JavaScript please click the link below

•  Playwright with Jest test runner:  Playwright-Jest

•  Playwright test runner:  Playwright Test (Without using Page fixture in Tests)  

•  Playwright test runner: Playwright Test Runner v1.16

For sample boiler plate code on GitHub:Playwright-TypeScript please click the link below

• Playwright test runner with TypeScript: Playwright with TypeScript            

 

Happy Automation Testing Guys !!! :)