Home Guide Nightwatch vs Protractor vs Mocha : The Preferred Testing Framework

Nightwatch vs Protractor vs Mocha : The Preferred Testing Framework

By Hamid Akhtar, Community Contributor and Pradeep Krishnakumar, Manager -

Amidst many commercial and open-source tools, Selenium is the most widely used open-source tool for automating testing suite for web applications across a range of platforms and browsers.

The significant advantages of using Selenium include –

  • No licensing costs
  • Supports multiple programming languages like Java, Python, Ruby, PHP, Perl, and C#
  • Easily integrate it with JUnit and TestNG for generating reports and managing test cases

Selenium tests instruct the browser what to do. Selenium enables teams to perform tests on Windows, Linux, or Mac, and in browsers like Chrome and Safari.

Selenium includes WebDriver API and RemoteWebDriverclass as a Selenium client, and the server part consists of –

  • A server component that receives from the client
  • WebDriver API
  • Selenium Grid

WebDriver is the core component of the Selenium framework. WebDriver enables one to perform automated cross browser testing irrespective of the programming language used. It provides certain best practices for conducting automation tests.

After a notable breakthrough in the javascript framework, it becomes inevitable to explore Selenium automation with the most popular test automation frameworks like Mocha, Nightwatch, and Protractor.

Javascript frameworks are end-to-end frameworks and, choosing the right one is a challenging task based on –

  • Assertion Library
  • Integration test
  • Cross Browser Layout Testing
  • BDD/TDD Integration
  • Regular updates from the community
  • End-to-End Testing (E2E)

Advantages of using Javascript frameworks include –

  • One can efficiently conduct end-to-end tests
  • Gives the flexibility of unit testing, to enable faster shipping
  • It establishes an active collaboration between Dev and QA team
  • The selection of the right tool in the same language gives extra benefits

Let us get started with the intended frameworks of this article!

Mocha

Mocha is a Nodejs testing library that enables one to run tests sequentially. Mocha provides a great feature and an API that structures the testing code into test case modules and test suites. For running Mocha, one can use the command line(CLI) or run programmatically.

To install Mocha use the npm command:

$ npm install -g mocha
Also, as a project dependency, use the command:
$ npm install --save-dev mocha

One can observe the regular use of Mocha in application tests. For executing the test suite, Mocha has ‘describe‘ keyword. The ‘describe‘ keyword accepts a callback function.

For each describe block, Mocha provides the hooks which can be invoked with any optional description that makes it easy to pinpoint errors. before(), after(), beforeEach(), afterEach() are hooks that run before and after each test case as well as the whole suite.

The easiest way to implement Selenium tests, is probably the “async/await” approach. The selenium web driver which returns Promises makes the approach easy. Given below is package.json that includes dependencies for creating necessary tests. In such a case, make sure the driver and browser version should be compatible with each other. In this case, use the chrome version >= 67.0.3396.0 and if you are using an older version of chrome, then change the driver

{
"name": "selenium-mocha-chai-testing",
"scripts": {
"test": "node Tests/DefaultTest.js"
},
"private": true,
"devDependencies": {
"chromedriver": "^2.41.0",
"selenium-webdriver": "^4.0.0-alpha.1"
}
}

Chai is an assertion library that helps to implement test suites. When you add chai dependencies, the addition would look like:

"devDependencies": {
"chai": "^4.1.2",
"chromedriver": "^2.41.0",
"mocha": "^5.2.0",
"selenium-webdriver": "^4.0.0-alpha.1"
}

An example on their website would be:

var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is not present', function() {
assert.equal([1,2,3].indexOf(4), -1);
});
});
});

Pros of using Mocha –

  • Mocha is flexible. Mocha doesn’t come with the mocking framework or assertion library. That means one can select any library or framework. One popular choice is Chai and Sinon for mocking
  • Mocha is mature with widespread adoption. Around Mocha, a lot of tools have been built
  • As Mocha has an active community, one can get firm support through blog posts, videos, libraries

Cons of using Mocha –

  • Mocha requires more configuration, which makes it weaker
  • If one can’t afford flexibility, one has to choose an assertion library explicitly
  • Mocha snapshot testing is not easy. For that one has to deal with a library called chai-jest-snapshot

Nightwatch

Nightwatch is another testing framework for web applications and websites. The main advantage of Nightwatch includes:

  • Clear syntax
  • Has built-in command-line test runner
  • Controls standalone selenium server in a separate child process. Nightwatch automatically manages the WebDriver services (Edge, Safari, GeckoDriver)
  • Flexible command and assertion framework that makes it easy to extend an application custom command and assertions.

Nightwatch is beneficial for e2e automation testing as it enables us to write end-to-end tests in node.js quickly and effortlessly. Nightwatch simplifies the process of continuous integration and set up automated tests. With Nightwatch, one can comprehend and configure the Selenium test suit and select the specific browser to execute tests. It allows executing the test cases in parallel that further reduces the build time.

Install Nightwatch through npm

Download Selenium JavaScript bindings from the official Selenium website.

Once downloaded, extract the zip file and add Selenium binding and all the dependent libraries to the classpath.

For selenium dependencies, use the command

npm i selenium-webdriver

Pros of using Nightwatch –

  • With simple and powerful syntax, one can write tests quickly
  • It works with cloud testing providers such as BrowserStack
  • Continuous Integration Support: Integrate your tests with systems such as Jenkins, Hudson, Teamcity

Cons of using Nightwatch –

  • It comes with its own testing framework and hence less flexibility to your own unit testing
  • Slightly lesser support

Protractor

Protractor is an essential framework for testing Angularjs applications. Also, it is beneficial for writing automated regression tests. Sometimes, it is hard to detect web elements in Angularjs application, and usually, selenium web driver fails with the extra HTML attributes like ng-controller, ng-model. Thus, Protractor on top of Selenium controls those elements and focus on testing the actual functionality.

Advantages of Protractor when it uses Selenium to automate browser behavior –

  • It provides a remarkable speed as compared to other libraries or tools
  • One can ignore the synchronization strategy as Protractor has built-in intelligence. Protractor connects with angular js framework and detects when protractor has finished rendering
  • Parallel testing through several browsers
  • Protractor has built-in support for many elements of angular.js application; the list is as follows-
    • by.binding
    • by.model
    • by.repeater
    • by.options
    • by.excatBinding
    • by.exactRepeater

Cons of using Protractor –

  • Debugging is tricky
  • One should know javascript as it is available in one client language
  • Doesn’t support mobile apps
  • It is a wrapper to WebdriverJs so if there is an issue with WebdriverJs, It will halt the protractor team

Comparing Mocha vs Nightwatch vs Protractor

MochaNightwatch Protractor
Simple, fun test framework with flexible and extendable configurationEasy to use E2E testing solution for websites and browser-based apps, using the W3C WebDriver APIWebDriver E2E test wrapper for angular 
Has many extensions to test unique scenarios, and many plugins tooNo typescript support, and the library seems to be less supportedTypeScript support is available. The library is maintained by the huge Angular team
An easy syntax that can be written for any testLanguage is simple, only basic knowledge of object-oriented programming requiredEasily write and manage page objects that help write cleaner tests
Mocha lets us specify what the test output looks like. One can define a –reporter flag to the package.jsonError report in XML or HTML formatGood Mechanism for error reporting

Conclusion

  • Test automation has empowered web testers. Cross browser testing is time-consuming, and automated selenium testing can save time during regression testing
  • Mocha is beneficial for both unit and integration testing. Mocha also has good documentation and strong community support
  • Protractor with Selenium is best if one needs to run a full end-to-end test, that opens a browser and performs a DOM manipulation. Also, Protractor is specifically good for angular apps.
  • In Nightwatch, asserts are done automatically, and Nightwatch cleans console output. Nightwatch automatically generates a test report and saves it to a /reports directory. With Protractor, some elements may be hard to detect, and in that case, you need element explorer.

It’s suggested to select a tool based on the application and knowledge levels of the team. Various questions should be analyzed before making the final choice like:

  • Is the application built on React, Angular, or any other framework?
  • Do you need a particular testing framework like Mocha?
  • Learning a new language enhances the skills of team members, and working on the common ground keeps them motivated.

BrowserStack Logo Run Selenium Tests on 2000+ Browsers & Devices Get Started Free