Page tree
Skip to end of metadata
Go to start of metadata

Last updated: Apr 04, 2019 11:43

by Prasant Sutaria

Problem:

An unexpected number of results in Digital Zoom's CI dashboard.

Cause:

Jasmine framework provides two options to disable some specs from a spec file:


OptionsExplanation
1xit

Mark all the spec with xit which needs to disable.

Example:

Disabled Spec
it("Validate Perfecto's title", function() {
  console.log("This spec will get execute.");
});


xit("Commented Scenario", function() {
  console.log("Its a commented Spec");
});
2fit

Mark the spec which needs to execute with fit, rest other tests are skipped.

Disabled Spec
fit("Validate Perfecto's title", function() {
  console.log("This spec will get execute.");
});


it("Commented Scenario", function() {
  console.log("This Spec will be ignored");
});

Even though the above specs are excluded, the jasmine hooks - specStarted and specDone are still executed for the specs, which leads to an unexpected number of reports in Digital Zoom's CI dashboard. Since these hooks are used to start and end Digital Zoom reports.

Expected test count is 1 but in Digital Zoom's CI dashboard we are getting unexpected count 2 test.

Solution:

To prevent unexpected reports in Digital Zoom's CI Dashboard following solution can be implemented:
  1. Use xit option to exclude specs in a spec file instead of fit option.
  2. When xit option is used, Jasmine provides a pending reason - "Temporarily disabled with xit" for the excluded specs. We can use the pending reason to check whether reporting in Digital zoom should be started or not

    spec started
    // Start of Spec
    specStarted: function (result) {
       if(result.pendingReason!=='Temporarily disabled with xit'){
          browser.reportingClient.testStart(result.fullName);
       }
    },

3. The specs can have 3 states - Passed, Failed and Pending. The excluded specs have "Pending" status. We can perform no action in case of "Pending" status.

spec done
      // End of Spec
      specDone: function (result) {
        switch(result.status){
          // Spec Passed
          case "passed":
            browser.reportingClient.testStop({
              status: Reporting.Constants.results.passed
            });
            break;
          // Spec Failed
          case "failed":
            const failure = result.failedExpectations[result.failedExpectations.length - 1];
            var failureType = getFailureReason(failure.message);
            console.log(failureType);
            var failedOptions = {
              status: Reporting.Constants.results.failed,
              message: `${failure.message} ${failure.stack}`
            };

            if(failureType!=""){
              failedOptions['failureReason']=failureType;
            }

            browser.reportingClient.testStop(failedOptions);
            break;
          // Spec Pending - No actions needed
          default:
        }
      }


The sample project can be downloaded from here.

How to run the tests:

  1. Extract the project
  2. Open command prompt
  3. Execute npm install
  4. Execute npm test
  • No labels