Splunk | Quantum project with Splunk integration

Review a Quantum Project with the ability to feed test result data to Splunk.

Important: This document includes references to a third-party product, Splunk. The user interface and usage of third-party products are subject to change without notice. For the latest published information about Splunk, see https://docs.splunk.com/Documentation.

On this page:

Pre-req

To use this project, follow the series HTTP Event Collector to ensure your Splunk instance is configured. For use with this series, ensure the index you create is named "testindex" for compatibility of code samples and dashboards.

GitHub project

A Quantum project containing everything you need to get started as well as an example test can be found here.

The project that is found in the Github directory is ready to be used out of the box. You only need to modify a couple of files. When using this project, it is important to import it as a Maven project.

Modify application.properties (/resources) file with your Perfecto cloud, username, password, and add your Splunk properties to match your Splunk server configuration.
Modify the TestNG.xml to match the devices you'd like to select based on either deviceName or device capabilities (src/main/resources/config/AmazonSearch.xml)

You can take a look at the scenario file that executes against Amazon.com and searches for an item (src/main/resources/scenarios/AmazonSearch.feature) to see how the Splunk transactions are performed and generated.

Integration overview

You can add this capability to an existing Quantum framework by reviewing the below integration guide. Further below, you'll find each of the files stand-alone, or you can copy them from the GitHub project and add to your existing project.

com.quantum.listeners (src/main/java)
- SplunkHelper Class
  No modifications should be required here. This is a helper class for the TestListener and PerfectoSplunkSteps classes.
- TestListener Class
  This is a standard TestNG listener class. All of the JSON data that is fed to Splunk is gathered in this listener. To add your own custom fields, you can use the below syntax:
```
SplunkHelper.getCollector().reporting.put("nodeName","nodeValue");
```
- com.quantum.steps.CommonSteps (src/main/java)
  - PerfectoSplunkSteps Class
    This class contains both an OCR and Image Analysis Splunk Transaction that you can utilize in your BDD test cases. No modifications to this class should be made. Behind the scenes transaction time will be captured and compared against the SLA value you inserted to consider the test a pass or fail. Example usage for a scenario (src/main/resources/scenarios) file below:
    Copy
    Then I perform a Splunk transaction using the name "2 searched item" and description of "Searching for an item" with an SLA of "60000" - I'm also utilizing an OCR checkpoint for the word "Best Books of the Month" with a timeout of "60" and threshold of "100" Then I perform a Splunk transaction using the name "2 Audio Played?" and description of "Verifying audio loaded" with an SLA of "60000" - I'm also utilizing an Image checkpoint for the image "PUBLIC:Audio/Stop_Button_GS4.png" with a timeout of "60" and threshold of "50"

OCR Transaction. Read more here on the checkpoint command.

Parameter	Value
Transaction name	The name of the step
Transaction description	A short description explaining what the step is attempting to validate
SLA	The value you wish for the transaction time to be below to consider the transaction a pass or failure based on performance
Word	The word you are searching for on the screen
Timeout	The amount of time you wish to wait for the word to appear before throwing and error
Threshold	How "off" the word can be for what you are expecting to consider it a pass

Image Transaction. Read more here on the checkpoint command.

Parameter	Value
Transaction name	The name of the step
Transaction description	A short description explaining what the step is attempting to validate
SLA	The value you wish for the transaction time to be below to consider the transaction a pass or failure based on performance
Repository	The location for the source image you are wanting to compare to the screen
Timeout	The amount of time you wish to wait for the word to appear before throwing and error
Threshold	How "off" the image can be for what you are expecting to consider it a pass

TestNG XML (src/main/resources/config)

Register the listener class from the com.quantum.listerners package in the TestNG.xml configuration file by modifying the listeners section as seen below:
Copy
```
<listeners>
    <listener class-name="com.quantum.listerners.QuantumReportiumListener" />
    <listener class-name="com.quantum.listerners.TestListener" />
</listeners>
```

Application.properties

Add the below to the top of application.properties (/resources) and modify to match your Splunk server setup

Copy

#Splunk Config
globalSLA=20000
splunkSchema=http
splunkHost=127.0.0.1
splunkPort=8088
splunkToken=65984A70-7F01-441C-9EE8-112836EC4D62
#end Splunk Config

Pom.xml

Need to add the following repository

Copy

 <repository>
     <id>jitpack.io</id>
     <url>https://jitpack.io</url>
 </repository>

Add the below dependency

Copy

<dependency>
    <groupId>com.github.PerfectoCode</groupId>
    <artifactId>PerfectoSplunk</artifactId>
    <version>8e9425e</version>
</dependency>

Standalone files

TestNG.xml needs to be added/modified in the src/main/resources/config directory.

Copy

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Appium Demo Suite" verbose="0" thread-count="2" parallel="tests">
    <listeners>
        <listener class-name="com.quantum.listerners.QuantumReportiumListener" />
        <listener class-name="com.quantum.listerners.TestListener" />
        </listeners>
        <parameter name="driver.name" value="perfectoRemoteDriver" />
        <test name="Android Amazon Search Test" enabled="true">
            <parameter name="perfecto.env.resources" value="src/main/resources/android" />
            <parameter name="perfecto.capabilities.deviceName" value="04157DF43A656B1A" />
            <parameter name="perfecto.capabilities.browserName" value="mobileOS"></parameter>
            <groups>
                <run>
                    <include name="@amazonSearch" />
                </run>
            </groups>
            <classes>
                <class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
            </classes>
        </test>
        <test name="iOS Amazon Search Test" enabled="true">
            <parameter name="perfecto.env.resources" value="src/main/resources/ios" />
            <parameter name="perfecto.capabilities.deviceName" value="1C3B401545D2CDBEC9D323460D914AD7319F31D9" />
            <parameter name="perfecto.capabilities.browserName" value="mobileOS"></parameter>
            <groups>
                <run>
                    <include name="@amazonSearch" />
                </run>
            </groups>
            <classes>
                <class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
            </classes>
        </test>

application.properties need to be added or modified in the /resources directory:

https://github.com/PerfectoCode/Community-Samples/blob/master/QuantumSplunk/resources/application.properties
Create a package (com.quantum.steps.CommonSteps) in src/main/java and add the below class

https://github.com/PerfectoCode/Community-Samples/blob/master/QuantumSplunk/src/main/java/com/quantum/steps/CommonSteps/PerfectoSplunkSteps.java
Create a package (com.quantum.listerners) in src/main/java and add the below class

https://github.com/PerfectoCode/Community-Samples/blob/master/QuantumSplunk/src/main/java/com/quantum/listerners/SplunkHelper.java
Create a package (com.quantum.listerners) in src/main/java and add the below class

https://github.com/PerfectoCode/Community-Samples/blob/master/QuantumSplunk/src/main/java/com/quantum/listerners/TestListener.java
Modify your scenarios (src/main/resources/scenarios) to utilize the Splunk Transactions as defined in the AmazonSearch feature file

https://github.com/PerfectoCode/Community-Samples/blob/master/QuantumSplunk/src/main/resources/scenarios/AmazonSearch.feature