One of the best ways to increase productivity in software development is to automate manual processes and tests. In this article, I will walk you through how to run tests on demand and alert the team using the slack app regarding the test results. This will boost the confidence of the client, product owner, project manager, developers, and the QA team. This will definitely further help them make appropriate decisions.
There are three important stages of the API testing life cycle
- Write good tests
- Run the tests on demand
- Send an alert or notification with test summary reports
This figure provides the resilient process that you need and ensures that all the functions work with each other. API testing pipeline shows how these three pieces fit together and gives you an overall bigger picture.
What is Postman Collection and Environment?
A collection is a group of saved API requests. The collections can be organized into folders and they can be easily shared and exported when necessary. All the API requests can be categorized and saved within a collection. You can add the test scripts to the API requests.
The environment is a set of variables that you can use for API requests. Let’s assume we have got three different URLs of our applications. One for production, another for staging and another for development. Since there will be three different URLs we can make three environments, one for each.
All the collections are stored in the cloud of the Postman. The beauty of Postman is that we can access the collection on the fly just using the Postman API. In order to achieve this, we need three things.
- Collection
- Collection ID and it’s environment ID if it has any
- Postman API
Generate Postman API Key
- Go to the Postman API Key Page.
- Click on the “Generate API Key”.
- Enter a name for your key and click on “Generate API Key”.
- Copy the Key and click on “Close”.
Once you have generated the Postman API key, you can paste it somewhere you can access it for the next step
GET IDs of Collection and Environment
Collection ID
Hover over the collection, you will see the three-dot menu.
Click on the three-dot menu of the collection.
Click on the Edit.
Copy the collection UID and paste it where you can access it.
GET Environment ID:
Click on the Environment which is on the left sidebar.
Click on the three-dot menu of the environment.
Click on Share.
A pop-up appears. Similar to the screenshot below.
Scroll to the right and copy the UID of the environment. Paste the environment UID where you can access it. We need this UID to access the environment file at a later time.
Note: DO NOT share the API key with anyone.
Link: https://learning.postman.com/docs/developer/intro-api/
Run the following command to run the test scripts in the command line.
newman run https://api.getpostman.com/collections/{{collection_id}}?apikey={{postman-api-key-here}}
-e https://api.getpostman.com/environments/{{environment_id}}?apikey={{postman-api-key-here}}
Create a Slack App for Notifications
We need a slack app in order to send the test reports. So it’s now time to create a slack app.
First, navigate to the URL below.
Click on “Create an app”.
Click on the “Create New App.”
A kind of dialog appears on the screen. Select the first option.
Let’s provide the slack name as “newman-runner” and its workspace name as “My Workspace.”
Click on “Create App” to create an app. Now, we have created an app called “newman-runner” and specified its workspace. Since we need to send reports to the slack app, we need to activate the webhook.
Click on the “Incoming Webhooks” from the sidebar. Click on the button to activate the incoming webhooks.
Scroll down and click on the “Add New Webhook to Workspace.”
You can select the channel. You have now successfully created a webhook URL. Copy the URL and paste it where you can access it. We need this webhook URL at a later time.
You will see a message in which the app is integrated. Something similar to this. We have created a slack app and webhook is also integrated.
What are GitHub Actions?
GitHub actions help automate tasks in the software development life cycle. GitHub actions make it easier to automate how you build, test, and deploy your applications on any platform such as Windows, macOS, and Linux. GitHub actions workflow allows you to perform certain kinds of actions when an event occurs on the repository. The trigger events can be something like “a push is made to any branch of a repo” or ”a push is made to the master branch” or “when a pull request is created on dev or master branch”. GitHub Actions workflow can be triggered based on these events. The great thing about the GitHub action is you can customize the code the way you want your test scripts to run. For more information, visit. https://github.com/features/actions
GitHub Actions uses YAML syntax to define a workflow. Each workflow must have a .yml or yaml file extension. Each workflow is stored as a separate YAML file in the code repository, in a directory called .github/workflows.
Now, let’s get started.
- Create a folder structure .github/workflow in your project.
- Create a new file ‘.yml’ and add the following code.
Workflow Steps
- Name and Workflow events actions
We can build our workflow by specifying the name of the GitHub workflow. We need to define the event that triggers our test to run. In our case, let’s run our tests when there is a push event in the dev branch.
# Name of the GitHub Actions
name: Continuous Test
# Set the actions that triggers the workflow
on:
# Trigger the workflow on the push event, only in dev branch
push:
branches: [ dev ]
All we need to do is define the job, where we will run the Postman collection with its environment.
2. Spin the virtual machine – in our case Ubuntu
The type of machine to run the job on. The machine can be either a GitHub-hosted runner or a self-hosted runner. For now, let’s go with the “ubuntu-latest”.
jobs:
build:
runs-on: ubuntu-latest
3. Defining Node Version
We need to define the node version that we need to install.
strategy:
matrix:
node-version: [14.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
4. Checking out the code
We can check out the code by using the following sets of commands.
steps:
# Check out the repo, so the job can access it.
- uses: actions/checkout@v2
5. Installing Node and Newman
Newman is the command-line interface of Postman that lets you run and test the Postman collection from the command line. The “newman-reporter-slackmsg” installs a Newman reporter to send messages to Slack.
The “newman-reporter-htmlextra” installs a newman HTML reporter, which creates an HTML report of the collection that you run.
# Install Node on the runner
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
# Install newman globally, the reporter for slack and install the html extra reporter
- name: Install newman
run: |
npm install -g newman
npm install -g newman-reporter-slackreporter
npm install -g newman-reporter-htmlextra
6. Creating Directory for Test Artifacts
We need to upload the artifacts generated during the test run. Github actions provide a way to upload the logs from the job to the workspace. For that, create a directory ‘testArtifacts’, you can name the folder what you want. But for now, let’s keep the name as it is. This is the directory where the test results will be uploaded after the test is completed.
# Create directory to upload test results
- name: Make Directory for Test Results
run: mkdir -p testArtifacts
7. Running Postman Collection using URL and sending reports to slack app using the webhook
We just need the URL to run the collection.
# Run postman collection
- name: Run Postman Collection
run: |
newman run https://api.getpostman.com/collections/{{collection_id}}?apikey={{postman-api-key-here}} -e https://api.getpostman.com/environments/{{environment_id}}?apikey={{postman-api-key-here}} --suppress-exit-code -r slackreporter,htmlextra --reporter-htmlextra-export testArtifacts/htmlreport.html --reporter-slackreporter-webhookurl {{webhook_url}}
8. Uploading Test Artifacts
We need to upload the test artifacts so that we can view them at a later time when needed. Use the code snippet below to upload the artifacts in the directory (testArtifacts) we created earlier on.
# Upload the reports to TestArtifacts directory
- name: Output the results
uses: actions/upload-artifact@v2
with:
name: Reports
path: testArtifacts
9. Complete Continuous Integration Workflow.
# Name of the GitHub Actions
name: Continuous Test
# Set the actions that triggers the workflow
on:
# Trigger the workflow on the push event, only in dev branch
push:
branches: [ dev ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
# Check out the repo, so the job can access it.
- uses: actions/checkout@v2
# Install Node on the runner
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
# Install newman globally, the reporter for slack and install the html extra reporter
- name: Install newman
run: |
npm install -g newman
npm install -g newman-reporter-slackreporter
npm install -g newman-reporter-htmlextra
# Create directory to upload test results
- name: Make Directory for Test Results
run: mkdir -p testArtifacts
# Run postman collection
- name: Run Postman Collection
run: |
newman run https://api.getpostman.com/collections/{{COLLECTION_ID}}?apikey={{POSTMAN-API-KEY-HERE}} -e https://api.getpostman.com/environments/{{ENVIRONMENT_ID}}?apikey={{POSTMAN-API-KEY-HERE}} --suppress-exit-code -r slackreporter,htmlextra --reporter-htmlextra-export testArtifacts/htmlreport.html --reporter-slackreporter-webhookurl {{SLACK_WEBHOOK_URL}}
# Upload the reports to TestArtifacts directory
- name: Output the results
uses: actions/upload-artifact@v2
with:
name: Reports
path: testArtifacts
Security Hardening for GitHub Actions
Security is one of the hot topics nowadays. This is not a good practice to share your sensitive information in plain text in the workflow. Never do that. Let’s hide these API keys from our code using the secrets feature of GitHub Actions.
So we will use the secrets feature of GitHub Actions in our YML file. To create secrets in Github, follow these steps.
- Navigate to the repo.
- Click on the Settings.
- Click on the Secrets.
- Click on the New Repository Secret.
Let’s revisit the YML file that we have created earlier on. Update the file with the following code.
# Name of the GitHub Actions
name: Continuous Test
# Set the actions that triggers the workflow
on:
# Trigger the workflow on the push event, only in dev branch
push:
branches: [ dev ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
# Check out the repo, so the job can access it.
- uses: actions/checkout@v2
# Install Node on the runner
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
# Install newman globally, the reporter for slack and install the html extra reporter
- name: Install newman
run: |
npm install -g newman
npm install -g newman-reporter-slackreporter
npm install -g newman-reporter-htmlextra
# Create directory to upload test results
- name: Make Directory for Test Results
run: mkdir -p testArtifacts
# Run postman collection
- name: Run Postman Collection
env:
COLLECTION_URL: ${{secrets.COLLECTION_URL}}
ENVIRONMENT_URL: ${{secrets.ENVIRONMENT_URL}}
SLACK_WEBHOOK_URL: ${{secrets.SLACK_WEBHOOK_URL}}
run: |
newman run ${{COLLECTION_URL}} -e ${{ENVIRONMENT_URL}} --suppress-exit-code -r slackreporter,htmlextra --reporter-htmlextra-export testArtifacts/htmlreport.html --reporter-slackreporter-webhookurl ${{SLACK_WEBHOOK_URL}}
# Upload the reports to TestArtifacts directory
- name: Output the results
uses: actions/upload-artifact@v2
with:
name: Reports
path: testArtifacts
Commit these changes and push them to the repository that you are working on. GitHub will automatically recognize if there are any trigger events (in our case push events on the dev branch) and run the test scripts accordingly. The test report will be sent to the Slack channel, where your team can view them. Another report will also be generated at the end of the cycle, where you can download the file.
Try it Out
So, this was some useful information on how to run Postman collections using GitHub Actions, its workflow and sending test summary reports to the slack application. Let this be your first step, start building more complex situations, and you can always adapt to your project’s nature and implement them as .
If you get stuck implementing this, have any questions, or just want to say hello, feel free to connect with me on Linkedin.
Happy Testing!!!
