Skip to main content

Integrate Azure Pipelines with Quality Dashboard

Integrate your Azure Pipelines CI setup with BrowserStack Quality Dashboard and view insights on different pipelines and jobs configured in Azure Pipelines.

Connect to Azure Pipelines using Azure DevOps PAT

You can configure the integration using an Azure DevOps Personal Access Token (PAT).

Create an Azure DevOps Personal Access Token (PAT)

  1. Follow this Azure DevOps guide to create a personal access token within your Azure DevOps user account.
  2. Create a token for all organizations by choosing ‘All accessible organizations’ from the Organization dropdown and grant the following permissions to the token so that BrowserStack Quality Dashboard can collect data to generate insights:
    • Build: Read & Write
    • Code: Read
    • Graph: Read
    • Project and Team: Read
    • User Profile: Read
    • Work Items: Read & Write
Important:
  • Set an expiration date far in the future, or ideally, no expiration date at all. If an Azure token expires, there may be data inconsistencies during the period between the token’s expiration and the issuance of a new one.

Connect to Azure Pipelines

  • Step 1: Click on the Connect button on the Azure Pipelines tile on the Integrations page.
  • Step 2: Enter the PAT created in previous section and click on Continue button.

Connect

  • Step 3: Select organizations and Azure DevOps projects to import to BrowserStack Quality Dashboard. All of the pipelines of the selected projects would be imported on Quality Dashboard.
  • Step 4: Click on View Insights button to start viewing rich insights from your Azure Pipelines setup on Quality Dashboard.

Select projects

Set up Automation metrics

BrowserStack Quality Dashboard provides insights into various CI/CD Automation metrics such as job frequency, stability, build flakiness and many more.

Configure Job Type for different jobs

From the Automation tab on the Settings page, you can configure the job type for all the jobs (i.e. pipelines) fetched from your Azure Pipelines setup.
You can tag your jobs according to testing type, such as Regression, Functional, Deployments, Sanity etc. All jobs labeled with job types other than ‘Deployments’ will be considered Testing jobs and will be used to populate Testing-related insights. Jobs labeled ‘Deployments’ will be considered Deployment jobs and will be used to populate Deployment-related insights. The following metrics will start getting computed upon the configuration of job types:

  • Job Stability
  • Total Jobs Configured
  • Job Frequency
  • Build Duration

Configure job type

Store test output as build artifacts

BrowserStack Quality Dashboard cannot track tests related metrics such as Build Stability, Total Automated Tests, Build Flakiness unless you publish the test output artifacts on Azure. Azure Pipelines provides the PublishBuildArtifacts@1 task to publish build artifacts to Azure Pipelines. The following script demonstrates the use of this task to publish artifacts:

steps:
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '11'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/surefire-reports/junitreports/*.xml'
    goals: 'clean test -Dsurefire.suiteXmlFiles=config/passtest.xml'
- task: CopyFiles@2
  inputs:
    Contents: 'target/surefire-reports/junitreports/*'
    TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts@1
  inputs:
    pathToPublish: '$(Build.ArtifactStagingDirectory)'
    artifactName: 'JUnitReports'
    publishLocation: 'Container'


Important:
  • Make sure to add these steps in all pipelines that you want to track on Quality Dashboard to ensure you are getting complete insights on the metrics mentioned under this section.

Troubleshooting

Need help? Reach out to us here.

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback!

Talk to an Expert
Download Copy