Pull Request Comments
Add the Replay bot to GitHub so PR comments include a summary of the latest test run with a dashboard link, root-cause analysis and suggested fixes for failures with recording links—updating on each push.
Overview
Once you have your test suites integrated with Replay in CI, you can add our bot to your GitHub repository. The bot will post comments:
- A summary of the most recent test run, with a link to view them in your Replay dashboard
- Root cause analysis and suggested fix for each failed test, with a link to view the Replay recording
The comment will automatically update with the most recent results every time a new commit is pushed.
Install the app
Go to the Replay Github App page and click on Configure. You will be prompted to select the organization and repositories that the PR Comments bot will access to.
Set up your workflow file
The PR Comment bot listens to the pull_request event. It will collect links from your test runs and show them as a comment in your pull request conversation tab on GitHub. This means that your workflow file needs to have run on the pull_request event.
name: Playwright Tests
on: [pull_request]
jobs:
### ...your jobs
Run your tests
After you run your test, PR comment bot will collect links from your test step, and show them as a comment in your pull request conversation tab on GitHub.
Test Run summary comment
Failed test root-cause analysis & fix
A comment will be posted for the first failed test, with time-travel root-cause analysis and suggested fix.
Once you push a fix, Replay will move on to the next failed test, and update the comment with analysis and a fix.
PR comment bot currently only listens to pull_request event, which means that comments will not appear on events such as push or workflow_dispatch. If you are running your tests against Vercel preview deployments, you can check out this page to learn how to set up your workflow file.