Commenting on PRs from GitHub Actions

This is a short note which explains how to set up a GitHub Action to post a comment on a pull request.

The GitHub Marketplace has many actions which create pull request comments. Most of these solutions trigger on pull_request events and use the GITHUB_TOKEN secret to create the comment.

When the pull request originates in a fork, the GITHUB_TOKEN is read-only. This means that none of the actions that use the GITHUB_TOKEN to push comments will work correctly on pull requests from forks. There is a long-running, still open discussion about this on the GitHub community forums.

People have proposed some workarounds to this issue for pull requests from forked repositories:

  1. Add PR Comment has a separate proxy server-based workflow for PRs from forks.
  2. nyurik/auto_pr_comments_from_forks contains an epic 220 line bash script that:
    • runs on a 15 minute schedule
    • checks all open pull requests that haven’t taken a comment
    • downloads artifacts from uploads triggered by the pull requests
    • uses those artifacts as the comment body

We found these solutions high friction and wanted something better.

Luckily, GitHub released two new Action-triggering events in August 2020: pull_request_target, and workflow_run. The documentation is confusing. The documentation claims that both pull_request_target and workflow_run can be used to comment on PRs that originate from forks. Worst of all, there are no examples for how to do this with either trigger.

The problem with workflow_run is that it runs on a separate workflow from the initial pull_request event. Artifacts are a natural way to share data across jobs in a single workflow, but the artifact sharing experience is not as good when you are doing it across workflows. In our experience, workflow_run is a dead end for solving the PR commenting issue.

pull_request_target has its own problems. Since pull_request_target runs on the base repository, you don’t have direct access to the head of the pull request. This is a problem because you typically generate the body of your PR comment from scripts run on the head of the pull request.

The solution is to parse the pull_request_target event payload to find the repository from which the pull request was created. Once you have this information, you can use an actions/checkout step to check out the pull request head, generate the comment body, and post it to the pull request.

This is the process we use in the GitHub Action for our code review tool, Locust.

Here is a sample workflow definition that you can start from.

Happy reviewing! 🙂