Manual setup of Code Reviews webhooks in Azure DevOps (ADO)

Code Reviews normally creates the webhooks it needs automatically when you turn on a protection mode. If your Azure DevOps (ADO) organization restricts permissions so the Code Reviews integration user can't create service hooks, that automatic step fails, and you see an error when you try to save a protection mode.

This guide covers how to create those service hooks by hand instead.

When you need this

Follow this guide if all of the below apply:

You use Azure DevOps as your git provider for Code Reviews.
Your ADO organization restricts admin permissions, so the integration user can't create service hooks on the repository.
Saving a protection mode in Code Reviews fails with a message that the webhook can't be created.

If none of these apply, let Code Reviews create the webhooks for you (see Create a quality gate with Azure DevOps.)

Before you start

You need someone with the Project Administrator role, or an equivalent permission to create service hooks, on the ADO project that holds your repository.
Have the name of the repository you use with Code Reviews ready, so you can set the filter on each event.

Create the service hooks in Azure DevOps

Repeat these steps once for each of the four events in the table below.

In Azure DevOps, go to Project settings for the project that holds your repository.
Open the Service Hooks tab and click Create subscription.
Select Web Hooks, then click Next.
Select the trigger for the event you're setting up (for example, Pull request created).
Leave each event's parameters as their defaults (Any / All), then set the Repository filter to the repository you use in Code Reviews.
Click Next.
In the URL field, enter https://app.clayton.io/webhook/azure.
Leave the remaining settings as their defaults, then click Finish.

Note: the webhook URL uses the clayton.io domain simply because Code Reviews was formerly known as Clayton, and the hooks still point to that address.

Events to create

Create a separate Web Hooks subscription for each of these events, with the Event filter set to your repository:

Consumer	Event	Event filter
Web Hooks	Pull request created	Repository
Web Hooks	Pull request updated	Repository
Web Hooks	Pull request commented on	Repository
Web Hooks	Pull request merge attempted	Repository

Once you're done, you should see four new Web Hooks subscriptions listed on the Service Hooks tab.

Finish the setup with Gearset

Creating the service hooks is only half of the process. Because the automatic step failed, Code Reviews still needs the protection mode applied on our side. You can't set it yourself, as doing so tries to create the webhook again and fails.

So, once the service hooks are in place, start a chat with our support team and let us know:

the ADO project, or projects, you've set up, and
the repository each one uses.

We'll then apply your chosen protection mode (commonly Make Suggestions ) for you.

Note: If you use Code Reviews across more than one ADO project, tell us about all of them, as the protection mode needs to be applied to each project you want scanned.

Test the setup

Once the protection mode is in place, trigger a scan by making a change to a pull request. For example, create a PR or push a new commit to an open one. A scan runs automatically, and the results appear in Code Reviews.

Good to know

Disabled service hooks: if a service hook fails several times, Azure DevOps disables it automatically. You don't need to recreate it, you can re-enable it from the Service Hooks tab in Azure, then trigger a new PR event to test.
Manual scans still work: without the webhooks in place, Code Reviews still runs scans you start manually. The webhooks are only needed for scans to run automatically on pull requests.

You can confirm the setup of these service hooks from the Service Hooks tab in Azure DevOps. For more on how ADO handles web hooks, see Microsoft's web hooks documentation.