Create a webhook automation

This feature requires a Pro or Enterprise plan.

This page shows how to create a webhook automation. To create a Slack automation, refer to Create a Slack automation instead.

At a high level, to create a webhook automation, you take these steps:

1. If necessary, create a W&B secret for each sensitive string required by the automation, such as an access token, password, or SSH key. Secrets are defined in your Team Settings.
2. Create a webhook to define the endpoint and authorization details and grant the integration access to any secrets it needs.
3. Create the automation to define the event to watch for and the payload W&B will send. Grant the automation access to any secrets it needs for the payload.

Create a webhook

A team admin can add a webhook for the team.

If the webhook requires a Bearer token or its payload requires a sensitive string, create a secret that contains it before creating the webhook. You can configure at most one access token and one other secret for a webhook. Your webhook’s authentication and authorization requirements are determined by the webhook’s service.

1. Log in to W&B and go to Team Settings page.

2. In the Webhooks section, click New webhook.

3. Provide a name for the webhook.

4. Provide the endpoint URL for the webhook.

5. If the webhook requires a Bearer token, set Access token to the secret that contains it. When using the webhook automation, W&B sets the Authorization: Bearer HTTP header to the access token, and you can access the token in the ${ACCESS_TOKEN} payload variable. Learn more about the structure of the POST request W&B sends to the webhook service in Troubleshoot your webhook.

6. If the webhook requires a password or other sensitive string in its payload, set Secret to the secret that contains it. When you configure the automation that uses the webhook, you can access the secret as a payload variable by prefixing its name with $.

   If the webhook’s access token is stored in a secret, you must also complete the next step to specify the secret as the access token.

7. To verify that the W&B can connect and authenticate to the endpoint:

   1. Optionally, provide a payload to test. To refer to a secret the webhook has access to in the payload, prefix its name with $. This payload is only used for testing and is not saved. You configure an automation’s payload when you create the automation. See Troubleshoot your webhook to view where the secret and access token are specified in the POST request.
   2. Click Test. W&B attempts to connect to the webhook’s endpoint using the credentials you configured. If you provided a payload, W&B sends it.

   If the test does not succeed, verify the webhook’s configuration and try again. If necessary, refer to Troubleshoot your webhook.

Now you can create an automation that uses the webhook.

Create an automation

After you configure a webhook, select Registry or Project, then follow these steps to create an automation that triggers the webhook.

Registry

A Registry admin can create automations in that registry. Registry automations are applied to all collections in the registry, including those added in the future.

1. Log in to W&B.

2. Click the name of a registry to view its details,

3. To create an automation scoped to the registry, click the Automations tab, then click Create automation.

4. Choose the event to watch for.

   Fill in any additional fields that appear. For example, if you select An artifact alias is added, you must specify the Alias regex.

   Click Next step.

5. Select the team that owns the webhook.

6. Set Action type to Webhooks. then select the webhook to use.

7. If you configured an access token for the webhook, you can access the token in the ${ACCESS_TOKEN} payload variable. If you configured a secret for the webhook, you can access it in the payload by prefixing its name with $. Your webhook’s requirements are determined by the webhook’s service.

8. Click Next step.

9. Provide a name for the automation. Optionally, provide a description. Click Create automation.

View and manage automations

Registry

Manage a registry’s automations from the registry’s Automations tab.

• To view an automation’s details, click its name.
• To edit an automation, click its action ... menu, then click Edit automation.
• To delete an automation, click its action ... menu, then click Delete automation. Confirmation is required.

Payload reference

Use these sections to construct your webhoook’s payload. For details about testing your webhook and its payload, refer to Troubleshoot your webhook.

Payload variables

This section describes the variables you can use to construct your webhook’s payload.

Variable	Details
${project_name}	The name of the project that owns the mutation that triggered the action.
${entity_name}	The name of the entity or team that owns the mutation that triggered the action.
${event_type}	The type of event that triggered the action.
${event_author}	The user that triggered the action.
${alias}	Contains an artifact’s alias if the automation is triggered by the An artifact alias is added event. For other automations, this variable is blank.
${tag}	Contains an artifact’s tags if the automation is triggered by the An artifact tag is added event. For other automations, this variable is blank.
${artifact_collection_name}	The name of the artifact collection that the artifact version is linked to.
${artifact_metadata.<KEY>}	The value of an arbitrary top-level metadata key from the artifact version that triggered the action. Replace <KEY> with the name of a top-level metadata key. Only top-level metadata keys are available in the webhook’s payload.
${artifact_version}	The Wandb.Artifact representation of the artifact version that triggered the action.
${artifact_version_string}	The string representation of the artifact version that triggered the action.
${ACCESS_TOKEN}	The value of the access token configured in the webhook, if an access token is configured. The access token is automatically passed in the Authorization: Bearer HTTP header.
${SECRET_NAME}	If configured, the value of a secret configured in the webhook. Replace SECRET_NAME with the name of the secret.

Payload examples

This section includes examples of webhook payloads for some common use cases. The examples demonstrate how to use payload variables.

GitHub repository dispatch

Verify that your access tokens have required set of permissions to trigger your GHA workflow. For more information, see these GitHub Docs.

Send a repository dispatch from W&B to trigger a GitHub action. For example, suppose you have a GitHub workflow file that accepts a repository dispatch as a trigger for the on key:

on:
repository_dispatch:
  types: BUILD_AND_DEPLOY

The payload for the repository might look something like:

{
  "event_type": "BUILD_AND_DEPLOY",
  "client_payload": 
  {
    "event_author": "${event_author}",
    "artifact_version": "${artifact_version}",
    "artifact_version_string": "${artifact_version_string}",
    "artifact_collection_name": "${artifact_collection_name}",
    "project_name": "${project_name}",
    "entity_name": "${entity_name}"
    }
}

The event_type key in the webhook payload must match the types field in the GitHub workflow YAML file.

The contents and positioning of rendered template strings depends on the event or model version the automation is configured for. ${event_type} will render as either LINK_ARTIFACT or ADD_ARTIFACT_ALIAS. See below for an example mapping:

${event_type} --> "LINK_ARTIFACT" or "ADD_ARTIFACT_ALIAS"
${event_author} --> "<wandb-user>"
${artifact_version} --> "wandb-artifact://_id/QXJ0aWZhY3Q6NTE3ODg5ODg3""
${artifact_version_string} --> "<entity>/model-registry/<registered_model_name>:<alias>"
${artifact_collection_name} --> "<registered_model_name>"
${project_name} --> "model-registry"
${entity_name} --> "<entity>"

Use template strings to dynamically pass context from W&B to GitHub Actions and other tools. If those tools can call Python scripts, they can consume the registered model artifacts through the W&B API.

• For more information about repository dispatch, see the official documentation on the GitHub Marketplace.

• Watch the videos Webhook Automations for Model Evaluation and Webhook Automations for Model Deployment, which guide you to create automations for model evaluation and deployment.

• Review a W&B report, which illustrates how to use a Github Actions webhook automation for Model CI. Check out this GitHub repository to learn how to create model CI with a Modal Labs webhook.

Troubleshoot your webhook

Interactively troubleshoot your webhook with the W&B App UI or programmatically with a Bash script. You can troubleshoot a webhook when you create a new webhook or edit an existing webhook.

For details about the format W&B uses for the POST request, refer to the Bash script tab.

W&B App UI

A team admin can test a webhook interactively with the W&B App UI.

1. Navigate to your W&B Team Settings page.
2. Scroll to the Webhooks section.
3. Click on the horizontal three docs (meatball icon) next to the name of your webhook.
4. Select Test.
5. From the UI panel that appears, paste your POST request to the field that appears.
   
6. Click on Test webhook. Within the W&B App UI, W&B posts the response from your endpoint.
   

Watch the video Testing Webhooks in W&B for a demonstration.