> ## Documentation Index
> Fetch the complete documentation index at: https://docs.waystone.run/llms.txt
> Use this file to discover all available pages before exploring further.

# Setup a global fallback for model requests

> Ensure all requests to a model provider fallback to different models in case of errors or provider limits.

## Overview

Sometimes the model providers (OpenAI, Anthropic, etc.) you are using in your application become overloaded or you hit rate limits. Unless you have fallbacks and retry mechanisms in place, this may cause your application to fail too.

This cookbook shows how to add a global fallback that retries each request with an equivalent model on a different provider if the request fails. In this particular example, we'll be switching to Claude 3.7 if GPT-4o fails.

### Prerequisites

* You've configured **at least 2** model providers on your Waystone project.
* You've created an API key for your Waystone project.
* You know how to make requests to the Waystone API. See the [quickstart](/quickstart) if not!

### Step 1 - Create a rule to retry with a fallback

In Waystone, rules are atomic components consisting of a condition and an action.
Rules do not inherently apply to any users, but do so by being added to any number of *rulesets*.

1. Head to your project on [Waystone](https://waystone.run) and click the *rules* button on the nav bar.
2. Create a new rule, giving it an appropriate name and description.
3. For the *condition*, select the `Model` condition and pick `GPT-4o`. This will apply the retry mechanism to and request made with 4o.
4. Under *action*, select `Change retry config` and select `Claude 3.7` as the fallback model. You can set a number of retries for each model - sometimes requests will randomly fail the first time even if the model provider is still available.

If your application makes requests with several different models and you want a different fallback for each,
you will need to create a separate rule for each model and its corresponding fallback.

### Step 2 - Creating a ruleset

We'll now create a ruleset to apply to everyone and trigger our "fallback" rule.

1. Head to the *rulesets* page (again accessible from the nav bar).
2. Click to create a new ruleset, giving it a name and description.
3. Under *conditions*, select the `Is Group` type. This applies to any request where a "group" is defined in the metadata. *N.b. Waystone currently doesn't have an "always" ruleset condition, but will soon.*
4. Select the rule we just created and add it to the ruleset.

### Step 3 - Include user metadata in request

The Waystone API is a superset of the OpenAI API. We make use of the `metadata` field to identify users and apply the arbitrary metadata you define to them.

To assign user metadata, simply add the user's unique ID in your application to the Waystone request and the "group" (e.g. a team or organisation. If you application does not have these entities, you can reuse the user ID). This metadata is sent as a JSON string in the `metadata` field of the request.

<CodeGroup>
  ```typescript query.ts theme={null}
    const client = new OpenAI({
      apiKey: process.env['WAYSTONE_API_KEY'],
      baseURL: 'https://waystone.run/api/gateway/v1',
    });

    const userMetadata = {
      user: userId,
      group: groupId,
    }

    const response = await client.responses.create({
      model: "openai/gpt-4.1",
      input: "Your super special prompt",
      metadata: { waystone: JSON.stringify(userMetadata) }
    });
  ```

  ```python query.py theme={null}
  client = OpenAI(
      api_key=os.environ.get("WAYSTONE_API_KEY"),
      base_url="https://waystone.run/api/gateway/v1",
  );

  user_metadata = {
     "user": userId,
     "group": groupId,
  }

  response = client.responses.create(
      model="openai/gpt-4.1",
      input="Your super special prompt",
      metadata={"waystone": json.dumps(userMetadata)}
  );
  ```
</CodeGroup>

You don't need any special code in your application to handle the fallback - Waystone manages the retry and model fallback for you.