Skip to main content

​
Overview

Adding LLMs to your product can get expensive fast. You want new users to get the full experience of your product, but new signups and free plan users can churn through tokens (and dollars). What’s more, not all free users are alike - some have a genuine chance of converting, while others just want to abuse the free tier. Managing conditions for which users should get access to which models and when can add complex, messy code to your application. You can use Waystone rulesets to simply and extensibly manage model access for your different users without needing to change any application code. This cookbook will show you how to use rulesets to route free users to cheaper models, but same recipe can be used to:
  • Upgrade higher tier customers to the most powerful (and expensive) models
  • Block access for users by country
  • Route users in sensitive domains (e.g. medtech) to more compliant models (region-based or self-hosted)

​
Prerequisites

  • You’ve configured model providers on your Waystone project
  • You’ve created an api key for your Waystone project
  • You know how to make requests to the Waystone API. See the quickstart if not!

​
Step 1 - Create a rule to downgrade a model

We’ll start by creating a rule to downgrade a model. In Waystone, rules are atomic components consisting of a condition and an action. Rules do not inherently apply to any users, but do so by being added to any number of rulesets.
  1. Head to your project on Waystone and click the rules button on the nav bar.
  2. Create a new rule, giving it an appropriate name and description
  3. For the condition, select the model condition and the model you make requests with
  4. Under action, select Change model to and the downgraded model
If your application makes requests with several different models, you will need to create a separate rule for each model and its corresponding downgrade.

​
Step 2 - Creating a ruleset

We’ll now create a ruleset to apply to free-tier users, and trigger our “downgrade” rule. Rulesets define which users to apply the associated rules to; in this cookbook we’ll be applying a ruleset on users with metadata indicating that they’re in the free-tier.
  1. Head to the rulesets page (again accessible from the nav bar)
  2. Click to create a new ruleset, giving it a name and description
  3. Under conditions, select the metadata type
  4. Define the metadata value you want to trigger, e.g. “free”, and the field you want to check, e.g. “tier” or “plan”
  5. Select the rule we just created and add it to the ruleset

​
Step 3 - Include user metadata in request

The Waystone API is a superset of the OpenAI API. We make use of the metadata field to identify users and apply the arbitrary metadata you define to them. To assign user metadata, simply add the user’s unique ID in your application to the Waystone request and provide their tier in the metadata. This is sent as a JSON string in the metadata field of the request. 
  const client = new OpenAI({
    apiKey: process.env["WAYSTONE_API_KEY"],
    baseURL: "https://waystone.run/api/gateway/v1",
  });

  const userMetadata = {
    user: { id: userId, metadata: { tier: "free" }},
  }

  const response = await client.responses.create({
    model: "openai/gpt-4.1-mini",
    input: "Your super special prompt",
    metadata: { waystone: JSON.stringify(userMetadata) }
  });
N.B this example uses the “tier” metadata field - make sure to change it to the field you set in the ruleset! Importantly, note that we don’t have the change the model we use in the request - model routing is handled by Waystone, based on the metadata you send!

​
Going further

  • (ROADMAP) Give new or returning free users access to the more powerful models. You want to put your best foot forward to new users! Extend the rule with a time-based condition so that users who have only been active for ~2 weeks are given access to the more powerful model, after which they are downgraded to the cheaper model.
  • (ROADMAP) Give users a monthly quota on the more powerful model before downgrading. Soon you’ll be able to set usage-based rule conditions. You can add a quota (token or dollar amount) to each free user before they get downgraded to the cheaper model.
⌘I