Configure how requests are distributed across compute providers
| Rule Name | Pri | Condition | Primary | Fallback | Status |
|---|---|---|---|---|---|
| Cost Optimizer | 1 | cost < $0.01/1K units | Together AI | Fireworks AI | Active |
| Low Latency | 2 | latency < 100ms | DGX CloudNIM | Together AI | Active |
| GPU Required | 3 | model.requires_gpu | DGX CloudNIM | Baseten | Active |
| Regional — EU | 4 | region == "eu-west" | DeepInfra | Nebius | Active |
| Regional — APAC | 5 | region == "ap-southeast" | Modal | CoreWeave | Active |
| Fallback Default | 99 | * | Together AI | Fireworks AI | Active |
| Time | Agent | Model | Provider | Reason | Latency | Cost |
|---|---|---|---|---|---|---|
| 14:32:01 | @acme-corp/email-drafter-pro | llama-3.3-70b | Together AI | Cost Optimizer | 48ms | $0.004 |
| 14:31:47 | @fintech-labs/data-analyst | nemotron-super-49b | DGX CloudNIM | Low Latency | 62ms | $0.011 |
| 14:31:22 | @acme-corp/code-reviewer | llama-3.3-70b | Fireworks AI | Cost Optimizer | 55ms | $0.003 |
| 14:30:58 | @devtools/doc-generator | mixtral-8x22b | DeepInfra | Regional — EU | 71ms | $0.005 |
| 14:30:41 | @fintech-labs/risk-scorer | nemotron-super-49b | DGX CloudNIM | GPU Required | 58ms | $0.012 |
| 14:30:19 | @acme-corp/email-drafter-pro | llama-3.3-70b | Together AI | Fallback Default | 51ms | $0.004 |
| 14:29:55 | @devtools/doc-generator | qwen-2.5-72b | Modal | Regional — APAC | 79ms | $0.006 |
| 14:29:33 | @fintech-labs/data-analyst | mixtral-8x22b | Fireworks AI | Cost Optimizer | 44ms | $0.003 |
| 14:29:10 | @acme-corp/code-reviewer | nemotron-super-49b | DGX CloudNIM | GPU Required | 61ms | $0.011 |
| 14:28:47 | @fintech-labs/risk-scorer | llama-3.3-70b | Together AI | Cost Optimizer | 53ms | $0.004 |