Skip to main content
Each clip below routes one workload through us. Every request fires twice on the same model. One call opens a flex window with start_within. The other sends default. The video plays the flex answer. We compare the two runs on cost per one million tokens and on time to first token. Those are the fair measures when two runs can return different tokens. We promise the cheaper result, not the faster one. Inside the window you set we either get the cheap tier or move up to standard. So a run bills at one tier and never waits past your deadline. Send your first request with the quickstart. Deadline routing covers the window and the move up to standard.

​
Integration demos

Each tool below is a stock open-source agent, changed to route through us. Each one now ships as a skill. Add the skill to your coding agent and ask for the integration in your own checkout. The skill covers the provider entry, the flex deadline, and the /flex session command, all through config.

​
OpenScience

OpenScience is an open-source alternative to Claude Science. This run asks it to find research papers on more efficient truck routing. It runs on GPT-5.6 Terra with low thinking and a five-second flex window. The clip runs 4 minutes 1 second. Flex rates covered the whole task, so it cost half as much. The first token came back 9.3 percent later, and the task finished about thirty seconds behind. Add our OpenScience skill to your coding agent, then ask it to point your OpenScience checkout at FlexInference. For Claude Code that’s one command:
Install the OpenScience skill
FlexInference demo transcript: OpenScienceHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have OpenScience, an open-source alternative to Claude Science. It’s been changed to route all its requests through FlexInference, and you can have your OpenScience do the same: we’ve added a prompt right below this demo that you can copy and paste into any agentic coding tool you use, and it’ll upgrade your OpenScience to also use FlexInference. For this demo we’re using GPT 5.6 Terra. I’ll set thinking to low, set a flex race so it searches for cheaper inference, and give it five seconds to find cheaper inference, otherwise upgrade to a default request. We’re going to be looking at cost per one million tokens and time to first token, because in the background we spin up two parallel requests every time I ask a question. That’s just so we can compare, and since the results could be non-deterministic, what we really care about is when the first token came back and how much each side costs per one million tokens. I’d like to say that FlexInference supports all Claude models, all Gemini models, and all GPT models you see here.Now let’s ask it: I work in logistics and I own three trucks. I want my routing to be more efficient, to deliver faster and for cheaper. Find me research papers that are relevant.While this goes, we have two parallel processes running in the background to compare against. What we do guarantee is that within the time you set, we’ll find you cheaper inference or escalate to a normal default request; in this case we’re comparing against always asking default requests. We sometimes see that we’re somehow faster, and obviously cheaper, but we don’t guarantee the faster part. We do guarantee the cheaper part. You can obviously make default requests through us and get the speed you’d like, along with priority requests.Why would you want an SLA for a science-style hypothesis and repetitive process task? Having worked in research, there are often cases where you have ten hypotheses and a limited amount of API credits, dev time, or just time before a deadline. So you only try the top three and shelve the bottom seven. Eventually the top three fail to deliver the results you want, you start going through the rest, you get to the eighth one on that list, and you think, I wish I’d tried this sooner - it just didn’t make sense in that moment. This way, the hypotheses you’re considering that don’t seem like a good return on investment can run cheaper in the background. Even if it takes a little longer, that’s fine, because you weren’t going to prioritize it anyway - but now you’ll get the results sooner. And when you’re doing research, or any open-ended task where you iterate and do trial and error, you’re getting a bunch of people’s inputs and you want to give them results too, but you can’t dedicate time or API credits. You can push all their requests and hypotheses onto different OpenScience sessions, make sure it comes out in enough time, and they’ll get their results much cheaper while you continue to prioritize. Your more prioritized requests you can keep doing through default or priority requests.As you can see, it completed, and we were able to complete our task with only thirty extra seconds - at fifty percent of the price. It makes sense to give up ten percent or less latency, in this case, for a fifty percent cost reduction. I hope you liked it and implement it in your own version of OpenScience. Thank you.

​
OpenCode

OpenCode is an open-source coding agent. This run asks it to build an HTML page whose visualizations explain derivatives. It runs on GPT-5.4 with low thinking and a five-second flex window. The clip runs 3 minutes 34 seconds. Flex rates halved the cost. The first token beat the default run by 37.5 percent, which happens but we never promise it. Add our OpenCode skill to your coding agent, then ask it to point your OpenCode checkout at FlexInference. The /flex plugin it configures sets start_within on each request because it owns the request hook. A tool with no hook needs no plugin. Put the deadline on the key instead. For Claude Code the install is one command:
Install the OpenCode skill
FlexInference demo transcript: OpenCodeHi, my name is Adi. I’m the founder of FlexInference, and here we have OpenCode, and it’s been changed to route model requests through FlexInference. You can actually change it yourself: there’s a prompt below this demo that you can copy and paste into any agentic coding tool, point it at your OpenCode, and say “I want to start using FlexInference,” and it’ll make it so you can have the same tool we have right here. It’s already set to GPT 5.4; we’ll set it to low thinking, flex on, five-second SLA, and I’ll start the demo and explain how things are going. So we’ll say: I’m having a hard time understanding derivatives. I would like a simple HTML page with dynamic visualizations that explain it to me.OpenCode is going to create this, and on the right-hand side a comparison will be running: the flex race rate, which is cost per one million tokens, then the default rate, and then flex race time to first token and default time to first token. On the front end, what you’re seeing through this TUI is just what flex is responding with. On the back end, we’re sending two parallel requests, so this is really an apples-to-apples comparison, and you can see the time-to-first-token and cost-per-one-million-token comparison as well.The point behind FlexInference is that we give you access to a flex race that searches for cheaper inference within the time you set, and we set it to five seconds; if that isn’t fulfilled, we escalate it up. We can do every other type of request as well. In slash models you can see we support all major models. What sets us apart is the flex race. There are a lot of cases with non-latency-sensitive tasks where it’s great. Imagine you get a page at 8 PM. It’s a sev 4 and not the biggest deal. You think it would be nice if it could be implemented, but you’re fine if it takes ten minutes longer. Those cases, it’s great. Or cases where you have background projects you’re interested in, but you don’t want to spend all your API credits at once: you can queue up a bunch of tasks, they slowly churn out, and you spend a lot less.Now it’s already created this derivatives HTML page, which we can open up, but what I’m interested in is the comparison. Time to first token was actually faster on GPT 5.4, which happens from time to time. We don’t guarantee it’ll be faster than default, but it does happen, and it’s great in our case. Second, it is cheaper: a fifty percent cost reduction here. Setting up FlexInference within OpenCode itself means you or your devs can spend a lot more time trying out different projects and ideas without worrying whether your credits are getting used up in the wrong area, because now you have a fifty percent cost reduction. Hopefully you enjoy this and end up using it as well. Thank you.

​
OpenWork

OpenWork drives an OpenCode sidecar for its routing. This run asks it to look up Netflix’s price and weekly change, then explain the move. It runs on GPT-5.4 with low thinking and a five-second flex window. The clip runs 2 minutes 57 seconds. Flex rates cut the cost 48.5 percent. The first token came back eight hundredths of a second later. Add our OpenWork skill to your coding agent, then ask it to point your OpenWork checkout at FlexInference. For Claude Code that’s one command:
Install the OpenWork skill
FlexInference demo transcript: OpenWorkHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have OpenWork, and it’s been changed so that it uses FlexInference in the background for its router. There’s a prompt down below that you can paste into any CLI or agentic coding tool you use, and it should change your OpenWork to start using FlexInference as well. Let’s start the demo and I’ll explain how it works. All of these models are supported through FlexInference, but let’s say I want a task that needs a smart model: GPT 5.4. I’ll set it to low thinking with flex on and give it a five-second SLA to find cheaper inference within, otherwise escalate to a default request. So let’s say: I want you to use the browser tool to find the current price for Netflix and the week change as well, and tell me why it is rising or falling.While this is happening, I’ll explain the cases where this is really helpful. Yes, you can use FlexInference for default requests, priority requests, auto requests, but what makes us different is the flex race, given an SLA you set. Think about cases where you have a daily brief in the morning: at 8 AM I want a stock market brief. Does it really matter if it created the brief within one minute compared to one minute and ten seconds? In my opinion, not really. If you’re willing to give up that ten-second or five-second SLA, those cases are incredible, because you’re going to save fifty percent for free, just by giving a few seconds here and there.In fact, in our case, the front end shows only the flex response, but on the back end, for every request in this demo, we send out two parallel requests - one default and one with the five-second SLA - and compare them on cost per one million tokens and time to first token. These are just really good comparison metrics, because the responses between the two requests could be different, but cost per one million tokens is fair, and time to first token is also fair.What we’re seeing here is that you saved fifty percent, and all you really did was spend .08 extra seconds getting a response. You can imagine that aggregated across 365 days and all the asynchronous processes you might have running in the background that are not latency-sensitive, it’s really helpful. And then you can think of the other use cases that start being helpful because of the fifty percent cost savings you get. So that’s one of the great ways you can use FlexInference. We hope you like it, and be sure to copy the prompt down below and use it on your own OpenWork. Thank you.

​
More demos

These three clips predate the integration skills, so there is no tool to wire. Each one runs the same two-request test on a different task.

​
Gemini image classification

Two Gemini 2.5 Flash runs sort a set of fifty color images. One opens a ten-second flex window. The other sends default. The clip runs 1 minute 46 seconds. Flex rates cut the cost 38.9 percent. The first token came back 20.2 percent later.
FlexInference demo transcript: Gemini image classificationHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have an image classification demo. I’m going to start this and then explain what’s going on.We have two parallel processes, both running Gemini 2.5 Flash, that are going to classify a dataset of fifty images of different colors, identifying what color each one is. We’re not trying to compare which side classifies better; it’s the same model on both sides, and both use FlexInference’s SDK and router. The left-hand side has been given a ten-second SLA to find cheaper inference through the start_within parameter. The right-hand side is also using the FlexInference SDK and router, but its start_within parameter is set to default, which asks for an instant response. What we’re trying to show is that given a small latency budget that you get to set, in this case ten seconds, you can get dramatically lower costs, approximately fifty percent. I’m going to fast-forward to when this dataset has been classified and completed, and then we can look at the trade-offs.Now that we’re back, we can see that fifty out of fifty images have been classified. The total costs were about half: $0.0183 compared to $0.0304. Cost per one million tokens, input and output together, is $0.58 versus $0.95, which is also approximately half. The total latency given up was about twenty-seven seconds, roughly fifteen percent. Essentially, given a small latency budget, you were able to get dramatically lower costs. Thank you.

​
OpenAI deep research

Two research agents draft a report on public CPU companies. One opens a ten-second flex window, and the other sends default. The clip runs 2 minutes 35 seconds. Flex rates cut the cost 44.8 percent on GPT-5 Mini. The first token came back 30.1 percent sooner.
FlexInference demo transcript: OpenAI deep researchHi everyone, my name is Adi. I’m the founder of FlexInference, and here is a demo for deep research. I’m going to start the demo as I explain how it works. We have two agents running as parallel processes, both using the FlexInference SDK and router, that are going to try to find public CPU companies with an edge in model training and inference. The topic itself is not that relevant: each agent creates some questions, does some web searches, and creates a report. What we’re really looking at is that the left-hand agent has been given a ten-second SLA to find cheaper inference, and if it can’t within that time, it escalates to a default request and still fulfills the request. The right-hand side’s start_within parameter is set to default instead of ten seconds, which asks for an instant response. What we’re trying to see is that the left-hand side, given a small latency budget of up to ten seconds, has dramatically lower costs, about fifty percent lower. That makes FlexInference far cheaper and far more affordable for tasks without very low latency requirements. I’m now going to fast-forward to the very end so we can compare the trade-offs.Now we’re back. Both reports have been made; the agents did web searches and some tertiary planning. The left-hand side, Flex, was not only cheaper but also completed faster, which sometimes happens. We do not guarantee that latency will be lower; we guarantee that within the SLA you set, we will either find you cheaper inference or escalate up to a default request. Here the left-hand side finished about sixty seconds faster, and cost per one million tokens was about twenty-six cents lower on GPT-5 Mini, nearly half the cost. We use cost per one million tokens because the two sides used a different number of tokens and citations, so it is a more accurate comparison. Total cost came out to about a $0.0265 difference. Thank you.

​
OpenAI browser agent

Two GPT-5 browser agents buy a T-shirt on an e-commerce clone. One opens a ten-second flex window, and the other sends default. The clip runs 1 minute 59 seconds. Flex rates cut the cost 51.5 percent. The first token came back 9.7 percent later.
FlexInference demo transcript: OpenAI browser agentHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have a browser agent demo. I’m going to start this demo as I explain how it works. On the back end we have an e-commerce clone, and our agents are going to try to accomplish the task of buying one Medusa T-shirt in size M, color black. On the left-hand side we have GPT-5 being routed through FlexInference’s SDK and router, given ten seconds to find cheaper inference; if it does, it fulfills the request that way, and otherwise it escalates to a default request. On the right-hand side we also have GPT-5 using the FlexInference SDK and router, but the start_within parameter is set to default, so its requests are fulfilled instantly. The goal is to show that on the left-hand side you can get the same task done, giving up a small latency budget, for dramatically lower total cost and cost per one million tokens. We look at total cost and cost per one million tokens because step counts can vary, since this task is non-deterministic. These are two parallel processes, and input prompt caching is turned off. Now I’m going to fast-forward to the very end so we can compare costs and latency and see the trade-offs.Coming back to this, what we notice is that by giving up about fifteen extra seconds, roughly ten percent more latency, we were able to reduce total costs by about half, and cost per one million tokens by slightly more. There is a difference in step count, but generally speaking this is what you see from FlexInference: dramatically lower costs for a slight increase in latency. Thank you.