start_within 打开一个 flex 窗口。另一个发送 default。视频播放的是 flex 响应。
我们比较了两次运行的每百万 token 成本和首个 token 响应时间。当两次运行可能返回不同的 token 时，这些是公平的衡量标准。
我们承诺提供更便宜的结果，而不是更快的。在您设置的窗口内，我们要么获得便宜的层级，要么升级到标准层级。因此，一次运行会按一个层级计费，并且永远不会超过您的截止时间。
使用快速入门发送您的第一个请求。截止时间路由涵盖了窗口和升级到标准层级的情况。
集成演示
以下每个工具都是一个现成的开源代理，经过修改后通过我们进行路由。现在每个工具都作为一个技能发布。将该技能添加到您的编程代理中，并在您自己的部署中请求集成。该技能通过配置涵盖了提供商入口、flex 截止时间以及/flex 会话命令。
OpenScience
OpenScience 是 Claude Science 的开源替代品。此运行要求它查找有关更高效卡车路线规划的研究论文。它在 GPT-5.6 Terra 上运行，思考程度较低，并具有五秒的 flex 窗口。 该片段运行 4 分 1 秒。Flex 费率覆盖了整个任务，因此成本减半。首个 token 响应时间晚了 9.3%，任务大约晚了三十秒完成。 将我们的 OpenScience skill 添加到您的编程代理中，然后要求它将您的 OpenScience 部署指向 FlexInference。对于 Claude Code，只需一个命令：Install the OpenScience skill
Transcript
Transcript
FlexInference demo transcript: OpenScienceHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have OpenScience, an open-source alternative to Claude Science. It’s been changed to route all its requests through FlexInference, and you can have your OpenScience do the same: we’ve added a prompt right below this demo that you can copy and paste into any agentic coding tool you use, and it’ll upgrade your OpenScience to also use FlexInference. For this demo we’re using GPT 5.6 Terra. I’ll set thinking to low, set a flex race so it searches for cheaper inference, and give it five seconds to find cheaper inference, otherwise upgrade to a default request. We’re going to be looking at cost per one million tokens and time to first token, because in the background we spin up two parallel requests every time I ask a question. That’s just so we can compare, and since the results could be non-deterministic, what we really care about is when the first token came back and how much each side costs per one million tokens. I’d like to say that FlexInference supports all Claude models, all Gemini models, and all GPT models you see here.Now let’s ask it: I work in logistics and I own three trucks. I want my routing to be more efficient, to deliver faster and for cheaper. Find me research papers that are relevant.While this goes, we have two parallel processes running in the background to compare against. What we do guarantee is that within the time you set, we’ll find you cheaper inference or escalate to a normal default request; in this case we’re comparing against always asking default requests. We sometimes see that we’re somehow faster, and obviously cheaper, but we don’t guarantee the faster part. We do guarantee the cheaper part. You can obviously make default requests through us and get the speed you’d like, along with priority requests.Why would you want an SLA for a science-style hypothesis and repetitive process task? Having worked in research, there are often cases where you have ten hypotheses and a limited amount of API credits, dev time, or just time before a deadline. So you only try the top three and shelve the bottom seven. Eventually the top three fail to deliver the results you want, you start going through the rest, you get to the eighth one on that list, and you think, I wish I’d tried this sooner - it just didn’t make sense in that moment. This way, the hypotheses you’re considering that don’t seem like a good return on investment can run cheaper in the background. Even if it takes a little longer, that’s fine, because you weren’t going to prioritize it anyway - but now you’ll get the results sooner. And when you’re doing research, or any open-ended task where you iterate and do trial and error, you’re getting a bunch of people’s inputs and you want to give them results too, but you can’t dedicate time or API credits. You can push all their requests and hypotheses onto different OpenScience sessions, make sure it comes out in enough time, and they’ll get their results much cheaper while you continue to prioritize. Your more prioritized requests you can keep doing through default or priority requests.As you can see, it completed, and we were able to complete our task with only thirty extra seconds - at fifty percent of the price. It makes sense to give up ten percent or less latency, in this case, for a fifty percent cost reduction. I hope you liked it and implement it in your own version of OpenScience. Thank you.
OpenCode
OpenCode 是一个开源编程代理。此运行要求它构建一个 HTML 页面，其可视化效果解释导数。它在 GPT-5.4 上运行，思考程度较低，并具有五秒的 flex 窗口。 该片段运行 3 分 34 秒。Flex 费率将成本减半。首个 token 响应时间比default 运行快 37.5%，这种情况偶尔发生，但我们从不承诺。
将我们的 OpenCode skill 添加到您的编程代理中，然后要求它将您的 OpenCode 部署指向 FlexInference。它配置的 /flex 插件会在每个请求上设置 start_within，因为它拥有请求钩子。没有钩子的工具不需要插件。而是将截止时间设置在密钥上。对于 Claude Code，安装只需一个命令：
Install the OpenCode skill
Transcript
Transcript
FlexInference demo transcript: OpenCodeHi, my name is Adi. I’m the founder of FlexInference, and here we have OpenCode, and it’s been changed to route model requests through FlexInference. You can actually change it yourself: there’s a prompt below this demo that you can copy and paste into any agentic coding tool, point it at your OpenCode, and say “I want to start using FlexInference,” and it’ll make it so you can have the same tool we have right here. It’s already set to GPT 5.4; we’ll set it to low thinking, flex on, five-second SLA, and I’ll start the demo and explain how things are going. So we’ll say: I’m having a hard time understanding derivatives. I would like a simple HTML page with dynamic visualizations that explain it to me.OpenCode is going to create this, and on the right-hand side a comparison will be running: the flex race rate, which is cost per one million tokens, then the default rate, and then flex race time to first token and default time to first token. On the front end, what you’re seeing through this TUI is just what flex is responding with. On the back end, we’re sending two parallel requests, so this is really an apples-to-apples comparison, and you can see the time-to-first-token and cost-per-one-million-token comparison as well.The point behind FlexInference is that we give you access to a flex race that searches for cheaper inference within the time you set, and we set it to five seconds; if that isn’t fulfilled, we escalate it up. We can do every other type of request as well. In slash models you can see we support all major models. What sets us apart is the flex race. There are a lot of cases with non-latency-sensitive tasks where it’s great. Imagine you get a page at 8 PM. It’s a sev 4 and not the biggest deal. You think it would be nice if it could be implemented, but you’re fine if it takes ten minutes longer. Those cases, it’s great. Or cases where you have background projects you’re interested in, but you don’t want to spend all your API credits at once: you can queue up a bunch of tasks, they slowly churn out, and you spend a lot less.Now it’s already created this derivatives HTML page, which we can open up, but what I’m interested in is the comparison. Time to first token was actually faster on GPT 5.4, which happens from time to time. We don’t guarantee it’ll be faster than default, but it does happen, and it’s great in our case. Second, it is cheaper: a fifty percent cost reduction here. Setting up FlexInference within OpenCode itself means you or your devs can spend a lot more time trying out different projects and ideas without worrying whether your credits are getting used up in the wrong area, because now you have a fifty percent cost reduction. Hopefully you enjoy this and end up using it as well. Thank you.
OpenWork
OpenWork 驱动一个 OpenCode 边车进行路由。此运行要求它查找 Netflix 的价格和每周变化，然后解释其变动。它在 GPT-5.4 上运行，思考程度较低，并具有五秒的 flex 窗口。 该片段运行 2 分 57 秒。Flex 费率将成本降低了 48.5%。首个 token 响应时间晚了 0.08 秒。 将我们的 OpenWork skill 添加到您的编程代理中，然后要求它将您的 OpenWork 部署指向 FlexInference。对于 Claude Code，只需一个命令：Install the OpenWork skill
Transcript
Transcript
FlexInference demo transcript: OpenWorkHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have OpenWork, and it’s been changed so that it uses FlexInference in the background for its router. There’s a prompt down below that you can paste into any CLI or agentic coding tool you use, and it should change your OpenWork to start using FlexInference as well. Let’s start the demo and I’ll explain how it works. All of these models are supported through FlexInference, but let’s say I want a task that needs a smart model: GPT 5.4. I’ll set it to low thinking with flex on and give it a five-second SLA to find cheaper inference within, otherwise escalate to a default request. So let’s say: I want you to use the browser tool to find the current price for Netflix and the week change as well, and tell me why it is rising or falling.While this is happening, I’ll explain the cases where this is really helpful. Yes, you can use FlexInference for default requests, priority requests, auto requests, but what makes us different is the flex race, given an SLA you set. Think about cases where you have a daily brief in the morning: at 8 AM I want a stock market brief. Does it really matter if it created the brief within one minute compared to one minute and ten seconds? In my opinion, not really. If you’re willing to give up that ten-second or five-second SLA, those cases are incredible, because you’re going to save fifty percent for free, just by giving a few seconds here and there.In fact, in our case, the front end shows only the flex response, but on the back end, for every request in this demo, we send out two parallel requests - one default and one with the five-second SLA - and compare them on cost per one million tokens and time to first token. These are just really good comparison metrics, because the responses between the two requests could be different, but cost per one million tokens is fair, and time to first token is also fair.What we’re seeing here is that you saved fifty percent, and all you really did was spend .08 extra seconds getting a response. You can imagine that aggregated across 365 days and all the asynchronous processes you might have running in the background that are not latency-sensitive, it’s really helpful. And then you can think of the other use cases that start being helpful because of the fifty percent cost savings you get. So that’s one of the great ways you can use FlexInference. We hope you like it, and be sure to copy the prompt down below and use it on your own OpenWork. Thank you.
更多演示
这三个片段早于集成技能，因此没有工具可供连接。每个片段都在不同的任务上运行相同的双请求测试。Gemini 图像分类
两个 Gemini 2.5 Flash 运行对一组五十张彩色图像进行分类。一个打开一个十秒的 flex 窗口。另一个发送default。
该片段运行 1 分 46 秒。Flex 费率将成本降低了 38.9%。首个 token 响应时间晚了 20.2%。
Transcript
Transcript
FlexInference demo transcript: Gemini image classificationHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have an image classification demo. I’m going to start this and then explain what’s going on.We have two parallel processes, both running Gemini 2.5 Flash, that are going to classify a dataset of fifty images of different colors, identifying what color each one is. We’re not trying to compare which side classifies better; it’s the same model on both sides, and both use FlexInference’s SDK and router. The left-hand side has been given a ten-second SLA to find cheaper inference through the start_within parameter. The right-hand side is also using the FlexInference SDK and router, but its start_within parameter is set to default, which asks for an instant response. What we’re trying to show is that given a small latency budget that you get to set, in this case ten seconds, you can get dramatically lower costs, approximately fifty percent. I’m going to fast-forward to when this dataset has been classified and completed, and then we can look at the trade-offs.Now that we’re back, we can see that fifty out of fifty images have been classified. The total costs were about half: $0.0183 compared to $0.0304. Cost per one million tokens, input and output together, is $0.58 versus $0.95, which is also approximately half. The total latency given up was about twenty-seven seconds, roughly fifteen percent. Essentially, given a small latency budget, you were able to get dramatically lower costs. Thank you.
OpenAI 深度研究
两个研究代理起草一份关于上市 CPU 公司的报告。一个打开一个十秒的 flex 窗口，另一个发送default。
该片段运行 2 分 35 秒。Flex 费率将 GPT-5 Mini 的成本降低了 44.8%。首个 token 响应时间快了 30.1%。
Transcript
Transcript
FlexInference demo transcript: OpenAI deep researchHi everyone, my name is Adi. I’m the founder of FlexInference, and here is a demo for deep research. I’m going to start the demo as I explain how it works. We have two agents running as parallel processes, both using the FlexInference SDK and router, that are going to try to find public CPU companies with an edge in model training and inference. The topic itself is not that relevant: each agent creates some questions, does some web searches, and creates a report. What we’re really looking at is that the left-hand agent has been given a ten-second SLA to find cheaper inference, and if it can’t within that time, it escalates to a default request and still fulfills the request. The right-hand side’s start_within parameter is set to default instead of ten seconds, which asks for an instant response. What we’re trying to see is that the left-hand side, given a small latency budget of up to ten seconds, has dramatically lower costs, about fifty percent lower. That makes FlexInference far cheaper and far more affordable for tasks without very low latency requirements. I’m now going to fast-forward to the very end so we can compare the trade-offs.Now we’re back. Both reports have been made; the agents did web searches and some tertiary planning. The left-hand side, Flex, was not only cheaper but also completed faster, which sometimes happens. We do not guarantee that latency will be lower; we guarantee that within the SLA you set, we will either find you cheaper inference or escalate up to a default request. Here the left-hand side finished about sixty seconds faster, and cost per one million tokens was about twenty-six cents lower on GPT-5 Mini, nearly half the cost. We use cost per one million tokens because the two sides used a different number of tokens and citations, so it is a more accurate comparison. Total cost came out to about a $0.0265 difference. Thank you.
OpenAI 浏览器代理
两个 GPT-5 浏览器代理在一个电商克隆网站上购买一件 T 恤。一个打开一个十秒的 flex 窗口，另一个发送default。
该片段运行 1 分 59 秒。Flex 费率将成本降低了 51.5%。首个 token 响应时间晚了 9.7%。
Transcript
Transcript
FlexInference demo transcript: OpenAI browser agentHi everyone, my name is Adi. I’m the founder of FlexInference, and here we have a browser agent demo. I’m going to start this demo as I explain how it works. On the back end we have an e-commerce clone, and our agents are going to try to accomplish the task of buying one Medusa T-shirt in size M, color black. On the left-hand side we have GPT-5 being routed through FlexInference’s SDK and router, given ten seconds to find cheaper inference; if it does, it fulfills the request that way, and otherwise it escalates to a default request. On the right-hand side we also have GPT-5 using the FlexInference SDK and router, but the start_within parameter is set to default, so its requests are fulfilled instantly. The goal is to show that on the left-hand side you can get the same task done, giving up a small latency budget, for dramatically lower total cost and cost per one million tokens. We look at total cost and cost per one million tokens because step counts can vary, since this task is non-deterministic. These are two parallel processes, and input prompt caching is turned off. Now I’m going to fast-forward to the very end so we can compare costs and latency and see the trade-offs.Coming back to this, what we notice is that by giving up about fifteen extra seconds, roughly ten percent more latency, we were able to reduce total costs by about half, and cost per one million tokens by slightly more. There is a difference in step count, but generally speaking this is what you see from FlexInference: dramatically lower costs for a slight increase in latency. Thank you.