Meet FluxAI: The Model We Trained to Deploy for You

Fluxers! The assistant in the new FluxCloud preview is not a general-purpose chatbot with a Flux prompt stapled on. It is FluxAI — a model we fine-tuned ourselves, for one job: understanding what you want to run, and turning it into a correct, priced deployment on Flux.

It is live today as fluxai:tiny on the Flux LLM hub, served from a pool of 40 instances on Flux nodes. This is how we built it — five training rounds in a week, each for about the price of a coffee.

Why fine-tune at all

When we tested general models as deploy agents, the mistakes they made were not about intelligence. They were about conventions: megabytes versus gigabytes, quoting before deploying, never inventing a key, knowing what a twenty-player game server actually needs, asking instead of guessing.

Conventions are exactly what supervised fine-tuning teaches best. And there is a second payoff that matters enormously on CPU: a model with the conventions built in needs a few hundred tokens of instructions instead of three to seven thousand tokens of tool schemas. On a CPU node, that is the difference between seconds and minutes to the first word.

We started from IBM’s granite-4.0-h-tiny — the same fast, compact mixture-of-experts model behind our docs assistant — and taught it Flux.

Training data that is correct by construction

The most important decision was how to make the training data. If you ask a large model to write example conversations, you get fluent examples with the occasional wrong number in them — and a fine-tune faithfully learns the wrong number.

So we generate the conversations from code. A scenario is sampled — a preset, resources, instance count, region, term, and a conversational flow — and every tool call in the dialogue is produced by code from that scenario. Only the user’s phrasing varies, drawn from banks of templates with placeholders, so every figure survives by construction. The flows cover the real shape of these conversations: asking for an estimate, deploying after confirmation, changing your mind, declining, being vague, asking to skip the quote (it quotes anyway), unit traps, and managing an app after it is running.

Documentation questions are generated the same careful way: a question and answer are only kept if every citation exists and every number appears in a cited source.

Five rounds in a week

Round one trained on 4,515 conversations, on a laptop, overnight — nine and a half hours on an M3 Max. It learned the protocol immediately: build the spec in the right units, quote, ask, and deploy only after a yes. It did it in 4 to 12 seconds per turn, against 40 to 100 seconds for a 20-billion-parameter general model. And strict grounding on our documentation evaluation went from 7 out of 9 to 9 out of 9.

Each round after that added what the evaluation asked for next:

• Round two varied the system prompts and tool descriptions, so the model behaves the same whatever surface it is plugged into.
• Round three added app updates, pasted specifications, Docker Compose conversion, edits after a quote, other languages, noisy input — and safety flows for prompt injection, secrets and large spends.
• Round four added long multi-turn sessions, filling in details over several messages, and arithmetic computed from the documentation: blocks, pricing and limits.
• Round five taught real multi-component applications — up to the network’s maximum of ten components, from app-plus-database to full stacks with a proxy, workers, a queue, search, storage and metrics, each wired to the others by its real internal hostname.

Round five trained on 12,853 conversations in 260 minutes on a rented A100, for about $4.50. On our 28-case deploy evaluation it scores 27 of 28 on its compact tool surface and 24 of 28 against all fifteen real MCP tool schemas, with documentation grounding holding at 9 of 9.

The newest datasets go wider still: deploy-from-Git with Orbit, documentation and web search tools, node operator questions, wallet knowledge, and marketplace specifications taken straight from the live catalogue.

Behaviour in the model, knowledge in retrieval

One design principle runs through all of it. The fine-tune carries behaviour — how to hold a deploy conversation, when to ask, how to use a tool. Knowledge lives in retrieval, pulled fresh from the documentation and the live network at answer time.

That split is what keeps FluxAI accurate as Flux changes. Prices, limits and features move; a model’s weights do not. By keeping facts out of the weights, a documentation update reaches the assistant the moment it is indexed, with no retraining.

Inside the new FluxCloud

In the new FluxCloud preview, FluxAI works with a set of in-app tools: navigating to a page, opening an app or a template, and prefilling the deploy form. It deliberately has no deploy tool there, because that application’s rule is that the assistant can never sign. It prepares everything; you approve it with your own wallet.

The same model, pointed at the Flux Cloud MCP server instead, can drive a full deployment for an AI agent — quote, confirm, deploy, watch it come up — with the plan-by-default safety that server enforces.

Served on Flux, tuned for speed

FluxAI runs where everything else we build runs: on Flux nodes. Its pool has 40 instances, each with 12 cores and two key-value cache slots, and a few serving details make it fast:

• The hub routes by measured serving speed, and every instance benchmarks itself hourly and publishes its tokens per second on its health line.
• Sessions stick per conversation, so the prompt cache is reused turn after turn instead of rebuilt.
• The common tool prefix is kept warm, so the first turn does not pay for it.
• Each instance reports the exact model file it is serving by digest, so the weights on any node can be verified rather than trusted.

Try it

The easiest way is to open the new FluxCloud preview and tell it what you want to run. If you would rather call it directly, fluxai:tiny is available through the OpenAI-compatible API at llm.runonflux.com.

A model trained for Flux, on data generated from Flux, served by Flux nodes, deploying to Flux. It is the most Flux thing we have built — and it keeps getting better every round.