A GPU That Never Sleeps Costs $584 to $2,913 a Month. The API Sells the Same Month for a Fraction, Until You Pass a Billion Tokens

Why this pays

A small team wants an open model of its own: a private endpoint its scripts call the way they call an API, running weights it chose, on a GPU it rents by the minute and switches off at six. The question is not whether it can be done, it can, in ten minutes, but whether the GPU month beats the API month, and both numbers are printed.

An idle GPU costs the same per minute as a busy one until it is scaled to zero. A team that will not push a billion tokens a month, and does not need the model, the data or the latency on one machine, is paying for an empty GPU. A team that does, or runs a fine-tune no API serves, has its answer in the other column.

The play, in three lines

1. Read the licence on the weights, then deploy from Hugging Face's Inference Endpoints Catalog with the defaults, or with one tg beta endpoints deploy command on Together.
2. Point your code at the Endpoint URL with the same chat request it already sends, run the day's job, and read the minutes on the billing page.
3. Stop it before you leave, check which idle clock your endpoint actually uses, and put your real monthly token volume next to $584 and $2,913.

The path above walks every step. What follows is the reasoning behind each and the numbers you quote from.

Where the money comes from

• Nowhere, directly. This is a cost line that changes shape: from per token to per GPU minute.
• The saving, at volume. Past about a billion output tokens a month at Mistral Large's rate, the on-demand H100 is cheaper than the API; below it, the API is.
• What no API sells. A fine-tune of your own, data that cannot leave one machine, latency you control. Those have no token price to compare against.

The pain you are selling the cure for

The team's scripts already work against a hosted API, and three things itch: the bill grows with every call, the data goes to someone else's server, and the model can change under them. The endpoint fixes all three for the price of a GPU that never sleeps, which is why the whole page turns on stopping it.

Step one: the licence on the weights

• The GPU costs the same whatever runs on it, so the first read is the model's licence. Mistral's open-weight models are Apache 2.0 for research and individual use; commercial deployments require a Mistral license with separate terms for derivatives and production, and you can self-host them anywhere.
• Every model card on Hugging Face carries its own licence line, and the catalog does not read it for you. If the output will be sold, that line decides before the first minute is billed.

Path A: in the browser, from the Catalog

• Inference Endpoints needs a card on file in billing settings. Open the Inference Endpoints UI, click Catalog, filter by name, task and hardware price.
• The guide deploys meta-llama/Llama-3.2-3B-Instruct on the recommended Nvidia L4, keeping every pre-selected option. Create Endpoint; initialising takes about 3 to 5 minutes.
• The Overview tab shows the Endpoint URL; the Playground tab tests the model and gives a code snippet you paste an access token into. Endpoints are private by default, every request authenticated and encrypted in transit.
• Afterwards: pause it, delete it, or leave it and it scales to zero on its own, on a clock discussed below.

Path B: one command on Together

tg beta endpoints deploy google/gemma-4-E4B-it --endpoint quickstart-endpoint

• It creates an endpoint, attaches a deployment on the model's default hardware, routes all traffic to it, and prints the endpoint string as project-slug/quickstart-endpoint. First-time provisioning takes about 5 to 10 minutes for a model that size; tg beta endpoints get with the deployment id reports ready.
• The console does the same from Endpoints, New endpoint: name, deployment name, model, quantization, hardware, region, min and max replicas, both defaulting to 1.

Pointing your tools at it

• Together: base URL api-inference.together.ai/v1, the endpoint string as the model parameter of the ordinary chat completions call, the same messages and max_tokens. Prototype on serverless, move to dedicated without changing application code.
• Hugging Face: the Playground snippet does the same for the Endpoint URL with your token.
• Run the day's job: the batch of documents, the evaluation set, the internal tool.

The day by the minute

• Inference Endpoints, per hour, billed by the minute, charged while initialising and while running: Nvidia T4 $0.5, L4 $0.8, A10G $1, L40S $1.8, A100 80 GB $2.5, H200 $5; CPU instances from $0.033. Hourly rate times hours times minimum replicas: $6.40 for an eight-hour day on the L4, $20 on an A100.
• Together, per minute per running replica: H100 $3.99 on demand or $1.99 preemptible, H200 $5.99, B200 $8.99. fal rents an H100 for your own deployments from $1.89 an hour.
• The AI job cost calculator multiplies any of these by the hours you type.

The month in hours

The chart above multiplies each hourly rate by 730. Hugging Face's small T4 at $0.40 an hour is $292; an Inference Endpoints L4 $584. fal's H100 $1,380, Together's preemptible H100 $1,453, on-demand H100 $2,913, B200 $6,563, and eight A100s on Hugging Face $14,600. Those are the prices of a GPU that never sleeps.

The same month, in tokens

• Mistral Large: $0.5 per million tokens in and $1.5 out, batch at half price. $2,913 buys about 1.9 billion output tokens.
• Together serverless: DeepSeek V4 Flash $0.14 in and $0.28 out, MiniMax M3 $0.30 and $1.20, Kimi K3 $3.00 and $15.00.
• The drafting page has the hosted API price lists in full.

Stopping it, and the two idle clocks

• Together: a deployment runs until you stop it. Stop, in the console or by setting min and max replicas to 0, scales it to zero and releases the hardware; in STOPPED it is neither billing nor serving, and Start brings it back. Deletion is permanent and ordered: stop the deployment, delete it, then the endpoint. The quick start's last step is tg beta endpoints rm with --force.
• Hugging Face, two pages, two idle periods: the quick start says the endpoint scales down to zero after 1 hour of inactivity; the autoscaling guide says an endpoint with no requests for over 15 minutes is scaled to 0 replicas. The contradiction block above shows both. The 45 minutes between them is 60 cents on an L4 and $3.75 on an H200, per idle stretch.
• The price of zero: a new request triggers a cold start whose length depends on the model's size, the server answers 502 Bad Gateway while the replica initialises, and there is no request queue, so your client must retry. Check the Settings page of your own endpoint for the period actually applied, then pause by hand when the day is done.

The clauses that bite a small team

• Together: pay as you go with non-refundable fees; credits funded by ACH or wire are not usable until they clear; access can be suspended on notice if any amount is 5 days past due; you may not terminate the agreement without Together's written consent.
• Hetzner, for a dedicated box without a GPU: AX42-1 at EUR 97.30 a month plus a one-off EUR 49.00 setup fee, net of 19% VAT for German customers; backups are the customer's duty and there is no claim to a particular IP address.

What leaves with you

• Mistral's open weights can be self-hosted anywhere, so a workflow built on them does not depend on any one host at any one price.
• Hugging Face says you own the Content you create and will not sell it.
• DigitalOcean: snapshots and backups cannot be downloaded off the platform; getting data out means copying it yourself with rsync or SFTP. Keep the weights, the system prompts and the data on a disk you control.

What ends it

• Hugging Face reserves the right to suspend or terminate access at any time, with or without cause, with or without notice, and may remove your Content at its sole discretion.
• Together's agreement cannot be ended by you without its written consent.
• DigitalOcean can terminate access at any time, effective immediately, and deletes Services Content on termination.
• The answer is the same as for the support bot: keep the pieces in your own folder and know the second host's deploy command.

What to do today

1. Put a card on file, deploy the catalog's Llama 3.2 3B on an L4 with the defaults, and run one real job through the Endpoint URL from the Playground snippet.
2. Note the minutes on the billing page, then pause it.
3. Write down your team's real token volume for the month. Against $584 to $2,913, it is the only figure on this page that decides anything.