Clustering

Choose a cluster to control which of your computers an API request can use. This guide covers the public API and website controls; a shared cluster name is not a guarantee that one model runs across several computers.

Find a cluster#

On Computers, select the computer group you intend to use. Check that its computers are online and the desired model is loaded.

From an authenticated computer, inspect your cluster list:

actual clusters --format json

For the public API, discover model and cluster availability through GET /v1/models. Each model can include clusters[].cluster_id and a per-cluster loaded value. See API Quickstart.

Target an API request#

Send the returned cluster ID in a header:

X-Cluster-ID: cl_example

This scopes both model inventory and inference. The chosen model must be loaded and ready on that cluster. A model reported loaded elsewhere in your account is not enough.

Without a cluster header, the API can select an eligible online computer with the requested model and capabilities. It does not promise to use the first computer or a particular hardware type.

Add or share computers#

Use Add lead computer on Computers for a new lead. Select an existing cluster and use Add child computer for another computer in that group. Follow Machine Installation to install, authorize, start, and verify each computer.

Organizations let members access shared computers. Sharing does not transfer ownership or grant every management action. See Organizations.

Different hardware configurations#

A cluster may expose computers with different hardware, including DGX Spark and Apple Silicon Macs. Check the exact model, client build, available memory, and loaded state on the computer that will answer your request.

Seeing both computers in the website does not prove that a single inference runs across both. Bare actual starts a standalone node; signing into the same account does not automatically configure distributed execution.

Explicit --offline operation is standalone and disables public relay access and LAN clustering. It can start without a model, but inference still requires one to be installed and loaded.

Troubleshooting#

If a pinned request fails, refresh model inventory for that cluster. Restore offline computers, load the requested model, and check whether the selected hardware supports the request's input capabilities.

If a cluster or action is missing, check the signed-in account, selected organization, and ownership permissions. If the computer is online but unavailable through the public API, inspect actual relay status and follow Private Relay.