First steps on DGX Spark

Your DGX Spark runs the OS image you chose, DGX OS by default. DGX OS is NVIDIA's Ubuntu-based operating system for DGX systems, with the NVIDIA driver, CUDA, Docker and the NVIDIA Container Toolkit already installed. The commands below are standard ones from NVIDIA and the projects they belong to. For everything else, NVIDIA's DGX Spark documentation and DGX Spark playbooks are the reference.

Look around

Log in as described in SSH access, then check what you're running on:

cat /etc/os-release     # the OS and its version
uname -m                # aarch64: the Spark has an Arm CPU
free -h                 # memory: 128 GB, shared by the CPU and GPU
df -h /                 # disk space

The CPU is Arm (aarch64), so pick Arm64 builds of software and container images. Most NVIDIA containers on NGC publish one.

Check the GPU

nvidia-smi

It shows the GPU, the driver and CUDA versions, and the processes using the GPU. To check that containers can reach the GPU too:

docker run --rm --gpus all ubuntu nvidia-smi

If both show the GPU, the machine is ready for work.

Keep DGX OS up to date

The OS is yours to maintain. Install updates with:

apt update
apt full-upgrade

Reboot when an update asks for it (reboot). NVIDIA's DGX Spark documentation describes system and firmware updates. If the machine doesn't come back after a reboot, ask us for a reboot.

Run a model

Three common ways to start, from the simplest:

Ollama

Ollama installs with one command and pulls models by name:

curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1:8b

Ollama's API listens on localhost:11434. Reach it from your computer through an SSH tunnel: ssh -L 11434:localhost:11434 root@<tailscale-name>.

vLLM

vLLM serves models through an OpenAI-compatible API. NVIDIA publishes vLLM containers on NGC; pick the current tag there and run:

docker run --rm -it --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -e HF_TOKEN \
  nvcr.io/nvidia/vllm:<tag> \
  vllm serve <hugging-face-model-id>

-p 127.0.0.1:8000:8000 keeps the API on the machine only; reach it with ssh -L 8000:localhost:8000 …. Set HF_TOKEN in your shell first for gated models.

NVIDIA NIM

NIM microservices are prebuilt, optimized model containers from NVIDIA. The DGX Spark playbooks list the ones that run on Spark, with step-by-step instructions. You need an NGC API key to pull them.

Linked units over ConnectX-7

A bundle of two, three or four DGX Sparks arrives cabled as configured: two units linked directly, three in a triangle, or four in a ring, with no switch. Each unit is a separate machine, with its own entry and access details in My hardware and its own Tailscale name.

Each Spark has a ConnectX-7 network card with two QSFP ports, up to 200 Gb/s each. To see the interfaces and their state on a unit:

ip -br link
ip -br addr

We'll tell you how the links between your units are addressed with your connection details.

To run one job across units, NVIDIA's DGX Spark playbooks include guides to connecting two Sparks and to testing communication between them with NCCL. Start there, and write to us if something about the cabling looks wrong.