From API to GPU, Week 1: Understanding NVIDIA DGX Spark Environment
System Inventory
The Spark runs Linux on aarch64 (ARM64), confirmed by uname -a:
Linux spark-66b9 6.17.0-1026-nvidia ... aarch64 aarch64 aarch64 GNU/Linux
My control machine is an Intel x86_64 MacBook Pro:
x86_64
Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
| Machine | Role | Architecture | Verified by |
|---|---|---|---|
| MacBook Pro | control / authoring | x86_64 (Intel i5) | uname -m |
| DGX Spark | model + GPU work | aarch64 (ARM64) | uname -a |
This architecture mismatch means Python wheels or Docker images built for Intel won't necessarily run on the ARM64 Spark - all real work happens over ssh spark.
CPU and GPU Architecture
The Spark's CPU has 20 cores:
ssh spark 'lscpu'
Architecture: aarch64
CPU(s): 20
Model name: Cortex-X925 (10 performance cores)
Model name: Cortex-A725 (10 efficiency cores)
For model inference, the CPU handles orchestration while the GPU does the heavy lifting. A neural network layer boils down to matrix multiplication - billions of independent multiply-add operations per token. This is embarrassingly parallel work:
| Aspect | CPU (20 ARM cores) | GPU (NVIDIA GB10) |
|---|---|---|
| Parallel workers | a few strong cores | thousands of small cores |
| Good at | branching logic, one-at-a-time | same math on huge data at once |
| Analogy | a few expert chefs | a stadium of line cooks |
Memory bandwidth - not raw compute - usually limits how fast a model runs.
Reading nvidia-smi
ssh spark 'nvidia-smi'
NVIDIA-SMI 580.159.03
Driver Version: 580.159.03
CUDA Version: 13.0
GPU 0: NVIDIA GB10
Persistence-M: On
Temp Perf Pwr:Usage/Cap Memory-Usage GPU-Util Compute M.
35C P8 4W / N/A Not Supported 0% Default
| Field | Value | Meaning |
|---|---|---|
| Driver Version | 580.159.03 | kernel driver talking to the GPU |
| CUDA Version | 13.0 | max CUDA the driver supports |
| GPU name | NVIDIA GB10 | the device (Grace-Blackwell) |
| Temp | 35C | die temperature (heat → throttling) |
| Perf | P8 | clock state, P0 = max … P8 = idle |
| Pwr:Usage/Cap | 4W / N/A | current vs max power draw |
| Memory-Usage | Not Supported | would show VRAM used/total |
| GPU-Util | 0% | % of last sample the GPU was busy |
The process table shows Xorg, GNOME, and Firefox using the GPU for the desktop.
Unified Memory Surprise
Memory-Usage: Not Supported is not a bug - it's the defining feature of the Grace-Blackwell superchip. The CPU and GPU share one memory pool:
ssh spark 'free -h'
total used free available
Mem: 121Gi 6.2Gi 67Gi 115Gi
| Aspect | Traditional discrete GPU | DGX Spark (GB10) |
|---|---|---|
| Memory | System RAM + separate VRAM | one shared 121 GiB pool |
| Data movement | Copy RAM → VRAM over PCIe | CPU and GPU read the same memory |
| "Will it fit?" | limited by VRAM (24 GB) | limited by total RAM (121 GB) |
Unified memory removes the PCIe copy tax. The practical ceiling on model size is 121 GB, not 24 GB of VRAM. The trade-off is lower peak bandwidth than a top-end discrete card's dedicated VRAM.
Driver vs CUDA Runtime vs CUDA Toolkit
"CUDA" is three separate layers:
| Layer | What it is | Infra analogy | Who needs it |
|---|---|---|---|
| NVIDIA driver | kernel module that talks to the GPU | a device driver | everyone using the GPU |
| CUDA runtime (libcudart) | shared libs an app calls to run GPU work | the .so libs you link | anyone running GPU programs |
| CUDA toolkit | the nvcc compiler, headers, profilers |
gcc + headers + build tools | only people compiling CUDA |
PyTorch ships its own CUDA runtime inside its wheel. To run models you need the driver plus a CUDA-enabled PyTorch - not nvcc.
The header CUDA Version: 13.0 is the maximum CUDA the driver supports. When installing PyTorch, you must pick a CUDA build ≤ 13.0.
nvcc --version returns "command not found" - but the toolkit is installed:
ssh spark 'ls -d /usr/local/cuda*'
/usr/local/cuda
/usr/local/cuda-13
/usr/local/cuda-13.0
The full path confirms a healthy stack:
ssh spark '/usr/local/cuda/bin/nvcc --version'
Cuda compilation tools, release 13.0, V13.0.88
The toolkit version 13.0.88 matches the driver's CUDA 13.0 ceiling. To add nvcc to PATH: export PATH=/usr/local/cuda/bin:$PATH.
Installing PyTorch
Use a virtual environment - never install into the system Python:
ssh spark 'python3 -m venv ~/venvs/w1'
ssh spark '~/venvs/w1/bin/python -m pip install --upgrade pip'
ssh spark '~/venvs/w1/bin/python -m pip install torch numpy'
The install script in the companion repo (setup.sh) runs these three steps against requirements.txt:
#!/usr/bin/env bash
set -euo pipefail
VENV="${VENV:-$HOME/venvs/w1}"
HERE="$(cd "$(dirname "$0")" && pwd)"
python3 -m venv "$VENV"
"$VENV/bin/python" -m pip install --upgrade pip
"$VENV/bin/python" -m pip install -r "$HERE/requirements.txt"
PyPI served an ARM64 + CUDA 13 build automatically:
Successfully installed torch-2.13.0
nvidia-cuda-runtime-13.0.96
nvidia-cudnn-cu13-9.20.0.48
nvidia-cublas-13.1.1.3
... (aarch64 wheels)
Comments
No comments yet. Start the discussion.