Knowledge Base
Modern GPUs provide tremendous computational throughput, but that throughput is useful only when applications can supply data at the required rate. As AI models, vector indexes, graph datasets, and retrieval corpora grow, their working sets can exceed the capacity of local GPU memory. GPU HBM and GDDR memory are fast but capacity-limited. [S1]
Adding more GPU compute does not solve this problem by itself. If an application must wait for small pieces of data to travel through a CPU-managed storage path, its GPUs may remain underutilized despite having available compute resources.
The challenge is particularly visible in applications that generate large numbers of small, concurrent requests. These access patterns differ from conventional storage workloads dominated by large sequential transfers. They require an I/O model that can better match the parallel execution model of the GPU.
NVIDIA® SCADA™ stands for Scaled Accelerated Data Access. It is a storage I/O architecture in which GPUs directly initiate and control high-throughput, fine-grained storage operations. SCADA is designed for requests smaller than 4 KB issued by thousands of parallel GPU threads to NVMe storage. [S2]
In conventional GPU-storage workflows, the CPU typically initiates I/O operations and coordinates data movement. SCADA changes this control model by allowing storage requests to originate from GPU execution. This aligns the initiator of an I/O request more closely with the application threads that need the data.
A SCADA deployment includes a client library on the GPU side and a server daemon responsible for managing the NVMe drives. [S2] This does not mean that computation takes place inside the SSD. GPU threads run application logic and initiate storage requests, while the NVMe devices continue to store and retrieve data.
SCADA is a GPU-oriented storage I/O architecture that depends on an appropriate combination of software, GPUs, NVMe storage, and system connectivity.
A simplified conventional read path may look like this:
NVMe storage → CPU-managed I/O stack → system memory → GPU memory
In a conventional POSIX-based storage workflow, data is first read from storage into a buffer in CPU system memory and then copied into GPU memory. This intermediate host-memory region—known as a “bounce buffer”—adds a data-movement step that consumes CPU cycles and memory bandwidth while increasing end-to-end I/O latency. [S3]
The CPU also performs software work related to I/O submission, completion processing, memory management, and synchronization. This model is practical for many applications, but it can become difficult to scale when thousands of GPU threads generate fine-grained requests.
NVIDIA GPUDirect Storage enables a direct DMA data path between NVMe storage and GPU memory, eliminating the intermediate copy through CPU system memory in a supported direct-data path. The data path is direct, while I/O setup and control remain host-driven through CPU-executed software and drivers. [S3] NVIDIA SCADA extends this model by moving storage-operation initiation and control to GPU threads. [S2]
This architectural shift applies specifically to the SCADA storage I/O path; it does not remove the host CPU from the system. The CPU continues to support platform initialization, operating-system services, system management, monitoring, and other control-plane functions outside the GPU-initiated I/O path.
The central change introduced by SCADA is who initiates the storage request. Instead of waiting for a CPU thread to translate application demand into storage operations, GPU threads can issue and control fine-grained I/O directly through the SCADA programming environment. [S2]
GPUs are designed to expose massive populations of lightweight work. A CUDA kernel can launch a grid containing hundreds of millions of logical threads, but the hardware schedules those threads in waves; only a hardware- and workload-dependent subset executes concurrently. [S10]
For SCADA, the relevant advantage is not hundreds of millions of simultaneously resident threads. It is the ability of thousands of parallel GPU threads to generate large numbers of independent, fine-grained storage operations, providing far greater request concurrency than a CPU-centric I/O path. CUDA threads and CPU hardware threads are different execution abstractions, so their counts should not be treated as a direct performance ratio. [S2] [S4]
Asynchronous execution is critical because storage access has much higher latency than local GPU memory. Applications can overlap outstanding I/O with useful work when sufficient independent work is available. Realized performance depends on application design, request size, queue configuration, storage behavior, and the amount of concurrency available to cover I/O latency.
Figure 1. Conceptual view of asynchronous GPU-initiated storage access.
GPU initiation alone cannot make one SSD provide unlimited performance. Each NVMe SSD has finite capacity, bandwidth, IOPS, internal resources, and PCIe connectivity.
NVMe is designed for parallel operation and supports multiple I/O queues with deep command capacity. [S5] However, the performance available from a single device remains bounded. Distributing requests across multiple SSDs creates more physical media resources, controllers, PCIe links, and I/O paths that can operate concurrently.
Current high-end PCIe Gen6 enterprise SSDs operate in the approximately 5–6 million 4KB random-read IOPS class. The Micron 9650 PCIe Gen6 NVMe SSD used in Falcon 6048’s public SCADA implementation is rated at up to 5.5 million 4KB random-read IOPS per drive. [S8] [S11]
Next-generation GPU-oriented SSD roadmaps target substantially higher small-block performance. Reported first-generation targets reach approximately 25 million IOPS per drive, but this figure represents a future product target rather than the specification of the Micron 9650, a minimum SCADA requirement, or a result from the current Falcon 6048 implementation. IOPS figures should be compared only when block size, queue depth, and test conditions are aligned. [S12] [S13]
A multi-SSD subsystem can therefore provide:
Figure 2. Comparison of single-drive and multi-drive storage resources
Aggregate local-storage bandwidth depends strongly on the number of drives and how effectively requests are distributed across them. [S6]
NVMe is not equivalent to HBM. HBM remains the GPU’s high-bandwidth local memory tier. NVMe offers a different combination of capacity, persistence, cost, and scalable aggregate concurrency. SCADA’s purpose is to make this storage tier more directly accessible to GPU workloads—not to transform SSD media into HBM.
The following conceptual diagram illustrates the system-level relationships:
Figure 3. Falcon 6048 architecture
The GPU-side client provides the application-facing path for GPU-initiated requests, while a SCADA server daemon manages the NVMe drives. [S2] The PCIe fabric provides connectivity between accelerators and storage endpoints, and multiple NVMe SSDs supply capacity and parallel I/O resources.
The host CPU remains available for general system and management responsibilities. Exact component placement, data paths, storage configuration, and software behavior can vary by implementation.
Falcon 6048 is an integrated hardware platform for solutions that work with NVIDIA® SCADA™. The 3U GPU-accelerated server integrates GPUs, high-density NVMe storage, host compute, management capabilities, and Broadcom PCIe Gen6 switching. [S7]
By placing these components within a unified PCIe fabric, Falcon 6048 provides the device density and connectivity needed to build parallel paths between GPUs and multiple NVMe SSDs. The system supports up to 48 E1.S NVMe SSDs and works with NVIDIA® SCADA™ technology; available configurations depend on the selected devices and platform setup. [S7]
NVIDIA provides the SCADA storage I/O technology. Falcon 6048 integrates the PCIe connectivity and high-density NVMe resources needed to build and evaluate solutions that work with NVIDIA® SCADA™. Micron has published a SCADA reference implementation that combines NVIDIA GPUs, Micron NVMe SSDs, Broadcom PCIe Gen6 switches, and H3 Platform Falcon 6048. [S8]
In a conventional CPU-centric storage workflow, an AI or GNN application relies on CPU-executed software to initiate and coordinate storage requests. Data may move from NVMe storage into CPU system memory before being copied to GPU memory, adding host-side processing and an intermediate data-movement step.
With NVIDIA SCADA, GPU threads initiate and control fine-grained storage requests through the SCADA programming environment. This more closely aligns storage access with the GPU threads that require the data and reduces CPU involvement within the SCADA I/O control and data paths.
Before: Conventional CPU-Centric I/O
With NVIDIA SCADA
In a conventional workflow, the host CPU initiates and coordinates storage I/O. With NVIDIA SCADA, GPU threads initiate and control fine-grained storage requests. The diagram is conceptual; exact component placement, data paths, and software behavior may vary by implementation.
NVIDIA® SCADA™ represents a shift from CPU-initiated storage operations toward fine-grained storage access initiated and controlled by GPU threads. In a conventional CPU-centric workflow, the host initiates and coordinates storage requests. With SCADA, the GPU threads that require the data can initiate those requests through the SCADA programming environment, more closely aligning storage access with GPU execution.
SCADA does not make NVMe storage equivalent to GPU HBM. Instead, multiple NVMe SSDs can provide the capacity and parallel I/O resources needed to serve highly concurrent requests for datasets that extend beyond local GPU memory. Realized performance remains dependent on the workload, request pattern, software integration, SSD characteristics, and system topology.
Falcon 6048 integrates GPUs, host compute, Broadcom PCIe Gen6 switching, management capabilities, and support for up to 48 E1.S NVMe SSDs in a unified platform for solutions that work with NVIDIA® SCADA™.
To explore system configurations and deployment considerations, learn more about Falcon 6048.
Within SCADA storage operations, CPU involvement is removed from the I/O control and data paths. [S2] This scope is specific to the SCADA I/O path; the server still uses the host CPU for boot, operating-system services, management, monitoring, and other system responsibilities.
No. Application computation remains on the GPU. GPU threads initiate and control storage operations, while the SSDs store and retrieve data.
No. NVMe and HBM serve different tiers. HBM provides local GPU memory performance, while NVMe provides persistent, high-capacity storage. SCADA makes the NVMe tier more directly accessible; it does not make its latency or bandwidth equivalent to HBM.
Each NVMe SSD has finite capacity, bandwidth, IOPS, controller resources, and PCIe connectivity. Using multiple drives adds capacity and creates more physical I/O resources across which concurrent GPU requests can be distributed. Realized aggregate performance depends on the SSD configuration, request pattern, queue settings, software integration, and PCIe topology.
NVIDIA GPUDirect Storage enables a direct DMA data path between supported storage and GPU memory, avoiding an intermediate copy through CPU system memory. However, I/O setup and control remain host-driven. NVIDIA SCADA extends this model by allowing GPU threads to initiate and control fine-grained storage operations. [S2] [S3]
Falcon 6048 integrates GPUs, host compute, management capabilities, Broadcom PCIe Gen6 switching, and support for up to 48 E1.S NVMe SSDs in a single platform. Its PCIe connectivity and storage density support the development and evaluation of solutions that work with NVIDIA® SCADA™. Available configurations depend on the selected devices and platform setup. [S7]
