NCA-AIIO Study Guide & Cheat Sheet (NVIDIA AI Infrastructure and Operations)

NCA-AIIO Exam Guide

Exam domains

Who it’s for: Data center technicians, DevOps and networking engineers, IT managers, sysadmins, solution architects, and technical sales — anyone who runs, supports, or sells AI infrastructure.

Study & test-day tips

• Budget your time: 50 questions in 60 minutes leaves about 70 seconds each. Answer what you know fast, flag the rest, and come back.
• The exam is conceptual, not a spec sheet. Know what NVLink, DPUs, MIG, and DCGM are FOR — not their generation-by-generation bandwidth numbers.
• Read for the qualifier words: 'most likely', 'primary purpose', 'best fits'. Two options are usually defensible; the qualifier picks the winner.
• Map every study hour to the blueprint: AI Infrastructure is 40% of the exam — if your practice accuracy is weakest there, that's where hours pay off most.
• Think 'what problem does this solve?' for every technology. Exam distractors are usually real technologies attached to the wrong problem.
• Training vs inference is a recurring lens: throughput vs latency, batch vs real-time, datacenter vs edge. Many questions hinge on knowing which side you're on.
• For scenario questions, identify the bottleneck first (compute, memory, network, storage, data pipeline) — the right answer almost always addresses that specific bottleneck.
• Don't leave blanks. There's no penalty for guessing — eliminate two options and pick the more specific remaining answer.
• Take the timed mock in this app at least twice before booking. The time pressure, not the content, is what surprises most first-time candidates.
• Book the exam when your practice readiness is consistently 80%+. The real exam is pass/fail, so you want margin, not a coin flip.

Cheat sheet

Hardware & interconnects

• GPU vs CPU: thousands of simple parallel cores + high memory bandwidth vs few complex cores optimized for sequential work
• NVLink: direct GPU-to-GPU interconnect inside a server — far faster than PCIe
• NVSwitch: switch fabric that lets every GPU in a node talk to every other at full NVLink speed
• DPU (BlueField): offloads networking, storage, and security from the host CPU; isolates infrastructure from tenant workloads
• MIG: partitions one GPU into up to 7 hardware-isolated instances (compute + memory)

Networking

• InfiniBand: lowest-latency, highest-bandwidth fabric for multi-node training
• RoCE: RDMA over Converged Ethernet — direct memory-to-memory transfers on Ethernet hardware
• East-west traffic (node-to-node) dominates AI clusters; design for non-blocking bandwidth between GPU nodes
• Distributed training = frequent gradient synchronization (all-reduce) — network becomes the bottleneck as you scale nodes

Facilities

• AI racks draw several times the power of enterprise racks — plan power delivery, floor loading, and cooling per rack
• Liquid cooling (direct-to-chip, rear-door) appears when rack density exceeds what air can remove
• On-prem: control, predictable cost at high utilization. Cloud: elasticity, no capex, fast start — best for bursty or exploratory work

NVIDIA software stack

• CUDA: the parallel computing platform everything else builds on
• NGC: catalog of GPU-optimized containers, pretrained models, SDKs
• TensorRT: optimizes trained models for fast inference
• Triton Inference Server: serves models from any framework at scale
• NVIDIA AI Enterprise: the supported, validated enterprise software suite
• DCGM: fleet-level GPU monitoring, health, and diagnostics

Operations

• Slurm: HPC-style batch job scheduler (queues, partitions, priorities)
• Kubernetes + GPU Operator: container orchestration with automated GPU driver/runtime/monitoring setup
• Low GPU utilization usually means a starved data pipeline, not broken GPUs — check input/storage/CPU preprocessing first
• Monitor utilization, memory, temperature, power, ECC errors; heat = throttling = silent performance loss
• vGPU: share GPUs across many light VMs (VDI, notebooks). Passthrough/bare metal: maximum performance for training

Core AI concepts

• AI ⊃ ML ⊃ DL (deep learning uses many-layered neural networks)
• Training: heavy, iterative, throughput-oriented. Inference: serving predictions, latency-oriented
• AI's rise = big data + GPU compute + better algorithms (all three together)
• AI lifecycle: data prep → train/validate → deploy → monitor → retrain

Glossary

All-reduce

The collective communication step in distributed training where every node exchanges and combines gradient updates each iteration.

BlueField

NVIDIA's DPU product line — a programmable processor on the network card that offloads infrastructure tasks from the host CPU.

CUDA

NVIDIA's parallel computing platform and programming model that lets software run general-purpose computation on GPUs.

cuDNN

NVIDIA's GPU-accelerated library of deep learning primitives (convolutions, attention, etc.) used by frameworks like PyTorch and TensorFlow.

DCGM

Data Center GPU Manager — NVIDIA's tool for monitoring GPU health, telemetry, and diagnostics across a fleet.

DGX

NVIDIA's line of purpose-built AI systems combining multiple GPUs, NVLink/NVSwitch, and tuned software.

DPU

Data Processing Unit — offloads networking, storage, and security from the host CPU and isolates infrastructure from workloads.

East-west traffic

Server-to-server traffic inside the data center; dominates AI clusters due to gradient synchronization.

Gradient synchronization

Exchanging weight updates between workers in distributed training so all copies of the model stay consistent.

HBM

High Bandwidth Memory — the stacked, very fast memory on data center GPUs that feeds their thousands of cores.

InfiniBand

A high-bandwidth, low-latency network fabric widely used to connect nodes in AI and HPC clusters.

Inference

Running a trained model on new inputs to produce predictions; typically latency-sensitive.

Kubernetes

Container orchestration platform; with NVIDIA's GPU Operator it automates the GPU software stack for AI workloads.

MIG

Multi-Instance GPU — partitions one physical GPU into up to seven isolated instances with dedicated compute and memory.

MLOps

The practice and tooling for taking ML models through data prep, training, deployment, and monitoring repeatably.

NCCL

NVIDIA Collective Communications Library — optimized multi-GPU/multi-node communication primitives (like all-reduce) used in distributed training.

NGC

NVIDIA's catalog of GPU-optimized containers, pretrained models, and SDKs.

NVIDIA AI Enterprise

The supported, validated enterprise suite of NVIDIA's AI software for production deployments.

NVLink

NVIDIA's direct GPU-to-GPU interconnect, much faster than PCIe, for multi-GPU systems.

NVSwitch

A switch chip that connects many GPUs at full NVLink bandwidth so all GPUs in a system can communicate simultaneously.

Parallel file system

Shared storage (e.g., Lustre-class) that delivers high-throughput data access to many compute nodes at once.

PCIe

The standard expansion bus connecting GPUs to CPUs; slower than NVLink for GPU-to-GPU communication.

PUE

Power Usage Effectiveness — total facility power divided by IT equipment power; closer to 1.0 means a more efficient data center.

RDMA

Remote Direct Memory Access — one server reads/writes another's memory directly, bypassing the CPU and OS network stack.

RoCE

RDMA over Converged Ethernet — brings RDMA's low latency to Ethernet fabrics.

Slurm

A widely used HPC job scheduler that queues, prioritizes, and places batch jobs (including GPU training) across a cluster.

TensorRT

NVIDIA's SDK that optimizes trained models (precision, fusion, tuning) for fast inference.

Throttling

A GPU automatically reducing clock speeds to protect itself when too hot or power-limited — silently degrading performance.

Training

The compute-intensive process of learning model weights from data; throughput-oriented, often distributed.

Triton Inference Server

NVIDIA's open-source server for deploying models from any framework at scale with dynamic batching.

vGPU

NVIDIA virtual GPU — shares physical GPUs across multiple virtual machines, suited to lighter, fragmented workloads.

Utilization

The share of time a GPU is doing work; persistently low utilization usually signals a starved data pipeline.