The demand for high-quality video continues to accelerate across industries, powering everything from immersive streaming experiences to remote collaboration, generative AI media tools, and large-scale content delivery.
Behind these experiences is a growing need for video pipelines that are faster, more efficient, and capable of handling increasingly complex formats and workloads. NVIDIA Video Codec SDK helps developers meet that challenge by providing access to GPU-accelerated video encoding and decoding through NVIDIA dedicated hardware video engines.
NVIDIA Video Codec SDK 13.1 is now available. We encourage developers to explore the latest features in their video pipelines, take advantage of the redesigned sample apps, and share feedback via the NVIDIA Developer forums.
What’s new in SDK 13.1
Encode features:
· Hierarchical Reference Mode for AV1 with up to 31 B-frames
· UHQ tuning info combined with iterative encoding
Decode features:
· Per-macroblock decode statistics for H.264 and HEVC
· View information during MV-HEVC decode
· Seek to a specific frame
Transcode features:
· Application-allocated CUarray as NVIDIA Video Encoder (NVENC) input and NVIDIA Video Decoder (NVDEC) output
· Redesigned transcoder samples with a modular, queue-based architecture
Other features:
· Official Docker-based development environment
Encode improvements
Why AV1 hierarchical reference mode
Using B-frames as references improves encode quality. Hierarchical Reference Mode improves it further by arranging B-frames in a tree-like reference structure: leaf nodes are non-reference B-frames, and the root is the middle B-frame. This structure increases NVENC’s maximum number of B-frames from 7 to 31, enabling the encoder to exploit temporal redundancy and improve overall quality. Use of this mode does not add any performance penalty, but video memory consumption goes higher.
SDK 13.1 adds Hierarchical Reference Mode for AV1, supporting 1, 3, 7, 15, and 31 B-frames; H.264 and HEVC will follow in future driver versions. The mode is most effective at seven or more B-frames and has minimal performance impact. For configuration details, see the NVENC Programming Guide.
Figure 2, below, shows bitrate savings when encoding 15 B-frames in Constant Quality (CQ) mode, compared with NVENC’s High Quality (HQ) tuning in preset p7.
Figure 3, below, shows bitrate savings when encoding 15 B-frames in Variable Bitrate (VBR).
UHQ tuning info with iterative encoding
Iterative encoding (introduced in Video Codec SDK 12.1) freezes the encoder’s automatic state advancement and lets the user re-encode the same frame with different parameters. NVENC tracks each iteration’s state, and can stop and commit to any one of them.
UHQ tuning info (introduced in Video Codec SDK 12.2) combines lookahead level and temporal filtering to deliver the best quality-vs-performance tradeoffs in latency-tolerant encoding. Temporal filtering reduces noise in natural video by using motion estimation to find matching patches in adjacent frames and applying them to filter the current frame, yielding average coding gains of 4–5% on natural content. The lookahead level feature analyzes future frames and uses coding tree units (CTUs) and other statistics to allocate bits efficiently for rate control. Four lookahead levels with different performance and quality tradeoffs are currently supported..
Version 13.1 combines UHQ tuning info with iterative encoding, so lookahead level and temporal filtering now work alongside per-iteration re-encode.
Decode improvements
Per-macroblock decode statistics
The NVDECODE API now retrieves detailed per-macroblock decode statistics for each decoded frame of H.264 and H.265 (HEVC) content. For every 16×16 block, the decoder exposes the luma quantization parameter (QP), the coding unit type (Intra, Inter, Skip, or PCM), and up to two motion vectors (forward and backward), all extracted as a natural byproduct of hardware decode with no additional CPU overhead.
These statistics unlock GPU-accelerated video-analytics workflows that previously required CPU-side bitstream parsing. Motion vectors enable scene-change detection, object tracking, and shot-boundary analysis. QP values give a per-block view of encoding quality for adaptive bitrate optimization and quality monitoring. Macroblock types reveal coding structure useful for content classification and compression research.
The workflow is straightforward: applications query decoder capabilities via cuvidGetDecoderCaps(), enable statistics collection at decoder creation, and retrieve a GPU-resident statistics buffer with each decoded frame from cuvidMapVideoFrame(). The user can copy the statistics to host memory or process them directly on the GPU with CUDA kernels for real-time pipelines. The SDK ships a ready-to-use sample, AppDec -dumpstats, that demonstrates the full flow.
Frame-accurate seek
AI workflows—from inference pipelines for object detection, content moderation, and video summarization, to training-data preparation that samples diverse frames across large datasets—frequently need specific frames rather than sequential decode.
Video editing and non-linear post-production have the same requirement. The Video Codec SDK now provides a comprehensive seeking and random-frame-access API through the NvVideoDecoder class, making frame-accurate access as simple as array indexing while fetching only the frames you need.
A GOP-aware seek architecture handles the request. For frame N, the SDK locates the nearest IDR frame before the target, seeks the demuxer there, and flushes decoder state via CUVID_PKT_DISCONTINUITY.
From that IDR forward, only frames needed to reach N are processed: the parser flags and skips non-reference frames, and reference frames decode but bypass mapping and post-processing (format conversion, scaling, cropping via cuvidMapVideoFrame()) through PTS-based filtering. Only frame N runs the full decode, mapping, and post-processing pipeline.
The NvVideoDecoder class wraps the low-level SeekUtils engine and exposes a clean, operator-based interface. Figure 5, below, shows the internal seek flow when a frame is requested.
Key capabilities:
- Decoder caching for playlists: NvVideoDecoder caches decoder instances by codec, bit depth, and chroma format, and reuses them via LRU eviction to avoid repeated creation overhead
- Non-seekable streams: Elementary streams, network streams, and pipes are auto-detected. Forward seeking works efficiently; backward seeking is supported with an automatic decoder reset
- Flexible frame specification: The AppDecVideoDecoder sample supports individual indices (0,10,20), ranges with step (0:100:10), time-based access (-t 1.5,3.0), and playlist files for batch processing
- Open GOP support: Streams with non-IDR I-frames seek to the nearest IDR rather than the nearest keyframe, ensuring all reference frames are available
MV-HEVC stereoscopic enhancements
The Video Codec SDK now offers better 3D-video support: view-ID and layer metadata from the decoder; multi-GPU encoding at high resolutions; and improved compatibility with third-party software.
MV-HEVC decode updates:
- View-information reporting: The decoder outputs specific layer and reference info such as nuh_layer_id, letting applications identify and route frames by view for stereo handling
- Wider bitstream support: The decoder now handles MV-HEVC 3D bitstreams created by third-party encoders, so both left and right views play correctly
MV-HEVC encode updates:
- Simplified metadata for FFmpeg: HEVC 3D display metadata now appears correctly in the bitstream when encoding via FFmpeg, making 3D-video creation easier
- Split Frame Encoding (SFE): Multiple encoders can work on a single frame, boosting speed for high-resolution 3D and XR video that exceeds a single encoder’s capacity
Redesigned transcoder pipeline
The transcoding samples now favor flexibility, performance, and customization. The updated suite ships four applications:
- AppTransPerf. Benchmarks the maximum throughput of NVDEC and NVENC
- AppTrans. 1:1 transcoding with optional bit-depth conversion
- AppTransOneToN. 1:N transcoding with scaling
- AppTransZeroCopy (new). 1:1 zero-copy pure-transcoding application optimized for the lowest possible latency
Modular, queue-based architecture
The previous implementation was fast but monolithic: the user had to understand the entire pipeline before modifying it. Version 13.1 redesigns the samples around a strictly modular, queue-based architecture that guarantees concurrency, simplifies customization, and maximizes hardware utilization.
The redesign allocates a dedicated CPU thread per pipeline stage, operating as a producer-consumer system where threads communicate through explicitly sized input and output queues. The core AppTrans pipeline (Figure 6, below) breaks into four isolated execution contexts: a Decode Thread (NVDEC) handles demux and decode; a Compute Thread (CUDA) handles processing; an Encode Thread (NVENC) handles encode; and an Output Thread gathers output and muxes.
Each thread isolates one step, producing a decoupled design that handles synchronization natively. Frames pass through the queues sequentially, ensuring safe data flow.
Key advantages of the redesign
- Modularity: Copy and tweak only the pipeline steps you need. Decoupled components keep errors and exceptions isolated within their threads
- Performance: Full concurrency between pipeline steps is guaranteed. Separating the CPU submission threads ensures the GPU is never starved; once saturated, the hardware engines (NVDEC, CUDA, NVENC) run their stages in parallel on different frames
- Customizability: Decoupled queues give you total control. Need encoding only? Remove the decode thread and feed the compute and encode queues directly. Want ultra-low latency over throughput? Reduce queue sizes to minimize buffer wait. Have a custom AI filter? Modify the Compute Thread to dispatch your CUDA kernels in isolation, without blocking the decoder or encoder submission loops
Zero-copy transcode with CUarray
In a traditional transcoding pipeline (as in AppTrans), the decoded frame passes through multiple internal format conversions and copies before reaching the encoder. These are inherent to the standard NvDecoder and NvEncoder APIs. The decoder output converts into an application-accessible surface; the application copies it into the encoder’s input buffer; the encoder converts that input again into its required format.
AppTransZeroCopy eliminates this copy chain by having NVDEC and NVENC operate directly on the same GPU memory in a format both engines understand natively. The mechanism has four parts:
- Shared buffer pool allocation: At startup, the application allocates a pool of CUDA arrays (
CUarray) usingcuArray3DCreatewith theCUDA_ARRAY3D_VIDEO_ENCODE_DECODEflag, which tells the CUDA driver these surfaces will be shared between both video-codec engines. EachCUarrayholds one frame (luma plus chroma planes) in a format both NVDEC and NVENC access natively, bypassing the traditional pipeline’s intermediate conversions. - Dual registration: The same
CUarrayregister with both codecs. On the decoder side, they are provided as external output surfaces viaSetExternalOutputArrays(), telling NVDEC to write decoded frames directly into them. On the encoder side, they register as input resources via NVENC’snvEncRegisterResourceAPI with resource typeNV_ENC_INPUT_RESOURCE_TYPE_CUDAARRAY, letting the encoder read them directly without input conversion. - Pipelined execution: Three threads—decode, encode, and output—connect through concurrent queues with token-based flow control to manage
CUarrayownership between decoder and encoder. - Stream-ordered synchronization: NVDEC and NVENC share the same CUDA stream, guaranteeing correct ordering between decode writes and encode reads without explicit CPU-GPU synchronization.
Key advantages
- Lower SM utilization: The traditional pipeline uses multiple CUDA copy and conversion kernels on Streaming Multiprocessors to shuttle frames between stages. Zero-copy eliminates these intermediate kernels, freeing SM resources for CUDA preprocessing, inference, or rendering
- Reduced GPU memory footprint: The traditional pipeline keeps separate buffers at each stage. Zero-copy collapses these into a single shared pool, significantly reducing per-session memory consumption
- Higher throughput with concurrent sessions: Lower SM utilization plus a smaller memory footprint improve scalability. The GPU can sustain more concurrent transcode sessions before hitting SM saturation or memory exhaustion
Figure 8, below, shows lower SM utilization in the new AppTransZeroCopy sample application compared to legacy AppTransPerf sample application.
Figure 9, below, shows lower video memory bandwidth utilization in the new AppTransZeroCopy sample application compared to legacy AppTransPerf sample application.
Docker development environment
Setting up the Video Codec SDK traditionally requires installing the CUDA toolkit, Vulkan SDK, system libraries, and FFmpeg, then building the SDK samples—often across different host distributions and driver versions. Video Codec SDK 13.1 introduces an official Docker-based development environment that packages a consistent, pre-configured stack into a single container. The image builds from an open Dockerfile, so you can reproduce the environment locally or in the cloud and customize it via build arguments.
The image is built in two stages: a builder stage compiles the SDK samples and installs the Vulkan SDK and FFmpeg; the runtime stage keeps only what is needed to run and develop. The stack pins CUDA 12.3.2, Vulkan SDK 1.4.304.1, and Ubuntu 22.04 LTS.
Use the SDK_ZIP build argument to point at Video_Codec_SDK_13.1.x.zip, and the optional FFMPEG_URL argument to supply a custom FFmpeg tarball (for example, an LGPL build with NVENC) instead of the default BtbN LGPL build.
Inside the container, pre-built samples live in /video-codec-sdk/Samples/build/, including AppDec and AppEncCuda, ready to run with test vectors. FFmpeg (LGPL) is installed at /opt/ffmpeg for YUV generation, MJPEG and MPEG encoding, and inspection of encoded streams.
A test-vector script produces raw YUV in multiple formats, JPEG, MPEG-1/2/4, and — with GPU access — H.264 and HEVC using AppEncCuda, so you can exercise the full pipeline. The container runs as a non-root user and includes a HEALTHCHECK for orchestrators.
Prerequisites
- an NVIDIA GPU with video encode/decode support
- Docker with GPU support enabled
- the NVIDIA Container Toolkit, and
- the SDK package file
Place the SDK zip (for example, Video_Codec_SDK_13.1.x.zip) in the Docker build context and build:
cd ubuntu22.04 docker build -t nvidia/video-codec-sdk:13.1-ubuntu22.04 \ --build-arg SDK_ZIP=Video_Codec_SDK_13.1.x.zip \ .
Launch with GPU access. You can generate test vectors at startup or open a shell and run samples directly:
# Standard launch docker run --gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04 # Generate the full test-vector suite during startup (~10-15 min) docker run --gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04 \ --generate-vectors full # Generate only H.264 vectors at 720p docker run --gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04 \ --generate-vectors h264 --resolution 1280x720
Generation modes are full, h264, hevc, vp8, vp9, and av1. Inside the container, the aliases sdk-samples, test-decode, and test-encode jump to the samples directory and run quick tests with the generated vectors.
Key advantages
- Reproducibility: The same CUDA, Vulkan, FFmpeg, and SDK versions run everywhere, reducing “works on my machine” drift
- Fast onboarding: Clone the repository, add the SDK zip, run
docker buildanddocker run --gpus all; no host-side SDK or Vulkan install required - CI- and cloud-friendly: A single image drives pipelines and cloud workloads wherever the NVIDIA Container Toolkit and GPU support are available
The Dockerfile and helper scripts live in the video-codec-sdk-docker repository. For detailed build options, environment variables, and troubleshooting, see the repository README.
Get started with Video Codec SDK 13.1
Download the SDK, try the new encode, decode, and transcode features in your pipelines, and share how they work for you.
The redesigned samples make it easy to drop the new features into existing workflows or build a custom pipeline from scratch.
· Download Video Codec SDK 13.1