1) cuMemGetAllocationGranularity returns the required size granularity (alignment) for a given device. 2) cuMemAddressReserve reserves a contiguous virtual address range (with no physical memory). 3) cuMemCreate creates a handle to physical memory (backing store) whose size is a multiple of the granularity. 4) cuMemMap maps the physical handle into the reserved address range. 5) cuMemSetAccess grants access rights (read/write) for specified devices, which also controls peer-to-peer access. The inverse operations are cuMemUnmap, cuMemAddressFree and cuMemRelease. To grow an allocation, a larger address range is reserved and additional physical handles are mapped in without moving existing data. Cross-process sharing uses cuMemExportToShareableHandle and cuMemImportFromShareableHandle operating on OS handles: file descriptors on Linux and HANDLE / D3DKMT_HANDLE on Windows. CUDA does not support mapping part of a single physical allocation, so sizes must match.
High-level cudaMalloc fuses address reservation with physical allocation, so an existing allocation cannot be grown without reallocating and copying data, and freeing memory forces a full-device synchronization. This makes it hard to build growing buffers, memory pools, and to control fragmentation as well as memory sharing across devices and processes.
Reserves a contiguous range of virtual GPU addresses without backing it with physical memory.
Creates a handle to a physical memory allocation (backing store) sized as a multiple of the granularity.
Maps a physical memory handle into the reserved virtual address range.
Sets access rights (read/write) to the mapped memory per device, also controlling peer-to-peer access.
Returns the required size alignment (granularity); allocation sizes must be a multiple of it.
Export and import allocations as OS handles (file descriptor on Linux, HANDLE/D3DKMT_HANDLE on Windows), enabling sharing across processes and with graphics APIs.
Official
Sizes must be a multiple of cuMemGetAllocationGranularity; CUDA cannot map part of a single physical allocation.
The developer is responsible for the reserve โ create โ map โ setAccess order and the unmap โ release โ addressFree teardown; mistakes cause leaks or access errors.
Another GPU's access to the memory requires an explicit cuMemSetAccess; without it access faults occur.
CUDA 10.2 adds cuMemCreate, cuMemAddressReserve, cuMemMap and cuMemSetAccess, decoupling address reservation from physical allocation.
The official guide describes usage patterns: growing allocations, memory pools, IPC and selective peer access.
PyTorch's caching allocator gains the expandable_segments mode that uses VMM to reduce fragmentation under changing allocation sizes.
Physical memory size alignment required by the device; allocation sizes must be a multiple of it.
Type of OS handle used for sharing: POSIX file descriptor (Linux) or HANDLE / D3DKMT_HANDLE (Windows).
Read/write rights set per device via cuMemSetAccess, also governing peer-to-peer access.
The CUDA driver API runs only on NVIDIA GPUs supporting virtual memory management (from CUDA 10.2).
