a NSic_Z@s^ddlZddlZddlZddlmZmZmZddlZddlm Z m Z m Z ddl m Z gdZddZejd d Zd8ee efd d d ZddZd9ee efddddZddddZd:ee efeeefdddZd;ee efeeefdddZdee efddddZd?ee efddd d!Zd@ee efedd"d#ZdAee efedd$d%ZdBee efedd&d'Z dCee efedd(d)Z!dDee efedd*d+Z"dEee efedd,d-Z#d.d/Z$dFee efe%ed1d2d3Z&dGee efedd4d5Z'dHee efedd6d7Z(dS)IN)AnyDictUnion)is_initialized_get_device_index _lazy_init)Device)caching_allocator_alloccaching_allocator_deleteset_per_process_memory_fraction empty_cache memory_statsmemory_stats_as_nested_dictreset_accumulated_memory_statsreset_peak_memory_statsreset_max_memory_allocatedreset_max_memory_cachedmemory_allocatedmax_memory_allocatedmemory_reservedmax_memory_reserved memory_cachedmax_memory_cachedmemory_snapshotmemory_summarylist_gpu_processes mem_get_infocCsttjSN)rtorch_C_cuda_cudaHostAllocatorr"r"M/var/www/html/django/DPS/env/lib/python3.9/site-packages/torch/cuda/memory.py_host_allocatorsr$ccs0tjzdVWtjn tj0dSr)rr _cuda_lock_mutex_cuda_unlock_mutexr"r"r"r# _free_mutexs r'devicecCs|durtj}t|}|dur.tj|}t|tjjjrD|j}t|t sVt dtj |tj ||WdS1s0YdS)aPerforms a memory allocation using the CUDA memory allocator. Memory is allocated for a given device and a stream, this function is intended to be used for interoperability with other frameworks. Allocated memory is released through :func:`~torch.cuda.caching_allocator_delete`. Args: size (int): number of bytes to be allocated. device (torch.device or int, optional): selected device. If it is ``None`` the default CUDA device is used. stream (torch.cuda.Stream or int, optional): selected stream. If is ``None`` then the default stream for the selected device is used. .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. NzsInvalid type for stream argument, must be `torch.cuda.Stream` or `int` representing a pointer to a exisiting stream)rcudacurrent_devicercurrent_stream isinstancestreamsStream cuda_streamint TypeErrorr)r $_cuda_cudaCachingAllocator_raw_alloc)sizer)streamr"r"r#r s   r cCstj|dS)aDeletes memory allocated using the CUDA memory allocator. Memory allocated with :func:`~torch.cuda.caching_allocator_alloc`. is freed here. The associated device and stream are tracked inside the allocator. Args: mem_ptr (int): memory address to be freed by the allocator. .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. N)rr %_cuda_cudaCachingAllocator_raw_delete)Zmem_ptrr"r"r#r Asr )r)returncCsbt|durtj}t|}t|ts2td|dksB|dkrPtd |tj ||dS)aSet memory fraction for a process. The fraction is used to limit an caching allocator to allocated memory on a CUDA device. The allowed value equals the total visible memory multiplied fraction. If trying to allocate more than the allowed value in a process, will raise an out of memory error in allocator. Args: fraction(float): Range: 0~1. Allowed memory equals total_memory * fraction. device (torch.device or int, optional): selected device. If it is ``None`` the default CUDA device is used. .. note:: In general, the total available free memory is less than the total capacity. Nz3Invalid type for fraction argument, must be `float`rrz.Invalid fraction value: {}. Allowed range: 0~1) rrr*r+rr-floatr2 ValueErrorformatr _cuda_setMemoryFraction)fractionr)r"r"r#r Rs  r )r7cCstrtjdS)aReleases all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in `nvidia-smi`. .. note:: :func:`~torch.cuda.empty_cache` doesn't increase the amount of GPU memory available for PyTorch. However, it may help reduce fragmentation of GPU memory in certain cases. See :ref:`cuda-memory-management` for more details about GPU memory management. N)rrr _cuda_emptyCacher"r"r"r#r ms r cs8gfddt|d}d|tS)aX Returns a dictionary of CUDA memory allocator statistics for a given device. The return value of this function is a dictionary of statistics, each of which is a non-negative integer. Core statistics: - ``"allocated.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: number of allocation requests received by the memory allocator. - ``"allocated_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: amount of allocated memory. - ``"segment.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: number of reserved segments from ``cudaMalloc()``. - ``"reserved_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: amount of reserved memory. - ``"active.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: number of active memory blocks. - ``"active_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: amount of active memory. - ``"inactive_split.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: number of inactive, non-releasable memory blocks. - ``"inactive_split_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``: amount of inactive, non-releasable memory. For these core statistics, values are broken down as follows. Pool type: - ``all``: combined statistics across all memory pools. - ``large_pool``: statistics for the large allocation pool (as of October 2019, for size >= 1MB allocations). - ``small_pool``: statistics for the small allocation pool (as of October 2019, for size < 1MB allocations). Metric type: - ``current``: current value of this metric. - ``peak``: maximum value of this metric. - ``allocated``: historical total increase in this metric. - ``freed``: historical total decrease in this metric. In addition to the core statistics, we also provide some simple event counters: - ``"num_alloc_retries"``: number of failed ``cudaMalloc`` calls that result in a cache flush and retry. - ``"num_ooms"``: number of out-of-memory errors thrown. The caching allocator can be configured via ENV to not split blocks larger than a defined size (see Memory Management section of the Cuda Semantics documentation). This helps avoid memory framentation but may have a performance penalty. Additional outputs to assist with tuning and evaluating impact: - ``"max_split_size"``: blocks above this size will not be split. - ``"oversize_allocations.{current,peak,allocated,freed}"``: number of over-size allocation requests received by the memory allocator. - ``"oversize_segments.{current,peak,allocated,freed}"``: number of over-size reserved segments from ``cudaMalloc()``. Args: device (torch.device or int, optional): selected device. Returns statistics for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. csRt|tr@t|dkr|d7}|D]\}}|||q&n||fdS)Nr.)r-dictlenitemsappend)prefixobjkv_recurse_add_to_resultresultr"r#rHs   z,memory_stats.._recurse_add_to_resultr()rsort collections OrderedDict)r)statsr"rGr#r|s F  rcCs"ts iSt|dd}tj|S)zNReturns the result of :func:`~torch.cuda.memory_stats` as a nested dictionary.Toptional)rrrr _cuda_memoryStatsr(r"r"r#rs rcCst|dd}tj|S)a~Resets the "accumulated" (historical) stats tracked by the CUDA memory allocator. See :func:`~torch.cuda.memory_stats` for details. Accumulated stats correspond to the `"allocated"` and `"freed"` keys in each individual stat dict, as well as `"num_alloc_retries"` and `"num_ooms"`. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. TrO)rrr !_cuda_resetAccumulatedMemoryStatsr(r"r"r#rs rcCst|dd}tj|S)aResets the "peak" stats tracked by the CUDA memory allocator. See :func:`~torch.cuda.memory_stats` for details. Peak stats correspond to the `"peak"` key in each individual stat dict. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. TrO)rrr _cuda_resetPeakMemoryStatsr(r"r"r#rs rcCstdtt|dS)aResets the starting point in tracking maximum GPU memory occupied by tensors for a given device. See :func:`~torch.cuda.max_memory_allocated` for details. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. warning:: This function now calls :func:`~torch.cuda.reset_peak_memory_stats`, which resets /all/ peak memory stats. .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. zytorch.cuda.reset_max_memory_allocated now calls torch.cuda.reset_peak_memory_stats, which resets /all/ peak memory stats.r(warningswarn FutureWarningrr(r"r"r#rs rcCstdtt|dS)aResets the starting point in tracking maximum GPU memory managed by the caching allocator for a given device. See :func:`~torch.cuda.max_memory_cached` for details. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. warning:: This function now calls :func:`~torch.cuda.reset_peak_memory_stats`, which resets /all/ peak memory stats. .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. zvtorch.cuda.reset_max_memory_cached now calls torch.cuda.reset_peak_memory_stats, which resets /all/ peak memory stats.r(rTr(r"r"r#rs rcCst|dddS)a`Returns the current GPU memory occupied by tensors in bytes for a given device. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: This is likely less than the amount shown in `nvidia-smi` since some unused memory can be held by the caching allocator and some context needs to be created on GPU. See :ref:`cuda-memory-management` for more details about GPU memory management. r(zallocated_bytes.all.currentrrgetr(r"r"r#r7srcCst|dddS)aReturns the maximum GPU memory occupied by tensors in bytes for a given device. By default, this returns the peak allocated memory since the beginning of this program. :func:`~torch.cuda.reset_peak_memory_stats` can be used to reset the starting point in tracking this metric. For example, these two functions can measure the peak allocated memory usage of each iteration in a training loop. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. r(zallocated_bytes.all.peakrrXr(r"r"r#rIsrcCst|dddS)aReturns the current GPU memory managed by the caching allocator in bytes for a given device. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. r(zreserved_bytes.all.currentrrXr(r"r"r#r_s rcCst|dddS)aReturns the maximum GPU memory managed by the caching allocator in bytes for a given device. By default, this returns the peak cached memory since the beginning of this program. :func:`~torch.cuda.reset_peak_memory_stats` can be used to reset the starting point in tracking this metric. For example, these two functions can measure the peak cached memory amount of each iteration in a training loop. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. r(zreserved_bytes.all.peakrrXr(r"r"r#rosrcCstdtt|dS)z4Deprecated; see :func:`~torch.cuda.memory_reserved`.zGtorch.cuda.memory_cached has been renamed to torch.cuda.memory_reservedr()rUrVrWrr(r"r"r#rs rcCstdtt|dS)z8Deprecated; see :func:`~torch.cuda.max_memory_reserved`.zOtorch.cuda.max_memory_cached has been renamed to torch.cuda.max_memory_reservedr()rUrVrWrr(r"r"r#rs rcCs tjS)a)Returns a snapshot of the CUDA memory allocator state across all devices. Interpreting the output of this function requires familiarity with the memory allocator internals. .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. )rr _cuda_memorySnapshotr"r"r"r#rs rF)r) abbreviatedr7c Csxt|dd}t|d}dd}dd}dd |fd d |fd d |fdd|fdd|fdd|fdd|fdd|fg}g}|d|d|d|d|d|d|D]\}}} |dd|fg} |s| d| dd \} } } }| D]\}}|d!|d!}||d"}||d#}||d$}||d%}| d&urV|} |} |} |}|d'|| || | || | || | ||qqd(d)|fd*d+|fg}|D]~\}}} |d|d!}||d"}||d#}||d$}||d%}|d'|| ||| ||| ||| ||q|dd,|d-}|D]\}}|||d!d.<q>d/d0|jfi|d1S)2aReturns a human-readable printout of the current memory allocator statistics for a given device. This can be useful to display periodically during training, or when handling out-of-memory exceptions. Args: device (torch.device or int, optional): selected device. Returns printout for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). abbreviated (bool, optional): whether to return an abbreviated summary (default: False). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. TrOr(cSsNgd}|d}|ddD]$}|dkr,qB|}|d}|d}qd||S)N)zB KBMBGBTBPBrri iz{:7d} {}r:)szZpref_szprefixesrC new_prefixr"r"r# _format_sizes z$memory_summary.._format_sizecSsNgd}|d}|ddD]$}|dkr,qB|}|d}|d}qd||S)N) KMrriq iz {:7d} {} ra)cntZpref_cntrcrCrdr"r"r# _format_counts z%memory_summary.._format_countZallocated_byteszAllocated memoryZ active_bytesz Active memoryZreserved_byteszGPU reserved memoryZinactive_split_byteszNon-releasable memoryZ allocationZ Allocationsactivez Active allocssegmentzGPU reserved segmentsZinactive_splitzNon-releasable allocszK===========================================================================z= {_:16} PyTorch CUDA memory summary, device ID {device:<17d} zK---------------------------------------------------------------------------zX {_:9} CUDA OOMs: {num_ooms:<12d} | {_:6} cudaMalloc retries: {num_alloc_retries:<8d} zK Metric | Cur Usage | Peak Usage | Tot Alloc | Tot Freed all)Z large_poolz from large pool)Z small_poolz from small pool)NNNNr>currentpeak allocatedfreedNz {:<21} | {} | {} | {} | {} Zoversize_allocationszOversize allocationsZoversize_segmentszOversize GPU segmentsrJ)_r)-|z| |z| )rrrBr:rAreplacejoin)r)r[rNrerjZmetrics_to_displaylinesZ metric_keyZ metric_name formatterZ submetricsZcurrent_prefvalZ peak_prefvalZallocated_prefvalZ freed_prefvalZ submetric_keyZsubmetric_namerCrnrorprqZfmt_dictrErFr"r"r#rs                               rcCsz ddl}Wnty YdS0ddlm}z |Wn|yNYdS0t|dd}||}||}g}|d|t|dkr|d |D],}|j d }|d |j d d |ddqd |S)aReturns a human-readable printout of the running processes and their GPU memory use for a given device. This can be useful to display periodically during training, or when handling out-of-memory exceptions. Args: device (torch.device or int, optional): selected device. Returns printout for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). rNz.pynvml module not found, please install pynvml)NVMLError_DriverNotLoadedz-cuda driver can't be loaded, is cuda enabled?TrOzGPU:zno processes are runningizprocess z>10dz uses z>12.3fz MB GPU memory ) pynvmlModuleNotFoundErrorrynvmlInitrnvmlDeviceGetHandleByIndexZ$nvmlDeviceGetComputeRunningProcessesrBr@Z usedGpuMemorypidrv)r)r{ryhandleZprocsrwpZmemr"r"r#rs(             rcCs*|durtj}t|}tj|S)aReturns the global free and total GPU memory occupied for a given device using cudaMemGetInfo. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). .. note:: See :ref:`cuda-memory-management` for more details about GPU memory management. N)rr*r+rcudartcudaMemGetInfor(r"r"r#r@s  r)NN)N)N)N)N)N)N)N)N)N)N)N)N)N)NF)N)N))rL contextlibrUtypingrrrrrJrrr torch.typesr __all__r$contextmanagerr'r1r r r r strrrrrrrrrrrrrrboolrrrr"r"r"r#s<  ""X" |"