a NSicjã @sLddlZddlZddlZddlmZmZmZgd¢ZdZ dd„Z dd„Z dd „Z d d „Z eejeejfdd œd d„Zde ddfdd„Zdde ddfddœeejeeejeejfeeeeejjeeejddœdd„Zdeejeddœdd„Zdeejeejddœdd„Ze ddfeejeejeddœdd„ZdS) éN)ÚOptionalÚSequenceÚUnion)Ú all_reduceÚreduceÚ broadcastÚ all_gatherÚreduce_scattercCstttjdƒst d¡dStƒ}|D]J}|jr4dS| ¡sBdS|jsNdS|  ¡}||vrddS|  |¡q$dS)NÚ_nccl_all_reducez)PyTorch is not compiled with NCCL supportFT) ÚhasattrÚtorchÚ_CÚwarningsÚwarnÚsetÚ is_sparseÚ is_contiguousÚis_cudaÚ get_deviceÚadd)ÚtensorsÚdevicesÚtensorÚdevice©rúK/var/www/html/django/DPS/env/lib/python3.9/site-packages/torch/cuda/nccl.pyÚ is_available s    rcCs0tj ¡}|d?}|d?d@}|d@}|||fS)Né éiÿÿ)r r Ú _nccl_version)ÚverÚmajorÚminorÚpatchrrrÚversion"s   r$cCs tj ¡S©N)r r Ú_nccl_unique_idrrrrÚ unique_id*sr'cCstj |||¡Sr%)r r Ú_nccl_init_rank)Z num_ranksÚuidÚrankrrrÚ init_rank.sr+)ÚinputsÚreturncCs&t|tjjƒrt|tjƒr"tdƒ‚dS)Nz(Inputs should be a collection of tensors)Ú isinstanceÚ collectionsÚabcÚ Containerr ÚTensorÚ TypeError)r,rrrÚ_check_sequence_type2sr4cCs4t|ƒ|dur|}t|ƒtj |||||¡dSr%)r4r r r ©r,ÚoutputsÚopÚstreamsÚcommsrrrr7s r)r6)r,ÚoutputÚrootr7r8r6r-cCs’t|ƒ|dur6|dur"tdƒ‚qxt d¡||}nBt|tjƒsdt|tjj ƒrdt d¡||}n|durt||n|}tj   ||||||¡dS)Nzä'output' and 'outputs' can not be both specified. 'outputs' is deprecated in favor of 'output', taking in a single output tensor. The signature of reduce is: reduce(inputs, output=None, root=0, op=SUM, streams=None, comms=None).zƒnccl.reduce with an output tensor list is deprecated. Please specify a single output tensor with argument 'output' instead instead.z\nccl.reduce with an output tensor list is deprecated. Please specify a single output tensor.) r4Ú ValueErrorrrr.r r2r/r0rr Ú _nccl_reduce)r,r:r;r7r8r9r6Ú_outputrrrrAs"ÿÿ ÿ r)r,r;r-cCst|ƒtj ||||¡dSr%)r4r r Ú_nccl_broadcast)r,r;r8r9rrrr`sr)r,r6r-cCs&t|ƒt|ƒtj ||||¡dSr%)r4r r Ú_nccl_all_gather)r,r6r8r9rrrresr)r,r6r7r-cCs(t|ƒt|ƒtj |||||¡dSr%)r4r r Ú_nccl_reduce_scatterr5rrrr ksr )rNN)NN)r/rÚ torch.cudar ÚtypingrrrÚ__all__ÚSUMrr$r'r+r2r4rÚintÚcudaÚStreamrrrr rrrrÚsD  ûú ú  ý ý