a 8SicäNã@s¼ddlZddlZddlmZddlZdd„Zdd„Zdd„Zd d „Zd d „Zd@d d„Z dAee e edœdd„Z dBee e edœdd„Z dCee e e e edœdd„Z ee edœdd„Zeedœd d!„Zeedœd"d#„Zd$d%„ZdDd'd(„Zd)d*„ZdEee ed+œd,d-„ZdFee ed+œd.d/„Zd0d1„ZdGee eed4œd5d6„ZdHee eed4œd7d8„ZdId9d:„ZdJdd?„Zee ƒZee ƒZeeƒZeeƒZ eeƒZ!eeƒZ"eeƒZ#eeƒZ$eeƒZ%eeƒZ&eeƒZ'dS)KéN)ÚTensorcCs8t ¡| ||¡WdƒS1s*0YdS©N)ÚtorchÚno_gradÚuniform_©ÚtensorÚaÚb©r úI/var/www/html/django/DPS/env/lib/python3.9/site-packages/torch/nn/init.pyÚ_no_grad_uniform_ s r cCs8t ¡| ||¡WdƒS1s*0YdSr)rrÚnormal_©rÚmeanÚstdr r r Ú_no_grad_normal_s rcCsÖdd„}||d|ks(||d|kr6tjdddt ¡„||||ƒ}||||ƒ}| d|dd|d¡| ¡| |t d¡¡|  |¡|j ||d|WdƒS1sÈ0YdS) NcSsdt |t d¡¡dS)Nçð?ç@)ÚmathÚerfÚsqrt)Úxr r r Únorm_cdfsz(_no_grad_trunc_normal_..norm_cdfézjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.©Ú stacklevelér)ÚminÚmax) ÚwarningsÚwarnrrrÚerfinv_Úmul_rrÚadd_Úclamp_)rrrr r rÚlÚur r r Ú_no_grad_trunc_normal_s þ  r(cCs6t ¡| |¡WdƒS1s(0YdSr)rrÚfill_©rÚvalr r r Ú_no_grad_fill_9s r,cCs4t ¡| ¡WdƒS1s&0YdSr)rrÚzero_©rr r r Ú_no_grad_zero_>s r/cCs¶gd¢}||vs|dkrdS|dkr(dS|dkr:t d¡S|dkr˜|d urPd }n2t|tƒsdt|tƒsnt|tƒrt|}ntd  |¡ƒ‚t dd|d ¡S|d kr¤dStd |¡ƒ‚d S)aÝReturn the recommended gain value for the given nonlinearity function. The values are as follows: ================= ==================================================== nonlinearity gain ================= ==================================================== Linear / Identity :math:`1` Conv{1,2,3}D :math:`1` Sigmoid :math:`1` Tanh :math:`\frac{5}{3}` ReLU :math:`\sqrt{2}` Leaky Relu :math:`\sqrt{\frac{2}{1 + \text{negative\_slope}^2}}` SELU :math:`\frac{3}{4}` ================= ==================================================== .. warning:: In order to implement `Self-Normalizing Neural Networks`_ , you should use ``nonlinearity='linear'`` instead of ``nonlinearity='selu'``. This gives the initial weights a variance of ``1 / N``, which is necessary to induce a stable fixed point in the forward pass. In contrast, the default gain for ``SELU`` sacrifices the normalisation effect for more stable gradient flow in rectangular layers. Args: nonlinearity: the non-linear function (`nn.functional` name) param: optional parameter for the non-linear function Examples: >>> gain = nn.init.calculate_gain('leaky_relu', 0.2) # leaky_relu with negative_slope=0.2 .. _Self-Normalizing Neural Networks: https://papers.nips.cc/paper/2017/hash/5d44ee6f2c3f71b73125876103c8f6c4-Abstract.html )ÚlinearÚconv1dÚconv2dÚconv3dÚconv_transpose1dÚconv_transpose2dÚconv_transpose3dÚsigmoidrÚtanhg«ªªªªªú?ÚrelurÚ leaky_reluNç{®Gáz„?z$negative_slope {} not a valid numberrÚselugè?zUnsupported nonlinearity {})rrÚ isinstanceÚboolÚintÚfloatÚ ValueErrorÚformat)Ú nonlinearityÚparamZ linear_fnsÚnegative_sloper r r Úcalculate_gainCs"! rFçr)rr r ÚreturncCs0tj |¡r$tjjt|f|||dSt|||ƒS)adFills the input Tensor with values drawn from the uniform distribution :math:`\mathcal{U}(a, b)`. Args: tensor: an n-dimensional `torch.Tensor` a: the lower bound of the uniform distribution b: the upper bound of the uniform distribution Examples: >>> w = torch.empty(3, 5) >>> nn.init.uniform_(w) r)rÚ overridesÚhas_torch_function_variadicÚhandle_torch_functionrr rr r r rzs r)rrrrHcCs0tj |¡r$tjjt|f|||dSt|||ƒS)azFills the input Tensor with values drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)`. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution Examples: >>> w = torch.empty(3, 5) >>> nn.init.normal_(w) r)rrIrJrKrrrr r r rŒs rçÀr)rrrr r rHcCst|||||ƒS)aÏFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) )r()rrrr r r r r Ú trunc_normal_srM)rr+rHcCs,tj |¡r"tjjt|f||dSt||ƒS)zÿFills the input Tensor with the value :math:`\text{val}`. Args: tensor: an n-dimensional `torch.Tensor` val: the value to fill the tensor with Examples: >>> w = torch.empty(3, 5) >>> nn.init.constant_(w, 0.3) r*)rrIrJrKÚ constant_r,r*r r r rN³s rN)rrHcCs t|dƒS)z¿Fills the input Tensor with the scalar value `1`. Args: tensor: an n-dimensional `torch.Tensor` Examples: >>> w = torch.empty(3, 5) >>> nn.init.ones_(w) r)r,r.r r r Úones_Ãs rOcCst|ƒS)zÀFills the input Tensor with the scalar value `0`. Args: tensor: an n-dimensional `torch.Tensor` Examples: >>> w = torch.empty(3, 5) >>> nn.init.zeros_(w) )r/r.r r r Úzeros_Ðs rPcCsV| ¡dkrtdƒ‚t ¡&tj|j||jdœŽWdƒn1sH0Y|S)a=Fills the 2-dimensional input `Tensor` with the identity matrix. Preserves the identity of the inputs in `Linear` layers, where as many inputs are preserved as possible. Args: tensor: a 2-dimensional `torch.Tensor` Examples: >>> w = torch.empty(3, 5) >>> nn.init.eye_(w) rú,Only tensors with 2 dimensions are supported)ÚoutÚ requires_gradN)Ú ndimensionrArrÚeyeÚshaperSr.r r r Úeye_Ýs  4rWrc Cs<| ¡}|dvrtdƒ‚| ¡}|d|dkr8tdƒ‚|d|}t||dƒ}t ¡Ì| ¡t|ƒD]ª}t|ƒD]œ}|dkr¦d|||||| d¡df<qx|dkrÜd|||||| d¡d| d¡df<qxd|||||| d¡d| d¡d| d¡df<qxqlWd ƒn1s.0Y|S) aAFills the {3, 4, 5}-dimensional input `Tensor` with the Dirac delta function. Preserves the identity of the inputs in `Convolutional` layers, where as many input channels are preserved as possible. In case of groups>1, each group of channels preserves identity Args: tensor: a {3, 4, 5}-dimensional `torch.Tensor` groups (optional): number of groups in the conv layer (default: 1) Examples: >>> w = torch.empty(3, 16, 5, 5) >>> nn.init.dirac_(w) >>> w = torch.empty(3, 24, 5, 5) >>> nn.init.dirac_(w, 3) )éééz5Only tensors with 3, 4, or 5 dimensions are supportedrz!dim 0 must be divisible by groupsrrXrrYN)rTrAÚsizerrrr-Úrange)rÚgroupsÚ dimensionsÚsizesZout_chans_per_grpZmin_dimÚgÚdr r r Údirac_ñs0    "ÿ ÿÿÿ(rbcCsp| ¡}|dkrtdƒ‚| d¡}| d¡}d}| ¡dkrX|jdd…D] }||9}qJ||}||}||fS)NrzNFan in and fan out can not be computed for tensor with fewer than 2 dimensionsrr)ÚdimrAr[rV)rr^Znum_input_fmapsZnum_output_fmapsZreceptive_field_sizeÚsÚfan_inÚfan_outr r r Ú_calculate_fan_in_and_fan_outs    rg)rÚgainrHcCsBt|ƒ\}}|t dt||ƒ¡}t d¡|}t|| |ƒS)a©Fills the input `Tensor` with values according to the method described in `Understanding the difficulty of training deep feedforward neural networks` - Glorot, X. & Bengio, Y. (2010), using a uniform distribution. The resulting tensor will have values sampled from :math:`\mathcal{U}(-a, a)` where .. math:: a = \text{gain} \times \sqrt{\frac{6}{\text{fan\_in} + \text{fan\_out}}} Also known as Glorot initialization. Args: tensor: an n-dimensional `torch.Tensor` gain: an optional scaling factor Examples: >>> w = torch.empty(3, 5) >>> nn.init.xavier_uniform_(w, gain=nn.init.calculate_gain('relu')) rç@)rgrrr@r )rrhrerfrr r r r Úxavier_uniform_/s rjcCs2t|ƒ\}}|t dt||ƒ¡}t|d|ƒS)a•Fills the input `Tensor` with values according to the method described in `Understanding the difficulty of training deep feedforward neural networks` - Glorot, X. & Bengio, Y. (2010), using a normal distribution. The resulting tensor will have values sampled from :math:`\mathcal{N}(0, \text{std}^2)` where .. math:: \text{std} = \text{gain} \times \sqrt{\frac{2}{\text{fan\_in} + \text{fan\_out}}} Also known as Glorot initialization. Args: tensor: an n-dimensional `torch.Tensor` gain: an optional scaling factor Examples: >>> w = torch.empty(3, 5) >>> nn.init.xavier_normal_(w) rrG)rgrrr@r)rrhrerfrr r r Úxavier_normal_Js rkcCsD| ¡}ddg}||vr(td ||¡ƒ‚t|ƒ\}}|dkr@|S|S)Nrerfz+Mode {} not supported, please use one of {})ÚlowerrArBrg)rÚmodeZ valid_modesrerfr r r Ú_calculate_correct_fands  rnrer:©rr rmrCcCs¨tj |¡r&tjjt|f||||dSd|jvr>t d¡|St||ƒ}t ||ƒ}|t   |¡}t   d¡|}t  ¡|  | |¡WdƒS1sš0YdS)a¬Fills the input `Tensor` with values according to the method described in `Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification` - He, K. et al. (2015), using a uniform distribution. The resulting tensor will have values sampled from :math:`\mathcal{U}(-\text{bound}, \text{bound})` where .. math:: \text{bound} = \text{gain} \times \sqrt{\frac{3}{\text{fan\_mode}}} Also known as He initialization. Args: tensor: an n-dimensional `torch.Tensor` a: the negative slope of the rectifier used after this layer (only used with ``'leaky_relu'``) mode: either ``'fan_in'`` (default) or ``'fan_out'``. Choosing ``'fan_in'`` preserves the magnitude of the variance of the weights in the forward pass. Choosing ``'fan_out'`` preserves the magnitudes in the backwards pass. nonlinearity: the non-linear function (`nn.functional` name), recommended to use only with ``'relu'`` or ``'leaky_relu'`` (default). Examples: >>> w = torch.empty(3, 5) >>> nn.init.kaiming_uniform_(w, mode='fan_in', nonlinearity='relu') rorú,Initializing zero-element tensors is a no-opriN)rrIrJrKÚkaiming_uniform_rVr r!rnrFrrrr)rr rmrCÚfanrhrÚboundr r r rqns$ ú     rqcCsrd|jvrt d¡|St||ƒ}t||ƒ}|t |¡}t ¡|  d|¡WdƒS1sd0YdS)a”Fills the input `Tensor` with values according to the method described in `Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification` - He, K. et al. (2015), using a normal distribution. The resulting tensor will have values sampled from :math:`\mathcal{N}(0, \text{std}^2)` where .. math:: \text{std} = \frac{\text{gain}}{\sqrt{\text{fan\_mode}}} Also known as He initialization. Args: tensor: an n-dimensional `torch.Tensor` a: the negative slope of the rectifier used after this layer (only used with ``'leaky_relu'``) mode: either ``'fan_in'`` (default) or ``'fan_out'``. Choosing ``'fan_in'`` preserves the magnitude of the variance of the weights in the forward pass. Choosing ``'fan_out'`` preserves the magnitudes in the backwards pass. nonlinearity: the non-linear function (`nn.functional` name), recommended to use only with ``'relu'`` or ``'leaky_relu'`` (default). Examples: >>> w = torch.empty(3, 5) >>> nn.init.kaiming_normal_(w, mode='fan_out', nonlinearity='relu') rrpN) rVr r!rnrFrrrrr)rr rmrCrrrhrr r r Úkaiming_normal_Ÿs     rtc Csà| ¡dkrtdƒ‚| ¡dkr$|S| d¡}| ¡|}| ||¡ dd¡}||kr^| ¡tj  |¡\}}t  |d¡}|  ¡}||9}||krš| ¡t  ¡*|  |¡ |¡| |¡Wdƒn1sÒ0Y|S)a!Fills the input `Tensor` with a (semi) orthogonal matrix, as described in `Exact solutions to the nonlinear dynamics of learning in deep linear neural networks` - Saxe, A. et al. (2013). The input tensor must have at least 2 dimensions, and for tensors with more than 2 dimensions the trailing dimensions are flattened. Args: tensor: an n-dimensional `torch.Tensor`, where :math:`n \geq 2` gain: optional scaling factor Examples: >>> w = torch.empty(3, 5) >>> nn.init.orthogonal_(w) rz4Only tensors with 2 or more dimensions are supportedrrN)rTrAÚnumelr[ÚnewrÚt_rÚlinalgÚqrÚdiagÚsignrÚview_asÚcopy_r#) rrhÚrowsÚcolsÚ flattenedÚqÚrraÚphr r r Ú orthogonal_Æs&      (r„r;c Cs˜| ¡dkrtdƒ‚|j\}}tt ||¡ƒ}t ¡L| d|¡t |ƒD]&}t  |¡}|d|…}d|||f<qNWdƒn1sŠ0Y|S)aNFills the 2D input `Tensor` as a sparse matrix, where the non-zero elements will be drawn from the normal distribution :math:`\mathcal{N}(0, 0.01)`, as described in `Deep learning via Hessian-free optimization` - Martens, J. (2010). Args: tensor: an n-dimensional `torch.Tensor` sparsity: The fraction of elements in each column to be set to zero std: the standard deviation of the normal distribution used to generate the non-zero values Examples: >>> w = torch.empty(3, 5) >>> nn.init.sparse_(w, sparsity=0.1) rrQrN) rTrArVr?rÚceilrrrr\Úrandperm) rZsparsityrr~rZ num_zerosZcol_idxÚ row_indicesZ zero_indicesr r r Úsparse_òs       ,rˆcs<ˆj‰ˆdd…‰‡‡‡fdd„}djˆˆd|_ˆ|_|S)Néÿÿÿÿcs$tjd ˆˆ¡ddˆ|i|¤ŽS)Nz4nn.init.{} is now deprecated in favor of nn.init.{}.rr)r r!rB)ÚargsÚkwargs©ÚmethÚnew_nameÚold_namer r Údeprecated_inits ÿÿz(_make_deprecate..deprecated_initz² {old_name}(...) .. warning:: This method is now deprecated in favor of :func:`torch.nn.init.{new_name}`. See :func:`~torch.nn.init.{new_name}` for details.)rrŽ)Ú__name__rBÚ__doc__)rrr rŒr Ú_make_deprecates ùr“)N)rGr)rGr)rGrrLr)r)r)r)rrer:)rrer:)r)r;)(rr rrr rr(r,r/rFr@rrrMrNrOrPrWrbrgrjrkrnÚstrrqrtr„rˆr“ÚuniformÚnormalÚconstantrUÚdiracÚxavier_uniformÚ xavier_normalÚkaiming_uniformÚkaiming_normalÚ orthogonalÚsparser r r r ÚsV # 7   + ÿÿ 2ÿÿ ' ,