a 8Sici)@sddlZddlmZddlZddlmZddlmZmZddlm Z ddlm Z dd l m Z dd lmZgd ZGd d d e ZGddde ZGdddeZGddde ZGdddeeZdS)N)Any)Tensor) ParameterUninitializedParameter) functional)init)Module)LazyModuleMixin)BilinearIdentity LazyLinearLinearcs:eZdZdZeeddfdd ZeedddZZS) r aA placeholder identity operator that is argument-insensitive. Args: args: any argument (unused) kwargs: any keyword argument (unused) Shape: - Input: :math:`(*)`, where :math:`*` means any number of dimensions. - Output: :math:`(*)`, same shape as the input. Examples:: >>> m = nn.Identity(54, unused_argument1=0.1, unused_argument2=False) >>> input = torch.randn(128, 20) >>> output = m(input) >>> print(output.size()) torch.Size([128, 20]) N)argskwargsreturncstt|dSN)superr __init__)selfrr __class__S/var/www/html/django/DPS/env/lib/python3.9/site-packages/torch/nn/modules/linear.pyr)szIdentity.__init__inputrcCs|Srrrrrrrforward,szIdentity.forward) __name__ __module__ __qualname____doc__rrrr __classcell__rrrrr sr cs|eZdZUdZddgZeed<eed<eed<deeeddfdd Z dd d d Z eed ddZ e d ddZ ZS)ra&Applies a linear transformation to the incoming data: :math:`y = xA^T + b` This module supports :ref:`TensorFloat32`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. Args: in_features: size of each input sample out_features: size of each output sample bias: If set to ``False``, the layer will not learn an additive bias. Default: ``True`` Shape: - Input: :math:`(*, H_{in})` where :math:`*` means any number of dimensions including none and :math:`H_{in} = \text{in\_features}`. - Output: :math:`(*, H_{out})` where all but the last dimension are the same shape as the input and :math:`H_{out} = \text{out\_features}`. Attributes: weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in\_features}}` bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{in\_features}}` Examples:: >>> m = nn.Linear(20, 30) >>> input = torch.randn(128, 20) >>> output = m(input) >>> print(output.size()) torch.Size([128, 30]) in_features out_featuresweightTNr$r%biasrcsv||d}tt|||_||_ttj||ffi||_|r^ttj|fi||_ n | dd| dSNdevicedtyper() rrrr$r%rtorchemptyr&r(register_parameterreset_parameters)rr$r%r(r+r,factory_kwargsrrrrZs  zLinear.__init__rcCs`tj|jtdd|jdur\t|j\}}|dkrFdt|nd}t|j| |dS)N)arr )rkaiming_uniform_r&mathsqrtr(Z_calculate_fan_in_and_fan_outuniform_)rfan_in_boundrrrr0gs  zLinear.reset_parametersrcCst||j|jSr)Flinearr&r(rrrrrqszLinear.forwardcCsd|j|j|jduS)Nz(in_features={}, out_features={}, bias={})formatr$r%r(rrrr extra_reprtszLinear.extra_repr)TNNrr r!r"Z __constants__int__annotations__rboolrr0rstrr@r#rrrrr0s $  rcs*eZdZdeeeddfdd ZZS)NonDynamicallyQuantizableLinearTNr'cstj|||||ddS)N)r(r+r,)rr)rr$r%r(r+r,rrrrs z(NonDynamicallyQuantizableLinear.__init__)TNN)rr r!rBrDrr#rrrrrFs rFcseZdZUdZgdZeed<eed<eed<eed<deeeedd fd d Z dd d dZ eeedddZ e d ddZ ZS)r aApplies a bilinear transformation to the incoming data: :math:`y = x_1^T A x_2 + b` Args: in1_features: size of each first input sample in2_features: size of each second input sample out_features: size of each output sample bias: If set to False, the layer will not learn an additive bias. Default: ``True`` Shape: - Input1: :math:`(*, H_{in1})` where :math:`H_{in1}=\text{in1\_features}` and :math:`*` means any number of additional dimensions including none. All but the last dimension of the inputs should be the same. - Input2: :math:`(*, H_{in2})` where :math:`H_{in2}=\text{in2\_features}`. - Output: :math:`(*, H_{out})` where :math:`H_{out}=\text{out\_features}` and all but the last dimension are the same shape as the input. Attributes: weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in1\_features}, \text{in2\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in1\_features}}` bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in1\_features}}` Examples:: >>> m = nn.Bilinear(20, 30, 40) >>> input1 = torch.randn(128, 20) >>> input2 = torch.randn(128, 30) >>> output = m(input1, input2) >>> print(output.size()) torch.Size([128, 40]) ) in1_features in2_featuresr%rGrHr%r&TN)rGrHr%r(rcs~||d}tt|||_||_||_ttj|||ffi||_ |rfttj|fi||_ n | dd| dSr)) rr rrGrHr%rr-r.r&r(r/r0)rrGrHr%r(r+r,r1rrrrs  zBilinear.__init__r2cCsHdt|jd}t|j| ||jdurDt|j| |dS)Nr )r6r7r&sizerr8r()rr;rrrr0s zBilinear.reset_parameters)input1input2rcCst|||j|jSr)r<bilinearr&r()rrJrKrrrrszBilinear.forwardcCsd|j|j|j|jduS)Nz:in1_features={}, in2_features={}, out_features={}, bias={})r>rGrHr%r(r?rrrr@szBilinear.extra_repr)TNNrArrrrr s % r csbeZdZUdZeZeed<eed<dee ddfdd Z dd fd d Z dd d d Z Z S)raA :class:`torch.nn.Linear` module where `in_features` is inferred. In this module, the `weight` and `bias` are of :class:`torch.nn.UninitializedParameter` class. They will be initialized after the first call to ``forward`` is done and the module will become a regular :class:`torch.nn.Linear` module. The ``in_features`` argument of the :class:`Linear` is inferred from the ``input.shape[-1]``. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_features: size of each output sample bias: If set to ``False``, the layer will not learn an additive bias. Default: ``True`` Attributes: weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in\_features}}` bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{in\_features}}` r&r(TN)r%r(rcsH||d}tdddtfi||_||_|rDtfi||_dS)Nr*rF)rrrr&r%r()rr%r(r+r,r1rrrrs  zLazyLinear.__init__r2cs |s|jdkrtdS)Nr)has_uninitialized_paramsr$rr0r?rrrr0szLazyLinear.reset_parameterscCsv|rrtR|jd|_|j|j|jf|jdurL|j|jf| Wdn1sh0YdS)N) rMr-no_gradshaper$r& materializer%r(r0rrrrinitialize_parameterss   z LazyLinear.initialize_parameters)TNN)rr r!r"r cls_to_becomerrCrBrDrr0rRr#rrrrrs  r)r6typingrr-rZtorch.nn.parameterrrrr<rmoduler lazyr __all__r rrFr rrrrrs      OJ