a OSicZ @sddlmZddlmZmZddlZddlmZddlm Z m Z m Z m Z m Z mZmZmZmZmZmZmZmZmZddlmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(Z(Gdddedd d gZ)Gd d d ed d d gZ*e)e!e$d Z+e)e$edZ,e)e!e"d Z-e)ee$d Z.e)ej/ej0ej1dej/ej1dd Z2e)ej/ej1dej/ej1dd Z3e)ee"d Z4e)e#ed Z5e)e#e d Z6e)e ed Z7e)e ed Z8e)ejj9ed Z:e)e ejj9d Z;e)eed Ze)ej/ej?ddde%d Z@e)ej/ej?ddde&d ZAe)ej/ej0ded ZBe)ej/ej0ded ZCd1ddZDe)ej/eddej?ddded ZEe)ej/eddej?ddded ZFddZGd2ddZHd3d d!ZIee)ejjJdd"d#d$ZKee)ZLeLeejJed%d&d'ZMeLeLd(d)d*ZNe)d+d,d-ZOee)d+d.d/ZPdS)4) namedtuple)OptionalAnyN) FakeQuantizeFakeQuantizeBasedefault_fake_quantdefault_dynamic_fake_quant%default_per_channel_weight_fake_quantdefault_weight_fake_quantdefault_fused_act_fake_quantdefault_fused_wt_fake_quantFusedMovingAvgObsFakeQuantize'default_fused_per_channel_wt_fake_quantdefault_embedding_fake_quant!default_embedding_fake_quant_4bit(fused_wt_fake_quant_range_neg_127_to_1274fused_per_channel_wt_fake_quant_range_neg_127_to_127)HistogramObserverMovingAverageMinMaxObserver NoopObserverPlaceholderObserverReuseInputObserverdefault_debug_observerdefault_dynamic_quant_observerdefault_float_qparams_observer#default_float_qparams_observer_4bitdefault_observer#default_per_channel_weight_observerdefault_placeholder_observerdefault_weight_observer$weight_observer_range_neg_127_to_1270per_channel_weight_observer_range_neg_127_to_127default_reuse_input_observercs eZdZdZfddZZS)QConfiga Describes how to quantize a layer or a part of the network by providing settings (observer classes) for activations and weights respectively. Note that QConfig needs to contain observer **classes** (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization preparation function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with `with_args` method (that behaves like functools.partial):: my_qconfig = QConfig( activation=MinMaxObserver.with_args(dtype=torch.qint8), weight=default_observer.with_args(dtype=torch.qint8)) cs4t|tjst|tjr tdtt||||S)NzQConfig received observer instance, please pass observer class instead. Use MyObserver.with_args(x=1) to override arguments to constructor if needed) isinstancennModule ValueErrorsuperr$__new__cls activationweight __class__Y/var/www/html/django/DPS/env/lib/python3.9/site-packages/torch/ao/quantization/qconfig.pyr*?szQConfig.__new__)__name__ __module__ __qualname____doc__r* __classcell__r1r1r/r2r$,sr$r-r.cs.eZdZdZejjejjffdd ZZS)QConfigDynamica Describes how to dynamically quantize a layer or a part of the network by providing settings (observer classes) for weights. It's like QConfig, but for dynamic quantization. Note that QConfigDynamic needs to contain observer **classes** (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with `with_args` method (that behaves like functools.partial):: my_qconfig = QConfigDynamic(weight=default_observer.with_args(dtype=torch.qint8)) cs2t|tjrtdtdtt||||S)NzQConfigDynamic received observer instance, please pass observer class instead. Use MyObserver.with_args(x=1) to override arguments to constructor if neededzTQConfigDynamic is going to be deprecated in PyTorch 1.12, please use QConfig instead) r%r&r'r(warningswarnr)r8r*r+r/r1r2r*Ws  zQConfigDynamic.__new__) r3r4r5r6torchr&Identityr*r7r1r1r/r2r8Gsr8r-r.)r.r-)dtype compute_dtype)r>fbgemmcCs|dkrh|dkr&ttjddtd}q||dkrDttjddtd}q||dkrbttjddtd}q|t}ntd t|d |S) z Returns the default PTQ qconfig for the specified backend. Args: * `backend`: a string representing the target backend. Currently supports `fbgemm`, `qnnpack` and `onednn`. Return: qconfig rr@T) reduce_ranger=qnnpackFonednnVersion number: zB in get_default_qconfig is not supported. Version number must be 0)r$r with_argsrr default_qconfigAssertionErrorstrbackendversionqconfigr1r1r2get_default_qconfigs$     rMFg0?)r>rAepscCs|dkrx|dkr,ttjtddddtd}nJ|dkrPttjtddddtd}n&|d krrttjtddd td}nt}n|d kr|dkrttjtddddtd}nJ|dkrttjtddddt d}n&|d krttjtddd td}nt }nt d t |d |S)a5 Returns the default QAT qconfig for the specified backend. Args: * `backend`: a string representing the target backend. Currently supports `fbgemm`, `qnnpack` and `onednn`. * `version`: version, for backwards compatibility. Can be `None` or `1`. Return: qconfig rr@T)observer quant_min quant_maxrAr=rBFrC)rPrQrRrrDzJin get_default_qat_qconfig is not supported. Version number must be 0 or 1) r$rrErr r default_qat_qconfigr rr default_qat_qconfig_v2rGrHrIr1r1r2get_default_qat_qconfigsl  rUi)rPrQrRr>rArNcCs|dtftjj|ftjj|ftjj|ftjj|ftjj|ftjj|ftjj |ftjj j |ftjj j |ftjj j |ftjj j|ftjj j|ftjj j|ftjj j|ftjj|ftjj j|ftj|ftjj|ftjj|ftjj|fgdS)Nreshape)Z object_type)default_reuse_input_qconfigr;r&Conv1dConv2dConv3dConvTranspose1dConvTranspose2dConvTranspose3dLinear functionalconv1dconv2dconv3dconv_transpose1dconv_transpose2dconv_transpose3dlinearReLUrelu BatchNorm1d BatchNorm2d BatchNorm3d)rLqconfig_transposer1r1r2 _get_default_qconfig_dict_helperRs0                   rocCs.t||}|}|dkr$t|jtd}t||SNr@r=)rMr$r-r rorJrKrLrnr1r1r2get_default_qconfig_dictks  rrcCs.t||}|}|dkr$t|jtd}t||Srp)rUr$r-r rorqr1r1r2get_default_qat_qconfig_dictvs  rs)rLmodreturncCs|dur dSt|tjjp4t|tjjp4t|tjj}|r||jdurHdS|}t|tjjj pnt|tjjj }|r|JddS)z0 Verifies that this `qconfig` is valid. NzGPer channel weight observer is not supported yet for ConvTranspose{n}d.) r%r;r&r]r^r_r.ao quantizationPerChannelMinMaxObserver%MovingAveragePerChannelMinMaxObserver)rLrtZis_conv_transpose_modZexample_observeris_per_channelr1r1r2assert_valid_qconfigs"   r{)rLmodulerucsTdus|dus|jdkr|Sfddfdd}||j}||j}t||S)aPThis is a helper function for use in quantization prepare that updates a qconfig so that the constructors stored in the qconfig will create observers on the same device that 'module' is on. This is intended to be used when the qconfigs are propagated to each module in order to avoid potential device alignment issues. Args: qconfig: QConfig with obs constructors stored in activation and weight module: module which the qconfig is related to Return: qconfig: configured so that obs constructors set to construct on the same device as module Nr=csfttjjsJddDddDB}t|dkrNtt|nd}|dur^dSd|iS)NcSsh|] }|jqSr1device.0pr1r1r2 zcadd_module_to_qconfig_obs_ctr..get_factory_kwargs_based_on_module_device..cSsh|] }|jqSr1r}rr1r1r2rrrr~) r%r;r&r' parametersbufferslennextiter)devicesr~)r|r1r2)get_factory_kwargs_based_on_module_devices zPadd_module_to_qconfig_obs_ctr..get_factory_kwargs_based_on_module_devicecsPz |jdd}||jdWSty6|YStyJ|YS0dS)N)factory_kwargs)rEwith_callable_argsAttributeError TypeError)Zoriginal_constructorcheck)rr1r21configure_constructor_to_put_obs_on_module_devices   zXadd_module_to_qconfig_obs_ctr..configure_constructor_to_put_obs_on_module_device)_fieldsr-r.r$)rLr|rr-r.r1)rr|r2add_module_to_qconfig_obs_ctrs   r)q1q2cCsdd}|dus|dur ||kS|dur0|dus4Jzrt|jtjjjjr^||jj|jj}n |j|jk}t|jtjjjjr||jj|jj}n |j|jk}|o|WSt y||kYS0dS)zD Returns `True` if `q1` equals `q2`, and `False` otherwise. cSs,|j|jk}|o|j|jk}|o*|j|jkSN)funcargskeywords)p1p2samer1r1r2partial_equalss z&qconfig_equals..partial_equalsN) r%r-r;rvrwrP_PartialWrapperrr.r)rrrZactivation_sameZ weight_samer1r1r2qconfig_equalss    rrLcCs:dd}|}t|tr.t|dr.||jS||SdS)z Return whether the observer for activations defined in the given QConfig is memoryless. This means a MovingAverage observer with averaging constant equal to 1. cSst|do|jdkS)Naveraging_constantr)hasattrr)rPr1r1r2_is_memorylesssz0activation_is_memoryless.._is_memorylessactivation_post_processN)r-r%rrr)rLrZactr1r1r2activation_is_memorylesss  rcCs$|duo"t|to"t|tSr)r%r-rr.rrr1r1r2is_reuse_input_qconfigs   r)r@r)r@r)r@r)r@r)Q collectionsrtypingrrr;torch.nnr&Z#torch.ao.quantization.fake_quantizerrrrr r r r r rrrrrrPrrrrrrrrrrrrr r!r"r#r9r$r8rFZdefault_debug_qconfigZdefault_per_channel_qconfigdefault_dynamic_qconfigrEfloat32float16float16_dynamic_qconfigZfloat16_static_qconfigZper_channel_dynamic_qconfig!float_qparams_weight_only_qconfigZ&float_qparams_weight_only_qconfig_4bitrSZdefault_dynamic_qat_qconfigr<Zdefault_weight_only_qconfigZdefault_activation_only_qconfigrTrYrMqint8Z!default_symmetric_qnnpack_qconfigZ-default_per_channel_symmetric_qnnpack_qconfigZdefault_embedding_qat_qconfigZ"default_embedding_qat_qconfig_4bitrUZ%default_symmetric_qnnpack_qat_qconfigZ1default_per_channel_symmetric_qnnpack_qat_qconfigrorrrsr'r{Z QConfigAnyrrrrr1r1r1r2s  @H    7   ?     *