a ==ic@slddlmZmZmZddlmZGdddeZGdddeZGdddeZ Gd d d eZ d d Z d S))DataType BlobReferenceScopedBlobReference) ParameterInfoc@s*eZdZdZd ddZddZddZdS) Initializerz This class abstracts out parameter creation. One can come up with a new Initializer in order to implement more complex parameter initialization logic NcKs||_||_dSN) operator_nameoperator_kwargsselfrkwargsr t/home/droni/.local/share/virtualenvs/DPS-5Je3_V2c/lib/python3.9/site-packages/caffe2/python/modeling/initializers.py__init__szInitializer.__init__cCs"|jdurtd||_||_dSNz(Operator name overwrites are not allowedr Exceptionr r r r rupdates zInitializer.updatecCs.||jg|fd|i|j}td||dS)Nshapeparam_idparamr) __getattr__rr rr param_nameinit_netrrr r r create_params zInitializer.create_param)N)__name__ __module__ __qualname____doc__rrrr r r rr s rc@seZdZdZddZdS)ExternalInitializerz This class is used in cases when the parameter should not be initialized by the initializer, but rather provided in the workspace when param_init_net is executed. Current version is not doing any real sanity checks to the parameter. cCsFt|trtt||}nt|tr0t||}ntdtd||dS)NzUnsupported type for param_namer) isinstancerstrr TypeErrorrrr r rr-s   z ExternalInitializer.create_paramN)rrrr rr r r rr!$sr!c@s eZdZdZddZddZdS)PseudoFP16Initializera Used in cases when the parameter should be used at half (16-bit) precision for compute purposes (i.e. on the forward and backward pass) but needs to be stored and optimized at single (32-bit) precision so tiny gradients with small learning rates don't underflow FP16 precision. A 32-bit copy of the 16-bit blob is stored in the ParameterInfo. This is helpful for mixed-precision training, see https://arxiv.org/abs/1710.03740 for details. cCs"|jdurtd||_||_dSrrr r r rrFs zPseudoFP16Initializer.updatecCsF||jg|dfd|i|j}|||}td||tj|idS)NZ_fp32rrrrZ blob_copy)rrr FloatToHalfrrFLOAT)r rrr param_fp32rr r rrLs  z"PseudoFP16Initializer.create_paramNrrrr rrr r r rr%<s r%c@s eZdZdZddZddZdS)ReversePseudoFP16Initializerz Like PseudoFP16Initializer above, except the primary blob is taken to be the 32-bit precision parameter, and the 16-bit version of the blob is stored in blob_copy instead. cCs"|jdurtd||_||_dSrrr r r rrcs z#ReversePseudoFP16Initializer.updatecCsF||jg|fd|i|j}|||d}td||tj|idS)NrZ_fp16r&)rrr r'rrZFLOAT16)r rrrr)Z param_fp16r r rris  z)ReversePseudoFP16Initializer.create_paramNr*r r r rr+]sr+csTfdd}|dur4||dfi|d}nt|dfi|d}|S)aV A helper function to convert from operator_name_and_kwargs to new object of type initializer_class. This function serves two purposes: 1. Support for custom initialization operators being passed in 2. Allow user to specify a custom Initializer without overwriting default operators used for initialization If initializer_class is None, creates a default initializer using the Initializer class and operator_name_and_kwargs provided If operator_name_and_kwargs is None, uses default_operator_name_and_kwargs returns an instantiated Initializer object cspSrr r  default_operator_name_and_kwargsoperator_name_and_kwargsr rget_initializer_argssz0update_initializer..get_initializer_argsNr)r)Zinitializer_classr.r-r/initr r,rupdate_initializerysr2N) Zcaffe2.python.corerrrZ%caffe2.python.modeling.parameter_inforobjectrr!r%r+r2r r r rs  !