a ==ic@s<ddlZddlZddlZddlZddlZddlmZmZddl m Z ddl m Z m Z mZddlmZeedZeejddZd d Zd d Zd dZddZddZddZddZGdddeZddZ ddZ!ddZ"dd Z#e$d!d"d#Z%e$d$d%d&Z&e$d'd(d)Z'd*d+Z(e$d,d-d.Z)e$d/d0d1Z*e$d2d3d4Z+e$d5d6d7Z,e$d8d9d:Z-e$d;dd?d@Z/e$dAdBdCZ0e$dDdEdFZ1e$dGdHdIZ2e$dJdKdLZ3e$dMdNdOZ4e$dPdQdRZ5e$dSdTdUZ6e$dVdWdXZ7e$dYdZd[Z8e$d\d]d^Z9e$d_d`daZ:e$dbdcddZ;e$dedfdgZe$dndodpZ?e$dqdrdsZ@e$dtdudvZAeBdwkr8ejCdxdyZDeDjEdzd{d|eDjEd}d~d|eDjEddddeDjEddddeDjEdddddeDjEdddeFgdeDGZHe IZJe IZKeHjLZMeHjNZOeHjPZQeHjRZSeTeM ZUeVeUWeJWdn1s0YeTeOdZUeKXeUWWdn1s0Ye eJeKdeHjYeHjZd\Z[Z\e[j]dj^dZ_e[j]dj`dZae[j_be_ge[j_bdde\jcDe[jabeage!e\e_ZPeTeSdZUeUde[eWdn1s0YeTeSddZUeUdefe[Wdn1s0YeTeQdZUeUdePeWdn1s.0YdS)N) caffe2_pb2caffe2_legacy_pb2) caffe_pb2)coreutils workspace) text_formatZcaffe_translatorcs|dr|j|jkrdS|dr4|j|jkr4dS|drN|j|jkrNdStt|jt|jrt fdd|jDrdSt|j rt fdd|j DrdSdS) z@A function that reproduces Caffe's StateMeetsRule functionality.phaseF min_level max_levelcsg|] }|vqSr .0sZ curr_stagesr o/home/droni/.local/share/virtualenvs/DPS-5Je3_V2c/lib/python3.9/site-packages/caffe2/python/caffe_translator.py z#_StateMeetsRule..csg|] }|vqSr r r rr rr!rT) HasFieldr levelr r setlistZstagelenanyZ not_stage)stateruler rr_StateMeetsRules""rcsVt|jdk}|tfdd|jD M}t|jrR|tfdd|jDO}|S)z@A function that reproduces Caffe's inclusion and exclusion rule.rcsg|]}t|qSr rrr net_stater rr+rz"_ShouldInclude..csg|]}t|qSr rrrr rr.r)rincluderexclude)r layerretr rr_ShouldInclude's  r%c Csi}tj}|jD]}||jt|q|j dj d}|||t t |j D]B}|j |} | | ||vr`| jd} || } | j||<q`|SNrrC Workspaceprotos create_blobnamefeedrCaffe2TensorToNumpyArrayopinputranger _run_operatorSerializeToStringoutput fetch_blobshape) net net_params dummy_inputlegacy_pad_opsdim_mapwsparamexternal_inputiop_defr4Z blob_legacyr r r_GetLegacyDims2s        rAcsHi}gd}dv}|r2|D]}dj||<qnfdd|D}|S)N)pad_lpad_tpad_rpad_bpadcsi|]}||jqSr r?)rxarg_mapr r Mrz%_GetLegacyPadArgs..rG)r@rJpadskeysis_padkr rIr_GetLegacyPadArgsEsrPcCs(|\}}}}|\} } } } || ks$J|| ks0Jd|v} || ksJ|| kr$|| dkrh|dd7<n|| kr|td|| || dkr|dd7<n|| krtd|| | r|j|dg}|D]&}t}||_|||_||q|j |n|D]}||||_qdS)NrFrEz!Unexpected dimensions for height:rDz Unexpected dimensions for width:) ExceptionargremoverMrArgumentr,r?appendextend)r@rJrLdim1dim2Zn1c1Zh1Zw1Zn2c2Zh2Zw2rNargsr,rSr r r _AdjustDimsQs2            r]cCsg}tt|jD]@}|j|}td|jr|jD]}|jdkr4||qq4q|r|\}}} } t j ||| |  t j } t||| |} tj} |jdjd}| || |jD]}| |jt|qtt|jD]}|j|}||vri}|jD]}|||j<q t||}tt|jD]*}|j|}|jdkr6|j|=qbq6|jd}|d}||jd<| || |}||jd<| |}|j}t|||||| |q|S)Nz0^(Conv|ConvTranspose|MaxPool|AveragePool)(\dD)?$ legacy_padrZ _nonlegacy)r1rr/rematchtyperSr,rVnprandomrandnastypefloat32rArr(r)r0r+Z feed_blobr*rr.rPr4r2r3r5r6r])r7r8 input_dimsr:r?r@rSnchwr9r;r<r>r=rJrLjr4Znonlegacy_outputZblob_nonlegacyrXrYr r r_RemoveLegacyPadnsX                   rmc Csi}tj}|jD]}||jt|q|j dj d}|||t t |j D]<}|j |}| ||jD]} || } | j|| <qq`|Sr&r') r7r8r9r;r<r=r>r?r@r4blobr r r_GetBlobDimMaps      rocCsLg}|jr|j}n6|jr&|jdj}n"|jdjjrH|jdjjdj}|Sr&)Z input_dimZ input_shapedimr#Z input_paramr6) caffe_netrgr r r _GetInputDimssrrc@s6eZdZiZeddZeddZed ddZdS) TranslatorRegistrycsfdd}|S)z.A decorator for registering gradient mappings.cs|j<|SN) registry_)funcclsop_namer rWrappers z,TranslatorRegistry.Register..Wrapperr )rxryrzr rwrRegisterszTranslatorRegistry.RegistercKsnz$|j|j|||fi|\}}Wn"tyFtdt|Yn0|durTg}t|turf|g}||fS)Nz+No translator registered for layer: %s yet.)ruraKeyErrorstrr)rxr#pretrained_blobsis_testkwargs caffe_opsparamsr r rTranslateLayers    z!TranslatorRegistry.TranslateLayerFNc sH|durtn|}t}|j|_t}t|jdkrBtd|sNt |}|j D]ʉt |svt djqTt djfdd|j Dfdd|jD} t| dkrtd n&t| dkrd d| djD} ng} |j| ||||d \} } |j| |j| qT|r@|s4Jd t|||}||fS) NrzuI think something is wrong. This translation script only accepts new style layers that are stored in the layer field.z(Current net state does not need layer {}zTranslate layer {}csg|]}|jjkr|qSr r,rlr#r rrs z5TranslatorRegistry.TranslateModel..csg|]}|jjkr|qSr rrrr rrs rQz0huh? more than one pretrained layer of one name?cSsg|]}t|qSr )rZCaffeBlobToNumpyArray)rrnr r rrs)r7r8rgz.Please specify input_dims to remove legacy_pad)rZNetStaterNetDefr,Z TensorProtosrZlayers ValueErrorrrr#r%loginfoformatZblobsrr/rWr*rm) rxrqZpretrained_netrr remove_legacy_padrgr7r8Zpretrained_layersr~ operatorsrr rrTranslateModelsX         z!TranslatorRegistry.TranslateModel)FNFN)__name__ __module__ __qualname__ru classmethodr{rrr r r rrss  rscOstj|i|Srt)rsr)r\rr r rrsrc Cst}|jD]Z}t|jdkr(tdtjdg|jgt dt |j t d|jgd}|j |gq|j tjdg|gdgd g|S) aaTakes the net_params returned from TranslateModel, and wrap it as an init net that contain GivenTensorFill. This is a very simple feature that only works with float tensors, and is only intended to be used in an environment where you want a single initialization file - for more complex cases, use a db to store the parameters. rz.Only float tensors are supported in this util.ZGivenTensorFillr6values)rSZ ConstantFillrQ)r6)rrr*rZ float_data RuntimeErrorrCreateOperatorr,r MakeArgumentrdimsr/rW)r8Z input_nameinit_netZtensorr/r r rConvertTensorProtosToInitNets     rcCs.t}||_|j|j|j|j|S)zBA simple translate interface that maps the layer input and output.)r OperatorDefrar0rWbottomr4top)r#Z caffe2_typeZ caffe2_opr r r BaseTranslate6s rcCs|jt||gdS)z*Makes an argument based on the value type.N)rSrWrr)r/keyvaluer r r AddArgument?srZInputcKsggfSrtr r#r~rrr r rTranslateInputHsrZ VideoDatacKsggfSrtr rr r rTranslateVideoDataMsrZDatacKsggfSrtr rr r r TranslateDataRsrcCsz~t|jdks,t|jdks,t|jdkr4tdt|jrH|jdnd}t|jr`|jdnd}t|jrx|jdnd}Wn$ty|j}|j}|j}Yn0|ds|drt|d|jt|d|j n t|d||ds|drP|j |j krt|d |j n8t|d |j t|d |j t|d |j t|d |j n t|d ||dst|drt|d|j t|d|j n t|d|dS)NrQzRTranslator currently does not support non-conventional pad/kernel/stride settings.rstride_hstride_wstridepad_hpad_wrFrCrErBrDkernel_hkernel_wkernel)rr kernel_sizerFNotImplementedError TypeErrorrrrrrrrr)r=caffe_oprrFrr r r_TranslateStridePadKernelHelperYs<     rZ Convolution3Dc Ks|j}t|d}|jd}|j|dt|d|j|j|jgt|d|j|j |j gd}d}t |drr|j }t |dr|j }t|d|||gd t |d|dg} t|d kr|j|d | t |d |d || fS) NConvr_wkernelsstrides temporal_padrFrL_brQ)Zconvolution3d_paramrr4r0rVr kernel_depthrtemporal_striderhasattrrrFrNumpyArrayToCaffe2Tensorrflatten) r#r~rrr=rr4r spatial_padrr r rTranslateConvNds>     rZ ConvolutioncKs|j}t|d}|jd}|j|dt||t|d|dg}t|dkr|j|d|t|d |d|j dkrt |d|j t|j dkrt|j dkrt |d|j dn2t|j dkrt |d |j dt |d |j d||fS) NrrrrrrQgroupdilationZ dilation_hZ dilation_w) convolution_paramrr4r0rVrrrrrrrr)r#r~rrr=rr4rr r r TranslateConvs.     rZ Deconvolutionc Ks|j}|jdkrtdt|d}|jd}t|||j|dgt|ddt |d|d}|j rt |d |d}|j|dg|||gfS||gfSdS) NrQz:Translator currently does not support group deconvolution.Z ConvTransposerrorderNCHWr) rrrrr4rr0rWrrrZ bias_termr r#r~rrr=rr4weightbiasr r rTranslateDeconvs$      rZCropc sf|d|d|d}}}|\}}} } tj||| | tj} t||| } |j} | j| j}t |d}|j d}| |gg}}t | |}t dksJdt |D]}| d| dqfd d t ||D}|||dgt |t}|j |j dg|j|j|j|j|j|_t|d |t|d ||gfS) Nr7r8rgSlicerQz@Caffe Translator for Crop only works for offset of 1 for nowrcs g|]}td|qS)r)int)rr?Z input_1_dimoffsetsr rrrz!TranslateCrop..startsends)rbrcrdrerfroZ crop_paramaxisoffsetrr0rr1rVrWrrr4rSrar)r#r~rrr7r8rgrhrirjrkr9r;r=rrZinput_1rrr_Z end_offsetr/r rr TranslateCrops4            rZReLUcKst|dgfS)NZRelurrr r r TranslateRelusrZPoolingcKs|j}|jtjjkr t|d}n|jtjjkr8t|d}t||t|ddz |j }Wnt ynd}Yn0|st|dt j |j rt|dd|gfS) NMaxPool AveragePoolrrFr^global_poolingrQ)Z pooling_parampoolrZPoolingParameterMAXrAVErrZ torch_poolingAttributeErrorrZCAFFE_LEGACY_POOLINGr)r#r~rrr=rZis_torch_poolingr r r TranslatePools$        rZ Pooling3DcKs|j}|jtjjkr t|d}n|jtjjkr8t|d}t|ddt|d|j|j |j gt|d|j |j |j gd}d}t |dr|j }t |d r|j}t|d |||gd |gfS) NrrrrrrrrrFrLr)Zpooling3d_paramrrZPooling3DParameterrrrrrrrrrrrF)r#r~rrr=rrrr r rTranslatePool3Ds0     rLRNcKst|d}|jd|jddg|j}|jtjjkrBtdt |dt |j t |dt |j t |dt |jt |d t |jt |d d |gfS) Nrrr_scalez8Does not support norm region other than across channels.sizealphabetarrr)rr4rWZ lrn_paramZ norm_regionrZ LRNParameterZACROSS_CHANNELSrrrZ local_sizefloatrrrOr#r~rrrr=r r r TranslateLRN7s  rZ InnerProductc Ks|j}z|jdks|jr tdWnty4Yn0t|d}|jd}|j|d|dg|dj dvrxtd|dj d krt |dj dd ddgkrtd t |dj t |dd |dj d |d}t |d|d}|||gfS) NrQzWe don't have testing case for non-default axis and transpose cases yet so we are disabling it for now. If you have a model with this, please do send us your model for us to update this support, and you are more than welcome to send a PR for this.FCrrr)rzUnexpected weight ndim.rrz_If pretrained blob has 4 dims (old-style Caffe), the first two should be of value 1, but I got r)Zinner_product_paramrZ transposerrrr4r0rWndimrr6r}rrZreshaperrr r rTranslateInnerProductGs:    rDropoutcKsRt|d}|jd|jddg|j}t|d|j|rJt|dd|gfS)NrrrZ_maskratiorrQ)rr4rWZ dropout_paramrZ dropout_ratiorr r rTranslateDropoutks  rSoftmaxcKst|d}|gfS)Nrrr#r~rrrr r rTranslateSoftmaxvs rZSoftmaxWithLosscKsptd|jdg|jdd}td|jd|jdg|jdd}td|jd|jd}|||ggfS)NrrZ_translator_autogen_softmaxZLabelCrossEntropyrQZ_translator_autogen_xentZ AveragedLoss)rrrr4r)r#r~rrZ softmax_opZxent_opZloss_opr r rTranslateSoftmaxWithLoss|s   rAccuracycKs.t|d}|jjdkr&t|d|jj|gfS)NrrQtop_k)rZaccuracy_paramrrrr r rTranslateAccuracys  rConcatcKs:t|d}|jd|jddgt|dd|gfS)Nrrr_dimsrr)rr4rWrrr r rTranslateConcats  rZTanHcKst|d}|gfS)NZTanhrrr r r TranslateTanHs r InstanceNormcKstt|d}|jd}t|d|d}t|d|d}|j|d|dgt|dd|||gfS)NrrrrQrrr)rr4rrrr0rWr)r#r~rrrr4rrr r rTranslateInstanceNorms   rZ BatchNormc Kst|d}|jd}|j}t|d|t|d|jt|dd|j|d|d|d |d g|s|j|d |d |d |d g|djd}|d ddkrt d|d d|d|d }t d|d d|d|d } nt dt |dkr| | t| t|ksTJ| |fSt|jd krvtdntddS)NZMulr broadcastTrQz$This path has not been verified yet.rZscale_wrAddZscale_b _internalz.Unexpected number of pretrained blobs in ScalezUnexpected number of inputs.)r scale_paramrrrr0Znum_axesrr4rVrrrcopydeepcopyra) r#r~rrZmul_opr r4Z mul_op_paramweightsZadd_opZ add_op_paramZ internal_blobrr r rTranslateScalesN                r ReshapecKsBt|d}|jd|jdd|j}t|d|jj|gfS)Nrrrrr6)rr4rVr0 reshape_paramrr6rp)r#r~rrrrr r rTranslateReshape$s  rFlattencKs\|j}|jdkrtd|jdkr.t|d}n&|jdkrDt|d}ntd|j|gfS)Nrz)flatten_param.end_axis not supported yet.rZ FlattenToVecrQrz,Not supported yet for flatten_param.axis {}.)Z flatten_paramZend_axisrrrrrr r rTranslateFlatten-s      rSigmoidcKst|d}|gfS)Nrrrr r rTranslateSigmoid?s rZ ROIPoolingcKst|d}t|dd|r(t|d|n|j|jdd|j}|dr\t|d|j|drtt|d|j|d rt|d |j|gfS) NZRoIPoolrrrrZ _argmaxespooled_hpooled_w spatial_scale) rrr4rVZroi_pooling_paramrrrrrr r rTranslateROIPoolingEs     rZPReLUcKsDt|d}|jd}|j|dgt|d|d}||gfS)NZPRelurZ_Slope)rr4r0rWrr)r#r~rrrr4Zsloper r rTranslatePRelu[s   rZ ReductioncKsp|j}|jtjjkr t|d}n"|jtjjkr:t|d}ntd|jdkrTtd|j }t |d||gfS)NZ ReduceBackSumZReduceBackMeanzNot yet supportedrnum_reduce_dim) Zreduction_paramrrZReductionParameterZSUMrZMEANrrr)r#r~rrr=rrr r rTranslateReductiones    r__main__z>Utilitity to convert pretrained caffe models to Caffe2 models.) description prototextzCaffe prototext.)help caffemodelzCaffe trained model.z --init_netzCaffe2 initialization net.z init_net.pb)rdefaultz --predict_netzCaffe2 prediction net.zpredict_net.pbz--remove_legacy_padzSRemove legacy pad (Only works for nets with one input blob) store_trueF)ractionr!z --input_dimszDimension of input blob+)rnargsrar!rbT)rrrgrcCsg|] }|jqSr r)rr=r r rrrrwbtxtrk)gargparser loggingr_numpyrbZ caffe2.protorrZ caffe.protorZ caffe2.pythonrrrZgoogle.protobufr basicConfig getLoggerrsetLevelINFOrr%rArPr]rmrorrobjectrsrrrrr{rrrrrrrrrrrrrrrrrrrrrrr rrrrrrrArgumentParserparser add_argumentr parse_argsr\Z NetParameterZcaffenetZcaffenet_pretrainedrZ input_protor Zinput_caffemodelrZoutput_init_netZ predict_netZoutput_predict_netopenfZMergereadZParseFromStringrrgr7Zpretrained_paramsr/r0r>r4Zexternal_outputrWr*writer3r}r r r rs      7 W     * "        #        5  3         0 .   ..