a /Sic0< @sdZddlZddlZddlZddlZddlmZddlZddl Z ddl m Z ddl m Z ddl mZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddl m!Z!ddl"m#Z#ddl"m$Z$ddl"m%Z%ddl"m&Z&ddl"m'Z'ddl"m(Z)ddl"m*Z*ddl"m+Z+ddl"m,Z,ddl"m-Z-ddl"m.Z.dd l/m0Z0dd!l/m1Z1dd"l/m2Z2dd#l/m3Z3dd$l/m4Z4dd%l/m5Z5dd&l6m7Z8dd'l9m:Z:dd(l;mm?Z?dd*l@mAZAdd+l@mBZBdd,l@mCZCdd-l@mDZDdd.l@mEZEdd/l@mFZFdd0l@mGZGdd1l@mHZHdd2l@mIZIdd3lJmKZKeDLd4eMd5ZNeDLd6eMd7ZOd8ZPd9ZQd:ZRd;ZSeTdd?ZVd@dAZWdBdCZXddEdFZYdGdHZZGdIdJdJe[Z\e]Z^dKdLZ_dMdNZ`dOdPZadQZbdRZcdSZddTdUZedVdWZfdXdYZgGdZd[d[e[ZhGd\d]d]e[Zid^d_ZjGd`dadae[ZkeldbdcZmGdddedee[ZnGdfdgdgenZoGdhdidienZpGdjdkdke[Zqe[ZrGdldmdme?jse=jtZseAudne+jveAudoe+jweAudpe*jxGdqdrdre[ZydsdtZzdudvZ{dwdxZ|ddzd{Z}eKd|gd}dd~dZ~Gddde[ZddZGddde[ZGddde[ZddZddZdS)z2Defun decorator for defining graph-mode functions.N)List)map)attr_value_pb2) function_pb2)function_cache) pywrap_tfe)pywrap_tf_session)backprop) backprop_util)context)execute)forwardprop_util)function_context)function_saved_model_utils) function_spec) monitoring)tape)graph_placeholder) c_api_util)composite_tensor)dtypes)error_interpolation)errors) func_graph)indexed_slices)ops) tensor_shape) tensor_spec) type_spec) array_ops)default_gradient)functional_ops)gradients_util)handle_data_util)resource_variable_ops) tf_logging)trace)base)core) _pywrap_utils)compat)function_utils) lazy_loader)memory)nest)object_identity) tf_decorator) tf_inspect) tf_exportag_ctxz'tensorflow.python.autograph.core.ag_ctx np_arraysz)tensorflow.python.ops.numpy_ops.np_arraysforward_function_nameZbackward_function_name _implementsZshared_rendezvousz6/tensorflow/core/tf_function/graph_building_time_usecsz+Time for tf.function to build a graph (us).cCsFt|tjrt|j|jSt|tjr,|St|t j r>|j S|SdS)zDReturns a TypeSpec for `x`, or `x` if `x` doesn't have a TensorSpec.N) isinstancerTensorr TensorSpecshapedtyperTypeSpecrCompositeTensor _type_specxrA\/var/www/html/django/DPS/env/lib/python3.9/site-packages/tensorflow/python/eager/function.py_type_spec_foras   rCcCst|tjr|||kSdS)zHReturns true if `b` is a subset of type `a` (or if a is not a TypeSpec.)T)r7rr<most_specific_compatible_type)abrArArB_is_type_subsetps rGcCs |dur|krdur8nntd|d|d|durDdSt|tjsdtdt|dt|tjstdt|d|j|jks|jdurtdSg}t|j|jD]H\}}||kst |dust |dur| dq| t |qt|S)z>Find a `TensorShape` that is compatible with both `x` and `y`.NzXCannot find a common shape when LHS shape is None but RHS shape is not (or vice versa): z vs. .z$`x` must be a TensorShape, got type z$`y` must be a TensorShape, got type ) RuntimeErrorr7r TensorShape TypeErrortyperankzipdimsdimension_valueappend)r@yrOdim_xdim_yrArArB common_shapews4"      rUFc Csvztj||ddWnttfy,YdS0|rrtj|dd}tj|dd}tddt||DrjdS||kSdS)zECheck two structures for equality, optionally of types and of values.Texpand_compositesFcss"|]\}}t|t|kVqdSNrL).0f1f2rArArB z$is_same_structure..)r.assert_same_structure ValueErrorrKflattenanyrN)Z structure1Z structure2 check_valuesZ flattened1Z flattened2rArArBis_same_structuresrdcCsi}|D]\}}t|tjr*|||<q t|trFtj|d||<q t|trbtj|d||<q t|tr~tj|d||<q t|ttt j frtjt |d||<q t d|dt|dq |S)aConvert the keyword arguments into function_def attributes. Currently only support primitive types: bool, int, float and string. Args: attributes: the dictionary of attributes. Returns: A dict of attributes where the key is the name of attribute and the value is the AttrValue proto. Raises: ValueError: If the kwargs contains unallowlisted name or unsupported value types. )rF)i)f)sz Attribute z5 must be bool, int, float, string, or AttrValue. Got rH)itemsr7r AttrValueboolintfloatstrbytessix text_typer*as_bytesr`rL) attributesattrskeyvaluerArArB_parse_func_attrss        rvc@s.eZdZdZdgZddZddZddZd S) _InterpolateFunctionErrorzFContext Manager that interpolates the exception from 'top_level_func'._funccCs ||_dSrX)rx)selfZtop_level_funcrArArB__init__sz"_InterpolateFunctionError.__init__cCsdSrXrAryrArArB __enter__sz#_InterpolateFunctionError.__enter__c Cs|rt|tjsdSt|j}t|\}}}d}|D]J}|jt |j jkrZ|j j }q8|r8| |j} | dur8t| t r8| j }q8|rt|||_dS)NF)r7rOpErrorr*as_textmessager parse_messagenameas_strrxgraph _get_function_EagerDefinedFunction interpolate_message) rytypexctbr_ func_tagsgZfunc_tag next_funcrArArB__exit__s"    z"_InterpolateFunctionError.__exit__N)__name__ __module__ __qualname____doc__ __slots__rzr|rrArArArBrws rwcCst|dS)aAdd a callback function for Function creation. The callback function has the signature: `def function_callback(function, name, graph, inputs, outputs):` where: - `function`: _EagerDefinedFunction being created before finalizing the graph. Do not modify the function directly but instead modify the graph. - `name`: name of the function. - `graph`: Graph of the function. - `inputs`: `tuple` of tensors used as inputs to the function. - `outputs`: `tuple` of tensors used as outputs from the function. The callback is at the top of the `_EagerDefinedFunction` construction, giving callback an opportunity to make the last edits to the graph. Do not make changes to `graph, inputs`, and `outputs` manually, but, instead, set the `graph` as the default then define ops. Repeated registration of the same callback function is idempotent. After a callback is added, it can be removed with the `remove_function_callback()` method. Args: function_callback: The callback to add. N)_function_callbacksaddfunction_callbackrArArBadd_function_callbacksrcCst|dS)zRemove an already-added function callback. See the doc string of `add_function_callback()` for more information. Args: function_callback: The callback to remove. N)rremoverrArArBremove_function_callbacksrcCs tdS)z9Clear all function callbacks, if any have been regisered.N)rclearrArArArBclear_function_callbackssrZ __forward_Z __backward_Z __inference_cCsdt|tfS)z.The name of a generated forward defun named n.%s%s_%s)_FORWARD_PREFIXruidnrArArB _forward_namesrcCsdt|tfS)z/The name of a generated backward defun named n.r)_BACKWARD_PREFIXrrrrArArB_backward_namesrcCsdt|tfS)z4The name of a forward-but-no-gradient defun named n.r)_INFERENCE_PREFIXrrrrArArB_inference_name"src@s&eZdZdZdgZddZddZdS)_EagerDefinedFunctionDeleterz'Unregister function from eager context.rcCs ||_dSrXr)ryrrArArBrz,sz%_EagerDefinedFunctionDeleter.__init__cCs8zt|jWn"ty"Ynty2Yn0dSrX)r remove_functionrrKAttributeErrorr{rArArB__del__/s   z$_EagerDefinedFunctionDeleter.__del__N)rrrrrrzrrArArArBr'src@sdeZdZdZddZeddZeddZdd Zdd d Z ed dZ eddZ dddZ d S)rarCallable with the interface of `framework.function._DefinedFunction`. `_EagerDefinedFunction` encapsulates a function definition and its properties, and it provides a method for calling the encapsulated function. Some Ops take functions as attributes, which have type `func`; an instance of this class may be provided as the value of these `func` attributes. cs<tD]}||||t|t|qtdd|Dfdd|D}|jdurtfdd|Drfdd|D}tt|t|krg}ng}|jd} t | t |dd d|Dd d|Dd d|D|d d|j Dgdt d  } Wdn1s0Yt| ||_|D]&\}} | } t | t || q4|j} t | j|_t>trtt| t|j|_ d|_!Wdn1s0Yt| j"|_#dd| j"D|_$dd|D|_%|j&|_'t(||_)d|_*d|_+d|_,||_-tdd|D|_.dS)aInitializes an eager defined function. Args: name: str, the name for the created function. graph: Graph, the graph containing the operations in the function inputs: the tensors in the graph to be used as inputs to the function outputs: the tensors in the graph which will be outputs from the function attrs: dict mapping names of attributes to their AttrValue values css|] }|jVqdSrX)oprZargrArArBr]Xr^z1_EagerDefinedFunction.__init__..csg|]}|vr|qSrArArZr) input_opsrArB Yr^z2_EagerDefinedFunction.__init__..Nc3s|]}t|vVqdSrXr tensor_idrZtgraph_output_namesrArBr]]r^cs g|]}tt|qSrA)r*rqrrrrrArBr^sFcSsg|] }|jqSrA_c_oprZorArArBrlr^cSsg|] }|qSrA _as_tf_outputrrArArBrmr^cSsg|] }|qSrArrrArArBrnr^cSsg|] }|jqSrArrrArArBrpr^TcSsg|] }|jqSrArYrrArArBrr^cSsg|] }|jqSrAr:rrArArBrr^css|]}|jr|VqdSrX) _is_statefulrrArArBr]r^)/rtuplesetget_operationsZ _output_namesalllen_c_graphgetrTF_GraphToFunction_wrapperr*rZcontrol_outputsrScopedTFFunction_c_funcrhSerializeToStringTF_FunctionSetAttrValueProto_get_definition signaturerqr_namer init_scoper executing_eagerlyensure_initialized add_functionr_function_deleterZ_registered_on_context output_arg _num_outputs _output_types_output_shapesZcontrol_captures_control_captureslist_func_graph_outputsgrad_func_namepython_grad_func _grad_funcr _stateful_ops)ryrrinputsoutputsrsr operations output_namesc_graphfn attr_value serializedrrA)rrrBrzKsh      $      &  z_EagerDefinedFunction.__init__cCs,z|jWSty$|jj|_Yn0|jSrX) _signaturer definitionrr{rArArBrs  z_EagerDefinedFunction.signaturecCs,z|jWSty$||_Yn0|jSrX) _definitionrrr{rArArBrs  z _EagerDefinedFunction.definitionc CstP}|j}t||Wdn1s60Yt|}Wdn1s^0Yt}| t ||SrX) r tf_bufferrrrTF_FunctionToFunctionDef TF_GetBufferr FunctionDefParseFromStringr*rq)rybuffer_func proto_data function_defrArArBrs  *(z%_EagerDefinedFunction._get_definitionNcCsp|s.tr.t}||jsl||jn>||jsD|||jj D]}||jsP||qPdS)zAdd the function to the current context or a graph, if supplied. Args: g: the graph to add the function to. If not supplied, the function will be added to the current context. N) r r has_functionradd_function_defr _is_function _add_functionr _functionsvalues)ryrctxrfrArArB add_to_graphs     z"_EagerDefinedFunction.add_to_graphcCs|jSrX)rr{rArArBrsz_EagerDefinedFunction.namecCs|jSrX)rr{rArArB stateful_opssz"_EagerDefinedFunction.stateful_opsc Cst|t|jjkr:tdtt|jjdt|d|j}|jdurTt}n|j}|j pbd}| }d|d|f}|rt |X|durt j t |jj|j|||d} n t jt |jj|j||||d } Wdn1s0Y| pd} nt |t|jRt(tj|||j|||d } Wdn1sF0YWdn1sf0YWdn1s0Yt|jD]\} } t| | | q|r| St|jD]\} } | | | q| SdS) aCalls this function with `args` as inputs. `ConcreteFunction` execution respects device annotations only if the function won't be compiled with xla. Args: ctx: a Context object args: a list of arguments to supply this function with. cancellation_manager: a `CancellationManager` object that can be used to cancel function execution. Returns: The outputs of the function call. Raises: ValueError: if the number of arguments is incorrect. FunctionAlreadyGarbageCollectedError: if the function is no longer available to be called because it has been garbage collected. zSignature specifies z arguments, got: rHNr executor_type config_proto) num_outputsrrsr)rrrsrcancellation_manager)argsrfZtoutrconfigr)rr input_argr`rfunction_call_optionsconfig_proto_serializedr+get_disabled_rewriter_configrrrwr rmrrexecute_with_cancellationrcontrol_dependenciesrrstop_recordingr!partitioned_callr enumeraterr#copy_handle_datar set_shape) ryrrrrrrrrsrrefunc_graph_outputr:rArArBcallsd       $   fz_EagerDefinedFunction.call)N)N) rrrrrzpropertyrrrrrrrrArArArBrBsL     rc Cstt|j}t|}|tdtt|i}||t||d}tt |ji}||t |||j |j |}||fS)z@Creates forward and backward functions from the function graphs.N)rs) rrdictpopIMPLEMENTS_ATTRIBUTE_NAMErvFORWARD_FUNCTION_ATTRIBUTE_NAMEupdateConcreteFunction BACKWARD_FUNCTION_ATTRIBUTE_NAMErrr) rs forward_graphbackwards_graphr5Zcommon_attributesZbackward_function_attrbackward_functionZforward_function_attrforward_functionrArArB#_create_forward_backward_with_graph$s(    rc@sTeZdZdZddZdddZddZd d Zd d Zdd dZ ddZ ddZ dS) _DelayedRewriteGradientFunctionszACaches forward/backward functions with a delayed forward rewrite.cCsTi|_||_tt|jj|j|jj|jj||_||_d|_ t |jj|_ ||_ dS)z6Construct an inference function and initialize caches.N) _cached_function_pairs _func_graphrrrrr_inference_function_attrsZ_gradient_namer_num_inference_outputs_func_graph_deleter)ryrrsfunc_graph_deleterrArArBrzAsz)_DelayedRewriteGradientFunctions.__init__NcCsJ|dur|j}|j|}|dur&|S||\}}||f|j|<||fS)z9A possibly-cached pair of forward and backward functions.N)r#rr_construct_forward_backward)ry num_doutputsforward_backwardforwardbackwardrArArBr(Rs z1_DelayedRewriteGradientFunctions.forward_backwardc sddjjd|Dg}D]}|tjt|q"fdd}jt t jj }tj |j |gi||d|j }fdd|D}tjj}|D]$} | |vr|| jj| qtjj|\} } | | fWdS1s0YdS)aCConstructs a pair of forward and backward functions. Args: num_doutputs: The constructed backprop function will take output gradients for the first `num_doutputs` outputs of the forward function. Defaults to the number of outputs for the inference function, but when higher-order gradients are computed this will increase to include side outputs. Returns: A pair of (forward_function, backward_function): forward_function: A re-generated inference function (an _EagerDefinedFunction) to account for new side outputs, if any extra were required when building the backward pass. backward_function: A ConcreteFunction that Takes `num_doutputs` arguments and returns gradients with respect to inputs of the forward function. cSsg|]}t|r|qSrAr IsTrainablerZoutputrArArBrps zP_DelayedRewriteGradientFunctions._construct_forward_backward..NcsFtd(tjjj|jdWdS1s80YdS)Ngrad_ysZ src_graph)rdevicer"_GradientsHelperr r)r0rytrainable_outputsrArB_backprop_functionys zX_DelayedRewriteGradientFunctions._construct_forward_backward.._backprop_function)rZ python_funcrkwargsrrcs(g|] }t|tjs|jjur|qSrAr7r EagerTensorrr rZcr{rArBrs)r rrQrr9r shape_and_dtype as_defaultfunc_graph_module FuncGraphrrfunc_graph_from_py_funcexternal_capturesr/ObjectIdentitySetrrr") ryr'rrr5rZbackwards_graph_capturescaptures_from_forwardexisting_outputscapturerrrAr3rBr&]sF     z<_DelayedRewriteGradientFunctions._construct_forward_backwardc sL|t|\}}|js|jS||j|d|j|d|j | |j t|jd|j t|jdt t|jD] }|j |}t||j|qttdd|jjD|jfdd|jD}g}t||jjD]X\} } t| rt| tjr|t| q| dur.|| q|t| q|||S)zCAdd outputs to the forward call and feed them to the grad function.rfToutNcss|]}t|VqdSrXrrrArArBr]r^zV_DelayedRewriteGradientFunctions._rewrite_forward_and_call_backward..csg|]}t||qSrArrrrZrDcapture_mappingrArBrszW_DelayedRewriteGradientFunctions._rewrite_forward_and_call_backward..)r(rrstructured_outputsrr_set_func_attrr_set_type_list_attrr _add_outputsrrangerr#r rrNr captured_inputsr r,r7r IndexedSlicesrQrconvert_to_tensorr zeros_like _call_flat) ryrZdoutputsrZbackwards_functionrerremapped_capturesZcleaned_doutputsdoutput placeholderrArHrB"_rewrite_forward_and_call_backwards<      zC_DelayedRewriteGradientFunctions._rewrite_forward_and_call_backwardcCs|jS)aReturns gradient function. The gradient rewrites an inference call op to a forward call op, but does not modify a pre-existing forward call op. It then computes the gradient from the output's gradients and the side outputs of the forward op. )rWr{rArArBget_gradient_functionsz6_DelayedRewriteGradientFunctions.get_gradient_functioncCs~|rtd|jS)a!A forward function with only user-specified outputs. The call operation for the returned inference function can be rewritten into a forward function. This only happens if the backward function (from the `backward` method) ends up being used to compute gradients. This approach avoids constructing unnecessary graphs, but it only works if we are calling this function when not executing eagerly. Args: inference_args: A flat list of Tensors, arguments to the inference function. Unused, but taken for compatibility with _TapeGradientFunctions. input_tangents: A flat list of Tensors, jvps associated with `inference_args`. Unused; if required, tape functions must be used instead. Returns: An _EagerDefinedFunction. zVunexpectedly got forwardprop information in a class that does not support forwardprop.)r InternalErrorr!ryinference_argsinput_tangentsrArArBr)s z(_DelayedRewriteGradientFunctions.forwardcsfdd}|fS)zBFetch a backward function for `outputs` from the forward function.csdj}j|g|RS)Nr)rrW)rZcall_oprryrArB_backward_functions zF_DelayedRewriteGradientFunctions._backward.._backward_functionrA)ryrr^rAr]rB _backwardsz*_DelayedRewriteGradientFunctions._backwardcCs,||\}}t|jjj||||dS)a<Record the function call operation. _DelayedRewriteGradientFunctions supports only first-order backprop tape gradients (and then only when graph building). It does not work with higher-order tape gradients or forward autodiff, but does work with higher-order symbolic gradients (tf.gradients). Args: flat_outputs: The result of running `forward`. inference_args: A flat list of Tensors with inference inputs to the operation. input_tangents: A flat list of Tensors with input tangents consumed by the operation. N)r_rrecord_operationr!rrry flat_outputsr[r\rZ to_recordrArArBrecords  z'_DelayedRewriteGradientFunctions.record)N)NN) rrrrrzr(r&rWrXr)r_rcrArArArBr>s =, r_ForwardWrapperrroutput_indicesoutput_tangentsc@sPeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ dS)_TapeGradientFunctionsa6Caches forward and backward functions compatible with eager gradients. In contrast to the delayed-rewrite approach in `_DelayedRewriteGradientFunctions` which only works with delayed execution, the forward function generated by this class has a fixed set of outputs which may be preserved by a tape in order to compute gradients later. This class is abstract; its child classes differ in how many side outputs of the forward function their backward function accepts gradients for, which determines whether higher-order tape gradients are possible. cCst||_d|_||_d|_d|_t|j|_||_||_ d|_ d|_ t|j|_ tdd|jD|_ ||_||_dS)NrcSsg|]}t|r|qSrAr+rrArArBr3r^z3_TapeGradientFunctions.__init__..)r _forward_graphr"_forwardr_rrrr$_forwardprop_input_indices_forwardprop_output_indices_num_forwardprop_outputsr# _num_trainable_inference_outputs_delayed_rewrite_functions_need_gradients_for_jvpsryrrsr%forwardprop_input_indicesdelayed_rewrite_functionsneed_gradients_for_jvpsrArArBrz$s   z_TapeGradientFunctions.__init__c sjj}g}g}t|D]6\}}t|r|t||}||||qt t jj } | g} |D]2}t |\} } t| | } t|| | | qttd(tj|jj| jd}Wdn1s0Y|rtdd|}fdd| jD}tjj}|D](}||vr$||jj|q$Wdn1sd0Y| | j| _| jddtj |d d D|| _!t"j#j| \}}|s|j|dd fS$||||}%|| |\}}&|}t"j#|j'|\}}t(|t(|t(|j'jkrht)*d t(|j'jd t(|t(|dt(|dt(|d ||j'||j+t(|j,fS)zFForward+backward functions where the backward function sees `outputs`.Nr/cSs|durt|SdSrX)rrQr?rArArB^r^zE_TapeGradientFunctions._build_functions_for_outputs..cs(g|] }t|tjs|jjur|qSrAr7r9r{rArBr`szG_TapeGradientFunctions._build_functions_for_outputs..css|]}|dur|VqdSrXrA)rZgradrArArBr]pszF_TapeGradientFunctions._build_functions_for_outputs..TrVrzThe forward graph had z inputs, but we expected  (z inference inputs and z input tangents).)-r Zvariable_capturesr r r,ridrQr=r>rrr<r r;rr#r rr1r"r2rr. map_structurer@r/rArrinternal_capturesextendrarJrr" _wrap_forward_function_with_jvps)_wrap_backward_function_with_jvp_backprop_shuffle_forward_outputsrrrrYrfrg)ryrr[r\Zhandles_to_variablesr4Ztrainable_indicesindexr.rgradients_wrt_outputsgradient_shapegradient_dtypeZgradient_placeholdergradients_wrt_inputsrBrCrDrrforward_wrapperwrapped_backwards_graphZwrapped_forward_functionZwrapped_backward_functionrAr{rB_build_functions_for_outputs7s         $   2      z3_TapeGradientFunctions._build_functions_for_outputsc stt|jjlt dd|jjD}t |jj D]d\}}t j |j |jd}|t|} | dur| || j tjkrt| |qHj |qHtj |D](\} } tjd| g| gddddd qt|} |jD]} | D]\}}j |}tj |kr:td |d tj d t |\}}t!||}||| }||t"#|j$|jtjd|g|gd dddd qqj d| }|j%j&}t'(||d2|)t**|}t+|tj,rg}Wdn1s0Y|-|j||\}}Wdn1s:0Yt.|j/j|||dt01|\}}fdd|D}Wdn1s0Yt2|||dS)z2Adds inline JVP computation to a forward function.cSsi|]\}}t||qSrAr)rZZexternalinternalrArArB szK_TapeGradientFunctions._wrap_forward_function_with_jvps..)r;r:NZcaptured_valuecSs|gSrXrAr?rArArBrur^zI_TapeGradientFunctions._wrap_forward_function_with_jvps..cSs|gSrXrAr?rArArBrur^)rrz Expected z forward graph inputs, got rHcSs|gSrXrAr?rArArBrur^cSs|gSrXrAr?rArArBrur^PartitionedCallStatefulPartitionedCallcsg|]}|qSrA)rDrZforward_wrapper_graphrArBrszK_TapeGradientFunctions._wrap_forward_function_with_jvps..re)3r=r>rr rr<r Zpush_forwardprop_statecapturesr rrrVr;r:rrrZ add_capturerresourcer#r rQrNrr`rrkrrYr r;rrrJassert_is_compatible_withrorWget_default_graph_override_gradient_functionrr r7 Operation_wrap_backward_function!record_operation_forwardprop_onlyrrTFE_Py_PackJVPsrd)ryrrr[r\Zforward_captures input_indexZ real_inputinput_placeholderrDinprZnum_inference_inputsZ tape_indicesZ jvp_indexrrZjvp_placeholderZ external_jvpZforward_inputsgradient_functionZforward_outputs py_backwardrrfrgrArrBr|s              $(  &z7_TapeGradientFunctions._wrap_forward_function_with_jvpsc sxtt|jj}|||j|j\}}d}g}g} |D]L} t | r||} t | j | j } | | | | |d7}qD| dqD|| } tjj}d}g}g}j}j}|jr|t|kr||dD]0} t| \}}t ||}| || |qt|}td(tj|jj|jd}Wdn1sZ0Yddt| |D} | |t| dfdd|jD}|D]&}||vr||j |qt !j\}}fdd|D}|D]}||qqWdn1s0Y|d|j"|||j"d|j#|_| |_$d d| D|_|j%||d fS) z8Wraps `backward_function` to include gradients for JVPs.rNr/cSs.g|]&\}}|dus|durt||fqSrX)r Z"aggregate_indexed_slices_gradients)rZexistingnewrArArBr"szT_TapeGradientFunctions._wrap_backward_function_with_jvp_backprop..cs(g|] }t|tjs|jjur|qSrAr7rr8rr9rrArBr's  csg|]}j|qSrA)rrDrrrArBr1scSsg|]}|dur|qSrXrArrArArBr;r^)rfrg)&r=r>rr rr<rrr r,rr;r:rQr/rArgrfrprr r;rr1r"r2rrrNr{r@rr pack_tangentsrnrzrJ_replace)ryrrrrrrecorded_outputsZtrainable_indexZforward_doutputsZ doutput_argsr.rUZdoutput_placeholderZdinputsrCZnum_processed_output_tangentsZgradients_wrt_output_tangentsZtangent_doutputsrgrfrrrVrrBrDrrArrBr}s             &     0  z@_TapeGradientFunctions._wrap_backward_function_with_jvp_backpropcsRfdd}t|j}jdjjjjdj_j|dS)z/Reorders function outputs so captures are last.cs>|jkr|S|tjkr0|tjjS|tjSrX)r#rrrg)originalrryrArB _index_mapBs  zC_TapeGradientFunctions._shuffle_forward_outputs.._index_mapN)rf)r.ryrfrr#rgrr)ryrrrfrArrBr~@sz/_TapeGradientFunctions._shuffle_forward_outputscCs0|jdur*|||\|_|_|_|_|_|jS)aConstruct or fetch a forward function with side-outputs. When graph building without a tape active, symbolic gradients rely on regenerating the backward function for higher-order gradients (to account for new side outputs of the rewritten forward function call). Thus there is no fixed backward function for this case. However, when a tape is active (eager or graph building), we generate fixed backward and forward functions at forward function call time. This difference between the tape and non-tape cases is to avoid building unneeded backward functions while graph building (where we may or may not eventually need gradients). Args: inference_args: A flat list of Tensors, arguments to the inference function. input_tangents: A flat list of Tensors, jvps associated with `inference_args`. Returns: A forward _EagerDefinedFunction. N)rj_forward_and_backward_functionsrir_rlrmrZrArArBr)Qs   z_TapeGradientFunctions.forwardc s:ttddjD|j}fdd|DtfddDrlfddD}td|ditjt|g}d }g|j r|j s|d |j ||j |j d }n|}t |D]R\} } |kr| | t| r|d 7}n  | | jtjkrt| | <qʇfd d } | |fS)zECreate a backward function given `outputs` from the forward function.css|]}t|VqdSrXrrrArArBr]qr^zA_TapeGradientFunctions._wrap_backward_function..csg|]}t||qSrArFrGrHrArBrsszB_TapeGradientFunctions._wrap_backward_function..c3s$|]}t|tjs|juVqdSrXrrrrArBr]ws cs&g|]}t|tjs|jur|qSrArrrrArBrys  zTFailed to map all backward graph captures to the forward graph. Incorrectly mapped: rHrNrcsjs jSg}d}t|D]~\}}t|tjr:t|}|vrDq|dur|j|}|j t j krl|}nt j t|}|||d7}|krqq|S)z8Process output gradients and call the backward function.rNr)rrJr r7rrPrrQrr;rvariantrzerosr r;rQrS)rZprocessed_argsr output_indexrr)r*backward_function_inputsrTskip_positionsvariant_zeros_likerArB_backward_function_wrappers2     zR_TapeGradientFunctions._wrap_backward_function.._backward_function_wrapper)rrNrrOrbrrYrrrmrpr#r rQr r,r;rrr rR) ryrr*rrOZincorrect_mappingrZtrainable_recorded_outputsZrelevant_outputsrr.rrA)r*rrIrrTrrrBrnsF         "z._TapeGradientFunctions._wrap_backward_functioncCsp||j|j|\}}|jrRt|jjj|||t |jjj|||||jnt |jjj||||dS)aRecord the function call operation. For backprop, indicates the backward function to use and which new Tensors must be watched. For forwardprop from eager, the function call itself will have produced tangents which need to be recorded. Args: flat_outputs: The result of running `forward`. inference_args: A flat list of Tensors with inference inputs to the operation. input_tangents: A flat list of Tensors with input tangents consumed by the operation. N) rrir_rlrrecord_operation_backprop_onlyrjrrrr`rarArArBrcs&  z_TapeGradientFunctions.recordN) rrrrrzrr|r}r~r)rrcrArArArBrhs _^LNrhcs(eZdZdZfddZddZZS) _FirstOrderTapeGradientFunctionsz9Caches tape-friendly functions for first-order gradients.cs*tt|||||||||_||_dSrX)superrrzr$rkrq __class__rArBrzs  z)_FirstOrderTapeGradientFunctions.__init__cCs |jjd|j}||||S)ayShortcut for when only first-order gradients are required. The returned backward function does not accept gradients with respect to side output of forward_function. This is fine as long as the user can't possibly request second order tape gradients, as when they've used a single non-persistent GradientTape. Since we don't need the backward function to take gradients with respect to side outputs, we can skip some potentially slow graph building. Args: inference_args: A flat list of Tensors, arguments to the inference function. input_tangents: A flat list of Tensors, jvps associated with `inference_args`. Returns: A tuple of (forward_function, backward_function): forward_function: Takes the same inputs as the inference function, but returns side outputs used by backward_function in addition to the inference function's outputs. backward_function: Takes side outputs from forward_function and gradients with respect to the "real" outputs of forward_function and returns gradients with respect to the inputs. N)r rr#r)ryr[r\rrArArBrsz@_FirstOrderTapeGradientFunctions._forward_and_backward_functions)rrrrrzr __classcell__rArArrBrs rc@seZdZdZddZdS)!_HigherOrderTapeGradientFunctionsz:Caches tape-friendly functions for higher-order gradients.c CsLg}d}t|t|jjkrtdd|jjt|dDr|d7}|dkr|ddkrd}d}|jjt|dD]}t|rv|d7}|j}qvtd |jj |||t |jj}| |||q| |||\}} } } } t|jjt|kr>td d|jjt|dDr>t d |jjt|dd || | | | fS) aForward and backward functions suitable for higher-order gradients. Unlike in `_FirstOrderTapeGradientFunctions`, the backward function built by this method accepts gradients for all of the outputs of the returned forward function, including side outputs. Args: inference_args: A flat list of Tensors, arguments to the inference function. input_tangents: A flat list of Tensors, jvps associated with `inference_args`. Returns: A tuple of (forward_function, backward_function): forward_function: Takes the same inputs as the inference function, but returns side outputs used by backward_function in addition to the inference function's outputs. backward_function: Takes side outputs from forward_function and gradients with respect to all of its outputs, real and side. Returns gradients with respect to the inputs. rcss|]}t|VqdSrXr+r-rArArBr]5szT_HigherOrderTapeGradientFunctions._forward_and_backward_functions..NraDetermining side outputs for the function '{}' is taking longer than expected ({} iterations, typically this converges in 5 or so). This could indicate that a gradient registration is adding new ops to the forward pass every time gradients are generated. {} new trainable output(s) were added this iteration, one from the following op: {} This may indicate a TensorFlow bug, or an issue in a tf.custom_gradient.css|]}t|VqdSrXr+r-rArArBr]Ssz\Unexpectedly added new outputs to the forward function when building the backward function: rH)rr rrbr r,rloggingwarningformatrrrrrY) ryr[r\rZiteration_countZnew_op_with_trainable_outputZnum_new_trainable_outputsr.rrrrfZnum_output_tangentsrArArBr sV  zA_HigherOrderTapeGradientFunctions._forward_and_backward_functionsN)rrrrrrArArArBrsrc@s0eZdZdZgdZddZddZddZd S) _ForwardBackwardCallzCHolds the state of a function call between execution and recording.r_inference_args_input_tangents_tape_watchingcCs||_||_||_||_dS)aCollects information about the function call. Args: functions: An object which produces forward and backward functions, either a _DelayedRewriteGradientFunctions or a _TapeGradientFunctions object. inference_args: A flat list of Tensors, arguments to the inference function. input_tangents: A flat list of Tensors, jvps associated with `inference_args`. tape_watching: Boolean, with True indicating that recording is necessary. Nr)ry functionsr[r\ tape_watchingrArArBrzds z_ForwardBackwardCall.__init__cCs"|j|j|j}||j|jfS)z5Builds or retrieves a forward function for this call.)rr)rr)ryrrArArBr)usz_ForwardBackwardCall.forwardcCs2|jr.t|tjs.|dur.|j||j|jdS)zEGiven outputs from the execution of `forward`, records the operation.N)rr7rrrrcrr)ryrbrArArBrc{s  z_ForwardBackwardCall.recordN)rrrrrrzr)rcrArArArBr]s rc@seZdZdZdZddZddZdd Zed d Zd d Z eddZ ddZ d[ddZ ddZ ddZddZddZddZddZd\d d!Zd"d#Zed$d%Zed&d'Zed(d)Zed*d+Zed,d-Zed.d/Zd0d1Zd]d2d3Zed4d5Zed6d7Zed8d9Zed:d;Z d^dd?Z"d@dAZ#dBdCZ$dDdEZ%edFdGZ&d`dIdJZ'dKdLZ(dadMdNZ)dOdPZ*dQdRZ+dbdTdUZ,dVdWZ-dXdYZ.dS)crzFA `tf.types.experimental.ConcreteFunction` created from `tf.function`.NTcCsd|_d|_||_|jj|jj|_|||r~t|vr~tdd|j D}t||jfr~Jdj |j t|t|j |jdt dd|jj D|_t|pi|_|rd|_n t||_t||j|j|_i|_i|_|j|_dS)aInitialize a `ConcreteFunction`. Args: func_graph: An instance of FuncGraph: the function body to wrap. attrs: (optional) dict mapping names of attributes to their AttrValue values. Attributes in `attrs` will be included in this function's definition. shared_func_graph: If False, the ConcreteFunction takes ownership of `func_graph` and will break reference cycles when it is deleted. This makes the FuncGraph inoperable. spec: FunctionSpec for the original function. If not specified, then this ConcreteFunction may only be called using the flat signature. Raises: ValueError: If number of input_placeholders is not equal to the number of function inputs. Ncss|]}|jtjkVqdSrX)r;rr)rZrrArArBr]sz,ConcreteFunction.__init__..aQFunction {name} has "{attr}={value}" attribute and thus can not depend on any tensors outside of its signature or modify variables. Note: variables are always captured and cause function re-tracing for every variable called. inputs: {inputs} captures: {captured} To pass a variable to such function use use variable.read_value().)rattrrurcapturedcss|] }|jVqdSrXrr-rArArBr]s) _arg_keywords_num_positional_argsr r@Zdeferred_external_captures_captured_inputs_set_function_specrrbrrrrrrrvr"_garbage_collector ConcreteFunctionGarbageCollectorrro_first_order_tape_functions_higher_order_tape_functionsr)r!)ryrrsshared_func_graphspecZhas_resource_varsrArArBrzs>      zConcreteFunction.__init__cCsd|_||_|dS)z5Enables the structured signature by supplying a spec.N)_function_spec_pre_initialized_function_spec_initialize_function_spec)ryrrArArBrsz#ConcreteFunction._set_function_specc Cs|jdurdS|jrJd|j}|jj}|j\}}tt|jt|}tj t |dd|Dddt gt|t t |t dd|D|jjd}tj||j|j|j|jjd|_dS) aUpdates `self._function_spec` to include varargs and bound variables. Adds new positional arguments for any varargs (i.e., for args that are in `structured_input_signature`, but not in the original fullargspec.args). Replaces `defaults` and `kwonlydefaults` with the `_BOUND_VALUE`, for all args and kwargs in `structured_input_signature`. Sets `varkw` and `varargs` to None. Nzalready initializedcSsg|]}d|dqS)rrrZrerArArBrr^z>ConcreteFunction._initialize_function_spec..css|]}|tfVqdSrX) _BOUND_VALUE)rZkrArArBr]r^z=ConcreteFunction._initialize_function_spec..)rvarargsvarkwdefaults kwonlyargskwonlydefaults annotationsr)rr fullargspecrstructured_input_signaturerNr arg_namesr1 FullArgSpecrrsortedrrr FunctionSpec is_methodinput_signatureis_purer r)ryrr arg_specs kwarg_specsZvararg_indicesrrArArBrs.    z*ConcreteFunction._initialize_function_speccCs t|jjS)z(Sequence of variables for this function.)rr  variablesr{rArArBrszConcreteFunction.variablescCs ||j_dSrX)r r)ryrrArArB set_variables szConcreteFunction.set_variablescCs t|jjS)z2Sequence of trainable variables for this function.)rr trainable_variablesr{rArArBr sz$ConcreteFunction.trainable_variablescOs |||S)a5Executes the wrapped function. ConcreteFunctions have two signatures: * The signature of the original function wrapped by this ConcreteFunction. * A flat signature, where each argument accepts a single Tensor. The original function signature is generally preferred, but the flat input signature is supported for backward compatibility. ### Original Function Signature When calling a ConcreteFunction with the signature of the original function, each argument must match the type or value that was used when the ConcreteFunction's graph was traced. In particular: * Tensor arguments (including CompositeTensors, such as RaggedTensor) must have matching `TypeSpec`s. * Non-Tensor arguments (such as booleans or ints) must have equal values. * Nested arguments (such as lists, tuples, or dictionaries) must have the same nesting structure; and each nested value must have a matching type or value. The default value for any arguments that were traced with non-Tensor values is the value that was used in the trace. Arguments that were traced with tensor arguments do not have a default value (even if the original function had a default value for that argument). ### Flat Signature When calling a ConcreteFunction with the flat signature, the arguments correspond to the flattened component tensors of the arguments that were used to construct the ConcreteFunction. Parameter names are assigned based on `TensorSpec.name` (when specified) or the original argument names (with suffixes automatically added for nested arguments or composite tensors with multiple components). Args: *args: Positional arguments to the concrete function. **kwargs: Keyword arguments to the concrete function. Returns: The result of applying the TF function on the given Tensors. Raises: AssertionError: If this `ConcreteFunction` was not created through `get_concrete_function`. TypeError: If the arguments do not match the function's signature.  _call_impl)ryrr6rArArB__call__s2zConcreteFunction.__call__c Cstj|jjdd|jdurz||||WWdSty}zNz*||||WWYd}~WdSty|Yn0WYd}~n d}~00||||WdS1s0YdS)zSee `__call__` for details.concrete)tf_function_callN)r&Tracer rr_call_with_structured_signaturerK_call_with_flat_signature)ryrr6rZstructured_errrArArBrFs    zConcreteFunction._call_implc Cst||jkr2t|d|jdt|dt|}t|}|jt|dD]}z||t |WqTt yt|jdt|t| }t t|jt|}t|dd|dYqT0qT|rFt|jdt|}|D]$}||vrt|d|dqt|d dt |dt|D]H\} } t| tjtjfsNt|d | d t| jd | d qN|||j|S)aExecutes the wrapped function with the flat signature. Args: args: Positional arguments to the concrete function. kwargs: Keyword arguments to the concrete function. cancellation_manager: A `CancellationManager` that can be used to cancel function invocation. Returns: The result of applying the function on the Tensors/Variables contained in `args` and `kwargs`. Raises: TypeError: if `args` and `kwargs` do not match the flat signature of this `ConcreteFunction`.  takes z positional arguments, got rHN missing required arguments: , z got two values for 'z'.# got unexpected keyword arguments: z: expected argument #z!(zero-based) to be a Tensor; got rw).)rrrK_flat_signature_summaryrrrrQrr*rKeyErrorkeysrrjoinr r7rr8r$BaseResourceVariablerLrrSrO) ryrr6rkeywordZspecified_keywordsZmissing_required_argsZpositional_arg_keywords unused_keyrerrArArBrXsT         z*ConcreteFunction._call_with_flat_signaturecCsN|jj|i|\}}}||||||||||j||j|dS)a$Executes the wrapped function with the structured signature. Args: args: Positional arguments to the concrete function. kwargs: Keyword arguments to the concrete function. cancellation_manager: A `CancellationManager` that can be used to cancel function invocation. Returns: The result of applying the function on the Tensors/Variables contained in `args` and `kwargs`. Raises: TypeError: if `args` and `kwargs` do not match the structured signature of this `ConcreteFunction`. )rOr)rcanonicalize_function_inputs(_structured_signature_check_missing_args+_structured_signature_check_unexpected_args%_structured_signature_check_arg_typesrSrO)ryrr6rfiltered_flat_argsrArArBrs   z0ConcreteFunction._call_with_structured_signaturec Cs|j\}}g}tt||D].\}\}}|turt|r||jj|q|D]&\} }|turTt|| rT|| qT|rt | dd t |ddS)z+Raises a TypeError if any args are missing.rrrHN) rr rNr_contains_type_specrQrrrhrK_structured_signature_summaryrr) ryrr6rrZmissing_argumentsrerrrrArArBrs    z9ConcreteFunction._structured_signature_check_missing_argscCs|j\}}t|t|krDt|dt|jjdt|dt|t|krt|t|}t|dd|ddS)z/Raises a TypeError if there are any extra args.rz positional arguments but got rHrrN)rrrKrrrrr)ryrr6rr extra_argsrArArBrs    z.cancellable_callrA)ryrr rArrB'_experimental_with_cancellation_managerXs z8ConcreteFunction._experimental_with_cancellation_managercCs |jjS)z`ConcreteFunction` name.)ror)rr{rArArBriszConcreteFunction.namecCs|jS)z;Returns the graph from which this function was constructed.r r{rArArBrnszConcreteFunction.graphcCs|jjS)z;Returns tensors in `self.graph` corresponding to arguments.)r rr{rArArBrsszConcreteFunction.inputscCs|jjS)a Returns structured signature for this concrete function. Returns: A tuple `(args, kwargs)`, where: * `args` is a tuple that specifies the expected type or value each for positional argument. * `kwargs` is a dictionary that specifies the expected type or value for each keyword-only argument. The type or value for each argument is specified using one of the following: * A `tf.TypeSpec`, indicating that a Tensor or other TensorFlow-native value is expected. * A Python value, such as an integer, indicating that an equal value is expected. * A nested structure of `tf.TypeSpec`s and Python values, indicating that a corresponding nested structure is expected. )r rr{rArArBrxsz+ConcreteFunction.structured_input_signaturecCs|jjS)zBReturns tensors in `self.graph` corresponding to returned tensors.)r rr{rArArBrszConcreteFunction.outputscCs|jjS)zEReturns outputs in `self.graph` as returned by the original function.)r rJr{rArArBrJsz#ConcreteFunction.structured_outputscCs ||_dS)aUpdates the function capture values. The new values must have tensor types and shapes consistent with the original captures of the concrete function, but it is allowed to change a value captured with a deferred one and vice-versa. Args: captures: A list of tensors or closures. Tensors are value captures, and closures are call-time (deferred captures). N)r)ryrrArArBset_external_capturessz&ConcreteFunction.set_external_capturesc Csd}t|jD] \}}t|t|kr|}q0q|durf|durPtd|d|jt|j |}||s||std|d|d|d|jj|||||d|dur||j|<dS) a Replaces existing capture `tensor` with a deferred capture `closure`. This API replaces the capture `tensor` from the concrete function's captured inputs list, and places the deferred capture `closure` in its spot so the order of captured inputs is preserved. This is important because the old `tensor` and the new `closure` will have the same internal placeholder, which can be passed through the `placeholder` argument, or skipped, in which case we find the placeholder from internal inputs by indexing `tensor` in the external captured inputs list. Thus, it is important that the new deferred capture has output spec (specified by the `spec` argument) compatible with the internal placeholder (`placeholder`) and the original capture (`tensor`). For example, ```python bool_captured_tensor = tf.constant(True) float_captured_tensor = tf.constant([3.], dtype=tf.float32) value = tf.constant([2.], dtype=tf.float32) @tf.function def fn(): deferred_tensor = ops.get_default_graph().capture_call_time_value( lambda: value, tf.TensorSpec(shape=(1,), dtype=tf.float32)) if bool_captured_tensor: return deferred_tensor else: return deferred_tensor + float_captured_tensor concrete_fn = fn.get_concrete_function() print(concrete_fn()) # tf.Tensor([2.], shape=(1,), dtype=float32) new_bool_captured_tensor = constant_op.constant(False) def bool_closure(): return new_bool_captured_tensor concrete_fn.replace_capture_with_deferred_capture( bool_captured_tensor, bool_closure, spec=tensor_spec.TensorSpec(shape=(), dtype=dtypes.bool)) print(concrete_fn()) # tf.Tensor([5.], shape=(1,), dtype=float32) ``` Args: tensor: Tensor already captured. This `tensor` should be listed in concrete_function.captured_inputs except when it's empty such as when the concrete function is restored from SavedModel. closure: function which takes no arguments, to be evaluated at function call time, returning a nest of tensors compatible with `spec`. spec: nest of TypeSpec for the value to capture. placeholder: optional. The internal placeholder corresponding to the captured `tensor` and the new `closure`. default_value: optional value to use in environments that cannot safely evaluate closure. NzDid not find `tensor` argument z in the ConcreteFunction's captured inputs list, and did not receive a placeholder argument. Thus we're unable to infer the internal placeholder. z+Attempting to substitute closure with spec z/ that's incompatible with the original capture z or the internal placeholder rH)tensorclosurerrV default_value) r rrxr`rrrr %replace_capture_with_deferred_capture) ryr$r%rrVr&Z capture_indexrerDrArArBr's<?  z6ConcreteFunction.replace_capture_with_deferred_capturecCstjdd|jDddS)zReturns external Tensors captured by this function. self.__call__(*args) passes `args + self.captured_inputs` to the function. cSsg|]}t|r|n|qSrA)callablerZr@rArArBrr^z4ConcreteFunction.captured_inputs..TrV)r.rarr{rArArBrO sz ConcreteFunction.captured_inputscCs |jjS)z:Returns a `FunctionDef` object representing this function.)ror)rr{rArArBrszConcreteFunction.function_defcCstjddt|jjddS)zThe function's output shapes.cSst|dtdS)Nr:)getattrrrJr?rArArBrur^z0ConcreteFunction.output_shapes..FrVr.ryr"replace_composites_with_componentsr rJr{rArArB output_shapesszConcreteFunction.output_shapescCstjddt|jjddS)NcSs|dur|jSdSrX)r;r?rArArBru(r^z0ConcreteFunction.output_dtypes..FrVr+r{rArArB output_dtypes$szConcreteFunction.output_dtypescCs(ts|st}|j|dS)zRegisters the function, adds it to the graph g or default graph. Args: g: If specified, registers the function with this graph. Defaults to the current context (either the default graph or the eager context). N)r rrrror)r)ryrrArArBr-s zConcreteFunction.add_to_graphcCsJts|st}|j||j\}}||||dS)zCAdd forward/backward functions to graph `g` or the current context.N)r rrrror)rr()ryrrrrArArBadd_gradient_functions_to_graph=s  z0ConcreteFunction.add_gradient_functions_to_graphcCs|jjS)zCReturns gradient function. It will be lazily created at first call.)rorWr{rArArBrGsz'ConcreteFunction._get_gradient_functioncCs|rt|}nt}t|j}||jf}|tjkr|jsB|r|j |d}|durt |j |j |j|j|j|d}||j |<t|||jddSt|j||jddSn\|tjkr|j |d}|durt|j |j |j|j|j|d}||j|<t|||jddSt|j||jddS)a`Selects forward and backward functions based on the calling context. The forward function computes the "real" function outputs, `self._outputs`, and any extra values needed by the corresponding backward function. Args: args: A flat list of Tensors with all of the inputs to the forward function (including user-specified and captured inputs). possible_gradient_type: One of gradients_util.POSSIBLE_GRADIENT_TYPES_*. executing_eagerly: Boolean, the value of context.executing_eagerly(). Returns: An object with a `forward` method returning a tuple of (forward_function : _EagerDefinedFunction, augmented_arguments : List), and a corresponding `record` method which takes outputs from the forward function and records the operation. forward_function should be called with augmented_arguments. N)rrrsrtT)rF)r rZ TangentInforshould_record_backproptangentsindicesr"Z#POSSIBLE_GRADIENT_TYPES_FIRST_ORDERrrrr r"rrorZ$POSSIBLE_GRADIENT_TYPES_HIGHER_ORDERrr)ryrrrr\rt cache_keyrrArArBrKs`            z7ConcreteFunction._select_forward_and_backward_functionscCs|jjdur|Stj|jjdd}d}t|D]:\}}|dur.t|j|||||||<|d7}q.tj|jj|dd}|S)zMaps the fdef output list to actual output structure. Args: result: Output lists defined by FunctionDef. Returns: The actual call output. NTrVrr) r rJr.rar r#r rpack_sequence_as)ryresultZ outputs_listjrerretrArArBrs    z$ConcreteFunction._build_call_outputscCs6tj|jd}|jD]\}}|j||q|S)z4Returns a `NameAttrList` representing this function.r)r NameAttrListrr"rhrCopyFrom)ryr7rrurArArB_as_name_attr_listsz#ConcreteFunction._as_name_attr_listFc Cs|jdusJ|j\}}t|jj}|dt|}|rptt|D]*}t||sD||d||7<qD|r|d| D]4\}}|||rt|s|dd|7<q|j j dd |d}|S)zReturns a string summarizing this function's structured signature. Args: default_values: If true, then include default values in the signature. Returns: A `string`. Nz={}*(r)) rrrrrrNrrrQrhr rr) rydefault_valuesrrrrerrrrArArBrs       z.ConcreteFunction._structured_signature_summarycCsj|jdusJ|jdusJ|j}|jt|krP|ddtt||jD|jjdd|dS)z.r=rr>)rrrr{rNr rr)ryrrArArBrs z(ConcreteFunction._flat_signature_summaryc s|s|jddSfdd|jddg}|j\}}t|jj}g}t|dt|t|D]&\}}t|r^|d ||q^|rt |D]*} || }t|r|d | |q|r|d| ||dd d } |d t | |jd |S) zEReturns a string summarizing the signature of this concrete function.T)r?cst|tjrd|jj|jSt|rtj |dd}ddt t |D}t ||}t j|dd}t||D]\}}|d||7}qp|St|Sd S) z;Returns a string describing the spec for a single argument.z{} Tensor, shape={}FrVcSsg|]}td|dqS)z<{}>r)_MarkerrrrArArBrr^zXConcreteFunction.pretty_printed_signature..pretty_print_spec..i')widthz {}: {}N)r7rr9rr;rr:r. is_nestedrarNrr4pprintpformatrNrepr)rpiecesmarkers structurer5markerpiecepretty_print_specrArBrLs   zDConcreteFunction.pretty_printed_signature..pretty_print_specNz {}: {}z Args:z Returns:cSst|tjr|St|SrX)r7rr<type_spec_from_valuerurArArBspec_from_value s zBConcreteFunction.pretty_printed_signature..spec_from_valuez {} )rrrrrrNrrrQrrr{r.ryrJr) ryverboselinesrrnamesZ arg_detailsrrkwargrOrArKrBpretty_printed_signatures8    "     z)ConcreteFunction.pretty_printed_signaturecCsX|jdur"d|jddt|S|jdusJ|jdusJd|t|St|SdS)NzF)rQ) rrrUrxrrrobject__repr__r{rArArBrW+ s  zConcreteFunction.__repr__cCs$|jdurd|S|SdS)NzConcreteFunction {})rrrUrWr{rArArB__str__5 s zConcreteFunction.__str__ checkpointcKsX|dkr iSi}t|jjD]6\}\}}|jtjtjfvrt|s||d|<q|S)zImplements `Trackable`.rYZcapture_) r rrr;rrrr$is_resource_variable)ry save_typer6Zcaptured_trackablesrrDrrArArB_trackable_children; s z$ConcreteFunction._trackable_childrencCs|SrXrA)rychildrenrArArB_deserialization_dependenciesT sz.ConcreteFunction._deserialization_dependenciescKsD|jjs(td|jdd|jj|t||||<gS)NzUnable to save function z for the following reason(s): rP) rrr`rrrrrZExportedConcreteFunction)ry object_map tensor_map unused_kwargsrArArB_export_to_saved_model_graphW s  z-ConcreteFunction._export_to_saved_model_graph)NTN)N)N)NN)N)N)F)T)rY)/rrrrrzrrrrrrrrrrrrrrrSr!rrrrrrJr#r'rOrr-r.rr/rrrr:rrrUrWrXr\r^rbrArArArBrsz O!  4 1  0 _       _      R  $ >  rr8r8rPc@seZdZdZd"ddZddZed d Zed d Zed dZ eddZ ddZ ddZ ddZ ddZeedddZddZddZd d!ZdS)#FunctionaWrapper class for the graph functions defined for a Python function. See the documentation for `defun` for more information on the semantics of defined functions. `Function` class is thread-compatible meaning that minimal usage of defuns (defining and calling) is thread-safe, but if users call other methods or invoke the base `python_function` themselves, external synchronization is necessary. In addition, Function is not reentrant, so recursive functions need to call the wrapped function, not the wrapper. NTFc Cs||_|ot|v} tjj||| | d|_||_||_||_||_ t |_ |pPi|_ ||_d|_t|_t|_t|_| |_| |_dS)aInitializes a `Function`. Args: python_function: the function to be wrapped. name: the name given to it. input_signature: a possibly nested sequence of `TensorSpec` objects specifying the input signature of this function. If `None`, a separate function is instantiated for each inferred input signature. attributes: dict, extra keyword arguments that will be added as attribute of the function. autograph: whether to use autograph to compile `python_function`. See https://www.tensorflow.org/guide/autograph for more information. autograph_options: Experimental knobs to control behavior `when autograph=True`. See https://www.tensorflow.org/guide/autograph for more information. reduce_retracing: When True, `tf.function` uses `tf.types.experimental.TraceType` to trace supertypes of arguments to reduce the number of traces. capture_by_value: Experimental. Whether to capture resource variables by value or reference. If None, will inherit from a parent context or default to False. jit_compile: Force-compile the function with XLA, cf. def_function.Function doc on jit_compile. experimental_follow_type_hints: See the documentation for `tf.function`. Raises: ValueError: if `input_signature` is not None and the `python_function`'s argspec has keyword arguments. )rexperimental_follow_type_hintsrN)_python_functionrrrfrom_function_and_signaturerr _autograph_autograph_options_reduce_retracingrZ FunctionCache_function_cache_function_attributes_capture_by_value tracing_countr=ZCapturesContainer_captures_container threadingRLock_lockweakrefWeakKeyDictionary_descriptor_cache _jit_compile_experimental_follow_type_hints) rypython_functionrrrr autographautograph_optionsreduce_retracingcapture_by_value jit_compilerdZ pure_functionrArArBrzx s*)      zFunction.__init__cOsF|j |||\}}Wdn1s,0Y|j||jdS)z1Calls a graph function specialized to the inputs.N)rO)rq_maybe_define_functionrSrO)ryrr6graph_functionrrArArBr s zFunction.__call__cCs|jS)z$Returns the wrapped Python function.)rer{rArArBrw szFunction.python_functioncCs|jSrX)rr{rArArBr szFunction.function_speccCs|jjS)zReturns the input signature.)rrr{rArArBr szFunction.input_signaturecCs|jjS)z&Returns the flattened input signature.)rflat_input_signaturer{rArArBr szFunction.flat_input_signaturecOsH|jrd\}}|j |||\}}Wdn1s:0Y|S)z?Returns a concrete function which cleans up its graph function.NNN)rrqr})ryrr6r~rrArArB1_get_concrete_function_internal_garbage_collected s .z:Function._get_concrete_function_internal_garbage_collectedcOs|j|i|}|j|S)z6Bypasses error checking when getting a graph function.)rrreleaseryrr6r~rArArB_get_concrete_function_internal s z(Function._get_concrete_function_internalcOs|jr|r"td|d|jd|rt|j|sJtd|d|jdtj|dd}tdd |Drrtd td d t|j|Drtd |d|jdd \}}|j| ||\}}t }t |j j}g|_i}d} |j jD]z} | |vrqj| d7} t| jd} | } | |vrR|| d} d| | } | d|| <q || |j| q| |_|WdS1s0YdS)aReturns a `ConcreteFunction` specialized to inputs and execution context. Unlike `get_concrete_function(...)`, the graph will be deleted when the returned function is deleted. It's useful to avoid creating a reference cycle when you know for sure that the graph will be no longer used without the returned function. Args: *args: inputs to specialize on. **kwargs: inputs to specialize on. zCannot define a TensorFlow function from a Python function with keyword arguments when input_signature is provided, got keyword arguments (z) with input_signature (rzLStructure of Python function inputs does not match input_signature: inputs (z), input_signature (TrVcss&|]}t|tjtjtjf VqdSrX)r7rr8rr r$rrrArArBr] s zDFunction._get_concrete_function_garbage_collected..zWhen input_signature is provided, all inputs to the Python function must be Tensors, Variables, tf.TensorSpec or tf.VariableSpec objects.css|]\}}|| VqdSrX)r)rZrotherrArArBr] sz9Python inputs incompatible with input_signature: inputs (rrrZ_user_specified_namez{}_{}N)rr`rdr.rarbrNrrqr}rr/rArrzrrr*rrget_attrrrrrQr)ryrr6 flat_inputsr~rZ seen_namesrZ prefix_countsZnum_positionalrZ user_arg_nameZproposalrrArArB(_get_concrete_function_garbage_collected sf            z1Function._get_concrete_function_garbage_collectedcOs|j|i|}|j|S)aAReturns a `ConcreteFunction` specialized to inputs and execution context. Args: *args: inputs to specialize on. Can be concrete values (e.g. 1) or `tf.Tensor` or `tf.TensorSpec`. **kwargs: keyword inputs to specialize on. Concrete values (e.g. 1) or `tf.Tensor` or `tf.TensorSpec`. )rrrrrArArBget_concrete_function) s  zFunction.get_concrete_function)returncCs |jSrX)rjrr{rArArB_list_all_concrete_functions7 sz%Function._list_all_concrete_functionscCs2~||jvr(|dur|St|||j|<|j|S)z,Makes it possible to defun instance methods.N)rtclass_method_to_instance_method)ryinstanceownerrArArB__get__: s  zFunction.__get__c Cs|jd7_|jdur"t|}n t|j}|jjd|}|t|jj}|jjg|}ddt|D}||}ttj |j |j |||j|j |j ||jd |j|jdd}|S)z5Create a `ConcreteFunction` from `args` and `kwargs`.rNcSsg|]\}}d||fqS)z%s_%drA)rZrerrArArBri sz3Function._create_graph_function..)rxryrr{F)rr)rmrrrrZ vararg_namer rr=r?rrergrhrlrkr) ryrr6arglenZbase_arg_namesZnum_missing_argsZmissing_arg_namesrr~rArArB_create_graph_function\ s8    zFunction._create_graph_functionc Csb|jdus|dus|dur4|jj|i|\}}}ng}|j}|jdurbt||f|\}}nt|j|\}}z t|Wn.t y}zt d|WYd}~n d}~00|j |d}|dur||fSt tntd@tdd|j|tdd|||jr tjjntjj} tj| |jd |jrn|jdurn|j |}|} | d \}}|||}|j j!} |j"| | }|jdurt||f|\} } nt|j|\} } |j #| | |||fWdWdWdS1s0YWdn1s40YWdn1sT0YdS) aGets a function for these inputs, defining it if necessary. `args` and `kwargs` can be None if this `Function` was created with an `input_signature`. Caller must hold self._lock. Args: args: The varargs for the Python function. kwargs: The keyword args for the Python function. Returns: A graph function corresponding to the input signature implied by args and kwargs, as well as filtered flattened inputs (only Tensors and Variables) that the object should be called with. Raises: ValueError: If inputs are incompatible with the input signature. TypeError: If the function inputs include non-hashable objects RuntimeError: If there's an internal bug (inconsistency) in handling shape relaxation retracing. NzCArguments supplied to `defun`-generated functions must be hashable.Tztf.function-graph_buildingrz7Creating new FuncGraph for Python function %r (key: %r)z1Python function signature [args: %s] [kwargs: %s])statusoptionsr)$rrrrnZ get_snapshotrZmake_cache_keyrhashrKrjlookuprMonitoredTimer_graph_building_time_counterget_cellr&rrr rergr3StatusENABLEDDISABLEDZControlStatusCtxrhriZ generalize_placeholder_valuerrZ_capture_func_librr)ryrr6rrZfunc_cache_keyrer~Z ag_statusZplaceholder_dictZgraph_capture_containerr3Zcache_key_deletion_observerrArArBr} sx            zFunction._maybe_define_function)NNTNFNNF)rrrrrzrrrwrrrrrrrrrrrrr}rArArArBrcj s6 C     @"%rccOsHt|ts$td|dt|d|j|i|}|||S)aRegister a specialization of a `Function` into the graph. This won't actually call the function with the inputs, and only put the function definition into graph. Register function with different input param will result into multiple version of functions registered in graph. Args: func: the `Function` instance that generated by a @defun *args: input arguments for the Python function. **kwargs: input keyword arguments for the Python function. Returns: a `ConcreteFunction` object specialized to inputs and execution context. Raises: ValueError: When the input function is not a defun wrapped python function. z5Only defun function is allowed to be registered. Got z with type rH)r7rcr`rLrrr/)rrr6Z concrete_funcrArArBregister s  rcCszt|ttfs"tdt|dtddtj|ddDrvddtj|ddD}td |d ttt|ddS) Nz6input_signature must be either a tuple or a list, got rHcss|]}t|tj VqdSrXr7rr rrArArBr] sz%validate_signature..TrVcSsg|]}t|tjs|qSrArrrArArBr s z&validate_signature..z[input_signature must be a possibly nested sequence of TensorSpec objects, got invalid args z with types ) r7rrrKrLrbr.rar)rZbad_argsrArArBvalidate_signature s   rcCst|st|ddS)Nz is not a callable object.)r(rK)rwrArArBvalidate_python_function srTcCst|||||dS)a>8Compiles a Python function into a callable TensorFlow graph. `defun` (short for "define function") compiles a Python function composed of TensorFlow operations into a callable that executes a `tf.Graph` containing those operations. The callable produced by `defun` contains only the subgraph of TensorFlow operations that were executed when the Python function was called with a particular input signature, defined as a list of the shapes and dtypes of the Python function's Tensor-valued arguments and the values of its non-Tensor Python objects. When eager execution is enabled, the ability to create graphs from Python functions makes it possible to incrementally trade off debuggability and interactivity for performance. Functions compiled with `defun` cannot be inspected with `pdb`; however, executing a graph generated by `defun` sometimes takes less time and memory than eagerly executing the corresponding Python function, since specifying computations as graphs allows for optimizations like automatic buffer reuse and parallelization among ops. Note that executing a `defun`-compiled function incurs a small constant overhead, so eagerly executing sufficiently small Python functions might take less time than executing their corresponding `defun`-generated graphs. For a Python function to be compatible with `defun`, all of its arguments must be hashable Python objects or lists thereof. The function itself may not modify the list/map structure of its arguments. Additionally, it must return zero or more `tf.Tensor` objects. If the Python function returns a `tf.Variable`, its compiled version will return the value of that variable as a `tf.Tensor`. Executing a graph generated by `defun` respects device annotations (i.e., all `with tf.device` directives present in a Python function will also be present in its corresponding graph), but it is not yet possible to execute the generated graphs across multiple machines. _Example Usage_ ```python import tensorflow as tf tf.compat.v1.enable_eager_execution() # A simple example. def f(x, y): return tf.reduce_mean(tf.multiply(x ** 2, 3) + y) g = tf.contrib.eager.defun(f) x = tf.constant([[2.0, 3.0]]) y = tf.constant([[3.0, -2.0]]) # `f` and `g` will return the same value, but `g` will be executed as a # TensorFlow graph. assert f(x, y).numpy() == g(x, y).numpy() # `defun` is capable of compiling Python functions that close over Python # objects, including Tensors and Variables. @tf.contrib.eager.defun def h(): return f(x, y) assert (h().numpy() == f(x, y).numpy()).all() # `defun` automatically lifts variables out of the graphs it creates, # allowing you to compile the `call` methods of `tf.keras.layers.Layer` and # `tf.keras.Model` objects. class MyModel(tf.keras.Model): def __init__(self, keep_probability=0.2): super(MyModel, self).__init__() self.dense1 = tf.keras.layers.Dense(4, activation=tf.nn.relu) self.dense2 = tf.keras.layers.Dense(5, activation=tf.nn.softmax) self.keep_probability = keep_probability @tf.contrib.eager.defun def call(self, inputs, training=True): x = self.dense2(self.dense1(inputs)) if training: return tf.nn.dropout(x, self.keep_probability) else: return x model = MyModel() model(x, training=True) # executes a graph, with dropout model(x, training=False) # executes a graph, without dropout # `defun`-compiled functions are differentiable. optimizer = tf.compat.v1.train.GradientDescentOptimizer(learning_rate=0.01) with tf.GradientTape() as tape: outputs = model(x) gradient = tape.gradient(outputs, model.trainable_variables) optimizer.apply_gradients((grad, var) for grad, var in zip(gradient, model.trainable_variables)) ``` When using `defun`, there are subtleties regarding inputs, Python control flow, and variable creation that one should be aware of. For concreteness, let `f` be a Python function that returns zero or more `tf.Tensor` objects and let `F = defun(f)`. `F` builds a graph for each unique input signature it sees, Python control flow is baked into graphs, and operations related to variable initialization are automatically lifted out of the graphs that `F` generates and placed in the eager context if executing eagerly or into an outer graph otherwise. _Input Signatures_ By default, `F = tf.contrib.eager.defun(f)` instantiates a separate graph for every unique sequence of the shapes and dtypes of Tensor arguments and the values of Python objects it is invoked with. For example, calling `F(tf.random.uniform([2])` will execute a different graph than `F(tf.random.uniform([3])` because the two inputs have different shapes. The first time that `F(*args, **kwargs)` is called with a particular sequence of Tensor shapes and dtypes and Python values, it constructs a graph by tracing the execution of `f(*args, **kwargs)`; this graph is bound to an input signature inferred from `(*args, **kwargs)` and cached for future reuse. NumPy arrays passed as inputs to `F` are converted to `tf.Tensor` objects before being passed to `f`, and are treated as Tensors for caching. This allows a function to be called multiple times with NumPy arrays having different values but the same shape and dtype without re-tracing each time. `tf.contrib.eager.defun` caches graphs for your convenience, letting you define TensorFlow functions without explicitly specifying their signatures. However, this policy is conservative and potentially expensive; for example, when different invocations of your function have differently-shaped Tensor inputs, this policy might generate more graph functions than necessary. To eliminate such costs, `tf.contrib.eager.defun` allows you to supply an optional `input_signature` argument specifying the shapes and dtypes of the inputs. In particular, the shapes may be partially unspecified, with `None`s in the unknown dimensions. When an input signature is provided, `tf.contrib.eager.defun` will only instantiate a single graph for the decorated Python function. The following is an example: ```python import tensorflow as tf # The first `TensorSpec` below describes the shape and dtype of `words`, # and the second describes the shape and dtype of `another_tensor`. Note that # the last dimension of the `words` `TensorSpec` is left unspecified. @tf.contrib.eager.defun(input_signature=[ tf.contrib.eager.TensorSpec(shape=[50, 300, None], dtype=tf.float32), tf.contrib.eager.TensorSpec(shape=[300, 100], dtype=tf.float32) ]) def my_sequence_model(words, another_tensor): ... # Note how the third dimension of the first input can vary freely. words = tf.random.uniform(([50, 300, 10]) second_input = tf.random.uniform([300, 100]) my_sequence_model(words, second_input) words = tf.random.uniform(([50, 300, 20]) my_sequence_model(words, second_input) # Passing an input with an incompatible shape will raise an error. words = tf.random.uniform(([50, 100, 20]) my_sequence_model(words, second_input) # <---- This will raise an error. ``` Python functions that are compiled with an `input_signature` must only accept Tensors as arguments and must not take unnamed keyword arguments (**kwargs). _Tracing_ Be aware that because `F` only logs TensorFlow operations, all the other Python code that `f` executes will only shape the _construction_ of the graphs that `F` executes: the Python code won't be executed when the graphs themselves are executed, though it will be executed every time the Python function is traced (and a given Python function might be traced multiple times, once for each input signature it is invoked with). For example, whereas the Python function ```python import tensorflow as tf import numpy as np tf.compat.v1.enable_eager_execution() def add_noise(): return tf.eye(5) + np.random.randn(5, 5) ``` will return a different output everytime it is invoked, the compiled function `compiled = tf.contrib.eager.defun(add_noise)` will return the same value every time it is called, since a particular random offset generated by NumPy will be inserted into the graph as a TensorFlow constant. The solution is to replace the call to `np.random.randn` with `tf.random.normal((5, 5))`. _Python Side-Effects_ A corollary of the previous discussion on tracing is the following: If a Python function `f` has Python side-effects, then executing `f` multiple times will not necessarily be semantically equivalent to executing `F = tf.contrib.eager.defun(f)` multiple times; this difference is due to the fact that `defun` only captures the subgraph of TensorFlow operations that is constructed when `f` is called in a graph-building context. _Python Control Flow_ The structure of many machine learning computations depend upon whether one is training or validating, and it is common to nest specialized logic under `if training:` blocks. By mapping each input signature to a unique graph, `defun` lets users transparently compile such code, as the following code snippet demonstrates: ```python import tensorflow as tf tf.compat.v1.enable_eager_execution() @tf.contrib.eager.defun def lossy_matmul(W, x, training=True): outputs = tf.matmul(W, x) if training: outputs = tf.nn.dropout(outputs, keep_probability=0.2) return outputs W = tf.random.normal((3, 5)) x = tf.random.normal((5, 1)) # Executes a graph that applies dropout. lossy_outputs = lossy_matmul(W, x, training=True) # Executes a graph that does not apply dropout. exact_outputs = lossy_matmul(W, x, training=False) ``` _TensorFlow Control Flow_ When `autograph` is `True`, data-dependent control flow is allowed as well. Control flow statements that depend on `Tensor` values are staged into corresponding TensorFlow ops. For example, the following code will work as expected: ```python @tf.contrib.eager.defun def dynamic_rnn_loop(cell, seq): state, output = cell.zero_state() for input in seq: state, output = cell(input, state) return output ``` For more information see `tf.autograph`. _Variables_ TensorFlow operations related to variable creation and initialization are automatically lifted out of the graphs generated by `defun`. In practice, this implies that variable creation and initialization only happen the first time `F` is called, and that variables are reused every time thereafter. Many TensorFlow APIs, like `tf.keras.layers.Layer` objects, create variables the first time they are called and reuse them thereafter. Automatic variable lifting makes it possible to compile these APIs without extra effort, at the cost of introducing a discrepancy between the semantics of executing Python functions and their corresponding compiled functions. For example: ```python import tensorflow as tf tf.compat.v1.enable_eager_execution() def fn(): x = tf.Variable(0.0) x.assign_add(1.0) return x.read_value() # `fn` is a Python function, so x is created, initialized, and destroyed upon # every invocation assert fn().numpy() == fn().numpy() == 1.0 compiled = tf.contrib.eager.defun(fn) # Compiling `fn` with `defun` hoists all variables outside of the generated # graph, so initialization happens exactly once. assert compiled().numpy() == 1.0 assert compiled().numpy() == 2.0 ``` Finally, because each input signature is bound to a unique graph, if your Python function constructs `tf.Variable` objects, then each graph constructed for that Python function will reference a unique set of variables. To circumvent this problem, we recommend against compiling Python functions that create `tf.Variable` objects. Instead, Python functions should either lexically close over `tf.Variable` objects or accept them as arguments, preferably encapsulated in an object-oriented container. If you must create variables inside your Python function and you want each graph generated for it to reference the same set of variables, add logic to your Python function that ensures that variables are only created the first time it is called and are reused for every subsequent invocation; note that this is precisely what `tf.keras.layers.Layer` objects do, so we recommend using them to represent variable-bearing computations whenever possible. Args: func: function to be compiled. If `func` is None, returns a decorator that can be invoked with a single argument - `func`. The end result is equivalent to providing all the arguments up front. In other words, defun(input_signature=...)(func) is equivalent to defun(func, input_signature=...). The former allows the following use case: @tf.contrib.eager.defun(input_signature=...) def foo(...): ... input_signature: A possibly nested sequence of `tf.contrib.eager.TensorSpec` objects specifying the shapes and dtypes of the Tensors that will be supplied to this function. If `None`, a separate function is instantiated for each inferred input signature. If a signature is specified, every input to `func` must be a `Tensor`, and `func` cannot accept `**kwargs`. autograph: Whether `func` should be compiled before constructing the graph. See https://www.tensorflow.org/guide/autograph for more information. experimental_autograph_options: Experimental knobs (in the form of a tuple of tensorflow.autograph.Feature values) to control behavior when autograph=True. reduce_retracing: When True, `tf.function` uses `tf.types.experimental.TraceType` to trace supertypes of arguments to reduce the number of traces. Returns: If `func` is not None, returns a callable that will execute the compiled function (and return zero or more `tf.Tensor` objects). If `func` is None, returns a decorator that, when invoked with a single `func` argument, returns a callable equivalent to the case above. Raises: TypeError: If `input_signature` is neither `None` nor a sequence of `tf.contrib.eager.TensorSpec` objects. rrrxexperimental_autograph_optionsrz)defun_with_attributesrrArArBdefun sQrz+__internal__.function.defun_with_attributes)v1c s<durtfdd}|dur8||S|S)aCompiles a Python function into a callable TensorFlow graph. This function supports adding extra function attributes. See detailed documentation in defun(). Currently this is not exposed in public API since we don't expect user to directly use attributes, and attribute won't work by itself. This assumption might change in future. Args: func: function to be compiled. input_signature: same as defun()'s input_signature. attributes: A dictionary of arguments which will be added to function def as attributes. Currently only support primitive types as value, and only allowlisted attribute name is allowed. Unallowlisted attribute name or unsupported value will result into ValueError. `func_name` is also one of the allowlisted argument which is a python string, and sets the name for this `ConcreteFunction` in the graph. autograph: same as defun()'s autograph. experimental_autograph_options: same as defun()'s experimental_autograph_options. jit_compile: same as defun()'s jit_compile. reduce_retracing: same as defun()'s reduce_retracing experimental_follow_type_hints: see `tf.function`. Returns: Same as the return value of defun, with attributes added to the function in graph. NcsXzrd|j}n|j}Wnty4d}Yn0t|t||d S)N func_namefunction)rrrrxryr|rzrd)rrrr0make_decoratorrc)rrrrrxrrdrr|rzrArB decorated s(   z(defun_with_attributes..decorated)r) rrrrrxrr|rzrdrrArrBre s $ rc@s<eZdZdZdZddZeddZeddZd d Z d S) TfMethodTargetz:Binding target for methods replaced by function and defun.)weakrefself_target__weakrefself_func__cCs||_t||_dSrX)rrrrefr)rytargetZoriginal_python_functionrArArBrz szTfMethodTarget.__init__cCs|SrX)rr{rArArBr szTfMethodTarget.targetcCs |}t|r|S|jSdSrX)rr1isclassr)ryZ true_selfrArArB target_class s zTfMethodTarget.target_classcCs6|}t|rt|}||g|Ri|SrX)rr1ismethodroget_unbound_functionr)ryrr6 wrapped_fnrArArBr s  zTfMethodTarget.callN) rrrrrrzrrrrrArArArBr s  rcst|tjtj}tds.Jtds.bound_method_wrapper)rrxrrzr|)rrr types_lib MethodTyperwrrrLr0rrrgrriru)rr bound_methodrZ instance_funcZwrapped_instance_funcrArrBr s0     rc@s.eZdZdZdgZddZddZddZd S) rzGCleans up reference cycles when a `ConcreteFunction` goes out of scope.r cCs ||_dSrXr")ryrrArArBrz sz)ConcreteFunctionGarbageCollector.__init__cCs d|_dS)z Call off the FuncGraph deletion.Nr"r{rArArBr sz(ConcreteFunctionGarbageCollector.releasecCs@tdustdus|jdurdSzt|jWn Yn0dSrX)r=r-r dismantle_func_graphr{rArArBr s z(ConcreteFunctionGarbageCollector.__del__N)rrrrrrzrrrArArArBr s rc@s&eZdZdZdgZddZddZdS)r@z@Markers used to pretty-print nested args in function signatures._scCs ||_dSrX)r)ryrgrArArBrz% sz_Marker.__init__cCs t|jSrX)rmrr{rArArBrW( sz_Marker.__repr__N)rrrrrrzrWrArArArBr@ sr@cs0ddfddt|D}tt||S)z?Displays a summary of the nesting structure of the given value.cSs"t|tjr|jjSt|jSdSrX)r7rr< value_typerrLr?rArArB type_name/ s z%_structure_summary..type_namecsg|]}t|qSrA)r@)rZrrrArBr5 r^z&_structure_summary..)r.rarmr4)rHrGrArrBr, srcCstddt|DS)Ncss|]}t|tjVqdSrX)r7rr<r)rArArBr]: r^z&_contains_type_spec..)rbr.rarNrArArBr9 sr)F)NNTNF)NNNTNNFF)r collectionsrCrotypesrtypingrrrro six.movesrtensorflow.core.frameworkrrZ%tensorflow.core.function.polymorphismrtensorflow.pythonrtensorflow.python.clientrtensorflow.python.eagerr r r r r rrrrrZ&tensorflow.python.eager.graph_only_opsrtensorflow.python.frameworkrrrrrrr=rrrrrtensorflow.python.opsrr r!r"r#r$tensorflow.python.platformr%rtensorflow.python.profilerr&tensorflow.python.trackabler' trackabletensorflow.python.typesr(tensorflow.python.utilr)r*r+r,r-r.r/r0r1 tensorflow.python.util.tf_exportr2 LazyLoaderglobalsr3r4rrr SHARED_RENDEZVOUS_ATTRIBUTE_NAMECounterrrCrGrUrdrvrVrwrrrrrrrrrrrrrrr namedtuplerdrhrrrrr Trackable RegisterTyper8r8rPrcrrrrrrrrr@rrrArArArBs$                                                  ! cLH*W,^x Y  O9