a Sic @s~dZddlZddlZddlZddlZddlmmZddl m Z ddl m Z ddlmZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlm Z ddl!m"Z#ddl$m%Z%Gdddej&Z'ddZ(ddZ)ddZ*ddZ+ddZ,d d!Z-d"d#Z.d,d$d%Z/d-d&d'Z0d(d)Z1Gd*d+d+ej2Z3dS).zHA `Network` is way to compose layers: the topological form of a `Model`.N)backend) layout_map) base_layer)base_layer_utils)functional_utils) input_layer) input_spec)node)training)training_utils) json_utils)network_serialization) generic_utils) tf_inspect)tf_utils) tf_logging) doc_controlscseZdZdZeedejj Z e j j j dCfdd Ze j j j ddZed d Zed d Zed dZejddZeddZeddZddZeddZdDfdd ZfddZddZedd Zd!d"ZejdEd#d$Z d%d&Z!dFd'd(Z"dGd)d*Z#d+d,Z$d-d.Z%fd/d0Z&d1d2Z'd3d4Z(dHd5d6Z)d7d8Z*d9d:Z+d;d<Z,d=d>Z-ed?d@Z.dIfdAdB Z/Z0S)J Functionalak A `Functional` model is a `Model` defined as a directed graph of layers. Three types of `Model` exist: subclassed `Model`, `Functional` model, and `Sequential` (a special case of `Functional`). In general, more Keras features are supported with `Functional` than with subclassed `Model`s, specifically: - Model cloning (`keras.models.clone`) - Serialization (`model.get_config()/from_config`, `model.to_json()` - Whole-model saving (`model.save()`) A `Functional` model can be instantiated by passing two arguments to `__init__`. The first argument is the `keras.Input` Tensors that represent the inputs to the model. The second argument specifies the output tensors that represent the outputs of this model. Both arguments can be a nested structure of tensors. Example: ``` inputs = {'x1': keras.Input(shape=(10,)), 'x2': keras.Input(shape=(1,))} t = keras.layers.Dense(1, activation='relu')(inputs['x1']) outputs = keras.layers.Add()([t, inputs['x2']) model = keras.Model(inputs, outputs) ``` A `Functional` model constructed using the Functional API can also include raw TensorFlow functions, with the exception of functions that create Variables or assign ops. Example: ```python inputs = keras.Input(shape=(10,)) x = keras.layers.Dense(1)(inputs) outputs = tf.nn.relu(x) model = keras.Model(inputs, outputs) ``` A new `Functional` model can also be created by using the intermediate tensors. This enables you to quickly extract sub-components of the model. Example: ```python inputs = keras.Input(shape=(None, None, 3)) processed = keras.layers.RandomCrop(width=32, height=32)(inputs) conv = keras.layers.Conv2D(filters=2, kernel_size=3)(processed) pooling = keras.layers.GlobalAveragePooling2D()(conv) feature = keras.layers.Dense(10)(pooling) full_model = keras.Model(inputs, feature) backbone = keras.Model(processed, conv) activations = keras.Model(conv, feature) ``` Note that the `backbone` and `activations` models are not created with `keras.Input` objects, but with the tensors that are originated from `keras.Input` objects. Under the hood, the layers and weights will be shared across these models, so that user can train the `full_model`, and use `backbone` or `activations` to do feature extraction. The inputs and outputs of the model can be nested structures of tensors as well, and the created models are standard `Functional` model that support all the existing API. Args: inputs: List of input tensors (must be created via `tf.keras.Input()` or originated from `tf.keras.Input()`). outputs: List of output tensors. name: String, optional. Name of the model. trainable: Boolean, optional. If the model's variables should be trainable. )_layer_call_argspecs_compiled_trainable_state_output_mask_cache_output_tensor_cache_output_shape_cacheNTc sv|dd}|rdSt|itj||dtjjrft ddtj |Dsft ||\}}|||dS)N skip_initF)name trainablecSsg|]}t|qS)rZis_input_keras_tensor.0trrS/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/engine/functional.py sz'Functional.__init__..)poprvalidate_kwargssuper__init__tfcompatv1#executing_eagerly_outside_functionsallnestflattenrZclone_graph_nodes_init_graph_network)selfinputsoutputsrrkwargsr __class__rr r%s    zFunctional.__init__c Cstjddd|_t|tr>ttj |dkr>|d}t|trdttj |dkrd|d}||_ ||_ tj ||_ tj ||_tj |j sd|_n^t|j ttfrtdd|j Dsd|_n2t|j trtdd|j Dsd|_nd|_tjjs0td d|jDr0t|j |d|_tj d d ||_d|_d|j_ d|j_!d|_"g|_#g|_$g|_%g|_&i|_'i|_(i|_)|jD]0}|j*\}}}|j$+||j&+|||fq|j D]L}|j*\}}}|dksJ|dksJ|j#+||j%+|||fqt,|j |j\}}} } ||_-||_.| |_/i|_0|j/D]}t12|j3|j0|<qX|4g|_5g|_6g|_7g|_8|j#D]F}|j5+|j9|j:r|j6+|j9|j8+|j;|j7+|j<q|=|>|j t?@|jA|jBdurtCD||jBdS) NrTrcss|]}tj|VqdSNr&r+ is_nestedrrrr sz1Functional._init_graph_network..css|]}tj|VqdSr5r6rrrr r8sFcss|]}t|d VqdS)_keras_historyN)hasattrrtensorrrr r8scSs|jSr5)shapexrrr z0Functional._init_graph_network..)ErZkeras_api_gaugeget_cellset_is_graph_network isinstancelistlenr&r+r,_nested_inputs_nested_outputsr/r0r7_enable_dict_to_input_mappingtupleanydictvaluesr'r(r)rcreate_keras_history"_validate_graph_inputs_and_outputsbuilt map_structure_build_input_shape _compute_output_and_mask_jointly _call_specexpects_training_argexpects_mask_arg _autocast _input_layers_output_layers_input_coordinates_output_coordinatesrrrr9append_map_graph_network_network_nodes_nodes_by_depth_self_tracked_trackablesrrgetfullargspeccall_set_output_names input_names_feed_input_names _feed_inputs_feed_input_shapesris_placeholder_batch_input_shapeinput_compute_tensor_usage_count_set_save_specrassert_no_legacy_layerslayersZ _layout_maplayout_map_libZ_map_functional_model_variable) r.r/r0r?layer node_index tensor_indexnodesnodes_by_depthro_rrr r-s            zFunctional._init_graph_networkcCs|jS)afRetrieves the input tensor(s) of a layer. Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer. Returns: Input tensor or list of input tensors. Raises: RuntimeError: If called in Eager mode. AttributeError: If no inbound nodes are found. )rHr.rrr rk3szFunctional.inputcCstjtj|jS)aRetrieves the input shape(s) of a layer. Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape. Returns: Input shape, as an integer shape tuple (or list of shape tuples, one tuple per input tensor). Raises: AttributeError: if the layer has no defined input_shape. RuntimeError: if called in Eager mode. )r&r+rRr int_shaperkrwrrr input_shapeCszFunctional.input_shapecsztdrjStjtttfr:tjtjkr:dStjtrft j }fdd|DSddjDSdS)N_manual_input_speccs&g|]}tjtj|d|dqST)r=allow_last_axis_squeezer)r InputSpecshape_with_no_batch_sizerH)rrrwrr r!bs  z)Functional.input_spec..cSs&g|]}tjt|d|jjjdqSr{)rr}r~r9rqrrr?rrr r!ms ) r:rzrErHrMrFrKrGr/sortedkeys)r.namesrrwr rUs     zFunctional.input_speccCs ||_dSr5)rz)r.valuerrr rvscCs|jS)aRetrieves the output tensor(s) of a layer. Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer. Returns: Output tensor or list of output tensors. Raises: AttributeError: if the layer is connected to more than one incoming layers. RuntimeError: if called in Eager mode. )rIrwrrr outputzszFunctional.outputcCstjtj|jS)aRetrieves the output shape(s) of a layer. Only applicable if the layer has one output, or if all outputs have the same shape. Returns: Output shape, as an integer shape tuple (or list of shape tuples, one tuple per output tensor). Raises: AttributeError: if the layer has no defined output shape. RuntimeError: if called in Eager mode. )r&r+rRrrxrrwrrr output_shapeszFunctional.output_shapecCsrg}t}i}|jD]R}|j}||vrR||jd}d|j|}|d||j<q||||q||_dS)zAssigns unique names to the Network's outputs. Output layers with multiple output tensors would otherwise lead to duplicate names in self.output_names. r4z{}_{}N)rCrZrgetformataddr] output_names)r. uniquifiedr prefix_countrqproposalexisting_countrrr rds   zFunctional._set_output_namesc Csbd}t}t|jD]F\}}z|jr:||d|<|d7}WntyNYn0||d|<q|S)zBDictionary of layer dependencies to be included in the checkpoint.rzlayer_with_weights-%dr4zlayer-%d) collections OrderedDict enumerateroweights ValueError)r.weight_layer_index dependencies layer_indexrqrrr _layer_checkpoint_dependenciess  z)Functional._layer_checkpoint_dependencies checkpointc s$|j}|tj|fi||Sr5)rupdater$_trackable_children)r. save_typer1rr2rr rszFunctional._trackable_childrencs"|j}||vr||St|Sr5)rr$_lookup_dependency)r.rlayer_dependenciesr2rr rszFunctional._lookup_dependencycCs@|j}dd|D}|D]}||vr|j|||dqdS)z@Handles layer checkpoint dependencies that are added after init.cSsi|]\}}||qSrr)rkvrrr rAzBFunctional._handle_deferred_layer_dependencies..)r trackableN)ritems_handle_deferred_dependencies)r.rolayer_checkpoint_dependencies layer_to_namerqrrr #_handle_deferred_layer_dependenciessz.Functional._handle_deferred_layer_dependenciescCsdS)NTrrwrrr _should_compute_maskszFunctional._should_compute_maskcCs |j||d}tjdd|S)N)maskcSs t|ddS)N _keras_mask)getattr)rrrr r@rAz)Functional.compute_mask..)_run_internal_graphr&r+rR)r.r/routput_tensorsrrr compute_maskszFunctional.compute_maskcCs|j|||dS)aCalls the model on new inputs. In this case `call` just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs). Args: inputs: A tensor or list of tensors. training: Boolean or boolean scalar tensor, indicating whether to run the `Network` in training mode or inference mode. mask: A mask or list of masks. A mask can be either a tensor or None (no mask). Returns: A tensor if there is a single output, or a list of tensors if there are more than one outputs. )r r)r)r.r/r rrrr rcszFunctional.callcCsFtj|dd}ttj|ttj|jkrLtd|dt|jdz,ttj|dd}||j vrv|j |WSWntyYn0i}t |jtj|D]\}}|j d}|||<qt |j }|jddt|d krB|D]}|j |} | D]} | j}||jvrqg} | jd } tj| D]2} | j}|jj d |j|jf}| ||q0tj| | } tj| dd} || }tj|dd}|j| }ttj|D]$\}}|j d ||f}|||<qqqg}tt|jD]6}|j|\}}}|j d ||f}|||qtj|j|}||j |<|S) NF) to_tupleszInvalid `input_shape` argument z: the model expects z input tensors.T_0_0reverser4rz_%s_%s)rconvert_shapesrGr&r+r,rYrrKrziprrFr`rsortrq call_argsr9rrrsr]pack_sequence_ascompute_output_shape_inbound_nodesindexrrangerZr\rI)r.ry cache_keylayers_to_output_shapesrqr= shape_key depth_keysdepthrtr layer_input_shapes layer_inputs layer_inputkhinput_layer_keylayer_output_shapesrrj output_shapesirsrrr rs              zFunctional.compute_output_shapecCs<|s2|jj}|jtkrd}tjt||d|_n||_dS)NModel) zero_based)r3__name__rrunique_object_namer to_snake_case_name)r.rrcls_namerrr _init_set_namebs  zFunctional._init_set_namecs||}|dur"dgt|}n ||}t||D]\}}||_q6i|j}t|j|D]4\}}|j||d}tt|} |g|| | <q\|j } t | } | j dd| D]} | | } | D]v}|j rqtfdd|jDrq|\}}|j|i|}t|jtj|D]\} }|g|| | <qqqg}|jD]>}tt|} | vsrJdt||| qHtj|j|S)aYComputes output tensors for new inputs. # Note: - Can be run on non-Keras tensors. Args: inputs: Tensor or nested structure of Tensors. training: Boolean learning phase. mask: (Optional) Tensor or nested structure of Tensors. Returns: output_tensors N) ref_inputTrc3s|]}|vVqdSr5r)rt_id tensor_dictrr r8rAz1Functional._run_internal_graph..zCould not compute output )_flatten_to_reference_inputsrGrr_tensor_usage_countr/_conform_to_reference_inputstridr`rFrris_inputrLflat_input_ids map_argumentsrqflat_output_idsr&r+r,r0r]r"rrI)r.r/r rmasksinput_ttensor_usage_countr?yx_idrurrrtr argsr1r0rrrr rosD        zFunctional._run_internal_graphcs|jrttr|j}tj|s*|jg}t|trBt|ndd|Dt t krt j d fddDddzfddDWSt ytjYS0tjS)z1Maps `tensors` to their respective `keras.Input`.cSsg|]}|jjjqSr)r9rqr)rinprrr r!sz;Functional._flatten_to_reference_inputs..zdInput dict contained keys {} which did not match any model input. They will be ignored by the model.csg|]}|vr|qSrrrn)ref_input_namesrr r!rA) stacklevelcsg|] }|qSrrr)tensorsrr r!rA)rJrErMrHr&r+r7rrrGwarningswarnrKeyErrorr,)r.r ref_inputsr)rrr rs*   z'Functional._flatten_to_reference_inputsc CsZt|tjr|j}|j}|j}|j}t|dd}|dur|dur||dkrh|ddkrhtj|dd}n&||dkr|ddkrtj|dd}|dur||_t sz| |j |jWn*t yt d|j||jYn0tj||jd}n\t|rVt|dd}t|dd} | durV|durV|| krV| tjkrVtj|| d}|S) z,Set shape and dtype based on `keras.Input`s.r9Nr4)axiszkModel was constructed with shape {} for input {}, but it was called on an input with incompatible shape {}.dtyper)rEr&Tensorr=rankrsqueeze expand_dimsr9executing_eagerly set_shape merge_withrloggingwarningrcastrris_extension_typevariant) r.r<rt_shapet_rank ref_shaperef_rank keras_history tensor_dtyperef_input_dtyperrr rsH        z&Functional._conform_to_reference_inputcst}tt||dS)N)config)r$ get_configcopydeepcopyget_network_config)r.rr2rr r s zFunctional.get_configcCsni}|jD]^}tjj|\}}|D]>}t|tjr(||}ddd|D}|||j d|<q(q |S)N.cSsg|] }|jqSrrrrrr r!rAz/Functional.get_weight_paths..) ror& __internal__trackingObjectGraphViewbreadth_first_traversalrEVariablejoinr)r.resultrq descendantsZobject_paths_dict descendantZtrackable_references object_pathrrr get_weight_pathss    zFunctional.get_weight_pathsc Cs*tdd|jDt|jkr.td|j|jD]}t|ds`|jj}td|d|d|jj}t|jdks|jr4|jd j s4|jj}t |d |j d |j d |q4t d d|jD}|dt|dkrt dt||jD]*}t|ds|jj}td|d|qdS)z4Validates the inputs and outputs of a Graph Network.cSsh|] }t|qSr)r)rrrrr $rAz@Functional._validate_graph_inputs_and_outputs..zThe list of inputs passed to the model contains the same input multiple times. All inputs should only appear once.Received inputs=r9zInput tensors to a z8 model must come from `tf.keras.Input`. Received inputs=z# (missing previous layer metadata).r4rz model inputs must come from `tf.keras.Input` (thus holding past layer metadata). They cannot be the output of a previous non-Input layer. Here, a tensor specified as input to "z6" was not an Input tensor, it was generated by layer "zx". Note that input tensors are instantiated via `tensor = tf.keras.Input(shape)`. The tensor that caused the issue was: cSsg|]}t|jjqSr)r get_static_batch_sizer9rqrrrr r!KszAFunctional._validate_graph_inputs_and_outputs..NzEFound incompatible static batch sizes among the inputs. Batch sizes: zOutput tensors of a z] model must be the output of a TensorFlow `Layer` (thus holding past layer metadata). Found: )rGr/rr:r3rr9rqrrrrrrCdiscardrr0)r.r?rrqinput_batch_sizesrrr rP!sd        z-Functional._validate_graph_inputs_and_outputsc stj|}t|i|jD] \}fdd|Dq$|s`tjdd|D}t|t  fdd}t |}d}|r|d7}|d krt d | d}||d ur||qt|jj|jj|}||jvr|<|j||j|qt|j} g} |D]B} | | vr.|j| | | t| j|j| <| | q.|| |d S) a Inserts Layers into the Network after Network creation. This is only valid for Keras Graph Networks. Layers added via this function will be included in the `call` computation and `get_config` of this Network. They will not be added to the Network's outputs. Args: layers: Arbitrary nested structure of Layers. Layers must be reachable from one or more of the `keras.Input` Tensors that correspond to this Network's inputs. relevant_nodes: Nodes from the Layers that should be considered part of this Network. If `None`, all Nodes will be considered part of this Network. Raises: ValueError: If the layers depend on `Input`s not found in this Model. csi|] }|qSrrrr )rrr rvrAz-Functional._insert_layers..cSsg|] }|jqSr)rrrqrrr r!{rAz-Functional._insert_layers..csVd}|D]@\}}}}|j|}|vr:t||}q |vrFq q dSq |dS)z5Gets the minimum depth at which node can be computed.rNr4)iterate_inboundrmin)r min_depthrqnode_idrv inbound_node) network_nodes node_to_depthrr _get_min_depths z1Functional._insert_layers.._get_min_depthrr4i'z6Layers could not be added due to missing dependencies.N)r&r+r,rrnr`rrrCrFrrrr"r]_make_node_keyrqrrrr_rrarrbrcrrrl) r.rorelevant_nodesrtr$unprocessed_nodesrr node_key layer_setdeferred_layersrqr)rr"r#r _insert_layers`sV              zFunctional._insert_layersc Cst}tdd|jD}t|j}|jdd|dd}|D]}|j|D]x}ddtj |j D}| |rTtj |j D]}|t t|d7<qtj |jD]}|t t|qqTqF|jD]}|t t|d7<q||_dS) a%Compute the #. of tensor usages for all the output tensors of layers. The computed tensor usage count is saved as `self._tensor_usage_count`. This is later used for saving memory in eager computation by releasing no-longer-needed tensors as early as possible. css|]}tt|VqdSr5rrr;rrr r8rAz9Functional._compute_tensor_usage_count..Trr4NcSsh|]}tt|qSrr,r;rrr rsz9Functional._compute_tensor_usage_count..)rCounterrCr/rFr`rrr&r+r, keras_inputsissubsetrrr0rr) r.ravailable_tensorsrrr input_tensorsr< output_tensorrrr rls$     z&Functional._compute_tensor_usage_countcCsdSr5rrwrrr _assert_weights_createdsz"Functional._assert_weights_createdcCsPt|j|g\}}tjd|jd}||||j|||||dS)NF) unconditionalr) _map_subgraph_networkr/rAddLossrextend inbound_nodesr]r+)r. symbolic_loss new_nodes new_layersadd_loss_layerrrr _graph_network_add_losss  z"Functional._graph_network_add_losscCsRt|j|g\}}tj|||jd}||||j|||||dS)Nr) r5r/r AddMetricrr7r8r]r+)r.r aggregationrr:r;add_metric_layerrrr _graph_network_add_metrics  z$Functional._graph_network_add_metriccCs t|Sr5)r NetworkSavedModelSaverrwrrr _trackable_saved_model_saversz'Functional._trackable_saved_model_savercst|ddrd}t||S)N_has_explicit_input_shapeTF)rr$_get_save_spec)r. dynamic_batchZ inputs_onlyr2rr rEs zFunctional._get_save_spec)NT)r)NN)T)NN)N)TT)1r __module__ __qualname____doc__ frozenset itertoolschain training_libr_TF_MODULE_IGNORED_PROPERTIESr&r r  no_automatic_dependency_trackingr%r-propertyrkryrsetterrrrdrrrrrrrdo_not_doc_inheritablercrrrrrrrrPr+rlr3r=rArCrE __classcell__rrr2r r/sfO               b <(7 ? V   rcCs|dt|S)Nz_ib-)r) layer_namerrrrr r%sr%c st|\}dd|D}i}i}t|D]d}||d}||jd}t||}|||j<|||<|jD]"} || d}t|d||| <qjq*|D]F} | jd} | |vrd|| <d| <d|| jd<| t | j dqt t} |D]\}}| ||qt t} |D]\}}| ||qt| }|jddg}|D],}| |}|jfdd d ||qRt| }|jddt}|D]}| t|qg}|D]}| |D]}|j}|r|jstj|jD]0}t||vrtd |d |j d |qtj|jD]}| t|q2||j q̐qdd|D}|D]2}||dkrntd|d||dqn|| || fS)a5Validates a network's topology and gather its layers and nodes. Args: inputs: List of input tensors. outputs: List of outputs tensors. Returns: A tuple `(nodes, nodes_by_depth, layers, layers_by_depth)`. - nodes: list of Node instances. - nodes_by_depth: dict mapping ints (depth) to lists of node instances. - layers: list of Layer instances. - layers_by_depth: dict mapping ints (depth) to lists of layer instances. Raises: ValueError: In case the network is not valid (e.g. disconnected graph). cSs$h|]}t|jj|jj|qSr)r%rqrrrrrrr rsz%_map_graph_network..rr4rTrcs|Sr5rr> layer_indicesrr r@SrAz$_map_graph_network..)keyz3Graph disconnected: cannot obtain value for tensor z at layer "z>". The following previous layers were accessed without issue: cSsg|] }|jqSrr rrrr r!trAz&_map_graph_network..z The name "z " is used z6 times in the model. All layer names should be unique.) _build_mapreversed setdefaultrrqmax parent_nodesr9rrr%rr defaultdictrFrr]rrr7rCrrr&r+r,r.rr0count)r/r0nodes_in_decreasing_depthr" nodes_depths layers_depthsr rprevious_depthnode_deprrrulayers_by_depthrqrrolayers_for_depthcomputable_tensorsr?layers_with_complete_input all_namesrrrUr r^s               r^cCs>t}t}g}i}tj|D]}t|||||q ||fS)aThis method topologically sorts nodes in order from inputs to outputs. It uses a depth-first search to topologically sort nodes that appear in the _keras_history connectivity metadata of `outputs`. Args: outputs: the output tensors whose _keras_history metadata should be walked. This may be an arbitrary nested structure. Returns: A tuple like (ordered_nodes, layer_to_first_traversal_index) ordered_nodes: list of nodes appearing in the keras history, topologically sorted from original inputs to the `outputs`. (If outputs have different sets of ancestors, the inputs to one output may appear after a different output). layer_to_first_traversal_index: A dict mapping layer to the traversal index in the DFS where it is seen. Note: if a layer is shared by several nodes, the dict will only store the index corresponding to the *first* time the layer seen. )rCr&r+r,_build_map_helper)r0finished_nodesnodes_in_progressr_rVrrrr rX~srXc Cs|j\}}}|j|}||vr"dS||vrBtd|d|jd||vrVt|||<|||js|jD]}t|||||ql||| || |dS)z"Recursive helper for `_build_map`.NzTensor z from layer "z" is part of a cycle.) r9rrrrGrrr.riremover]) r<rjrkr_rVrqrrrvr rrr ris6       ricCsFtjjst|t||\}}}}tjdd| D|fS)zReturns the nodes and layers in the topology from `inputs` to `outputs`. Args: inputs: List of input tensors. outputs: List of output tensors. Returns: A tuple of List{Node] and List[Layer]. cSsg|]}|qSrr)rrtrrr r!rAz)_map_subgraph_network..) r&r'r(r)rrOr^r+r,rN)r/r0rvrurorrr r5s  r5cCs0|jr"t|to t|jdtjSt|tSdS)zCReturns True if the first layer node should not be saved or loaded.rN)rarErinput_layer_module InputLayer)rqrrr _should_skip_first_nodes  rocsDfdd|D}|r@tjdd|D}||S)z?Adds layers that are not connected to the outputs to the model.csg|]}|jvr|qSr)rormodelrr r!sz,connect_ancillary_layers..cSs(g|] }t|r|jddn|jqS)r4N)ror8rrrr r!s)rNr&r+r,r+)rqcreated_layersancillary_layersr&rrpr connect_ancillary_layerss  rtcsp tiittfddfddfdd}fdd}|d D] }||qdr|d D]P}|d }|vr~|}|r~|d }|||r|d q||<q~qq~qrg} g} tj|d d d} tj | D]T}| \} } }| vsJ| }|| } |j | j }| tj ||qtj|dd d}tj |D]V}| \} } }| vsJ| }|| } |j | j }| tj ||qltj| | } tj|| } | | fS)aReconstructs graph from config object. Args: config: Dictionary returned from Network.get_config() custom_objects: Optional dictionary mapping names (strings) to custom classes or functions to be considered during deserialization. created_layers: Optional dictionary mapping names to Layer objects. Any layer not in this dictionary will be created and added to the dict. This function will add new nodes to all layers (excluding InputLayers), instead of re-using pre-existing nodes in the layers. Returns: Tuple of (input tensors, output tensors, dictionary of created layers) cs"t|tjrdS|j|fdS)zBReturns node index in layer (might differ from config_node_index).rN)rErmrnrr)rqconfig_node_index)node_index_maprr get_node_index$s z/reconstruct_from_config..get_node_indexcs*fdd}tj|dd}tj||S)z-Deserializes Keras Tensors passed to `call`..csjt|tjrf|}|d}|d}|d}|}||}|durJt|j|}tj|j |S|S)z4Deserializes a single Keras Tensor passed to `call`.rr4rN) rEr ListWrapperas_list IndexErrorrr&r+r,r0)rrTrrrsrqnew_node_indexr )rw layer_maprr _deserialize_keras_tensor-s    z^reconstruct_from_config.._deserialize_keras_tensors.._deserialize_keras_tensorTwrap)rconvert_inner_node_datar&r+rR)r1r|r})rw)r|r _deserialize_keras_tensors*sz;reconstruct_from_config.._deserialize_keras_tensorsc sg}tj|D]@}|}t|dkr0i}nDt|dkrl|d}z|}WqttyhYdS0ntd|dtjkr|d}|d}|d}|}||}|durdS|j |} | tj| j |q|dd ksJt|dksJ|d} t | t rHt| dkrH| dtjkrH| t| dq| | qtj||}|dur|js~t|}||fi|} tj| djj} | |j|f<|d7<d S) alDeserialize a node. Args: layer: layer instance. node_data: Nested structure of `ListWrapper`. Returns: Whether the node was processed (i.e. the layer was called on the inputs specified by the node data) Raises: ValueError: In case of improperly formatted `node_data`. Fz"Improperly formatted model config.rr4rNrT)r&r+r,ryrGrzr node_module_CONSTANT_VALUErr]r0rErKZ_COMPOSITE_TYPEr decoder#_preserve_input_structure_in_configrunnest_if_single_tensorr9rrr) rq node_datar1 input_datar1inbound_layer_nameinbound_node_indexinbound_tensor_index inbound_layerr!Z const_valr output_index)rrrrwnode_count_by_layerrvrr process_nodeHsj           z-reconstruct_from_config..process_nodecs||d}|vr|}n ddlm}||d}||<tt||<|d}tj|dd}|D]}||qddS) zDeserializes a layer, then call it on appropriate inputs. Args: layer_data: layer config dict. Raises: ValueError: In case of improperly formatted `layer_data` dict. rr) deserialize)custom_objectsr8Tr~N) keras.layersrintrorrr]) layer_datarTrqdeserialize_layerinbound_nodes_datar)rrrrr'rr process_layers    z.reconstruct_from_config..process_layerrorr input_layersTr~ output_layers)rrr]rFr"rrr&r+r,ryrrr]r)rrrrrrrrqZ layer_nodesrr1rrrTrrrslayer_output_tensorsrr)rrrrrwrrvr'r reconstruct_from_configsd    Q#           rcCsb|ptj}|pi}|j|d<i}|jD]N}t|r6dnd}t|jD].\}}t|j|}||jvrD|||<|d7}qDq&g} t |jD]r}g} t|jD]:\}}t|j|}||jvr|j s| t|} | | q||} |j| d<| | d<| | q| |d<Wdn1s0Yg} t t|jD]R}|j|\}}}t|j|}||jvrhq8||}| t|j||gq8tj|j| } tj| s| g} t| } | |d<g}t t|jD]R}|j|\}}}t|j|}||jvrq||}| t|j||gqtj|j|}tj|sL|g}t|}||d<|S) aQBuilds the config, which consists of the node graph and serialized layers. Args: network: A Network object. serialize_layer_fn: Function used to serialize layers. config: A dict to append more config entries into. If None, start with a new dict for the config. Returns: Config dictionary. rr4rr8roNrr)rserialize_keras_objectrrororrr%r_SharedObjectSavingScoper serializer]rrGrYr[rrxr&r+rrHr7rrZr\rI)networkserialize_layer_fnrnode_conversion_maprq kept_nodesoriginal_node_indexr r( layer_configsfiltered_inbound_nodesr layer_config model_inputsrrrrsr{ model_outputsrrr r s~            (      r cCs*|jjdurdS|j}|r&d|d<|S)Nr)r=rry)r?r=rrr r~Js   r~cs*eZdZdZdfdd ZddZZS) ModuleWrapperzFWrapper for `tf.Module`s to support the Functional and Sequential API.Nc stjfi||dur8t|dr*d}nt|dr8d}|dusJt||sZtd||||_||_t||}t |}d|j vp|j du|j _ d|j vp|j du|j _dS)aInitializes the wrapper Layer for this module. Args: module: The `tf.Module` instance to be wrapped. method_name: (Optional) str. The name of the method to use as the forward pass of the module. If not set, defaults to '__call__' if defined, or 'call'. **kwargs: Additional keywrod arguments. See `tf.keras.layers.Layer`. Raises: ValueError: If `method` is not defined on `module`. N__call__rcz{} is not defined on object {}r r)r$r%r:rr_module _method_namerrrbrvarkwrUrVrW)r.module method_namer1methodmethod_arg_specr2rr r%Vs&       zModuleWrapper.__init__cOsHd|vr|js|dd|vr0|js0|dt|j|j|i|S)Nr r)_expects_training_argr"_expects_mask_argrrr)r.rr1rrr rc|s   zModuleWrapper.call)N)rrGrHrIr%rcrSrrr2r rSs&r)NN)NN)4rIrrrKrZtensorflow.compat.v2r'v2r&kerasr keras.dtensorrrp keras.enginerrrrrmrr rr rMr Zkeras.saving.saved_modelr r keras.utilsrrrtensorflow.python.platformrrtensorflow.tools.docsrrrr%r^rXrir5rortrr r~Layerrrrrr sT                 W{$. o W