a Sic*h@s>dZddlmZddlmZddlmZddlZddlZddlmm Z ddl m Z ddl mZddl mZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZdaedgdedgdejddZedgdedgdddZ ddZ!edgdedgdGdddej"Z"dd Z#dS)!z=Contains the base Layer class, from which all layers inherit.)absolute_import)division)print_functionN)backend)base_layer_utils) base_layer_v1)variable_scope_shim)policy) tf_contextlib)variable_scope) keras_export) tf_exportFz?keras.__internal__.legacy.layers.experimental.keras_style_scope)v1z%layers.experimental.keras_style_scopeccs"t}dazdVW|an|a0dS)aUUse Keras-style variable management. All tf.layers and tf RNN cells created in this scope use Keras-style variable management. Creating such layers with a scope= argument is disallowed, and reuse=True is disallowed. The purpose of this scope is to allow users of existing layers to slowly transition to a Keras layers API without breaking existing functionality. One example of this is when using TensorFlow's RNN classes with Keras Models or Networks. Because Keras models do not properly set variable scopes, users of RNNs may either accidentally share scopes between two different models, or get errors about variables that already exist. Example: ```python class RNNModel(tf.keras.Model): def __init__(self, name): super(RNNModel, self).__init__(name=name) self.rnn = tf.compat.v1.nn.rnn_cell.MultiRNNCell( [tf.compat.v1.nn.rnn_cell.LSTMCell(64) for _ in range(2)]) def call(self, input, state): return self.rnn(input, state) model_1 = RNNModel("model_1") model_2 = RNNModel("model_2") # OK output_1, next_state_1 = model_1(input, state) # Raises an error about trying to create an already existing variable. output_2, next_state_2 = model_2(input, state) ``` The solution is to wrap the model construction and execution in a keras-style scope: ```python with keras_style_scope(): model_1 = RNNModel("model_1") model_2 = RNNModel("model_2") # model_1 and model_2 are guaranteed to create their own variables. output_1, next_state_1 = model_1(input, state) output_2, next_state_2 = model_2(input, state) assert len(model_1.weights) > 0 assert len(model_2.weights) > 0 assert(model_1.weights != model_2.weights) ``` Yields: A keras layer style scope. TN_KERAS_STYLE_SCOPE)stackrW/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/legacy_tf_layers/base.pykeras_style_scope)s @rz=keras.__internal__.legacy.layers.experimental.set_keras_stylez#layers.experimental.set_keras_stylecCsdadS)aUse Keras-style variable management. All tf.layers and tf RNN cells created after keras style ha been enabled use Keras-style variable management. Creating such layers with a scope= argument is disallowed, and reuse=True is disallowed. The purpose of this function is to allow users of existing layers to slowly transition to Keras layers API without breaking existing functionality. For more details, see the documentation for `keras_style_scope`. Note, once keras style has been set, it is set globally for the entire program and cannot be unset. Example: ```python set_keras_style() model_1 = RNNModel(name="model_1") model_2 = RNNModel(name="model_2") # model_1 and model_2 are guaranteed to create their own variables. output_1, next_state_1 = model_1(input, state) output_2, next_state_2 = model_2(input, state) assert len(model_1.weights) > 0 assert len(model_2.weights) > 0 assert(model_1.weights != model_2.weights) ``` TNrrrrrset_keras_styleqs&rcCstSNrrrrr_is_in_keras_style_scopesrz&keras.__internal__.legacy.layers.Layerz layers.Layerc seZdZdZd"fdd ZddZedd Zd d Zd#ddZ eddZ d$fdd Z fddZ d%ddZ ddddddejjejjjjdf fdd ZfddZddZfddZed d!ZZS)&LayeraXBase layer class. It is considered legacy, and we recommend the use of `tf.keras.layers.Layer` instead. Args: trainable: Boolean, whether the layer's variables should be trainable. name: String name of the layer. dtype: Default dtype of the layer's weights (default of `None` means use the type of the first input). Read-only properties: name: The name of the layer (string). dtype: Default dtype of the layer's weights (default of `None` means use the type of the first input). trainable_variables: List of trainable variables. non_trainable_variables: List of non-trainable variables. variables: List of all variables of this layer, trainable and non-trainable. updates: List of update ops of this layer. losses: List of losses added by this layer. trainable_weights: List of variables to be included in backprop. non_trainable_weights: List of variables that should not be included in backprop. weights: The concatenation of the lists trainable_weights and non_trainable_weights (in this order). Mutable properties: trainable: Whether the layer should be trained (boolean). input_spec: Optional (list of) `InputSpec` object(s) specifying the constraints on inputs that can be accepted by the layer. TNc s d|_|dd}|dd|_g|_d|_|dur>td}d|vrNd|d<d|_tj f|||d|t r|durt d ||jdurt d |jd|_ nd|_ d |jjv|_|rtjj|}||_Wdn1s0Ynd|_d|_dS) NF_scope_reuse_inferautocastT) trainablenamedtypeKscope argument not allowed when keras style layers are enabled, but saw: {}zKreuse argument not allowed when keras style layers are enabled, but saw: {}scope)_use_resource_variablespopr_trainable_weightsbuiltr Policy_disable_keras_instrumentationsuper__init__r ValueErrorformat _keras_style _call_spec arg_names_call_has_scope_argtfcompatrr r_current_scope)selfrrrkwargsr!captured_scope __class__rrr)s@   &zLayer.__init__cOs||i|Srr)r3argsr4rrrapplysz Layer.applycCs"tjdddtrtddS)Nz`Layer.graph` is deprecated and will be removed in a future version. Please stop using this property because tf.layers layers no longer track their graph.) stacklevelz1Layer.graph not supported when executing eagerly.)warningswarnr0executing_eagerly RuntimeErrorr3rrrgraphsz Layer.graphcCsLt|tjjjr&|j}|\|_}n |}||_|sB|\|_}||_dSr) isinstancer0r1r VariableScoper_make_unique_name_name _base_name)r3r base_name_rrr_init_set_nameszLayer._init_set_nameFcCs*t|jj}tj|||||d}||fS)N) name_uid_map avoid_names namespace zero_based) base_layer to_snake_caser7__name__runique_object_name)r3rKrLrMrNrGrrrrrDszLayer._make_unique_namecCs,|js$td|jdddd|jjS)Nz5No name available for layer scope because the layer "z(" has not been used yet. The scope name z4 is determined the first time the layer instance is z1called. You must therefore call the layer before zquerying `scope_name`.)rr*rErr@rrr scope_name$szLayer.scope_namec st|j}t|j}tj||dts||j|d}|j|d}|D]}|}|durL||qLt|tj j j j dS)N)inputs) len_losses_callable_lossesr(add_lossr0r>append_add_elements_to_collectionr1r GraphKeysREGULARIZATION_LOSSES) r3lossesrTprevious_losses_lengthprevious_callable_losses_length new_lossesnew_callable_losses regularizer loss_tensorr6rrrX1s    zLayer.add_losscs|jrtS|jjS)z#Determines op naming for the Layer.)r,r( _name_scoper2original_name_scoper@r6rrrdCs zLayer._name_scopecCs|jdur|jrTtjj|dur$|n|j}||_Wdq1sH0Yn:tjjj||jd}||_Wdn1s0YdS)N) default_name)rrr0r1rr rF)r3r!r5rrr _set_scopeIs &zLayer._set_scopec  s| D]} | dkrtd| q|jr^tjf||||||o<|j||tjjtjj j j | d | S| tjj kr~|rxt dqd}n |durd}dd }d}tstjj }|jrt4tstjj }ttjj }Wdn1s0Yn|}ttjj }|dur.|jp,tj}|d|jpD|j}t|j}tjj j|j|dd |}||_t| D|p|j!p|j"}|dur|j#}tj||ft$|||o|j|| || | tjj j%d | }|r2tjj &s|||r2|'|||t()}t*|d r2|+|||dur|,tjj -}Wdn1sh0Y|r|jr||vr|j|d}|jd||_|j.|7_.Wdn1s0YWdn1s0Y|S) an Adds a new variable to the layer, or gets an existing one; returns it Args: name: variable name. shape: variable shape. dtype: The type of the variable. Defaults to `self.dtype` or `float32`. initializer: initializer instance (callable). regularizer: regularizer instance (callable). trainable: whether the variable should be part of the layer's "trainable_variables" (e.g. variables, biases) or "non_trainable_variables" (e.g. BatchNorm mean, stddev). Note, if the current variable scope is marked as non-trainable then this parameter is ignored and any added variables are also marked as non-trainable. `trainable` defaults to `True` unless `synchronization` is set to `ON_READ`. constraint: constraint instance (callable). use_resource: Whether to use `ResourceVariable`. synchronization: Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. If `synchronization` is set to `ON_READ`, `trainable` must not be set to `True`. aggregation: Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`. partitioner: (optional) partitioner instance (callable). If provided, when the requested variable is created it will be split into multiple partitions according to `partitioner`. In this case, an instance of `PartitionedVariable` is returned. Available partitioners include `tf.compat.v1.fixed_size_partitioner` and `tf.compat.v1.variable_axis_size_partitioner`. For more details, see the documentation of `tf.compat.v1.get_variable` and the "Variable Partitioners and Sharding" section of the API guide. **kwargs: Additional keyword arguments. Returns: The created variable. Usually either a `Variable` or `ResourceVariable` instance. If `partitioner` is not `None`, a `PartitionedVariable` instance is returned. Raises: RuntimeError: If called with partitioned variable regularization and eager execution is enabled. ValueError: When trainable has been set to True with synchronization set as `ON_READ`. experimental_autocastzUnknown keyword argument:) rshaper initializerrbr constraint use_resourcesynchronization aggregation partitionerzSynchronization value can be set to VariableSynchronization.ON_READ only for non-trainable variables. You have specified trainable=True and synchronization=VariableSynchronization.ON_READ.FNTcSs2t|r&|D]}||vrdSqdS||vSdS)NFT)ris_split_variable)variableexisting_variable_setvarrrr_should_add_regularizers  z1Layer.add_weight.._should_add_regularizerreuseauxiliary_name_scope) rrjrrkrorlrmrngetteradd_regularizer)/ TypeErrorr,r( add_weightrr0VariableSynchronizationAUTOr1rVariableAggregationNONEON_READr*r>get_default_graphbuilding_function init_scopesetglobal_variablesrfloat32rgr%rrUr$r rr2r name_scoperdr"rlrjas_dtype get_variable#executing_eagerly_outside_functions_handle_weight_regularizationvs_get_default_variable_storehasattrry as_defaulttrainable_variables_non_trainable_weights)r3rrirrjrbrrkrlrmrnror4kwargrt init_graph default_graphexisting_variablesrvprev_len_trainabler!rq var_storerextra_trainable_varsr6rrr{Ws?           $           ,NzLayer.add_weightc sx|dd}|jrB|dur(td|tj|g|Ri|S|||jrz |j}Wnt yrd}Yn0|durt j j j |jddd}t j j s||_nt j j j |j|jdd}|}||_z |j}Wn8t y t|j|j_d|jjv|_|j}Yn0|r||d<tj|g|Ri|}Wdn1sL0Yt stt|jt j j jj|S)aWraps `call`, applying pre- and post-processing steps. Args: inputs: input tensor(s). *args: additional positional arguments to be passed to `self.call`. **kwargs: additional keyword arguments to be passed to `self.call`. **Note**: kwarg `scope` is reserved for use by the layer. Returns: Output tensor(s). Note: - If the layer's `call` method takes a `scope` keyword argument, this argument will be automatically set to the current variable scope. - If the layer's `call` method takes a `mask` argument (as some Keras layers do), its default value will be set to the mask generated for `inputs` by the previous layer (if `input` did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support. Raises: ValueError: if the layer's `call` method returns None (an invalid value). r!Nr TFru)r#r,r*r+r(__call__rgr%_always_reuse_variable_scopeAttributeErrorr0r1rr rrrr2r/rfn_argscallr-r.r>rZupdatesr[ UPDATE_OPS)r3rTr8r4r!scope_context_managercall_has_scope_argoutputsr6rrrsT         : zLayer.__call__cCstgd}tddg}|j}||}||t|<|jD]f\}}||vr\t|||q>||vrxt||t|q>t |rt|||q>t||t ||q>|S)N)_graph _thread_local _metrics_lockrr) rr7__new__id__dict__itemssetattrcopyrOis_tensor_or_tensor_listdeepcopy)r3memono_copy shallow_copyclsresultkvrrr __deepcopy__ws     zLayer.__deepcopy__csttjjj|||dSr)r(r0 __internal__tracking Trackable __setattr__)r3valuerr6rrrszLayer.__setattr__cCsdS)zLUsed by keras to check compatibility. This should not be overridden.Trr@rrr_is_legacy_layerszLayer._is_legacy_layer)TNN)NNrJF)N)N)rQ __module__ __qualname____doc__r)r9propertyrArIrDrSrXrdrgr0r|r}r1rr~rr{rrrr __classcell__rrr6rrs@!0      H Y rcCsztrtd||ftj|}tj|}|D]@}tjj|}dd|D}|D]}t||vrX| |qXq4dS)NzPUsing collections from Layers not supported in Eager mode. Tried to add %s to %scSsh|] }t|qSr)r).0errr z._add_elements_to_collection..) r0r>r?nestflattenr1rget_collection_refrrY)elementscollection_listr collectioncollection_setelementrrrrZs   rZ)$r __future__rrrrr<Ztensorflow.compat.v2r1v2r0kerasr keras.enginerrrOZkeras.legacy_tf_layersrkeras.mixed_precisionr keras.utilsr tensorflow.python.opsr r tensorflow.python.util.tf_exportr r rcontextmanagerrrrrrZrrrrsF             C  %  v