a Sic1@sdZddlZddlmmZddlmZddlm Z ddl m Z ddl m Z ddlmZddlmZejjd d d Zed Gd dde ejdZddZdS)zEContains the base ProcessingLayer and a subclass that uses Combiners.N) data_adapter)Layer) version_utils)context) keras_export) doc_controlsz*/tensorflow/api/keras/layers/preprocessingz keras preprocessing layers usagemethodz:keras.layers.experimental.preprocessing.PreprocessingLayercseZdZdZdZfddZeddZej ddZ ej d d Z ej d d Z ej d dZ dddZdddZddZejjjddZddZZS)PreprocessingLayeraOBase class for Preprocessing Layers. **Don't use this class directly: it's an abstract base class!** You may be looking for one of the many built-in [preprocessing layers](https://keras.io/guides/preprocessing_layers/) instead. Preprocessing layers are layers whose state gets computed before model training starts. They do not get updated during training. Most preprocessing layers implement an `adapt()` method for state computation. The `PreprocessingLayer` class is the base class you would subclass to implement your own preprocessing layers. Tc s8tjfi|d|_d|_|j|_|j|_d|_dS)NF)super__init__ _is_compiled _is_adapted reset_state_reset_state_impl_reset_state_wrapper_adapt_function)selfkwargs __class__a/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/engine/base_preprocessing_layer.pyr 8s zPreprocessingLayer.__init__cCs|jS)z/Whether the layer has been fit to data already.)r rrrr is_adaptedCszPreprocessingLayer.is_adaptedcCstdS)zAccumulates statistics for the preprocessing layer. Arguments: data: A mini-batch of inputs to the layer. NNotImplementedError)rdatarrr update_stateHszPreprocessingLayer.update_statecCstdS)z1Resets the statistics of the preprocessing layer.NrrrrrrQszPreprocessingLayer.reset_statecCsdS)a3Finalize the statistics for the preprocessing layer. This method is called at the end of `adapt` or after restoring a serialized preprocessing layer's state. This method handles any one-time operations that should occur on the layer's state before `Layer.__call__`. Nrrrrrfinalize_stateVs z!PreprocessingLayer.finalize_statecs^jdurjSfddjdkr4}nfdd}jsRt|}|_jS)aCreates a function to execute one step of `adapt`. This method can be overridden to support custom adapt logic. This method is called by `PreprocessingLayer.adapt`. Typically, this method directly controls `tf.function` settings, and delegates the actual state update logic to `PreprocessingLayer.update_state`. This function is cached the first time `PreprocessingLayer.adapt` is called. The cache is cleared whenever `PreprocessingLayer.compile` is called. Returns: Function. The function created by this method should accept a `tf.data.Iterator`, retrieve a batch, and update the state of the layer. Ncs t|}||dSN)next_adapt_maybe_buildr)iteratorrrrr adapt_stepxs z:PreprocessingLayer.make_adapt_function..adapt_stepcstjD] }|q dSr)tfrange_steps_per_execution)r"_r#rrradapt_fnsz8PreprocessingLayer.make_adapt_function..adapt_fn)rr'numpyitem _run_eagerlyr%function)rr*rr)rmake_adapt_functionas   z&PreprocessingLayer.make_adapt_functionNcCs4|dur d}|||dur$|j}||_d|_dS)a`Configures the layer for `adapt`. Arguments: run_eagerly: Bool. Defaults to `False`. If `True`, this `Model`'s logic will not be wrapped in a `tf.function`. Recommended to leave this as `None` unless your `Model` cannot be run inside a `tf.function`. steps_per_execution: Int. Defaults to 1. The number of batches to run during each `tf.function` call. Running multiple batches inside a single `tf.function` call can greatly improve performance on TPUs or small models with a large Python overhead. Nr$T)_configure_steps_per_executiondynamicr-r )r run_eagerlysteps_per_executionrrrcompiles  zPreprocessingLayer.compilec Cstdtstd|js&||jr4|tj |||d|j dd}| |_ | D]V\}}|6|D]}| ||jrxtqxWdq^1s0Yq^|d|_dS)aFits the state of the preprocessing layer to the data being passed. After calling `adapt` on a layer, a preprocessing layer's state will not update during training. In order to make preprocessing layers efficient in any distribution context, they are kept constant with respect to any compiled `tf.Graph`s that call the layer. This does not affect the layer use when adapting each layer only once, but if you adapt a layer multiple times you will need to take care to re-compile any compiled functions as follows: * If you are adding a preprocessing layer to a `keras.Model`, you need to call `model.compile` after each subsequent call to `adapt`. * If you are calling a preprocessing layer inside `tf.data.Dataset.map`, you should call `map` again on the input `tf.data.Dataset` after each `adapt`. * If you are using a `tf.function` directly which calls a preprocessing layer, you need to call `tf.function` again on your callable after each subsequent call to `adapt`. `tf.keras.Model` example with multiple adapts: >>> layer = tf.keras.layers.Normalization( ... axis=None) >>> layer.adapt([0, 2]) >>> model = tf.keras.Sequential(layer) >>> model.predict([0, 1, 2]) array([-1., 0., 1.], dtype=float32) >>> layer.adapt([-1, 1]) >>> model.compile() # This is needed to re-compile model.predict! >>> model.predict([0, 1, 2]) array([0., 1., 2.], dtype=float32) `tf.data.Dataset` example with multiple adapts: >>> layer = tf.keras.layers.Normalization( ... axis=None) >>> layer.adapt([0, 2]) >>> input_ds = tf.data.Dataset.range(3) >>> normalized_ds = input_ds.map(layer) >>> list(normalized_ds.as_numpy_iterator()) [array([-1.], dtype=float32), array([0.], dtype=float32), array([1.], dtype=float32)] >>> layer.adapt([-1, 1]) >>> normalized_ds = input_ds.map(layer) # Re-map over the input dataset. >>> list(normalized_ds.as_numpy_iterator()) [array([0.], dtype=float32), array([1.], dtype=float32), array([2.], dtype=float32)] `adapt()` is meant only as a single machine utility to compute layer state. To analyze a dataset that cannot fit on a single machine, see [Tensorflow Transform]( https://www.tensorflow.org/tfx/transform/get_started) for a multi-machine, map-reduce solution. Arguments: data: The data to train on. It can be passed either as a tf.data Dataset, or as a numpy array. batch_size: Integer or `None`. Number of samples per state update. If unspecified, `batch_size` will default to 32. Do not specify the `batch_size` if your data is in the form of datasets, generators, or `keras.utils.Sequence` instances (since they generate batches). steps: Integer or `None`. Total number of steps (batches of samples) When training with input tensors such as TensorFlow data tensors, the default `None` is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. If x is a `tf.data` dataset, and 'steps' is None, the epoch will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the `steps` argument. This argument is not supported with array inputs. adaptz+`adapt` is only supported in tensorflow v2.r$F) batch_sizesteps_per_epochepochsr3 distributeNT)_disallow_inside_tf_functionr should_use_v2 RuntimeErrorr r4builtrr DataHandlerr'r/renumerate_epochscatch_stop_iterationsteps should_syncr async_waitrr )rrr6rA data_handlerr(r"rrrr5s0M    *zPreprocessingLayer.adaptcCs|d|_dS)z2Calls `reset_state` and sets `adapted` to `False`.FN)rr rrrrrsz'PreprocessingLayer._reset_state_wrappercCstj|dtjjd|_dS)Nint64)dtype aggregation)r%VariableVariableAggregationONLY_FIRST_REPLICAr')rr3rrrr0 s z1PreprocessingLayer._configure_steps_per_executioncCsn|jsjz|j}tdgt|j}Wnty>d}d}Yn0t|dd}|durZ||_||d|_dS)N_batch_input_shapeT)r=shapetuplelenAttributeErrorgetattrrKbuild)rr data_shapedata_shape_nonesbatch_input_shaperrrr!s    z%PreprocessingLayer._adapt_maybe_build)NN)NN)__name__ __module__ __qualname____doc___must_restore_from_configr propertyrrdo_not_generate_docsrrrr/r4r5rr% __internal__tracking no_automatic_dependency_trackingr0r! __classcell__rrrrr %s&      )  f r ) metaclasscCs trdj|d}t|dS)z1Disallow calling a method inside a `tf.function`.aDetected a call to `PreprocessingLayer.{method_name}` inside a `tf.function`. `PreprocessingLayer.{method_name} is a high-level endpoint that manages its own `tf.function`. Please move the call to `PreprocessingLayer.{method_name}` outside of all enclosing `tf.function`s. Note that you can call a `PreprocessingLayer` directly on `Tensor`s inside a `tf.function` like: `layer(x)`, or update its state like: `layer.update_state(x)`.) method_nameN)r%inside_functionformatr<)ra error_msgrrrr:+s  r:)rXabctensorflow.compat.v2compatv2r% keras.enginerkeras.engine.base_layerr keras.utilsrtensorflow.python.eagerr tensorflow.python.util.tf_exportrtensorflow.tools.docsrr\ monitoring BoolGaugeZkeras_kpl_gaugeABCMetar r:rrrrs$