a Sic}@s$dZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m m ZddlmZddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lm Z ddl!m"Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)z ddl*Z*Wne+y6dZ*Yn0ddddddde j,fddZ-dddddde j,fddZ.ddZ/dQddZ0e'dGdd d Z1e'd!Gd"d#d#Z2e'd$Gd%d&d&e2Z3e'd'Gd(d)d)e2Z4e'd*Gd+d,d,e2Z5e'd-Gd.d/d/e2Z6e'd0Gd1d2d2e2Z7e'd3gd4Gd5d6d6e2Z8e'd7gd4e%9d7Gd8d9d9e8Z:e'd:Gd;d<dd?d?e2Ze'dEgd4GdFdGdGe2ej?Z@e'dHGdIdJdJe2ZAe'dKGdLdMdMe2ZBe'dNGdOdPdPe2ZCdS)SzDCallbacks: utilities called at certain points during model training.N)backend)distributed_file_utils)worker_training_state)learning_rate_schedule) generic_utils)io_utils)tf_utils) version_utils)Sequence)Progbar)ModeKeys) tf_logging) deprecation) keras_export) doc_controlsFstepsc Cst|tr|S|sg}| tjkrRt|_tg|p4g|jg}|rR|t|t|} | } | | t | |||||||| d d| j _ | S)aConfigures callbacks for use in various training loops. Args: callbacks: List of Callbacks. model: Model being trained. do_validation: Whether or not validation loop will be run. batch_size: Number of samples per batch. epochs: Number of epoch to train. steps_per_epoch: Number of batches to run per training epoch. samples: Number of training samples. verbose: int, 0 or 1. Keras logging verbosity to pass to ProgbarLogger. count_mode: One of 'steps' or 'samples'. Per-batch or per-sample count. mode: String. One of ModeKeys.TRAIN, ModeKeys.TEST, or ModeKeys.PREDICT. Which loop mode to configure callbacks for. Returns: Instance of CallbackList used to control all Callbacks. ) do_validation batch_sizeepochssteps_per_epochsamplesverbosemodeF) isinstance CallbackListr TRAINHistoryhistory BaseLoggerappend ProgbarLogger_get_callback_model set_modelset_callback_parametersmodel stop_training) callbacksr%rrrrrr count_moder callback_listcallback_modelr+K/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/callbacks.pyconfigure_callbacks8s2    r-c Cs||j} |D] } t| ttfr | dd| _q g} |tjkrZt| } |rZ| dd| D7} ||||||| d} || dS)acSets callback parameters. Args: callback_list: CallbackList instance. model: Model being trained. do_validation: Whether or not validation loop will be run. batch_size: Number of samples per batch. epochs: Number of epoch to train. steps_per_epoch: Number of batches to run per training epoch. samples: Number of training samples. verbose: int, 0 or 1. Keras logging verbosity to pass to ProgbarLogger. mode: String. One of ModeKeys.TRAIN, ModeKeys.TEST, or ModeKeys.PREDICT. Which loop mode to configure callbacks for. rNcSsg|] }d|qSval_r+).0nr+r+r, z+set_callback_parameters..)rrrrrrmetrics) metrics_namesrrr!stateful_metricsr PREDICTcopy set_params) r)r%rrrrrrr metric_namescbkcallback_metricscallback_paramsr+r+r,r$ys$   r$cCs0t|dp.t|dp.t|ttjjjjtjjfS)z5Checks if data is a generator, Sequence, or Iterator.__next__next)hasattrrr tfcompatv1dataIterator)rDr+r+r,_is_generator_likes rFcCsH|j}|tjtjhvr<|rd?Z!dS)Srz*Container abstracting a list of callbacks.NFcKs|rtj|ng|_||||r0|||r>||tdd|jD|_tdd|jD|_ t dd|jD|_ t dd|jD|_ t dd|jD|_ |t dd|jD|_d|_i|_d |_g|_d S) aContainer for `Callback` instances. This object wraps a list of `Callback` instances, making it possible to call them all at once via a single endpoint (e.g. `callback_list.on_epoch_end(...)`). Args: callbacks: List of `Callback` instances. add_history: Whether a `History` callback should be added, if one does not already exist in the `callbacks` list. add_progbar: Whether a `ProgbarLogger` callback should be added, if one does not already exist in the `callbacks` list. model: The `Model` these callbacks are used with. **params: If provided, parameters will be passed to each `Callback` via `Callback.set_params`. css|]}t|ddVqdS_supports_tf_logsFN)getattrr0cbr+r+r, sz(CallbackList.__init__..css2|]*}|s|s|rt|ddVqdSrP)_implements_train_batch_hooks_implements_test_batch_hooks_implements_predict_batch_hooksrRrSr+r+r,rUs css|]}|VqdSN)rVrSr+r+r,rUscss|]}|VqdSrY)rWrSr+r+r,rUscss|]}|VqdSrY)rXrSr+r+r,rUscss|]}|jjtvVqdSrY) __class____name__globals)r0r;r+r+r,rUsN)rAnestflattenr'_add_default_callbacksr#r9allrQ_batch_hooks_support_tf_logsany_should_call_train_batch_hooks_should_call_test_batch_hooks _should_call_predict_batch_hooks$_disallow_batch_hooks_in_ps_strategy _check_timing_num_batches_for_timing_check _hook_times_batch_start_time _batch_times)selfr' add_history add_progbarr%paramsr+r+r,__init__s:         zCallbackList.__init__cCsd|_d|_|jD]&}t|tr(||_qt|tr||_q|jdur^|r^t|_|j|j|jdur|rtdd|_|j|jdS)z)Adds `Callback`s that are always present.Nrr()_progbar_historyr'rr!rr )rmrnrorTr+r+r,r` s    z#CallbackList._add_default_callbackscCs.|dur iS|jr|S|r$|jr$|St|S)z?Turns tensors into numpy arrays or Python scalars if necessary.N)rQrbrsync_to_numpy_or_python_type)rmrK is_batch_hookr+r+r, _process_logss zCallbackList._process_logscCs|j|dSrY)r'r )rmcallbackr+r+r,r &szCallbackList.appendcCs ||_|jD]}||q dSrY)rpr'r9)rmrprxr+r+r,r9)s zCallbackList.set_paramscCs.||_|jr|j|_|jD]}||qdSrY)r%rtrr'r#)rmr%rxr+r+r,r#.s  zCallbackList.set_modelcCsN|js dS|dkr"||||n(|dkr:||||ntd|ddS)z4Helper function for all batch_{begin | end} methods.NbeginendzUnrecognized hook: z&. Expected values are ["begin", "end"])r'_call_batch_begin_hook_call_batch_end_hook ValueError)rmrhookbatchrKr+r+r,_call_batch_hook5s zCallbackList._call_batch_hookcCs.dj|d}|||||jr*t|_dS)z/Helper function for `on_*_batch_begin` methods.on_{mode}_batch_beginrN)format_call_batch_hook_helperrhtimerk)rmrrrK hook_namer+r+r,r{Ds z#CallbackList._call_batch_begin_hookc Csdj|d}|jr4|dkr4t|j}|j|||||t|j|jkr|}dj|d}t |jt|j}t |j |t|j |} t |j |t|j |} d|} d} | | krt | j||| d| | krt | j||| dd|_d |_g|_i|_ d S) z-Helper function for `on_*_batch_end` methods.zon_{mode}_batch_endrrr?zCallback method `{hook}` is slow compared to the batch time (batch time: {batch_time:.4f}s vs `{hook}` time: {hook_time:.4f}s). Check your callbacks.)r~ batch_time hook_timeFN) rrhrrkrlr rlenrisumrjloggingwarning) rmrrrKrr end_hook_namebegin_hook_nameavg_batch_timeavg_end_hook_timeavg_begin_hook_timethreshold_time warning_msgr+r+r,r|LsL   z!CallbackList._call_batch_end_hookcCsr|jrt}|j|dd}|jD]}t||}|||q"|jrn||jvrVg|j|<|j|t|dS)z+Helper function for `on_*_batch_*` methods.T)rvN)rhrrwr'rRrjr )rmrrrK start_timerxr~r+r+r,r|s     z$CallbackList._call_batch_hook_helpercCs4|tjkr|n|tjkr(|n|dS)z:Helper function for on_{train|test|predict}_begin methods.N)r ron_train_beginrI on_test_beginon_predict_beginrmrr+r+r,_call_begin_hooks     zCallbackList._call_begin_hookcCs4|tjkr|n|tjkr(|n|dS)z8Helper function for on_{train|test|predict}_end methods.N)r r on_train_endrI on_test_endon_predict_endrr+r+r,_call_end_hooks     zCallbackList._call_end_hookcCs|jr|jtjd||ddS)NryrKrdrr rrmrrKr+r+r,on_batch_beginszCallbackList.on_batch_begincCs|jr|jtjd||ddS)Nrzrrrr+r+r, on_batch_endszCallbackList.on_batch_endcCs&||}|jD]}|||qdS)a?Calls the `on_epoch_begin` methods of its callbacks. This function should only be called during TRAIN mode. Args: epoch: Integer, index of epoch. logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. N)rwr'on_epoch_beginrmepochrKrxr+r+r,rs  zCallbackList.on_epoch_begincCs&||}|jD]}|||qdS)a|Calls the `on_epoch_end` methods of its callbacks. This function should only be called during TRAIN mode. Args: epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with `val_`. N)rwr' on_epoch_endrr+r+r,rs  zCallbackList.on_epoch_endcCs|jr|jtjd||ddS)adCalls the `on_train_batch_begin` methods of its callbacks. Args: batch: Integer, index of batch within the current epoch. logs: Dict, contains the return value of `model.train_step`. Typically, the values of the `Model`'s metrics are returned. Example: `{'loss': 0.2, 'accuracy': 0.7}`. ryrNrrr+r+r,on_train_batch_begins z!CallbackList.on_train_batch_begincCs|jr|jtjd||ddS)zCalls the `on_train_batch_end` methods of its callbacks. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Aggregated metric results up until this batch. rzrNrrr+r+r,on_train_batch_endszCallbackList.on_train_batch_endcCs|jr|jtjd||ddS)abCalls the `on_test_batch_begin` methods of its callbacks. Args: batch: Integer, index of batch within the current epoch. logs: Dict, contains the return value of `model.test_step`. Typically, the values of the `Model`'s metrics are returned. Example: `{'loss': 0.2, 'accuracy': 0.7}`. ryrNrerr rIrr+r+r,on_test_batch_begins z CallbackList.on_test_batch_begincCs|jr|jtjd||ddS)zCalls the `on_test_batch_end` methods of its callbacks. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Aggregated metric results up until this batch. rzrNrrr+r+r,on_test_batch_endszCallbackList.on_test_batch_endcCs|jr|jtjd||ddS)aQCalls the `on_predict_batch_begin` methods of its callbacks. Args: batch: Integer, index of batch within the current epoch. logs: Dict, contains the return value of `model.predict_step`, it typically returns a dict with a key 'outputs' containing the model's outputs. ryrNrfrr r7rr+r+r,on_predict_batch_begins z#CallbackList.on_predict_batch_begincCs|jr|jtjd||ddS)zCalls the `on_predict_batch_end` methods of its callbacks. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Aggregated metric results up until this batch. rzrNrrr+r+r,on_predict_batch_endsz!CallbackList.on_predict_batch_endcCs$||}|jD]}||qdS)zCalls the `on_train_begin` methods of its callbacks. Args: logs: Dict. Currently, no data is passed via this argument for this method, but that may change in the future. N)rwr'rrmrKrxr+r+r,rs  zCallbackList.on_train_begincCs$||}|jD]}||qdS)zCalls the `on_train_end` methods of its callbacks. Args: logs: Dict. Currently, no data is passed via this argument for this method, but that may change in the future. N)rwr'rrr+r+r,rs  zCallbackList.on_train_endcCs$||}|jD]}||qdS)zCalls the `on_test_begin` methods of its callbacks. Args: logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. N)rwr'rrr+r+r,rs  zCallbackList.on_test_begincCs$||}|jD]}||qdS)zCalls the `on_test_end` methods of its callbacks. Args: logs: Dict. Currently, no data is passed via this argument for this method, but that may change in the future. N)rwr'rrr+r+r,r%s  zCallbackList.on_test_endcCs$||}|jD]}||qdS)zCalls the 'on_predict_begin` methods of its callbacks. Args: logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. N)rwr'rrr+r+r,r0s  zCallbackList.on_predict_begincCs$||}|jD]}||qdS)zCalls the `on_predict_end` methods of its callbacks. Args: logs: Dict. Currently, no data is passed via this argument for this method, but that may change in the future. N)rwr'rrr+r+r,r;s  zCallbackList.on_predict_endcCs t|jSrY)iterr'rmr+r+r,__iter__FszCallbackList.__iter__cCsftj}|jrbg}|jD]4}t|ddr,q|sD|sD|r| |q|rbt d|dS)z>Error out if batch-level callbacks are passed with PSStrategy.rQFzgBatch-level `Callback`s are not supported with `ParameterServerStrategy`. Found unsupported callbacks: N) rA distribute get_strategy_should_use_with_coordinatorr'rRrVrWrXr r})rmstrategyunsupported_callbacksrTr+r+r,rgIs&    z1CallbackList._disallow_batch_hooks_in_ps_strategy)NFFN)F)N)N)N)N)N)N)N)N)N)N)N)N)N)N)N)N)N)"r[ __module__ __qualname____doc__rqr`rwr r9r#rr{r|rrrrrrrrrrrrrrrrrrrrrgr+r+r+r,rsF D  0      rzkeras.callbacks.Callbackc@speZdZdZddZddZddZeje j d/d d Z eje j d0d d Z ejd1d dZ ejd2ddZeje j d3ddZeje j d4ddZeje j d5ddZeje j d6ddZeje j d7ddZeje j d8ddZejd9ddZejd:dd Zejd;d!d"Zejdd'd(Zd)d*Zd+d,Zd-d.ZdS)?CallbackaAbstract base class used to build new callbacks. Callbacks can be passed to keras methods such as `fit`, `evaluate`, and `predict` in order to hook into the various stages of the model training and inference lifecycle. To create a custom callback, subclass `keras.callbacks.Callback` and override the method associated with the stage of interest. See https://www.tensorflow.org/guide/keras/custom_callback for more information. Example: >>> training_finished = False >>> class MyCallback(tf.keras.callbacks.Callback): ... def on_train_end(self, logs=None): ... global training_finished ... training_finished = True >>> model = tf.keras.Sequential([ ... tf.keras.layers.Dense(1, input_shape=(1,))]) >>> model.compile(loss='mean_squared_error') >>> model.fit(tf.constant([[1.0]]), tf.constant([[1.0]]), ... callbacks=[MyCallback()]) >>> assert training_finished == True If you want to use `Callback` objects in a custom training loop: 1. You should pack all your callbacks into a single `callbacks.CallbackList` so they can all be called together. 2. You will need to manually call all the `on_*` methods at the appropriate locations in your loop. Like this: Example: ```python callbacks = tf.keras.callbacks.CallbackList([...]) callbacks.append(...) callbacks.on_train_begin(...) for epoch in range(EPOCHS): callbacks.on_epoch_begin(epoch) for i, data in dataset.enumerate(): callbacks.on_train_batch_begin(i) batch_logs = model.train_step(data) callbacks.on_train_batch_end(i, batch_logs) epoch_logs = ... callbacks.on_epoch_end(epoch, epoch_logs) final_logs=... callbacks.on_train_end(final_logs) ``` Attributes: params: Dict. Training parameters (eg. verbosity, batch size, number of epochs...). model: Instance of `keras.models.Model`. Reference of the model being trained. The `logs` dictionary that callback methods take as argument will contain keys for quantities relevant to the current batch or epoch (see method-specific docstrings). cCsd|_d|_d|_d|_dS)NF)validation_datar%_chief_worker_onlyrQrr+r+r,rqszCallback.__init__cCs ||_dSrY)rprmrpr+r+r,r9szCallback.set_paramscCs ||_dSrY)r%rmr%r+r+r,r#szCallback.set_modelNcCsdS)z;A backwards compatibility alias for `on_train_batch_begin`.Nr+rr+r+r,rszCallback.on_batch_begincCsdS)z9A backwards compatibility alias for `on_train_batch_end`.Nr+rr+r+r,rszCallback.on_batch_endcCsdS)aeCalled at the start of an epoch. Subclasses should override for any actions to run. This function should only be called during TRAIN mode. Args: epoch: Integer, index of epoch. logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rmrrKr+r+r,rszCallback.on_epoch_begincCsdS)a0Called at the end of an epoch. Subclasses should override for any actions to run. This function should only be called during TRAIN mode. Args: epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with `val_`. For training epoch, the values of the `Model`'s metrics are returned. Example: `{'loss': 0.2, 'accuracy': 0.7}`. Nr+rr+r+r,rszCallback.on_epoch_endcCs|j||ddS)aCalled at the beginning of a training batch in `fit` methods. Subclasses should override for any actions to run. Note that if the `steps_per_execution` argument to `compile` in `tf.keras.Model` is set to `N`, this method will only be called every `N` batches. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. rN)rrr+r+r,rszCallback.on_train_batch_begincCs|j||ddS)aCalled at the end of a training batch in `fit` methods. Subclasses should override for any actions to run. Note that if the `steps_per_execution` argument to `compile` in `tf.keras.Model` is set to `N`, this method will only be called every `N` batches. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Aggregated metric results up until this batch. rN)rrr+r+r,rszCallback.on_train_batch_endcCsdS)a~Called at the beginning of a batch in `evaluate` methods. Also called at the beginning of a validation batch in the `fit` methods, if validation data is provided. Subclasses should override for any actions to run. Note that if the `steps_per_execution` argument to `compile` in `tf.keras.Model` is set to `N`, this method will only be called every `N` batches. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rr+r+r,rszCallback.on_test_batch_begincCsdS)a3Called at the end of a batch in `evaluate` methods. Also called at the end of a validation batch in the `fit` methods, if validation data is provided. Subclasses should override for any actions to run. Note that if the `steps_per_execution` argument to `compile` in `tf.keras.Model` is set to `N`, this method will only be called every `N` batches. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Aggregated metric results up until this batch. Nr+rr+r+r,r szCallback.on_test_batch_endcCsdS)aCalled at the beginning of a batch in `predict` methods. Subclasses should override for any actions to run. Note that if the `steps_per_execution` argument to `compile` in `tf.keras.Model` is set to `N`, this method will only be called every `N` batches. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rr+r+r,r szCallback.on_predict_batch_begincCsdS)aCalled at the end of a batch in `predict` methods. Subclasses should override for any actions to run. Note that if the `steps_per_execution` argument to `compile` in `tf.keras.Model` is set to `N`, this method will only be called every `N` batches. Args: batch: Integer, index of batch within the current epoch. logs: Dict. Aggregated metric results up until this batch. Nr+rr+r+r,r1szCallback.on_predict_batch_endcCsdS)zCalled at the beginning of training. Subclasses should override for any actions to run. Args: logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rmrKr+r+r,rAszCallback.on_train_begincCsdS)a.Called at the end of training. Subclasses should override for any actions to run. Args: logs: Dict. Currently the output of the last call to `on_epoch_end()` is passed to this argument for this method but that may change in the future. Nr+rr+r+r,rLszCallback.on_train_endcCsdS)aCalled at the beginning of evaluation or validation. Subclasses should override for any actions to run. Args: logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rr+r+r,rXszCallback.on_test_begincCsdS)aCCalled at the end of evaluation or validation. Subclasses should override for any actions to run. Args: logs: Dict. Currently the output of the last call to `on_test_batch_end()` is passed to this argument for this method but that may change in the future. Nr+rr+r+r,rcszCallback.on_test_endcCsdS)aCalled at the beginning of prediction. Subclasses should override for any actions to run. Args: logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rr+r+r,roszCallback.on_predict_begincCsdS)zCalled at the end of prediction. Subclasses should override for any actions to run. Args: logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. Nr+rr+r+r,rzszCallback.on_predict_endcCs8t|j p6t|j p6t|j p6t|j S)zBDetermines if this Callback should be called for each train batch.)r is_defaultrrrrrr+r+r,rVs   z&Callback._implements_train_batch_hookscCst|j pt|j S)zADetermines if this Callback should be called for each test batch.)rrrrrr+r+r,rWs  z%Callback._implements_test_batch_hookscCst|j pt|j S)zLDetermines if this Callback should be called for each predict batch.)rrrrrr+r+r,rXs  z(Callback._implements_predict_batch_hooks)N)N)N)N)N)N)N)N)N)N)N)N)N)N)N)N)r[rrrrqr9r#rfor_subclass_implementersrdefaultrrrrrrrrrrrrrrrrrVrWrXr+r+r+r,ras^;          rzkeras.callbacks.BaseLoggercs@eZdZdZd fdd Zd ddZd ddZdd d ZZS)raCallback that accumulates epoch averages of metrics. This callback is automatically applied to every Keras model. Args: stateful_metrics: Iterable of string names of metrics that should *not* be averaged over an epoch. Metrics in this list will be logged as-is in `on_epoch_end`. All others will be averaged in `on_epoch_end`. Ncstt|pg|_dSrY)superrqsetr6)rmr6rZr+r,rqs zBaseLogger.__init__cCsd|_i|_dSNr)seentotalsrr+r+r,rszBaseLogger.on_epoch_begincCs|pi}|dd}|dd}|j||7_|D]N\}}||jvrX||j|<q:||jvrz|j|||7<q:|||j|<q:dS)Nsizer num_stepsr)getritemsr6r)rmrrKrrkvr+r+r,rs     zBaseLogger.on_batch_endcCsT|durP|jdD]<}||jvr||jvr:|j|||<q|j||j||<qdS)Nr4)rprr6r)rmrrKrr+r+r,rs   zBaseLogger.on_epoch_end)N)N)N)N) r[rrrrqrrr __classcell__r+r+rr,rs    rzkeras.callbacks.TerminateOnNaNcs*eZdZdZfddZdddZZS)TerminateOnNaNzACallback that terminates training when a NaN loss is encountered.cstd|_dSNT)rrqrQrrr+r,rqs zTerminateOnNaN.__init__NcCsV|pi}|d}|durRt|}t|s8t|rRtd|dd|j_ dS)NlosszBatch z$: Invalid loss, terminating trainingT) rrrunpisnanisinfr print_msgr%r&)rmrrKrr+r+r,rs   zTerminateOnNaN.on_batch_end)N)r[rrrrqrrr+r+rr,rs rzkeras.callbacks.ProgbarLoggercseZdZdZd+edfdd ZddZd,d d Zd-d d Zd.d dZ d/ddZ d0ddZ d1ddZ d2ddZ d3ddZd4ddZd5ddZddZdd Zd!d"Zd#d$Zd%d&Zd6d'd(Zd)d*ZZS)7r!aDCallback that prints metrics to stdout. Args: count_mode: One of `"steps"` or `"samples"`. Whether the progress bar should count samples seen or steps (batches) seen. stateful_metrics: Iterable of string names of metrics that should *not* be averaged over an epoch. Metrics in this list will be logged as-is. All others will be averaged over time (e.g. loss, etc). If not provided, defaults to the `Model`'s metrics. Raises: ValueError: In case of invalid `count_mode`. rNrrcstd|_|dkr d|_n |dkr0d|_ntd|d|rLt|nt|_d|_d|_d|_ d|_ d|_ d \|_ |_ |_d|_d|_dS) NTrFrzUnknown `count_mode`: z*. Expected values are ["samples", "steps"]rr)NNN)rrqrQ use_stepsr}rr6rprogbartargetrr _train_step _test_step _predict_step_call_batch_hooks_called_in_fit)rmr(r6rr+r,rqs&  zProgbarLogger.__init__cCs|d|_|d|_|jr.d|vr.|d|_n |jsHd|vrH|d|_nd|_|jdk|_|jdurz"|jj|_|jj|_ |jj |_ Wnt yd|_Yn0dS)NrrrrrT) rrrrrr%_train_counterr _test_counterr_predict_counterrAttributeErrorrr+r+r,r9s          zProgbarLogger.set_paramscCs d|_dSr)rrr+r+r,r%szProgbarLogger.on_train_begincCs|js||dSrY)r_reset_progbar_maybe_init_progbarrr+r+r,r)szProgbarLogger.on_test_begincCs||dSrY)rrrr+r+r,r.szProgbarLogger.on_predict_begincCs@|||jr<|jdkr.Nstepsample)rrr6 unit_name) rr6r%unionr4rr rrr_update_stateful_metricsrr+r+r,rQs   z!ProgbarLogger._maybe_init_progbarcCs|jSrYrrr+r+r,rVksz+ProgbarLogger._implements_train_batch_hookscCs|jSrYrrr+r+r,rWnsz*ProgbarLogger._implements_test_batch_hookscCs|jSrYrrr+r+r,rXqsz-ProgbarLogger._implements_predict_batch_hookscCs|pi}||jr"|d|_nDt|}|dd}|dd}|dd||}|j|7_|jdkrt|}|jj |jt | dddS) zUpdates the progbar.rrrrrNFfinalize) rrrr8poprrrurupdatelistr)rmrrKrradd_seenr+r+r,rts       z#ProgbarLogger._batch_update_progbarcCstt|p i}|jdurT|dur>|}|js>||dd9}|pF|j|_|j|j_|jj|jt | dddS)NrrTr) rrurnumpyrrrrrrr)rmrKcounterr+r+r,rs   zProgbarLogger._finalize_progbar)rN)N)N)N)N)N)N)N)N)N)N)N)r[rrrstrrqr9rrrrrrrrrrrrrVrWrXrrrr+r+rr,r!s(           r!zkeras.callbacks.Historycs4eZdZdZfddZd ddZd ddZZS) raCallback that records events into a `History` object. This callback is automatically applied to every Keras model. The `History` object gets returned by the `fit` method of models. Example: >>> model = tf.keras.models.Sequential([tf.keras.layers.Dense(10)]) >>> model.compile(tf.keras.optimizers.SGD(), loss='mse') >>> history = model.fit(np.arange(100).reshape(5, 20), np.zeros(5), ... epochs=10, verbose=1) >>> print(history.params) {'verbose': 1, 'epochs': 10, 'steps': 1} >>> # check the keys of history object >>> print(history.history.keys()) dict_keys(['loss']) csti|_dSrY)rrqrrrr+r,rqs zHistory.__init__NcCs g|_dSrYrrr+r+r,rszHistory.on_train_begincCsF|pi}|j||D]\}}|j|g|q||j_dSrY)rr rr setdefaultr%)rmrrKrrr+r+r,rs  zHistory.on_epoch_end)N)N)r[rrrrqrrrr+r+rr,rs  rzkeras.callbacks.ModelCheckpointcseZdZdZd!eeeeedfd d Zd"d d Zd dZ d#ddZ d$ddZ d%ddZ ddZ ddZddZddZddZdd ZZS)&ModelCheckpointaCallback to save the Keras model or model weights at some frequency. `ModelCheckpoint` callback is used in conjunction with training using `model.fit()` to save a model or weights (in a checkpoint file) at some interval, so the model or weights can be loaded later to continue the training from the state saved. A few options this callback provides include: - Whether to only keep the model that has achieved the "best performance" so far, or whether to save the model at the end of every epoch regardless of performance. - Definition of 'best'; which quantity to monitor and whether it should be maximized or minimized. - The frequency it should save at. Currently, the callback supports saving at the end of every epoch, or after a fixed number of training batches. - Whether only weights are saved, or the whole model is saved. Note: If you get `WARNING:tensorflow:Can save best model only with available, skipping` see the description of the `monitor` argument for details on how to get this right. Example: ```python model.compile(loss=..., optimizer=..., metrics=['accuracy']) EPOCHS = 10 checkpoint_filepath = '/tmp/checkpoint' model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint( filepath=checkpoint_filepath, save_weights_only=True, monitor='val_accuracy', mode='max', save_best_only=True) # Model weights are saved at the end of every epoch, if it's the best seen # so far. model.fit(epochs=EPOCHS, callbacks=[model_checkpoint_callback]) # The model weights (that are considered the best) are loaded into the # model. model.load_weights(checkpoint_filepath) ``` Args: filepath: string or `PathLike`, path to save the model file. e.g. filepath = os.path.join(working_dir, 'ckpt', file_name). `filepath` can contain named formatting options, which will be filled the value of `epoch` and keys in `logs` (passed in `on_epoch_end`). For example: if `filepath` is `weights.{epoch:02d}-{val_loss:.2f}.hdf5`, then the model checkpoints will be saved with the epoch number and the validation loss in the filename. The directory of the filepath should not be reused by any other callbacks to avoid conflicts. monitor: The metric name to monitor. Typically the metrics are set by the `Model.compile` method. Note: * Prefix the name with `"val_`" to monitor validation metrics. * Use `"loss"` or "`val_loss`" to monitor the model's total loss. * If you specify metrics as strings, like `"accuracy"`, pass the same string (with or without the `"val_"` prefix). * If you pass `metrics.Metric` objects, `monitor` should be set to `metric.name` * If you're not sure about the metric names you can check the contents of the `history.history` dictionary returned by `history = model.fit()` * Multi-output models set additional prefixes on the metric names. verbose: Verbosity mode, 0 or 1. Mode 0 is silent, and mode 1 displays messages when the callback takes an action. save_best_only: if `save_best_only=True`, it only saves when the model is considered the "best" and the latest best model according to the quantity monitored will not be overwritten. If `filepath` doesn't contain formatting options like `{epoch}` then `filepath` will be overwritten by each new better model. mode: one of {'auto', 'min', 'max'}. If `save_best_only=True`, the decision to overwrite the current save file is made based on either the maximization or the minimization of the monitored quantity. For `val_acc`, this should be `max`, for `val_loss` this should be `min`, etc. In `auto` mode, the mode is set to `max` if the quantities monitored are 'acc' or start with 'fmeasure' and are set to `min` for the rest of the quantities. save_weights_only: if True, then only the model's weights will be saved (`model.save_weights(filepath)`), else the full model is saved (`model.save(filepath)`). save_freq: `'epoch'` or integer. When using `'epoch'`, the callback saves the model after each epoch. When using integer, the callback saves the model at end of this many batches. If the `Model` is compiled with `steps_per_execution=N`, then the saving criteria will be checked every Nth batch. Note that if the saving isn't aligned to epochs, the monitored metric may potentially be less reliable (it could reflect as little as 1 batch, since the metrics get reset every epoch). Defaults to `'epoch'`. options: Optional `tf.train.CheckpointOptions` object if `save_weights_only` is true or optional `tf.saved_model.SaveOptions` object if `save_weights_only` is false. initial_value_threshold: Floating point initial "best" value of the metric to be monitored. Only applies if `save_best_value=True`. Only overwrites the model weights already saved if the performance of current model is better than this value. **kwargs: Additional arguments for backwards compatibility. Possible key is `period`. val_lossrFautorN)monitorrsave_best_onlysave_weights_onlyrc  s td|_||_||_t||_||_||_ ||_ d|_ d|_ d|_ | |_|r|duslt|tjjr~|pxtj|_qtd|dn8|dust|tjjr|ptj|_ntd|dd| vr| d|_tdnd|_d | vr | d |_td nd |_|d vr,td |d}|dkrTtj|_|jdurtj|_n|dkr~tj|_|jdurtj |_nVd|jvs|j drtj|_|jdurtj |_ntj|_|jdurtj|_|j dkrt|j t!st"d|j dd|_#dS)NTrzfIf save_weights_only is True, then `options` must be either None or a tf.train.CheckpointOptions. Got .zgIf save_weights_only is False, then `options` must be either None or a tf.saved_model.SaveOptions. Got load_weights_on_restartz`load_weights_on_restart` argument is deprecated. Please use `model.load_weights()` for loading weights before the start of `model.fit()`.Fperiodzk`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.rrminmaxz:ModelCheckpoint mode %s is unknown, fallback to auto mode.rr r accfmeasurerzUnrecognized save_freq: z+. Expected save_freq are "epoch" or integer)$rrqrQrrrpath_to_stringfilepathrr save_freqepochs_since_last_save_batches_seen_since_last_saving_last_batch_seenbestrrAtrainCheckpointOptions_options TypeError saved_model SaveOptionsrrrrrless monitor_opInfgreater startswithintr}r) rmrrrrrrroptionsZinitial_value_thresholdkwargsrr+r,rq's                   zModelCheckpoint.__init__c Csv|jrr||j}|durr||rrz|j|Wn<ttfyp}z td|d|WYd}~n d}~00dS)NzError loading file from z . Reason: )r1_get_most_recently_modified_file_matching_patternr_checkpoint_existsr% load_weightsIOErrorr})rmrKfilepath_to_loader+r+r,rs  zModelCheckpoint.on_train_begincCs |jdkSNrrrr+r+r,rVsz-ModelCheckpoint._implements_train_batch_hookscCs ||r|j|j||ddS)NrrrK)_should_save_on_batch _save_model_current_epochrr+r+r,rs z"ModelCheckpoint.on_train_batch_endcCs ||_dSrYr.rr+r+r,rszModelCheckpoint.on_epoch_begincCs,|jd7_|jdkr(|j|d|ddS)Nrrr+)rrr-rr+r+r,rs zModelCheckpoint.on_epoch_endcCsZ|jdkrdS||jkr"|d}n ||j}|j|7_||_|j|jkrVd|_dSdS)z?Handles batch-level saving logic, supports steps_per_execution.rFrrT)rrr)rmr add_batchesr+r+r,r,s     z%ModelCheckpoint._should_save_on_batchc Cs|pi}t|jts"|j|jkrt|}d|_||||}zH|jr(| |j }|durpt d|j n| ||jr|jdkrtd|dd|j d|jdd |dd | ||_|jr|jj|d |jd n|jj|d |jd n2|jdkrtd|dd|j d |jdnX|jdkrNtd|dd||jrl|jj|d |jd n|jj|d |jd |Wnptytd|YnPty}z6dt|jdvrtd||WYd}~n d}~00dS)aSaves the model. Args: epoch: the epoch this iteration is in. batch: the batch this iteration is in. `None` if the `save_freq` is set to `epoch`. logs: the `logs` dict passed in to `on_batch_end` or `on_epoch_end`. rNz5Can save best model only with %s available, skipping. Epoch rz: z improved from z.5fz to z, saving model to T) overwriter!z did not improve from z: saving model to zePlease specify a non-directory filepath for ModelCheckpoint. Filepath used is an existing directory: zis a directoryzfPlease specify a non-directory filepath for ModelCheckpoint. Filepath used is an existing directory: f)rrr rrrru_get_file_pathrrrrrrrrrrrr% save_weightsrsave_maybe_remove_fileIsADirectoryErrorr&rargslower)rmrrrKrcurrentr(r+r+r,r-s          zModelCheckpoint._save_modelc CszP|dusd|vr.|jjfd|di|}n |jjf|d|dd|}Wn:ty}z"td|jd|WYd}~n d}~00t||jj|_|jS)z%Returns the file path for checkpoint.Nrrrrrz*Failed to format this callback filepath: "z ". Reason: )rrKeyErrorrwrite_filepathr%distribute_strategy_write_filepath)rmrrrK file_pathr(r+r+r,r3%s$   zModelCheckpoint._get_file_pathcCst|j|jjdSrY)rremove_temp_dir_with_filepathr?r%r>rr+r+r,r6=s z"ModelCheckpoint._maybe_remove_filecCs@|drtjj|Stjj|}tjj|d}|p>|S)z;Returns whether the checkpoint `filepath` refers to exists.z.h5z.index)endswithrAiogfileexists)rmrtf_saved_model_exists!tf_weights_only_checkpoint_existsr+r+r,r$Es z"ModelCheckpoint._checkpoint_existsc Cstj|}tj|}dtdd|d}tj|}|durZt |tj|rZ|Sd}d}d}d} tj j |rt |D]d} t || rtj|| } tj| } | dus| | kr| } | |kr| }| }d}q| |kr|d7}q|dkr|S| SdS)aWReturns the most recently modified filepath matching pattern. Pattern may contain python formatting placeholder. If `tf.train.latest_checkpoint()` does not return None, use that; otherwise, check for most recently modified one that matches the pattern. In the rare case where there are more than one pattern-matching file having the same modified time that is most recent among all, return the filepath that is largest (by `>` operator, lexicographically using the numeric equivalents). This provides a tie-breaker when multiple files are most recent. Note that a larger `filepath` can sometimes indicate a later time of modification (for instance, when epoch/batch is used as formatting option), but not necessarily (when accuracy or loss is used). The tie-breaker is put in the logic as best effort to return the most recent, and to avoid undeterministic result. Modified time of a file is obtained with `os.path.getmtime()`. This utility function is best demonstrated via an example: ```python file_pattern = 'f.batch{batch:02d}epoch{epoch:02d}.h5' test_dir = self.get_temp_dir() path_pattern = os.path.join(test_dir, file_pattern) file_paths = [ os.path.join(test_dir, file_name) for file_name in ['f.batch03epoch02.h5', 'f.batch02epoch02.h5', 'f.batch01epoch01.h5'] ] for file_path in file_paths: # Write something to each of the files self.assertEqual( _get_most_recently_modified_file_matching_pattern(path_pattern), file_paths[-1]) ``` Args: pattern: The file pattern that may optionally contain python placeholder such as `{epoch:02d}`. Returns: The most recently modified file's full filepath matching `pattern`. If `pattern` does not contain any placeholder, this returns the filepath that exactly matches `pattern`. Returns `None` if no match is found. ^z{.*}z.*$Nrr)ospathdirnamebasenameresubrArlatest_checkpointmatchrCrDrElistdirjoingetmtime) rmpatterndir_name base_namebase_name_regexlatest_tf_checkpointlatest_mod_timefile_path_with_latest_mod_timen_file_with_latest_mod_time file_path_with_largest_file_name file_namer@mod_timer+r+r,r#Os>0        zAModelCheckpoint._get_most_recently_modified_file_matching_pattern)rrFFrrNN)N)N)N)N)r[rrrrr boolrqrrVrrrr,r-r3r6r$r#rr+r+rr,rs6ll    [ rz keras.callbacks.BackupAndRestore)rCcs\eZdZdZdfdd ZdddZdd d Zd d Zdd dZdddZ dddZ Z S)BackupAndRestoreaCallback to back up and restore the training state. `BackupAndRestore` callback is intended to recover training from an interruption that has happened in the middle of a `Model.fit` execution, by backing up the training states in a temporary checkpoint file (with the help of a `tf.train.CheckpointManager`), at the end of each epoch. Each backup overwrites the previously written checkpoint file, so at any given time there is at most one such checkpoint file for backup/restoring purpose. If training restarts before completion, the training state (which includes the `Model` weights and epoch number) is restored to the most recently saved state at the beginning of a new `Model.fit` run. At the completion of a `Model.fit` run, the temporary checkpoint file is deleted. Note that the user is responsible to bring jobs back after the interruption. This callback is important for the backup and restore mechanism for fault tolerance purpose, and the model to be restored from an previous checkpoint is expected to be the same as the one used to back up. If user changes arguments passed to compile or fit, the checkpoint saved for fault tolerance can become invalid. Note: 1. This callback is not compatible with eager execution disabled. 2. A checkpoint is saved at the end of each epoch. After restoring, `Model.fit` redoes any partial work during the unfinished epoch in which the training got restarted (so the work done before the interruption doesn't affect the final model state). 3. This works for both single worker and multi-worker modes. When `Model.fit` is used with `tf.distribute`, it supports `tf.distribute.MirroredStrategy`, `tf.distribute.MultiWorkerMirroredStrategy`, `tf.distribute.TPUStrategy`, and `tf.distribute.experimental.ParameterServerStrategy`. Example: >>> class InterruptingCallback(tf.keras.callbacks.Callback): ... def on_epoch_begin(self, epoch, logs=None): ... if epoch == 4: ... raise RuntimeError('Interrupting!') >>> callback = tf.keras.callbacks.BackupAndRestore(backup_dir="/tmp/backup") >>> model = tf.keras.models.Sequential([tf.keras.layers.Dense(10)]) >>> model.compile(tf.keras.optimizers.SGD(), loss='mse') >>> try: ... model.fit(np.arange(100).reshape(5, 20), np.zeros(5), epochs=10, ... batch_size=1, callbacks=[callback, InterruptingCallback()], ... verbose=0) ... except: ... pass >>> history = model.fit(np.arange(100).reshape(5, 20), np.zeros(5), ... epochs=10, batch_size=1, callbacks=[callback], ... verbose=0) >>> # Only 6 more epochs are run, since first trainning got interrupted at >>> # zero-indexed epoch 4, second training will continue from 4 to 9. >>> len(history.history['loss']) 6 Args: backup_dir: String, path to store the checkpoint. e.g. backup_dir = os.path.join(working_dir, 'backup') This is the directory in which the system stores temporary files to recover the model from jobs terminated unexpectedly. The directory cannot be reused elsewhere to store other files, e.g. by BackupAndRestore callback of another training, or by another callback (ModelCheckpoint) of the same training. save_freq: `'epoch'` or integer. When set to `'epoch'` the callback saves the checkpoint at the end of each epoch. When set to an integer, the callback saves the checkpoint every `save_freq` batches. delete_checkpoint: Boolean, default to True. This `BackupAndRestore` callback works by saving a checkpoint to back up the training state. If `delete_checkpoint=True`, the checkpoint will be deleted after training is finished. Use `False` if you'd like to keep the checkpoint for future usage. rTcst||_d|_tjjtjjtjjj tjj tjjj f|_ ||_ ||_ d|_d|_tsxtrptdntdd|_dS)NTrz\This Callback's method contains Python state and should be called outside of `tf.function`s.zBackupAndRestore only supports eager mode. In graph mode, consider using ModelCheckpoint to manually save and restore weights with `model.load_weights()` and by providing `initial_epoch` in `model.fit()` for fault tolerance.F)rrq backup_dirrQrArMirroredStrategyMultiWorkerMirroredStrategy experimental TPUStrategyParameterServerStrategy_supported_strategiesrdelete_checkpoint_batches_countr.executing_eagerlyinside_functionr}r)rmrbrrirr+r,rqs,  zBackupAndRestore.__init__NcCs^|jjr.t|jj|js.tt|jjdt|j|j |j |j_ |jj |_ |j dS)Nz is not supported yet. Currently BackupAndRestore callback only supports empty strategy, MirroredStrategy, MultiWorkerMirroredStrategy and TPUStrategy.) r%_distribution_strategyrr>rhNotImplementedErrortyperWorkerTrainingStaterbr_training_staterestorerr+r+r,r"s    zBackupAndRestore.on_train_begincCs@|jdkr<|jd7_|j|jkrsz.BackupAndRestore._implements_train_batch_hookscCs|jr|j|`|j`dSrY)rirq delete_backupr%rr+r+r,rAs zBackupAndRestore.on_train_endcCs ||_dSrYr/rr+r+r,rJszBackupAndRestore.on_epoch_begincCs|jdkr|jj|ddS)Nrr)rrqrsrr+r+r,rMs zBackupAndRestore.on_epoch_end)rT)N)N)N)N)N) r[rrrrqrrrVrrrrr+r+rr,rasL#   raz-keras.callbacks.experimental.BackupAndRestorecs eZdZdZfddZZS)BackupAndRestoreExperimentala Deprecated. Please use `tf.keras.callbacks.BackupAndRestore` instead. Caution: `tf.keras.callbacks.experimental.BackupAndRestore` endpoint is deprecated and will be removed in a future release. Please use `tf.keras.callbacks.BackupAndRestore`. cs tdtj|i|dS)Nz`tf.keras.callbacks.experimental.BackupAndRestore` endpoint is deprecated and will be removed in a future release. Please use `tf.keras.callbacks.BackupAndRestore`.)rrrrq)rmr8r"rr+r,rq_sz%BackupAndRestoreExperimental.__init__r[rrrrqrr+r+rr,ruSsruzkeras.callbacks.EarlyStoppingcsPeZdZdZdfdd Zdd d Zdd d Zdd dZddZddZ Z S) EarlyStoppinga Stop training when a monitored metric has stopped improving. Assuming the goal of a training is to minimize the loss. With this, the metric to be monitored would be `'loss'`, and mode would be `'min'`. A `model.fit()` training loop will check at end of every epoch whether the loss is no longer decreasing, considering the `min_delta` and `patience` if applicable. Once it's found no longer decreasing, `model.stop_training` is marked True and the training terminates. The quantity to be monitored needs to be available in `logs` dict. To make it so, pass the loss or metrics at `model.compile()`. Args: monitor: Quantity to be monitored. min_delta: Minimum change in the monitored quantity to qualify as an improvement, i.e. an absolute change of less than min_delta, will count as no improvement. patience: Number of epochs with no improvement after which training will be stopped. verbose: Verbosity mode, 0 or 1. Mode 0 is silent, and mode 1 displays messages when the callback takes an action. mode: One of `{"auto", "min", "max"}`. In `min` mode, training will stop when the quantity monitored has stopped decreasing; in `"max"` mode it will stop when the quantity monitored has stopped increasing; in `"auto"` mode, the direction is automatically inferred from the name of the monitored quantity. baseline: Baseline value for the monitored quantity. Training will stop if the model doesn't show improvement over the baseline. restore_best_weights: Whether to restore model weights from the epoch with the best value of the monitored quantity. If False, the model weights obtained at the last step of training are used. An epoch will be restored regardless of the performance relative to the `baseline`. If no epoch improves on `baseline`, training will run for `patience` epochs and restore weights from the best epoch in that set. Example: >>> callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3) >>> # This callback will stop the training when there is no improvement in >>> # the loss for three consecutive epochs. >>> model = tf.keras.models.Sequential([tf.keras.layers.Dense(10)]) >>> model.compile(tf.keras.optimizers.SGD(), loss='mse') >>> history = model.fit(np.arange(100).reshape(5, 20), np.zeros(5), ... epochs=10, batch_size=1, callbacks=[callback], ... verbose=0) >>> len(history.history['loss']) # Only 4 epochs are run. 4 rrrNFcst||_||_||_||_t||_d|_d|_ ||_ d|_ |dvr\t d|d}|dkrntj|_nH|dkrtj|_n6|jds|jds|jd rtj|_ntj|_|jtjkr|jd 9_n|jd 9_dS) Nrr z8EarlyStopping mode %s is unknown, fallback to auto mode.rr r r accuracyaucr)rrqrpatiencerbaselineabs min_deltawait stopped_epochrestore_best_weights best_weightsrrrrrrrB)rmrr~r{rrr|rrr+r,rqs>         zEarlyStopping.__init__cCs8d|_d|_|jtjkrtjntj |_d|_d|_dSr) rrrrrrrr best_epochrr+r+r,rs zEarlyStopping.on_train_begincCs||}|durdS|jr2|jdur2|j|_|jd7_|||jr||_||_|jrl|j|_|j dus|||j rd|_|j|j kr|dkr||_ d|j_ |jr|jdur|j dkrtd|jdd|j|jdS)NrrTz8Restoring model weights from the end of the best epoch: r)get_monitor_valuerrr% get_weightsr_is_improvementrrr|r{rr&rrr set_weights)rmrrKr:r+r+r,rs4    zEarlyStopping.on_epoch_endcCs0|jdkr,|jdkr,td|jdddS)Nrrrz: early stopping)rrrrrr+r+r,rszEarlyStopping.on_train_endc Cs>|pi}||j}|dur:td|jdt||S)Nz[Early stopping conditioned on metric `%s` which is not available. Available metrics are: %s,)rrrrrSrkeys)rmrK monitor_valuer+r+r,rs zEarlyStopping.get_monitor_valuecCs|||j|SrY)rr~)rmrreference_valuer+r+r,r szEarlyStopping._is_improvement)rrrrrNF)N)N)N) r[rrrrqrrrrrrr+r+rr,rwhs80  "  rwzkeras.callbacks.RemoteMonitorcs,eZdZdZd fdd Zd d d ZZS) RemoteMonitora|Callback used to stream events to a server. Requires the `requests` library. Events are sent to `root + '/publish/epoch/end/'` by default. Calls are HTTP POST, with a `data` argument which is a JSON-encoded dictionary of event data. If `send_as_json=True`, the content type of the request will be `"application/json"`. Otherwise the serialized JSON will be sent within a form. Args: root: String; root url of the target server. path: String; path relative to `root` to which the events will be sent. field: String; JSON field under which the data will be stored. The field is used only if the payload is sent within a form (i.e. send_as_json is set to False). headers: Dictionary; optional custom HTTP headers. send_as_json: Boolean; whether the request should be sent as `"application/json"`. http://localhost:9000/publish/epoch/end/rDNFcs,t||_||_||_||_||_dSrY)rrqrootrKfieldheaders send_as_json)rmrrKrrrrr+r,rq's  zRemoteMonitor.__init__cCstdurtd|pi}i}||d<|D]0\}}t|tjtjfrT|||<q,|||<q,zL|jrtj |j |j ||j dn&tj |j |j |j t|i|j dWn*tjjytdt|j Yn0dS)Nz.RemoteMonitor requires the `requests` library.r)jsonr)rz6Warning: could not reach RemoteMonitor root server at )requests ImportErrorrrrndarraygenericitemrpostrrKrrrdumps exceptionsRequestExceptionrrr)rmrrKsendrrr+r+r,r7s2   zRemoteMonitor.on_epoch_end)rrrDNF)N)r[rrrrqrrr+r+rr,rsrz%keras.callbacks.LearningRateSchedulercs6eZdZdZd fdd Zd ddZd dd ZZS) LearningRateScheduleraLearning rate scheduler. At the beginning of every epoch, this callback gets the updated learning rate value from `schedule` function provided at `__init__`, with the current epoch and current learning rate, and applies the updated learning rate on the optimizer. Args: schedule: a function that takes an epoch index (integer, indexed from 0) and current learning rate (float) as inputs and returns a new learning rate as output (float). verbose: int. 0: quiet, 1: update messages. Example: >>> # This function keeps the initial learning rate for the first ten epochs >>> # and decreases it exponentially after that. >>> def scheduler(epoch, lr): ... if epoch < 10: ... return lr ... else: ... return lr * tf.math.exp(-0.1) >>> >>> model = tf.keras.models.Sequential([tf.keras.layers.Dense(10)]) >>> model.compile(tf.keras.optimizers.SGD(), loss='mse') >>> round(model.optimizer.lr.numpy(), 5) 0.01 >>> callback = tf.keras.callbacks.LearningRateScheduler(scheduler) >>> history = model.fit(np.arange(100).reshape(5, 20), np.zeros(5), ... epochs=15, callbacks=[callback], verbose=0) >>> round(model.optimizer.lr.numpy(), 5) 0.00607 rcst||_||_dSrY)rrqscheduler)rmrrrr+r,rq}s zLearningRateScheduler.__init__NcCst|jjdstdz$tt|jjj}|||}Wnt yV||}Yn0t |t j tt jt jfs~td|t |t j r|jjstd|jt|jjjt||jdkrtd|dd|d dS) Nlrz%Optimizer must have a "lr" attribute.zeZdZdZdJfdd Zd d Zd d ZeddZeddZ ddZ ddZ ddZ ddZ ddZddZddZd d!Zd"d#ZdKd$d%ZdLd&d'ZdMd(d)ZdNd*d+Zd,d-ZdOd.d/ZdPd0d1ZdQd2d3ZdRd4d5Zd6d7ZdSd8d9Zd:d;Zdd?Z d@dAZ!dBdCZ"dDdEZ#dFdGZ$dTdHdIZ%Z&S)U TensorBoarda5Enable visualizations for TensorBoard. TensorBoard is a visualization tool provided with TensorFlow. This callback logs events for TensorBoard, including: * Metrics summary plots * Training graph visualization * Weight histograms * Sampled profiling When used in `Model.evaluate`, in addition to epoch summaries, there will be a summary that records evaluation metrics vs `Model.optimizer.iterations` written. The metric names will be prepended with `evaluation`, with `Model.optimizer.iterations` being the step in the visualized TensorBoard. If you have installed TensorFlow with pip, you should be able to launch TensorBoard from the command line: ``` tensorboard --logdir=path_to_your_logs ``` You can find more information about TensorBoard [here](https://www.tensorflow.org/get_started/summaries_and_tensorboard). Args: log_dir: the path of the directory where to save the log files to be parsed by TensorBoard. e.g. log_dir = os.path.join(working_dir, 'logs') This directory should not be reused by any other callbacks. histogram_freq: frequency (in epochs) at which to compute weight histograms for the layers of the model. If set to 0, histograms won't be computed. Validation data (or split) must be specified for histogram visualizations. write_graph: whether to visualize the graph in TensorBoard. The log file can become quite large when write_graph is set to True. write_images: whether to write model weights to visualize as image in TensorBoard. write_steps_per_second: whether to log the training steps per second into Tensorboard. This supports both epoch and batch frequency logging. update_freq: `'batch'` or `'epoch'` or integer. When using `'batch'`, writes the losses and metrics to TensorBoard after each batch. The same applies for `'epoch'`. If using an integer, let's say `1000`, the callback will write the metrics and losses to TensorBoard every 1000 batches. Note that writing too frequently to TensorBoard can slow down your training. profile_batch: Profile the batch(es) to sample compute characteristics. profile_batch must be a non-negative integer or a tuple of integers. A pair of positive integers signify a range of batches to profile. By default, profiling is disabled. embeddings_freq: frequency (in epochs) at which embedding layers will be visualized. If set to 0, embeddings won't be visualized. embeddings_metadata: Dictionary which maps embedding layer names to the filename of a file in which to save metadata for the embedding layer. In case the same metadata file is to be used for all embedding layers, a single filename can be passed. Examples: Basic usage: ```python tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir="./logs") model.fit(x_train, y_train, epochs=2, callbacks=[tensorboard_callback]) # Then run the tensorboard command to view the visualizations. ``` Custom batch-level summaries in a subclassed Model: ```python class MyModel(tf.keras.Model): def build(self, _): self.dense = tf.keras.layers.Dense(10) def call(self, x): outputs = self.dense(x) tf.summary.histogram('outputs', outputs) return outputs model = MyModel() model.compile('sgd', 'mse') # Make sure to set `update_freq=N` to log a batch-level summary every N # batches. In addition to any `tf.summary` contained in `Model.call`, # metrics added in `Model.compile` will be logged every N batches. tb_callback = tf.keras.callbacks.TensorBoard('./logs', update_freq=1) model.fit(x_train, y_train, callbacks=[tb_callback]) ``` Custom batch-level summaries in a Functional API Model: ```python def my_summary(x): tf.summary.histogram('x', x) return x inputs = tf.keras.Input(10) x = tf.keras.layers.Dense(10)(inputs) outputs = tf.keras.layers.Lambda(my_summary)(x) model = tf.keras.Model(inputs, outputs) model.compile('sgd', 'mse') # Make sure to set `update_freq=N` to log a batch-level summary every N # batches. In addition to any `tf.summary` contained in `Model.call`, # metrics added in `Model.compile` will be logged every N batches. tb_callback = tf.keras.callbacks.TensorBoard('./logs', update_freq=1) model.fit(x_train, y_train, callbacks=[tb_callback]) ``` Profiling: ```python # Profile a single batch, e.g. the 5th batch. tensorboard_callback = tf.keras.callbacks.TensorBoard( log_dir='./logs', profile_batch=5) model.fit(x_train, y_train, epochs=2, callbacks=[tensorboard_callback]) # Profile a range of batches, e.g. from 10 to 20. tensorboard_callback = tf.keras.callbacks.TensorBoard( log_dir='./logs', profile_batch=(10,20)) model.fit(x_train, y_train, epochs=2, callbacks=[tensorboard_callback]) ``` rKrTFrNc  std|_|| t||_||_||_||_ ||_ |dkrJdn||_ ||_ | |_ ||d|_d|_d|_d|_i|_g|_dS)NTrrr)rrqrQ_validate_kwargsrrlog_dirhistogram_freq write_graph write_imageswrite_steps_per_second update_freqembeddings_freqembeddings_metadata_init_profile_batch_global_train_batch_previous_epoch_iterations_train_accumulated_timerk_writers_prev_summary_state) rmrrrrrr profile_batchrrr"rr+r,rqQ s$    zTensorBoard.__init__cCs|ddrtd|ddr,td|ddrBtd|ddrXtd hd }t||}|rtd |d |d S)z&Handle arguments were supported in V1. write_gradsFzO`write_grads` will be ignored in TensorFlow 2.0 for the `TensorBoard` Callback.rze`batch_size` is no longer needed in the `TensorBoard` Callback and will be ignored in TensorFlow 2.0.embeddings_layer_nameszp`embeddings_layer_names` is not supported in TensorFlow 2.0. Instead, all `Embedding` layers will be visualized.embeddings_datazl`embeddings_data` is not supported in TensorFlow 2.0. Instead, all `Embedding` variables will be visualized.>rrrrz2Unrecognized arguments in `TensorBoard` Callback: z. Supported kwargs are: N)rrrrrr})rmr"supported_kwargsunrecognized_kwargsr+r+r,rw s4    zTensorBoard._validate_kwargscCsz||_||_tj|jd|_|jj|_tj|jd|_ |jj |_ i|_ d|_ |jrh|d|_ |jrv|dS)z/Sets Keras model and writes graph if specified.r validationFTN)r%_get_log_write_dir_log_write_dirrJrKrS _train_dirrr_val_dirr _val_stepr_should_write_train_graphr_write_keras_model_summaryr_configure_embeddingsrr+r+r,r# s   zTensorBoard.set_modelcCs(d|jvrtj|j|jd<|jdS)Nr)rrArcreate_file_writerrrr+r+r, _train_writer s   zTensorBoard._train_writercCs(d|jvrtj|j|jd<|jdS)Nval)rrArrrrr+r+r, _val_writer s zTensorBoard._val_writercCst|j|jjS)zBFor multi-worker, only chief should write, others write to '/tmp'.)r write_dirpathrr%r>rr+r+r,r s zTensorBoard._get_log_write_dircCst|j|jjdS)z/Deletes tmp write directories for multi-worker.N)rremove_temp_dirpathrr%r>rr+r+r,_delete_tmp_write_dir s z!TensorBoard._delete_tmp_write_dirc Cs||j^tjd2|jj}t|dr r3z*TensorBoard._push_writer..rr) rrvaluerArrrr __enter__)rmwriterr should_recordsummary_contextr+r r, _push_writer s     zTensorBoard._push_writercCs@|jdkrdS|j}|djt|djtdS)zPops the current writer.rNrr)rrr__exit__sysexc_info)rmprevious_contextr+r+r, _pop_writer s   zTensorBoard._pop_writercCs|jD] }|q dSrY)rvaluesclose)rmr r+r+r,_close_writers) szTensorBoard._close_writerscCsd|}t|tr0t|d}tjt|}t|trH||_||_n0t|t t frpt |dkrp|\|_|_nt ||jdks|j|jkrt |d|_ |jdkr|jdd|jddd|_|jdko|jdk |_d S) aValidate profile_batch value and set the range of batches to profile. Sets values of _start_batch and _stop_batch attributes, specifying the start and stop batch to profile. Setting `profile_batch=0` disables profiling. Args: profile_batch: The range of batches to profile. Should be a non-negative integer or a comma separated string of pair of positive integers. A pair of positive integers signify a range of batches to profile. Raises: ValueError: If profile_batch is not an integer or a comma separated pair of positive integers. zprofile_batch must be a non-negative integer or 2-tuple of positive integers. A pair of positive integers signifies a range of batches to profile. Found: rrFrGlogdirr5N)rrsplitrAr^ map_structurer  _start_batch _stop_batchtuplerrr}_profiler_started_start_profiler_stop_profiler _is_tracing _should_trace)rmrprofile_batch_error_messager+r+r,r- s2       zTensorBoard._init_profile_batchcCs d|_d|_||j|jdSr)rrrrrrr+r+r,ri szTensorBoard.on_train_begincCs*||jr|||dSrY)rr% _stop_tracerrrr+r+r,rn s zTensorBoard.on_train_endcCs||j|jdSrY)rrrrr+r+r,rw szTensorBoard.on_test_beginc Cs|jjrt|jjdrtjdp|jF|D],\}}tjj d|d||jjj dq8Wdn1sz0YWdn1s0Y| dS)N iterationsT evaluation__vs_iterationsr) r%rr@rArrrrrscalarr) read_valuer)rmrKrr r+r+r,rz s  DzTensorBoard.on_test_endcCs |jp |jSrY)r&rrr+r+r,rV sz)TensorBoard._implements_train_batch_hookscCs@|jd7_|jrt|_|js(dS|j|jkr<|dS)Nr)rrrrkr&r _start_tracerr+r+r,r s  z TensorBoard.on_train_batch_begincCsh|jr|d|_|jr@t|j}tjjdd||jd|j sJdS|j rd|j |j krd| dS)NFbatch_steps_per_secondrr)rrrrrkrArr,rr&r%rr r()rmrrKbatch_run_timer+r+r,r szTensorBoard.on_train_batch_endcCs$|jr |jjj|_t|_dSrY)rr%rr)rrr_epoch_start_timerr+r+r,r s zTensorBoard.on_epoch_begincCsL||||jr*||jdkr*|||jrH||jdkrH||dS)z2Runs metrics and histogram summaries at epoch end.rN)_log_epoch_metricsr _log_weightsr_log_embeddingsrr+r+r,r s   zTensorBoard.on_epoch_endcCs(tjjddd|j|jdd|_dS)NTF)rprofilerr)rArtrace_onr#rr%rr+r+r,r. szTensorBoard._start_tracec Cs|dur|j}|jPtjd$tjjd||dWdn1sP0YWdn1sn0Y|d|_dS)z$Logs the trace graph to TensorBoard.NTzbatch_%d)rrF) r rrrArr trace_exportr$r%)rmrr+r+r,r( s PzTensorBoard._stop_tracecCs2t|jjdd}t|tjr.||jjj|d<|S)Nr learning_rate)rRr%rrrLearningRateScheduler))rmrK lr_scheduler+r+r,_collect_learning_rate s z"TensorBoard._collect_learning_ratecCs.|jjj}t|j}||j|}|SrY)r%rr)rrr1r)rmcurrent_iterationZtime_since_epoch_beginsteps_per_secondr+r+r,_compute_steps_per_second s z%TensorBoard._compute_steps_per_secondc Cs6|sdSdd|D}dd|D}||}|jrH||d<tjd|r|j8|D]\}}tjj d|||dqnWdn1s0Y|r|j D|D]*\}}|d d}tjj d|||dqWdn1s0YWdn1s(0YdS) zWrites epoch metrics out as scalar summaries. Args: epoch: Int. The global step to use for TensorBoard. logs: Dict. Keys are scalar summary names, values are scalars. NcSs i|]\}}|ds||qSr.rr0rrr+r+r, r3z2TensorBoard._log_epoch_metrics..cSs i|]\}}|dr||qSr.r?r@r+r+r,rA r3r=Tepoch_r) rr;rr>rArrrrr,r)rmrrK train_logsval_logsrr r+r+r,r2 s"   6  zTensorBoard._log_epoch_metricsc Cs|jtjdx|jjD]T}|jD]H}|j dd}|d}tjj |||d|j r,|d}| |||q,q"|j Wdn1s0YWdn1s0YdS)z-Logs the weights of the Model to TensorBoard.T:rz /histogramrz/imageN)rrrArrr%rweightsrreplace histogramr_log_weight_as_imageflush)rmrrweight weight_nameZhistogram_weight_nameZimage_weight_namer+r+r,r3 s   zTensorBoard._log_weightscCst|}t|}t|dkr:t|d|dddg}nt|dkr|d|dkrjt|}t|}t|d|d|ddg}nTt|dkrtdkrtj|gdd}t|}t||d|d|ddg}t|}t|dkr|d d vrtjj |||d d S) z%Logs a weight as a TensorBoard image.rrr channels_last)rrr)permrCrz)rrNrCrN) rAsqueezer int_shaperreshape transposeimage_data_formatrimage)rmrLrMrw_imgshaper+r+r,rJ s"           z TensorBoard._log_weight_as_imagecCs(tj|jdd|}|j|dS)Nrzkeras_embedding.ckpt-{})rJrKrSrrr%r4)rmrembeddings_ckptr+r+r,r4 s zTensorBoard._log_embeddingsc Cs`|jr dSztjjj|dd|_Wn6tjjyZ}ztd|j WYd}~n d}~00dS)zStarts the profiler if currently inactive. Args: logdir: Directory where profiler results will be saved. NrTzFailed to start profiler: %s) r"rAr5restarterrorsAlreadyExistsErrorrerrormessage)rmrr(r+r+r,r#% s zTensorBoard._start_profilerc Csn|js dSzVztjjj|dWn6tjjyV}ztd|j WYd}~n d}~00Wd|_nd|_0dS)zStops the profiler if currently active. Args: save: Whether to save the profiler results to TensorBoard. NrzFailed to stop profiler: %sF) r"rAr5restopr[UnavailableErrorrr]r^)rmr5r(r+r+r,r$4 s&zTensorBoard._stop_profiler) rKrTFFrrrN)N)N)N)N)N)N)N)N)N)T)'r[rrrrqrr#propertyrrrrrrrrrrrrrrrrVrrrrr.r(r;r>r2r3rJr4r#r$rr+r+rr,rs\&*    + <       rz!keras.callbacks.ReduceLROnPlateaucsFeZdZdZdfdd Zd d Zdd dZdddZddZZ S)ReduceLROnPlateauaReduce learning rate when a metric has stopped improving. Models often benefit from reducing the learning rate by a factor of 2-10 once learning stagnates. This callback monitors a quantity and if no improvement is seen for a 'patience' number of epochs, the learning rate is reduced. Example: ```python reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.001) model.fit(X_train, Y_train, callbacks=[reduce_lr]) ``` Args: monitor: quantity to be monitored. factor: factor by which the learning rate will be reduced. `new_lr = lr * factor`. patience: number of epochs with no improvement after which learning rate will be reduced. verbose: int. 0: quiet, 1: update messages. mode: one of `{'auto', 'min', 'max'}`. In `'min'` mode, the learning rate will be reduced when the quantity monitored has stopped decreasing; in `'max'` mode it will be reduced when the quantity monitored has stopped increasing; in `'auto'` mode, the direction is automatically inferred from the name of the monitored quantity. min_delta: threshold for measuring the new optimum, to only focus on significant changes. cooldown: number of epochs to wait before resuming normal operation after lr has been reduced. min_lr: lower bound on the learning rate. r皙? rr-C6?c  st||_|dkr&td|d| vrB| d}td||_||_||_ ||_ ||_ ||_ d|_ d|_d|_||_d|_|dS)Nrz8ReduceLROnPlateau does not support a factor >= 1.0. Got epsilonzN`epsilon` argument is deprecated and will be removed, use `min_delta` instead.r)rrqrr}rrrfactormin_lrr~r{rcooldowncooldown_counterrrrr_reset) rmrrgr{rrr~rirhr"rr+r,rqj s2  zReduceLROnPlateau.__init__cs|jdvrtdjd_jdks<jdkrTdjvrTfdd_tj_nfdd_tj _d _d _ d S) z)Resets wait counter and cooldown counter.r zBLearning rate reduction mode %s is unknown, fallback to auto mode.rr r cst||jSrY)rrr~abrr+r,r  r3z*ReduceLROnPlateau._reset..cst||jSrY)rrr~rlrr+r,r  r3rN) rrrrrrrrrjrrr+rr,rk s"    zReduceLROnPlateau._resetNcCs |dSrY)rkrr+r+r,r sz ReduceLROnPlateau.on_train_beginc Cs2|pi}t|jjj|d<||j}|durPtd|jd t | n| rl|j d8_ d|_|||jr||_d|_n| s.|jd7_|j|jkr.t|jjj}|t|jkr.||j}t||j}t|jjj||jdkr td|dd|d|j|_ d|_dS) NrzgLearning rate reduction is conditioned on metric `%s` which is not available. Available metrics are: %srrrr1z.: ReduceLROnPlateau reducing learning rate to r)rrr%rrrrrrrSrr in_cooldownrjrrrr{rrrhrgr rrrrri)rmrrKr:old_lrnew_lrr+r+r,r s>      zReduceLROnPlateau.on_epoch_endcCs |jdkSr)rjrr+r+r,ro szReduceLROnPlateau.in_cooldown)rrcrdrrrerr)N)N) r[rrrrqrkrrrorr+r+rr,rbE s%'  %rbzkeras.callbacks.CSVLoggercs@eZdZdZd fdd ZdddZdd d Zdd d ZZS) CSVLoggeraCCallback that streams epoch results to a CSV file. Supports all values that can be represented as a string, including 1D iterables such as `np.ndarray`. Example: ```python csv_logger = CSVLogger('training.log') model.fit(X_train, Y_train, callbacks=[csv_logger]) ``` Args: filename: Filename of the CSV file, e.g. `'run/log.csv'`. separator: String used to separate elements in the CSV file. append: Boolean. True: append if file exists (useful for continuing training). False: overwrite existing file. rFcs8||_t||_||_d|_d|_d|_t dSr) seprrfilenamer r r append_headerrrq)rmrt separatorr rr+r,rq s zCSVLogger.__init__NcCs~|jrbtjj|jr\tjj|jd$}tt|  |_ Wdn1sR0Yd}nd}tjj|j||_ dS)Nrrmr) r rArCrDrErtrr`rreadlinerucsv_file)rmrKrrr+r+r,r s2zCSVLogger.on_train_begincspiddjdur(t_jjrHtfddjDjsGfdddtj}dgj}tjj ||d_j rj t d|i}|fd djDj|j dS) NcSsRt|tjo|jdk}t|tr$|St|tjjrJ|sJddt t|S|SdS)Nrz"[%s]"z, ) rrrndimr collectionsabcIterablerSmap)ris_zero_dim_ndarrayr+r+r, handle_value s  z,CSVLogger.on_epoch_end..handle_valuec3s*|]"}|vr||fn|dfVqdS)NANr+)r0rrr+r,rU sz)CSVLogger.on_epoch_end..cseZdZjZdS)z-CSVLogger.on_epoch_end..CustomDialectN)r[rrrs delimiterr+rr+r, CustomDialect srr) fieldnamesdialectc3s|]}||fVqdSrYr+)r0key)rrKr+r,rU r3)rsortedr%r&dictr csvexcel DictWriterryru writeheaderr{ OrderedDictrwriterowrK)rmrrKrrrow_dictr+)rrKrmr,r s(     zCSVLogger.on_epoch_endcCs|jd|_dSrY)ryrr rr+r+r,r# s zCSVLogger.on_train_end)rF)N)N)N) r[rrrrqrrrrr+r+rr,rr s  *rrzkeras.callbacks.LambdaCallbackcs"eZdZdZdfdd ZZS)LambdaCallbackaCallback for creating simple, custom callbacks on-the-fly. This callback is constructed with anonymous functions that will be called at the appropriate time (during `Model.{fit | evaluate | predict}`). Note that the callbacks expects positional arguments, as: - `on_epoch_begin` and `on_epoch_end` expect two positional arguments: `epoch`, `logs` - `on_batch_begin` and `on_batch_end` expect two positional arguments: `batch`, `logs` - `on_train_begin` and `on_train_end` expect one positional argument: `logs` Args: on_epoch_begin: called at the beginning of every epoch. on_epoch_end: called at the end of every epoch. on_batch_begin: called at the beginning of every batch. on_batch_end: called at the end of every batch. on_train_begin: called at the beginning of model training. on_train_end: called at the end of model training. Example: ```python # Print the batch number at the beginning of every batch. batch_print_callback = LambdaCallback( on_batch_begin=lambda batch,logs: print(batch)) # Stream the epoch loss to a file in JSON format. The file content # is not well-formed JSON but rather has a JSON object per line. import json json_log = open('loss_log.json', mode='wt', buffering=1) json_logging_callback = LambdaCallback( on_epoch_end=lambda epoch, logs: json_log.write( json.dumps({'epoch': epoch, 'loss': logs['loss']}) + '\n'), on_train_end=lambda logs: json_log.close() ) # Terminate some processes after having finished model training. processes = ... cleanup_callback = LambdaCallback( on_train_end=lambda logs: [ p.terminate() for p in processes if p.is_alive()]) model.fit(..., callbacks=[batch_print_callback, json_logging_callback, cleanup_callback]) ``` Nc snt|j||dur$||_|dur2||_|dur@||_|durN||_|dur\||_|durj||_ dSrY) rrq__dict__rrrrrrr)rmrrrrrrr"rr+r,rq] s  zLambdaCallback.__init__)NNNNNNrvr+r+rr,r( s5r)rG)N)Drr{r8rrrJrNrrrrtensorflow.compat.v2rBv2rArrZkeras.distributerrkeras.optimizers.schedulesr keras.utilsrrrr keras.utils.data_utilsr keras.utils.generic_utilsr keras.utils.mode_keysr tensorflow.python.platformr rtensorflow.python.utilr tensorflow.python.util.tf_exportrtensorflow.tools.docsrrrrr-r$rFrOrrrrr!rrradeprecated_endpointsrurwrrrTensorBoardVersionSelectorrrbrrrr+r+r+r,s                  D 2 "<12'w " (FG 1 x V