a Sic}@sdZddlZddlZddlZddlZddlZddlmm Z ddl m Z ddl mZddl mZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlm Z edGdddej!ej"dZ#Gddde#Z$edGddde$Z%edGddde$Z&edGddde&Z'edGd d!d!e#Z(Gd"d#d#e$Z)Gd$d%d%e)Z*d&d'Z+d(d)Z,d*d+Z-dS),zBase Metric classes.N)backend) dtensor_api)utils) base_layer)base_layer_utils) keras_tensor)metric_serialization) generic_utils) losses_utils) metrics_utils)tf_utils) keras_export) doc_controlszkeras.metrics.MetriccseZdZdZd$fdd ZfddZddZd d Zd d Ze d dZ ddZ ddZ e jddZddZe jddZejdejjejjddffdd Ze ddZe ddZe d d!Zejejd"d#Z Z!S)%Metrica Encapsulates metric logic and state. Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. **kwargs: Additional layer keywords arguments. Standalone usage: ```python m = SomeMetric(...) for input in ...: m.update_state(input) print('Final result: ', m.result().numpy()) ``` Usage with `compile()` API: ```python model = tf.keras.Sequential() model.add(tf.keras.layers.Dense(64, activation='relu')) model.add(tf.keras.layers.Dense(64, activation='relu')) model.add(tf.keras.layers.Dense(10, activation='softmax')) model.compile(optimizer=tf.keras.optimizers.RMSprop(0.01), loss=tf.keras.losses.CategoricalCrossentropy(), metrics=[tf.keras.metrics.CategoricalAccuracy()]) data = np.random.random((1000, 32)) labels = np.random.random((1000, 10)) dataset = tf.data.Dataset.from_tensor_slices((data, labels)) dataset = dataset.batch(32) model.fit(dataset, epochs=10) ``` To be implemented by subclasses: * `__init__()`: All state variables should be created in this method by calling `self.add_weight()` like: `self.var = self.add_weight(...)` * `update_state()`: Has all updates to the state variables like: self.var.assign_add(...). * `result()`: Computes and returns a scalar value or a dict of scalar values for the metric from the state variables. Example subclass implementation: ```python class BinaryTruePositives(tf.keras.metrics.Metric): def __init__(self, name='binary_true_positives', **kwargs): super(BinaryTruePositives, self).__init__(name=name, **kwargs) self.true_positives = self.add_weight(name='tp', initializer='zeros') def update_state(self, y_true, y_pred, sample_weight=None): y_true = tf.cast(y_true, tf.bool) y_pred = tf.cast(y_pred, tf.bool) values = tf.logical_and(tf.equal(y_true, True), tf.equal(y_pred, True)) values = tf.cast(values, self.dtype) if sample_weight is not None: sample_weight = tf.cast(sample_weight, self.dtype) sample_weight = tf.broadcast_to(sample_weight, values.shape) values = tf.multiply(values, sample_weight) self.true_positives.assign_add(tf.reduce_sum(values)) def result(self): return self.true_positives ``` Nc sNtjf||d|d|_d|_tsJ|dur.update_state_fncs*tjj}tjj|}||i|Sr"r#)r(rr) ag_result) obj_resultr r! result_fns  z!Metric.__new__..result_fn)rr__new__ris_in_eager_or_tf_function is_built_in update_state isinstancerr$functionFunctiontypes MethodTyper update_state_wrapperresultresult_wrapper)clsr(robjr,r/r)r.r+r!r0s     zMetric.__new__cs0fdd}ddlm}|j|g|Ri|S)aAccumulates statistics and then computes metric result value. Args: *args: **kwargs: A mini-batch of inputs to the Metric, passed on to `update_state()`. Returns: The metric value tensor. cstddtj||fDr$d}nj|i|}g}|durJ||t|"}|_|WdS1s|0YdS)z;Updates the state of the metric in a replica-local context.css|]}t|tjVqdSr")r4r KerasTensor).0argr r r! sz.replica_local_fn..N) anyrnestflattenr3appendcontrol_dependenciesr: _metric_obj)r(r update_op update_opsresult_trr r!replica_local_fns   z)Metric.__call__..replica_local_fnr)distributed_training_utils)Zkeras.distributerMcall_replica_local_fn)rr(rrLrMr rKr!__call__s  zMetric.__call__cCs0ddd|D}|jjd|dS)N,css |]\}}|d|VqdS)=Nr r?kvr r r!rAz!Metric.__str__..())join get_configitemsr__name__)rr(r r r!__str__szMetric.__str__c Cst||j|jd}||t|<|jD]^\}}|dvr rUz&Metric.reset_state..N) r is_default reset_stateswarningswarnrr[rbatch_set_value variablesrKr r r! reset_states zMetric.reset_statecOs tddS)aAccumulates statistics for the metric. Note: This function is executed as a graph function in graph mode. This means: a) Operations on the same resource are executed in textual order. This should make it easier to do things like add the updated value of a variable to another, for example. b) You don't need to worry about collecting the update ops to execute. All update ops added to the graph by this function will be executed. As a result, code should generally work the same way with graph or eager execution. Args: *args: **kwargs: A mini-batch of inputs to the Metric. "Must be implemented in subclasses.NNotImplementedError)rr(rr r r!r3szMetric.update_statecCsbg}|D]T}t|jt|jkr4td|d|t|j|jD]\}}|||qBq|S)asMerges the state from one or more metrics. This method can be used by distributed systems to merge the state computed by different metric instances. Typically the state will be stored in the form of the metric's weights. For example, a tf.keras.metrics.Mean metric contains a list of two weight values: a total and a count. If there were two instances of a tf.keras.metrics.Accuracy that each independently aggregated partial state for an overall accuracy calculation, these two metric's states could be combined as follows: >>> m1 = tf.keras.metrics.Accuracy() >>> _ = m1.update_state([[1], [2]], [[0], [2]]) >>> m2 = tf.keras.metrics.Accuracy() >>> _ = m2.update_state([[3], [4]], [[3], [4]]) >>> m2.merge_state([m1]) >>> m2.result().numpy() 0.75 Args: metrics: an iterable of metrics. The metrics must have compatible state. Raises: ValueError: If the provided iterable does not contain metrics matching the metric's required specifications. zMetric z is not compatible with )lenweights ValueErrorziprE assign_add)rmetricsZassign_add_opsmetricweightZ weight_to_addr r r! merge_state#szMetric.merge_statecCs tddS)a,Computes and returns the scalar metric value tensor or a dict of scalars. Result computation is an idempotent operation that simply calculates the metric value using the state variables. Returns: A scalar tensor, or a dictionary of scalar tensors. ruNrvrKr r r!r:Ks z Metric.resultr c stjrtj}nd}t|r,tjj}t|dddurZdt j |j t |ji}ni}tj|dBtjf|||dur|jn|d|g||d|WdS1s0YdS)z0Adds state variable. Only for use by subclasses.N_meshlayout)layerF)rshaper trainable initializer collectionssynchronization aggregation)r distribute has_strategy get_strategyris_tpu_strategyVariableSynchronizationON_WRITEgetattrdtensorLayout replicatedr TensorShaperankr maybe_init_scoper add_weightr) rrrrrrrstrategyZadditional_kwargsrr r!rYs0    zMetric.add_weightcCs4|jr,|j}|jD]}||j7}q||SgSdSr")r_trainable_weights_metricstrainable_weights_dedup_weights)rrmr r r!rs    zMetric.trainable_weightscCsP|jr$|j}|jD]}||j7}qn"|j|j}|jD]}||j7}q6||Sr")r_non_trainable_weightsrnon_trainable_weightsrryr)rrrr r r!rs    zMetric.non_trainable_weightscCs t|Sr")rMetricSavedModelSaverrKr r r!_trackable_saved_model_saversz#Metric._trackable_saved_model_savercCs|Sr")rtrKr r r!roszMetric.reset_states)NN)"r[ __module__ __qualname____doc__rr0rOr\ripropertyrrYrtabcabstractmethodr3rr:rfor_subclass_implementersrVariableAggregationSUMrON_READrrrrr defaultdo_not_generate_docsro __classcell__r r rr!r,s>G */  ( +   r) metaclasscs4eZdZdZd fdd Zd ddZddZZS) ReducezEncapsulates metrics that perform a reduce operation on the values. Args: reduction: a `tf.keras.metrics.Reduction` enum value. name: string name of the metric instance. dtype: (Optional) data type of the metric result. NcsNtj||d||_|jddd|_|tjjtjjfvrJ|jddd|_ dS)Nrtotalzeros)rcount) rr reductionrrr ReductionSUM_OVER_BATCH_SIZE WEIGHTED_MEANr)rrrrrr r!rszReduce.__init__c Cs$t|g|\\}}zt||j}Wn<ttfybd|d}t|trV|d7}t |Yn0|durt||j}t j ||d\}}}ztj j ||}Wnhty t|}t|}|jtjjkrtj|tt||d}ntj|tt||d}Yn0t||}t|}t|g|j|}Wdn1sT0Y|jtjjkrr|S|jtjjkrtt||j} nN|jtjjkr|durtt||j} n t|} ntd|jdt|g|j | WdS1s0YdS) zAccumulates statistics for computing the metric. Args: values: Per-example value. sample_weight: Optional weighting of each example. Defaults to 1. Returns: Update op. zGThe output of a metric function can only be a single Tensor. Received: z. z?To return a dict of values, implement a custom Metric subclass.N sample_weightaxis Reduction "M" not implemented. Expected "sum", "weighted_mean", or "sum_over_batch_size".)!r ,ragged_assert_compatible_and_get_flat_valuesrcastrrz TypeErrorr4dict RuntimeErrorr squeeze_or_expand_dimensionsr$opsbroadcast_weightsrndimrrr reduce_sumlistrange reduce_meanmultiplyrFrr|rsizerrwr) rvaluesrmsg_r weight_ndim value_sumupdate_total_op num_valuesr r r!r3st        ,   zReduce.update_statecCsX|jtjjkrt|jS|jtjjtjjfvrBtj |j|j St d|jddS)Nrr) rr rrridentityrrrmath divide_no_nanrrwrKr r r!r:s  z Reduce.result)N)N)r[rrrrr3r:rr r rr!rs Rrzkeras.metrics.Sumcs(eZdZdZejdfdd ZZS)SumaYComputes the (weighted) sum of the given values. For example, if values is [1, 3, 5, 7] then the sum is 16. If the weights were specified as [1, 1, 0, 0] then the sum would be 4. This metric creates one variable, `total`, that is used to compute the sum of `values`. This is ultimately returned as `sum`. If `sample_weight` is `None`, weights default to 1. Use `sample_weight` of 0 to mask values. Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. Standalone usage: >>> m = tf.keras.metrics.Sum() >>> m.update_state([1, 3, 5, 7]) >>> m.result().numpy() 16.0 Usage with `compile()` API: ```python model.add_metric(tf.keras.metrics.Sum(name='sum_1')(outputs)) model.compile(optimizer='sgd', loss='mse') ``` sumNcstjtjj||ddSN)rrr)rrr rrrrrrr r!r?s z Sum.__init__)rNr[rrr dtensor_utils inject_meshrrr r rr!rsrzkeras.metrics.Meancs(eZdZdZejdfdd ZZS)MeanaBComputes the (weighted) mean of the given values. For example, if values is [1, 3, 5, 7] then the mean is 4. If the weights were specified as [1, 1, 0, 0] then the mean would be 2. This metric creates two variables, `total` and `count` that are used to compute the average of `values`. This average is ultimately returned as `mean` which is an idempotent operation that simply divides `total` by `count`. If `sample_weight` is `None`, weights default to 1. Use `sample_weight` of 0 to mask values. Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. Standalone usage: >>> m = tf.keras.metrics.Mean() >>> m.update_state([1, 3, 5, 7]) >>> m.result().numpy() 4.0 >>> m.reset_state() >>> m.update_state([1, 3, 5, 7], sample_weight=[1, 1, 0, 0]) >>> m.result().numpy() 2.0 Usage with `compile()` API: ```python model.add_metric(tf.keras.metrics.Mean(name='mean_1')(outputs)) model.compile(optimizer='sgd', loss='mse') ``` meanNcstjtjj||ddSr)rrr rrrrr r!rls z Mean.__init__)rNrr r rr!rFs$rzkeras.metrics.MeanMetricWrappercsReZdZdZejd fdd Zd fdd ZfddZe fd d Z Z S) MeanMetricWrapperauWraps a stateless metric function with the Mean metric. You could use this class to quickly build a mean metric from a function. The function needs to have the signature `fn(y_true, y_pred)` and return a per-sample loss array. `MeanMetricWrapper.result()` will return the average metric value across all samples seen so far. For example: ```python def accuracy(y_true, y_pred): return tf.cast(tf.math.equal(y_true, y_pred), tf.float32) accuracy_metric = tf.keras.metrics.MeanMetricWrapper(fn=accuracy) keras_model.compile(..., metrics=accuracy_metric) ``` Args: fn: The metric function to wrap, with signature `fn(y_true, y_pred, **kwargs)`. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. **kwargs: Keyword arguments to pass on to `fn`. Nc s tj||d||_||_dS)Nrrr_fn _fn_kwargsrfnrrrrr r!rszMeanMetricWrapper.__init__cst||j}t||j}t||g|\\}}}t||\}}tjj |j tjj }|||fi|j }t |}t||||j}tj||dS)a Accumulates metric statistics. `y_true` and `y_pred` should have the same shape. Args: y_true: Ground truth values. shape = `[batch_size, d0, .. dN]`. y_pred: The predicted values. shape = `[batch_size, d0, .. dN]`. sample_weight: Optional `sample_weight` acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If `sample_weight` is a tensor of size `[batch_size]`, then the metric for each sample of the batch is rescaled by the corresponding element in the `sample_weight` vector. If the shape of `sample_weight` is `[batch_size, d0, .. dN-1]` (or can be broadcasted to this shape), then each metric element of `y_pred` is scaled by the corresponding value of `sample_weight`. (Note on `dN-1`: all metric functions reduce by 1 dimension, usually the last axis (-1)). Returns: Update op. r)rrrr rr rr$r%r'rr&rget_maskapply_valid_maskrrr3ry_truey_predrag_fnmatchesmaskrr r!r3s*  zMeanMetricWrapper.update_statecsPdd|jD}t|tur*|j|d<t}tt|t|S)NcSs*i|]"\}}|t|r"t|n|qSr r is_tensor_or_variablerevalrRr r r! sz0MeanMetricWrapper.get_config..r) rrZr`rrrrYrrrconfig base_configrr r!rYs   zMeanMetricWrapper.get_configcsDddlm}|dd}|tur4|||fi|Stt||S)Nr)getr)Z keras.metricsrpoprr from_config)r<rrrrr r!rs   zMeanMetricWrapper.from_config)NN)N) r[rrrrrrr3rY classmethodrrr r rr!rus, rzkeras.metrics.MeanTensorcsbeZdZdZejdfdd ZddZedd Z ed d Z dd d Z ddZ ddZ ZS) MeanTensoraComputes the element-wise (weighted) mean of the given tensors. `MeanTensor` returns a tensor with the same shape of the input tensors. The mean value is updated by keeping local variables `total` and `count`. The `total` tracks the sum of the weighted values, and `count` stores the sum of the weighted counts. Args: name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. shape: (Optional) A list of integers, a tuple of integers, or a 1-D Tensor of type int32. If not specified, the shape is inferred from the values at the first call of update_state. Standalone usage: >>> m = tf.keras.metrics.MeanTensor() >>> m.update_state([0, 1, 2, 3]) >>> m.update_state([4, 5, 6, 7]) >>> m.result().numpy() array([2., 3., 4., 5.], dtype=float32) >>> m.update_state([12, 10, 8, 6], sample_weight= [0, 0.2, 0.5, 1]) >>> m.result().numpy() array([2. , 3.6363635, 4.8 , 5.3333335], dtype=float32) >>> m = tf.keras.metrics.MeanTensor(dtype=tf.float64, shape=(1, 4)) >>> m.result().numpy() array([[0., 0., 0., 0.]]) >>> m.update_state([[0, 1, 2, 3]]) >>> m.update_state([[4, 5, 6, 7]]) >>> m.result().numpy() array([[2., 3., 4., 5.]]) mean_tensorNcs>tj||dd|_d|_d|_d|_|dur:||dS)NrF)rr_shape_total_count_built_build)rrrrrr r!rszMeanTensor.__init__cCst||_|j|_|jd|dd|_|jd|dd|_t&tsXt t Wdn1sl0Yd|_ dS)Nrr)rrrrT) rrr_build_input_shaperrr init_scopeexecuting_eagerlyr_initialize_variables _get_sessionr)rrr r r!r s  ,zMeanTensor._buildcCs|jr |jSdSr")rrrKr r r!rszMeanTensor.totalcCs|jr |jSdSr")rrrKr r r!rszMeanTensor.countc Cs2t||j}|js"||jn&|j|jkrHtd|jd|jdt|}|durt||j}t j ||d\}}}ztj j ||}Wn>tyt|}t|}tj|tt||d}Yn0t||}t||}|j|}t|g|j|WdS1s$0YdS)zAccumulates statistics for computing the element-wise mean. Args: values: Per-example value. sample_weight: Optional weighting of each example. Defaults to 1. Returns: Update op. zeMeanTensor input values must always have the same shape. Expected shape (set during the first call): z. Got: .Nrr)rrrrrrrrz ones_liker rr$rrrrrrrrrr|rFr)rrrrrrrrr r r!r3#sH          zMeanTensor.update_statecCs |jstdtj|j|jS)NzMeanTensor does not have any value yet. Please call the MeanTensor instance or use `.update_state(value)` before retrieving the result.)rrzrrrrrrKr r r!r:Xs zMeanTensor.resultcCs |jrtdd|jDdS)NcSs g|]}|t|jfqSr )nprras_listrlr r r!rmdrUz*MeanTensor.reset_state..)rrrrrsrKr r r!rtaszMeanTensor.reset_state)rNN)N)r[rrrrrrrrrrr3r:rtrr r rr!rs#    5 rcs"eZdZdZdfdd ZZS)SumOverBatchSizea=Computes the weighted sum over batch size of the given values. For example, if values is [1, 3, 5, 7] then the metric value is 4. If the weights were specified as [1, 1, 0, 0] then the value would be 1. This metric creates two variables, `total` and `count` that are used to compute the average of `values`. This average is ultimately returned as sum over batch size which is an idempotent operation that simply divides `total` by `count`. If `sample_weight` is `None`, weights default to 1. Use `sample_weight` of 0 to mask values. sum_over_batch_sizeNcstjtjj||ddSr)rrr rrrrr r!rws zSumOverBatchSize.__init__)r N)r[rrrrrr r rr!r hsr cs<eZdZdZd fdd Zd fdd ZfddZZS) SumOverBatchSizeMetricWrapperzAWraps a function with the `SumOverBatchSizeMetricWrapper` metric.Nc s tj||d||_||_dS)arCreates a `SumOverBatchSizeMetricWrapper` instance. Args: fn: The metric function to wrap, with signature `fn(y_true, y_pred, **kwargs)`. name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result. **kwargs: The keyword arguments that are passed on to `fn`. rNrrrr r!rs z&SumOverBatchSizeMetricWrapper.__init__cst||j}t||j}t||\}}tjj|jtjj }|||fi|j }t |}t ||||j }tj||dS)Nr)rrrr rr$r%r'rr&rrrrrr3rrr r!r3s  z*SumOverBatchSizeMetricWrapper.update_statecs:dd|jD}t}tt|t|S)NcSs*i|]"\}}|t|r"t|n|qSr rrRr r r!rsz.)rrZrrYrrrrr r!rYs  z(SumOverBatchSizeMetricWrapper.get_config)NN)N)r[rrrrr3rYrr r rr!r sr cCsFt|trBt |j|WdS1s80Y|S)zFReturns a clone of the metric if stateful, otherwise returns it as is.N)r4rrrrrrY)r~r r r! clone_metrics  .r cCstjt|S)z"Clones the given metric list/dict.)rrC map_structurer )r}r r r! clone_metricssrcCs"|jdtjdddS)Nr)r startswithrXrsplit)r<r r r!r2sr2).rrrer7rpnumpyrtensorflow.compat.v2compatv2rkerasr keras.dtensorrrrr keras.enginerrrkeras.saving.saved_modelr keras.utilsr r r r tensorflow.python.util.tf_exportr tensorflow.tools.docsrLayerABCMetarrrrrrr r r rr2r r r r!sN             t&.g +