a Sic!@s.dZddlZddlZddlmZddlZddlmm Z ddl m Z ddl mZddl mZddlmZdZGd d d eZd d Zd dZddZddZd0ddZGdddeZGdddeZGdddeZd1ddZdd Zd2d!d"Zd#d$Zd3d%d&Z d'd(Z!d4d)d*Z"d+d,Z#d5d.d/Z$dS)6zUtils related to keras metrics.N)Enum)backend) losses_utils)tf_utils)to_listg _c@seZdZdZdZdZdZdS) Reductiona%Types of metrics reduction. Contains the following values: * `SUM`: Scalar sum of weighted values. * `SUM_OVER_BATCH_SIZE`: Scalar sum of weighted values divided by number of elements. * `WEIGHTED_MEAN`: Scalar sum of weighted values divided by sum of weights. sumsum_over_batch_size weighted_meanN)__name__ __module__ __qualname____doc__SUMSUM_OVER_BATCH_SIZE WEIGHTED_MEANrrU/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/utils/metrics_utils.pyr!s rcsfdd}tjj|S)zDecorator to wrap metric `update_state()` with `add_update()`. Args: update_state_fn: function that accumulates metric statistics. Returns: Decorated function that wraps `update_state_fn()` with `add_update()`. cstj}|jD],}t|r|j|stjst dqt j |i||i|}Wdn1sr0Y|dur| ||S)z'Decorated function with `add_update()`.zTrying to run metric.update_state in replica context when the metric was not created in TPUStrategy scope. Make sure the keras Metric is created in TPUstrategy scope. N) tf distribute get_strategyweightsris_tpu_strategyextendedvariable_created_in_scopein_cross_replica_context ValueErrorr"graph_context_for_symbolic_tensors add_update) metric_objargskwargsstrategyweight update_opupdate_state_fnrr decorated;s    , z'update_state_wrapper..decoratedr __internal__ decoratormake_decorator)r&r'rr%rupdate_state_wrapper1s r,csfdd}tjj|S)aDecorator to wrap metric `result()` function in `merge_call()`. Result computation is an idempotent operation that simply calculates the metric value using the state variables. If metric state variables are distributed across replicas/devices and `result()` is requested from the context of one device - This function wraps `result()` in a distribution strategy `merge_call()`. With this, the metric state variables will be aggregated across devices. Args: result_fn: function that computes the metric result. Returns: Decorated function that wraps `result_fn()` in distribution strategy `merge_call()`. c stj}|dustjjrtjj|}t|tjtjt t frVt |}n\t|t rtdd| D}n>zt |}Wn.ttfytd|jd|dYn0Wdq1s0Yndd}|j|f|d }||_|S) z#Decorated function with merge_call.NcSsi|]\}}|t|qSr)ridentity).0keyvaluerrr sz5result_wrapper..decorated..zqThe output of `metric.result()` can only be a single Tensor/Variable, or a dict of Tensors/Variables. For metric z , got result .cWs||d|}t|S)Nr)experimental_local_resultsrr-) distributionmerge_fnr resultrrrmerge_fn_wrappers z;result_wrapper..decorated..merge_fn_wrapper)r )rrget_replica_contextr)strategy_supports_no_merge_callvariable_sync_on_read_context isinstanceTensorVariablefloatintr-dictitemsr TypeError RuntimeErrorname merge_call _call_result)rr replica_context raw_resultresult_tr7 result_fnrrr'hs:    * z!result_wrapper..decoratedr()rKr'rrJrresult_wrapperUs WrLcs8|j|jt|jt|fdd}~|S)z-Creates a weak reference to the bound method.cs|i|SN)__get__)r r!clsfunc instance_refrrinnerszweakmethod..inner)im_classim_funcweakrefrefim_self functoolswraps)methodrSrrOr weakmethods r\cCs,|dur(dd|D}|r(td|dS)NcSs(g|] }|dus |dks |dkr|qS)Nrr)r.trrr sz+assert_thresholds_range..z.Threshold values must be in [0, 1]. Received: )r) thresholdsinvalid_thresholdsrrrassert_thresholds_rangesrb?cCs,|durtt|t|dur"|n|}|SrM)rbr)r`default_thresholdrrrparse_init_thresholdss  rec@seZdZdZdZdZdZdS)ConfusionMatrixtpfptnfnN)r r r TRUE_POSITIVESFALSE_POSITIVESTRUE_NEGATIVESFALSE_NEGATIVESrrrrrfsrfc@s$eZdZdZdZdZeddZdS)AUCCurvezType of AUC Curve (ROC or PR).ROCPRcCs0|dvrtjS|dvrtjStd|ddS)N)prrq)rocrpzInvalid AUC curve value: "z$". Expected values are ["PR", "ROC"])rorqrprr/rrrfrom_strs zAUCCurve.from_strN)r r r rrprq staticmethodrurrrrros roc@s(eZdZdZdZdZdZeddZdS)AUCSummationMethoda:Type of AUC summation method. https://en.wikipedia.org/wiki/Riemann_sum) Contains the following values: * 'interpolation': Applies mid-point summation scheme for `ROC` curve. For `PR` curve, interpolates (true/false) positives but not the ratio that is precision (see Davis & Goadrich 2006 for details). * 'minoring': Applies left summation for increasing intervals and right summation for decreasing intervals. * 'majoring': Applies right summation for increasing intervals and left summation for decreasing intervals. interpolationmajoringminoringcCs>|dvrtjS|dvrtjS|dvr*tjStd|ddS)N)rx Interpolation)ryMajoring)rzMinoringz%Invalid AUC summation method value: "z@". Expected values are ["interpolation", "majoring", "minoring"])rw INTERPOLATIONMAJORINGMINORINGrrtrrrrus zAUCSummationMethod.from_strN) r r r rr~rrrvrurrrrrws rwFcs|jd|durd}n.tjjtj||jd|}|sJt|dg}|durXd}n.t |d}tjj||}|st|dg}tt |||j}tj |ddd}tt|tj |j}|st|dg}t|dg}t ||} t d||} tj |dd} |r*tj| } t| tj} |rt| } t| } t| } fd d } t| | | f} t| | | f}ttj| d dd }ttj|d dd }n@tj j| | d } tj j| | d }tj| d d}tj|d d}tj|vstj|vrF|r2tj| dd}tj| dd}nt| }t| }g}tj|vrp|tj}|||tj|vr|tj}|||tj|vr|tj}||}|||tj|vr|tj}||}|||t|S)a.Update confusion matrix variables with memory efficient alternative. Note that the thresholds need to be evenly distributed within the list, eg, the diff between consecutive elements are the same. To compute TP/FP/TN/FN, we are measuring a binary classifier C(t) = (predictions >= t) at each threshold 't'. So we have TP(t) = sum( C(t) * true_labels ) FP(t) = sum( C(t) * false_labels ) But, computing C(t) requires computation for each t. To make it fast, observe that C(t) is a cumulative integral, and so if we have thresholds = [t_0, ..., t_{n-1}]; t_0 < ... < t_{n-1} where n = num_thresholds, and if we can compute the bucket function B(i) = Sum( (predictions == t), t_i <= t < t{i+1} ) then we get C(t_i) = sum( B(j), j >= i ) which is the reversed cumulative sum in tf.cumsum(). We can compute B(i) efficiently by taking advantage of the fact that our thresholds are evenly distributed, in that width = 1.0 / (num_thresholds - 1) thresholds = [0.0, 1*width, 2*width, 3*width, ..., 1.0] Given a prediction value p, we can map it to its bucket by bucket_index(p) = floor( p * (num_thresholds - 1) ) so we can use tf.math.unsorted_segment_sum() to update the buckets in one pass. Consider following example: y_true = [0, 0, 1, 1] y_pred = [0.1, 0.5, 0.3, 0.9] thresholds = [0.0, 0.5, 1.0] num_buckets = 2 # [0.0, 1.0], (1.0, 2.0] bucket_index(y_pred) = tf.math.floor(y_pred * num_buckets) = tf.math.floor([0.2, 1.0, 0.6, 1.8]) = [0, 0, 0, 1] # The meaning of this bucket is that if any of the label is true, # then 1 will be added to the corresponding bucket with the index. # Eg, if the label for 0.2 is true, then 1 will be added to bucket 0. If the # label for 1.8 is true, then 1 will be added to bucket 1. # # Note the second item "1.0" is floored to 0, since the value need to be # strictly larger than the bucket lower bound. # In the implementation, we use tf.math.ceil() - 1 to achieve this. tp_bucket_value = tf.math.unsorted_segment_sum(true_labels, bucket_indices, num_segments=num_thresholds) = [1, 1, 0] # For [1, 1, 0] here, it means there is 1 true value contributed by bucket # 0, and 1 value contributed by bucket 1. When we aggregate them to # together, the result become [a + b + c, b + c, c], since large thresholds # will always contribute to the value for smaller thresholds. true_positive = tf.math.cumsum(tp_bucket_value, reverse=True) = [2, 1, 0] This implementation exhibits a run time and space complexity of O(T + N), where T is the number of thresholds and N is the size of predictions. Metrics that rely on standard implementation instead exhibit a complexity of O(T * N). Args: variables_to_update: Dictionary with 'tp', 'fn', 'tn', 'fp' as valid keys and corresponding variables to update as values. y_true: A floating point `Tensor` whose shape matches `y_pred`. Will be cast to `bool`. y_pred: A floating point `Tensor` of arbitrary shape and whose values are in the range `[0, 1]`. thresholds: A sorted floating point `Tensor` with value in `[0, 1]`. It need to be evenly distributed (the diff between each element need to be the same). multi_label: Optional boolean indicating whether multidimensional prediction/labels should be treated as multilabel responses, or flattened into a single label. When True, the valus of `variables_to_update` must have a second dimension equal to the number of labels in y_true and y_pred, and those tensors must not be RaggedTensors. sample_weights: Optional `Tensor` whose rank is either 0, or the same rank as `y_true`, and must be broadcastable to `y_true` (i.e., all dimensions must be either `1`, or the same as the corresponding `y_true` dimension). label_weights: Optional tensor of non-negative weights for multilabel data. The weights are applied when calculating TP, FP, FN, and TN without explicit multilabel handling (i.e. when the data is to be flattened). thresholds_with_epsilon: Optional boolean indicating whether the leading and tailing thresholds has any epsilon added for floating point imprecisions. It will change how we handle the leading and tailing bucket. Returns: Update op. rN?dtype)clip_value_minclip_value_maxr]cs$|d|d}}tjj||dS)Nrr]data segment_ids num_segments)rmathunsorted_segment_sum)label_and_bucket_indexlabel bucket_indexnum_thresholdsrr gather_bucketszC_update_confusion_matrix_variables_optimized..gather_bucketT)reverseaxisr)rr)shapeas_listrr)opsbroadcast_weightscastrreshape expand_dimsmultiply clip_by_valueboolrceilnnreluint32 transposevectorized_mapcumsumrrfrmrn reduce_sumrkappend assign_addrlgroup)variables_to_updatey_truey_predr` multi_labelsample_weights label_weightsthresholds_with_epsilonr true_labels false_labelsbucket_indicesr tp_bucket_v fp_bucket_vrgrhtotal_true_labelstotal_false_labels update_opsvariablerirjrrr,_update_confusion_matrix_variables_optimized"sf                 rcCs@t|}|dkrdStj|tjd|d}tj||tdS)aCheck if the thresholds list is evenly distributed. We could leverage evenly distributed thresholds to use less memory when calculate metrcis like AUC where each individual threshold need to be evaluated. Args: thresholds: A python list or tuple, or 1D numpy array whose value is ranged in [0, 1]. Returns: boolean, whether the values in the inputs are evenly distributed. Frr])atol)lennparangefloat32allcloserepsilon)r`reven_thresholdsrrr is_evenly_distributed_thresholdssrc * Csv|r|durtd|dur dStdd|DsPtdttd|dt|dj} tj|| d }tj|| d }| r|dd kp|d d k} tj || d }|j d} |rtj tjd tj d t|dd} n&t||g|\\}}}tjdtjd } dd|D}|r2td|dttd|durNt||\}}n$tj|| d }tj|||d\}}}|j |j |durt||}|dur|d|f}|d|f}| rt|||||||| dSt |}|d}|j jd krd }ntjj|d ddd}t| |tjgtj d }|rXt|d}ttj|tjd d}n,t|d d g}ttj|tjd d d g}|r| d d g}d ||g}| d d g}n| d g}d ||g}| d g}tt||t|}t||}t||}t||}|dur>tj j!"tj|| d |}tt|||}nd}|dur|st|d}tj j!"||}tt|||}|dur|}n t#||}g}dd} tj$||fi}!tj%|v}"tj&|v}#tj'|v}$|$s|"rt(|}%||%f|!tj'<|#s|"r2t(|}&|&|f|!tj&<|"r2|&|%f|!tj%<|!)D]0\}'\}(})|'|vr:|*| |(|)|||'q:t+|S)a Returns op to update the given confusion matrix variables. For every pair of values in y_true and y_pred: true_positive: y_true == True and y_pred > thresholds false_negatives: y_true == True and y_pred <= thresholds true_negatives: y_true == False and y_pred <= thresholds false_positive: y_true == False and y_pred > thresholds The results will be weighted and added together. When multiple thresholds are provided, we will repeat the same for every threshold. For estimation of these metrics over a stream of data, the function creates an `update_op` operation that updates the given variables. If `sample_weight` is `None`, weights default to 1. Use weights of 0 to mask values. Args: variables_to_update: Dictionary with 'tp', 'fn', 'tn', 'fp' as valid keys and corresponding variables to update as values. y_true: A `Tensor` whose shape matches `y_pred`. Will be cast to `bool`. y_pred: A floating point `Tensor` of arbitrary shape and whose values are in the range `[0, 1]`. thresholds: A float value, float tensor, python list, or tuple of float thresholds in `[0, 1]`, or NEG_INF (used when top_k is set). top_k: Optional int, indicates that the positive labels should be limited to the top k predictions. class_id: Optional int, limits the prediction and labels to the class specified by this argument. sample_weight: Optional `Tensor` whose rank is either 0, or the same rank as `y_true`, and must be broadcastable to `y_true` (i.e., all dimensions must be either `1`, or the same as the corresponding `y_true` dimension). multi_label: Optional boolean indicating whether multidimensional prediction/labels should be treated as multilabel responses, or flattened into a single label. When True, the valus of `variables_to_update` must have a second dimension equal to the number of labels in y_true and y_pred, and those tensors must not be RaggedTensors. label_weights: (optional) tensor of non-negative weights for multilabel data. The weights are applied when calculating TP, FP, FN, and TN without explicit multilabel handling (i.e. when the data is to be flattened). thresholds_distributed_evenly: Boolean, whether the thresholds are evenly distributed within the list. An optimized method will be used if this is the case. See _update_confusion_matrix_variables_optimized() for more details. Returns: Update op. Raises: ValueError: If `y_pred` and `y_true` have mismatched shapes, or if `sample_weight` is not `None` and its shape doesn't match `y_pred`, or if `variables_to_update` contains invalid keys. Nz`label_weights` for multilabel data should be handled outside of `update_confusion_matrix_variables` when `multi_label` is True.css|]}|ttvr|VqdSrMlistrfr.r/rrr esz4update_confusion_matrix_variables..zhPlease provide at least one valid confusion matrix variable to update. Valid variable key options are: "z". Received: ""rrrrrr]one_set_of_thresholds_cond)rDTcSsg|]}|ttvr|qSrrrrrrr_sz5update_confusion_matrix_variables..zInvalid keys: "z$". Valid variable key options are: ") sample_weight.)rrrrrcSsFtjt|||jd}|dur4|tj||jd9}|t|dS)Nrr])rr logical_andrrr)rpredrvarlabel_and_predrrrweighted_assign_addsz>update_confusion_matrix_variables..weighted_assign_add),ranyrrfkeysvaluesrrrconvert_to_tensorrrequalrrank,ragged_assert_compatible_and_get_flat_valuesrrsqueeze_or_expand_dimensionsassert_is_compatible_with _filter_top_krndimsr reduce_prodwhereonesrrtilestackgreaterr)rrrrkrmrlrn logical_notrArr)*rrrr`top_kclass_idrrrthresholds_distributed_evenlyvariable_dtyperr one_thresh_ invalid_keys pred_shapenum_predictions num_labelsthresh_label_tilepredictions_extra_dimlabels_extra_dimthresh_pretile_shape thresh_tiles data_tiles thresh_tiled preds_tiled pred_is_pos label_is_pos weights_tiledlabel_weights_tiledrr loop_vars update_tn update_fp update_fn pred_is_neg label_is_neg matrix_condrrrrr!update_confusion_matrix_variablessE                                   rcCsNtjj||dd\}}tjtj|t|ddddd}||td|S)asFilters top-k values in the last dim of x and set the rest to NEG_INF. Used for computing top-k prediction values in dense labels (which has the same shape as predictions) for recall and precision top-k metrics. Args: x: tensor with any dimensions. k: the number of values to keep. Returns: tensor with same shape and dtype as x. F)sortedrrr])rrrrone_hotrNEG_INF)xkr top_k_idx top_k_maskrrrrs rc Cst|tr0tdd|D}tdd|D}nt|tj}|}|rB|dus\t|tjrBd}t|tst|g}d}dd|D}t|}t|tjrt|d |jg}t|t |j d }Wdn1s0Yg}|D]D} t|$| t | j d Wdq1s"0Yq|r<|d n|}n8|rXt d |n"t|tjrzt d |d |||fS)a#If ragged, it checks the compatibility and then returns the flat_values. Note: If two tensors are dense, it does not check their compatibility. Note: Although two ragged tensors with different ragged ranks could have identical overall rank and dimension sizes and hence be compatible, we do not support those cases. Args: values: A list of potentially ragged tensor of the same ragged_rank. mask: A potentially ragged tensor of the same ragged_rank as elements in Values. Returns: A tuple in which the first element is the list of tensors and the second is the mask tensor. ([Values], mask). Mask and the element in Values are equal to the flat_values of the input arguments (if they were ragged). css|]}t|tjVqdSrMr;r RaggedTensorr.rtrrrrFz?ragged_assert_compatible_and_get_flat_values..css|]}t|tjVqdSrMr r rrrrGrNFTcSsg|] }|jqSr)nested_row_splitsr rrrr_Trz@ragged_assert_compatible_and_get_flat_values..rrz.r]N)rr)r%r rr#rrws   rcCsBt|}t||j}t||k|j}tt||tS)aCreates int Tensor, 1 for label-prediction match, 0 for mismatch. Args: y_true: Ground truth values, of shape (batch_size, d0, .. dN). y_pred: The predicted values, of shape (batch_size, d0, .. dN). threshold: (Optional) Float representing the threshold for deciding whether prediction values are 1 or 0. Returns: Binary matches, of shape (batch_size, d0, .. dN). )rrrrrrfloatx)rr thresholdrrrbinary_matchess r(cCsd}t|}t|}t|}|jj}|jj}|durp|durptt|tt|krpt|dg}d}tjj |dd}t |t |krt |t |}t t ||t }|rtj||d}|S)afCreates float Tensor, 1.0 for label-prediction match, 0.0 for mismatch. You can provide logits of classes as `y_pred`, since argmax of logits and probabilities are same. Args: y_true: Integer ground truth values. y_pred: The prediction values. Returns: Match tensor: 1.0 for label-prediction match, 0.0 for mismatch. FNrTrr)rrrrrr int_shapesqueezerargmaxrrrr&r)rrreshape_matchesy_true_org_shape y_pred_rank y_true_rankmatchesrrrsparse_categorical_matchess*    r2cCsd}t|}t|}|jj}|jj}t|}|durz|durz|dkr`t|d|jdg}|dkrzd}t|dg}tjtjj|t|d|dt d }|rtj||d S|S) ajCreates float Tensor, 1.0 for label-TopK_prediction match, 0.0 for mismatch. Args: y_true: tensor of true targets. y_pred: tensor of predicted targets. k: (Optional) Number of top elements to look at for computing accuracy. Defaults to 5. Returns: Match tensor: 1.0 for label-prediction match, 0.0 for mismatch. FNrr]Tr) predictionstargetsrrr)) rrrrrrrin_top_krr&)rrrr-r0r/r.r1rrr sparse_top_k_categorical_matchess*    r8)rc)FNNF)NNNFNF)N)rc)r3)%rrYrVenumrnumpyrtensorflow.compat.v2compatv2rkerasr keras.utilsrrkeras.utils.generic_utilsrrrr,rLr\rbrerfrorwrrrrrrr(r2r8rrrrsR     $m ' _   D (