a SicA@sdZddlmmZddlmZddlmZddl m Z ddl m Z e dgdGd d d Z d"d d Zd#d dZddZddZe jfddZe dgdde jdfddZddZddZddZddZd d!ZdS)$z$Utilities related to loss functions.N)backend) keras_tensor)tf_utils) keras_exportzkeras.losses.Reduction)v1c@s8eZdZdZdZdZdZdZeddZ edd Z d S) ReductionV2aTypes of loss reduction. Contains the following values: * `AUTO`: Indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to `SUM_OVER_BATCH_SIZE`. When used with `tf.distribute.Strategy`, outside of built-in training loops such as `tf.keras` `compile` and `fit`, we expect reduction value to be `SUM` or `NONE`. Using `AUTO` in that case will raise an error. * `NONE`: No **additional** reduction is applied to the output of the wrapped loss function. When non-scalar losses are returned to Keras functions like `fit`/`evaluate`, the unreduced vector loss is passed to the optimizer but the reported loss will be a scalar value. Caution: **Verify the shape of the outputs when using** `Reduction.NONE`. The builtin loss functions wrapped by the loss classes reduce one dimension (`axis=-1`, or `axis` if specified by loss function). `Reduction.NONE` just means that no **additional** reduction is applied by the class wrapper. For categorical losses with an example input shape of `[batch, W, H, n_classes]` the `n_classes` dimension is reduced. For pointwise losses you must include a dummy axis so that `[batch, W, H, 1]` is reduced to `[batch, W, H]`. Without the dummy axis `[batch, W, H]` will be incorrectly reduced to `[batch, W]`. * `SUM`: Scalar sum of weighted losses. * `SUM_OVER_BATCH_SIZE`: Scalar `SUM` divided by number of elements in losses. This reduction type is not supported when used with `tf.distribute.Strategy` outside of built-in training loops like `tf.keras` `compile`/`fit`. You can implement 'SUM_OVER_BATCH_SIZE' using global batch size like: ``` with strategy.scope(): loss_obj = tf.keras.losses.CategoricalCrossentropy( reduction=tf.keras.losses.Reduction.NONE) .... loss = tf.reduce_sum(loss_obj(labels, predictions)) * (1. / global_batch_size) ``` Please see the [custom training guide]( https://www.tensorflow.org/tutorials/distribute/custom_training) for more details on this. autononesumsum_over_batch_sizecCs|j|j|j|jfSN)AUTONONESUMSUM_OVER_BATCH_SIZE)clsrT/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/utils/losses_utils.pyallQszReductionV2.allcCs*||vr&td|d|ddS)NzInvalid Reduction Key: z. Expected keys are "")r ValueError)rkeyrrrvalidateUs zReductionV2.validateN) __name__ __module__ __qualname____doc__r rrr classmethodrrrrrrrs. rc st|p d|ts&tts:tj}|j}j}|j}|dur|dur||}||dkr|jd drt dgn*||dkr|jd drt dgfWdSt t }|dus|jd dr0t t |d|fddfdd|dusL|jd drvt t |d|fddfd dfWdS1s0YdS) aHSqueeze last dim if ranks differ from expected by exactly 1. In the common case where we expect shapes to match, `expected_rank_diff` defaults to 0, and we squeeze the last dimension of the larger rank if they differ by 1. But, for example, if `labels` contains class IDs and `predictions` contains 1 probability per class, we expect `predictions` to have 1 more dimension than `labels`, so `expected_rank_diff` would be 1. In this case, we'd squeeze `labels` if `rank(predictions) - rank(labels) == 0`, and `predictions` if `rank(predictions) - rank(labels) == 2`. This will use static shape if available. Otherwise, it will add graph operations, which could result in a performance hit. Args: labels: Label values, a `Tensor` whose dimensions match `predictions`. predictions: Predicted values, a `Tensor` of arbitrary dimensions. expected_rank_diff: Expected result of `rank(predictions) - rank(labels)`. name: Name of the op. Returns: Tuple of `labels` and `predictions`, possibly with last dim squeezed. remove_squeezable_dimensionsNcstdgSNr tfsqueezer predictionsrrz.remove_squeezable_dimensions..csSr rrr%rrr'r(cstdgSr!r"rlabelsrrr'r(csSr rrr)rrr'r()r name_scoperis_tensor_or_extension_typer#convert_to_tensorshapendimsdimsis_compatible_withr$rankcondequal) r*r&expected_rank_diffnamepredictions_shapepredictions_rank labels_shape labels_rank rank_diffr)r*r&rr]sT          rc sj}|j}durj}|j}|durX|durX||dksH|ddkrt\ndttfddtdtdfdd}ttd|\dur̈fSj}|j} | dkrfS|durB| durB| |dkrtdgn|| dkr8tdgfSt} | tfddfd d fd d } tt| dfd d| fS)aSqueeze or expand last dimension if needed. 1. Squeezes last dim of `y_pred` or `y_true` if their rank differs by 1 (using `remove_squeezable_dimensions`). 2. Squeezes or expands last dim of `sample_weight` if its rank differs by 1 from the new rank of `y_pred`. If `sample_weight` is scalar, it is kept scalar. This will use static shape if available. Otherwise, it will add graph operations, which could result in a performance hit. Args: y_pred: Predicted values, a `Tensor` of arbitrary dimensions. y_true: Optional label `Tensor` whose dimensions match `y_pred`. sample_weight: Optional weight scalar or `Tensor` whose dimensions match `y_pred`. Returns: Tuple of `y_pred`, `y_true` and `sample_weight`. Each of them possibly has the last dimension squeezed, `sample_weight` could be extended by one dimension. If `sample_weight` is None, (y_pred, y_true) is returned. Nrr cs tSr )rry_predy_truerrr'r(z.squeeze_or_expand_dimensions..cstfddS)NcsfSr rrr<rrr'r(z@squeeze_or_expand_dimensions....)r#r3r) is_last_dim_1 squeeze_dimsr=r>rrr'srcstdgSr!r"r sample_weightrrr'r(cs*fdd}ttd|fddS)NcstdgSr!)r# expand_dimsrrArrr'r(zMsqueeze_or_expand_dimensions.._maybe_expand_weights..r csSr rrrArrr'r(r#r3r4)expand_weights)r;rBrr_maybe_expand_weightss z;squeeze_or_expand_dimensions.._maybe_expand_weightscsttdS)NrrDr)rFmaybe_squeeze_weightsr;rr_maybe_adjust_weightssz;squeeze_or_expand_dimensions.._maybe_adjust_weightscsSr rrrArrr'r() r.r/rr#r2r4r3r$rC) r=r>rB y_pred_shape y_pred_rank y_true_shape y_true_rankmaybe_squeeze_dims weights_shape weights_rankweights_rank_tensorrHr)rFr?rGr;rBr@r=r>rsqueeze_or_expand_dimensionssL      rQcCst|}tjj||ddS)a:Computes a safe mean of the losses. Args: losses: `Tensor` whose elements contain individual loss measurements. num_present: The number of measurable elements in `losses`. Returns: A scalar representing the mean of `losses`. If `num_present` is zero, then zero is returned. valuer6)r# reduce_summath divide_no_nan)losses num_present total_lossrrr _safe_means rZcCsHtd*}tjtj||d|jdWdS1s:0YdS)z3Computes the number of elements in `losses` tensor. num_elementsrS)dtypeN)rr+r#castsizer\)rWscoperrr _num_elementss r`cCs6|tjkr|}n"t|}|tjkr2t|t|}|S)z2Reduces the individual weighted loss measurements.)rrr#rTrrZr`)weighted_losses reductionlossrrrreduce_weighted_losss    rdz/keras.__internal__.losses.compute_weighted_lossc Cst||tjkrtj}|dur&d}t|p0d|tjj _ t |t j tjfs`t|}t |t j tjfs|t|}|jjs|j}t|d}d}nd}t||j}t|d|\}}}t||}t||}|rt||}|WdS1s0YdS)aComputes the weighted loss. Args: losses: `Tensor` of shape `[batch_size, d1, ... dN]`. sample_weight: Optional `Tensor` whose rank is either 0, or the same rank as `losses`, or be broadcastable to `losses`. reduction: (Optional) Type of `tf.keras.losses.Reduction` to apply to loss. Default value is `SUM_OVER_BATCH_SIZE`. name: Optional name for the op. Raises: ValueError: If the shape of `sample_weight` is not compatible with `losses`. Returns: Weighted loss `Tensor` of the same type as `losses`. If `reduction` is `NONE`, this has the same shape as `losses`; otherwise, it is scalar. N? weighted_lossfloat32TF)rrr rrr+r#compatrget_default_graph_last_loss_reduction isinstancer KerasTensor RaggedTensorr-r\ is_floatingr]rQmultiplyrd) rWrBrbr6 input_dtypeZ input_casted_rarcrrrcompute_weighted_loss!s:          rrcCs$tjj}|dkr |d|9}|S)zBScales and returns the given loss value by the number of replicas.rre)r# distribute get_strategynum_replicas_in_sync) loss_value num_replicasrrrscale_loss_for_distributionis  rxcstd|D]P}|jjrHdus*|jjjkr2|jn|jhddhkrHd|jjr|Sqrpfdd|D}|S)arCast a list of losses to a common dtype. If any loss is floating-point, they will all be casted to the most-precise floating-point loss. Otherwise the losses are not casted. We also skip casting losses if there are any complex losses. Args: losses: A list of losses. Returns: `losses`, but they have been casted to a common dtype. Nbfloat16float16rgcsg|]}t|qSr)r#r]).0rc highest_floatrr r(z/cast_losses_to_common_dtype..)r\rnr^ is_complex)rWrcrr|rcast_losses_to_common_dtypeqs rcCs t|ddS)zReturns Keras mask from tensor. _keras_maskN)getattr)y_prrrget_masksrcCsB|dur>t||j}|dur:t||d\}}}||9}n|}|S)z2Applies any mask on predictions to sample weights.NrA)r#r]r\rQ)rswmaskrqrrr apply_masks rcCs\|durPt||j}|tjtjfvrPtt||j}t|}|||9}t|||S)z;Redistribute sample weights considering only valid entries.N) r#r]r\rr rr^rTr)rWrrrbtotalvalidrrrapply_valid_masks   r)rN)NN)rtensorflow.compat.v2rhv2r#kerasr keras.enginer keras.utilsr tensorflow.python.util.tf_exportrrrrQrZr`rrdrrrxrrrrrrrrs0     A F \  G