a yf3@snddlZddlmZddlmmZddlmZddlm Z ddl m Z m Z Gdddej Zd d d ZdS)Nlinear_sum_assignment)bbox_iou) xywh2xyxy xyxy2xywhcs,eZdZdZd fdd Zd d d ZZS)HungarianMatchera A module implementing the HungarianMatcher, which is a differentiable module to solve the assignment problem in an end-to-end fashion. HungarianMatcher performs optimal assignment over the predicted and ground truth bounding boxes using a cost function that considers classification scores, bounding box coordinates, and optionally, mask predictions. Attributes: cost_gain (dict): Dictionary of cost coefficients: 'class', 'bbox', 'giou', 'mask', and 'dice'. use_fl (bool): Indicates whether to use Focal Loss for the classification cost calculation. with_mask (bool): Indicates whether the model makes mask predictions. num_sample_points (int): The number of sample points used in mask cost calculation. alpha (float): The alpha factor in Focal Loss calculation. gamma (float): The gamma factor in Focal Loss calculation. Methods: forward(pred_bboxes, pred_scores, gt_bboxes, gt_cls, gt_groups, masks=None, gt_mask=None): Computes the assignment between predictions and ground truths for a batch. _cost_mask(bs, num_gts, masks=None, gt_mask=None): Computes the mask cost and dice cost if masks are predicted. NTF1?@csJt|dur"dddddd}||_||_||_||_||_||_dS)zjInitializes a HungarianMatcher module for optimal assignment of predicted and ground truth bounding boxes.N)classbboxgioumaskZdice)super__init__ cost_gainuse_fl with_masknum_sample_pointsalphagamma)selfrrrrrr __class__X/var/www/html/django/DPS/env/lib/python3.9/site-packages/ultralytics/models/utils/ops.pyr"s zHungarianMatcher.__init__cs|j\}} } tdkr*ddt|DS|d| }|jrJt|n tj|dd}|dd}|dd|f}|jrd|j ||j d|d  } |j d||j |d  } | | } n| } | d| d d}d t| d| dd d d d}|jd | |jd||jd|}|jr`|||||7}d|||B<||| d}ddt|dD}tdgdddfddt|DS)a= Forward pass for HungarianMatcher. This function computes costs based on prediction and ground truth (classification cost, L1 cost between boxes and GIoU cost between boxes) and finds the optimal matching between predictions and ground truth based on these costs. Args: pred_bboxes (Tensor): Predicted bounding boxes with shape [batch_size, num_queries, 4]. pred_scores (Tensor): Predicted scores with shape [batch_size, num_queries, num_classes]. gt_cls (torch.Tensor): Ground truth classes with shape [num_gts, ]. gt_bboxes (torch.Tensor): Ground truth bounding boxes with shape [num_gts, 4]. gt_groups (List[int]): List of length equal to batch size, containing the number of ground truths for each image. masks (Tensor, optional): Predicted masks with shape [batch_size, num_queries, height, width]. Defaults to None. gt_mask (List[Tensor], optional): List of ground truth masks, each with shape [num_masks, Height, Width]. Defaults to None. Returns: (List[Tuple[Tensor, Tensor]]): A list of size batch_size, each element is a tuple (index_i, index_j), where: - index_i is the tensor of indices of the selected predictions (in order) - index_j is the tensor of indices of the corresponding selected ground truth targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) rcSs,g|]$}tjgtjdtjgtjdfqSdtypetorchtensorlong).0_rrr Jz,HungarianMatcher.forward..dimNr g:0yE>?T)ZxywhZGIoUrrrcSsg|]\}}t||qSrr)r&icrrrr(pr)cs<g|]4\}\}}tj|tjdtj|tjd|fqSrr")r&kr0j gt_groupsrrr(rs )shapesumrangedetachviewrFZsigmoidZsoftmaxrrlogZ unsqueezeabsrsqueezerrZ _cost_maskisnanisinfcpu enumeratesplitr#Z as_tensorZcumsum_)rZ pred_bboxesZ pred_scoresZ gt_bboxesgt_clsr5masksZgt_maskbsZnqncZneg_cost_classZpos_cost_classZ cost_classZ cost_bboxZ cost_giouCindicesrr4rforward.s:  &" &    zHungarianMatcher.forward)NTFrr r )NN)__name__ __module__ __qualname____doc__rrJ __classcell__rrrrr s rd?r.Fc"s|r |dkrdS|d}t|} t|dkr4dS|} | dkrHdn| } t|} |d} |d} |d}| d| }| d| d}|d| d }tj| | tj| jd | | }|dkr t |j |d k}t | d }tj |d||j|jd }|||<|dkrt|}|d dd fd dd|}t |dddd}t|}||d7<||9}|||7}|jdddt|}tj|dd}td| }||}tj| ||j d | jd}tj| |d| jd}tdd|Dtjfddt| Ddd}tfddtd| D|||f<|||f<||}tj||gtjd}d||d d |f<t| D]} | dkrd|d| d| dd| d|f<| | dkrd|d| d| dd | df<ndd|d| d| dd| d|f<d|d| d| dd d| f<qdd|jt|ddD| ||gd}!||j||j||j|!fS)a Get contrastive denoising training group. This function creates a contrastive denoising training group with positive and negative samples from the ground truths (gt). It applies noise to the class labels and bounding box coordinates, and returns the modified labels, bounding boxes, attention mask and meta information. Args: batch (dict): A dict that includes 'gt_cls' (torch.Tensor with shape [num_gts, ]), 'gt_bboxes' (torch.Tensor with shape [num_gts, 4]), 'gt_groups' (List(int)) which is a list of batch size length indicating the number of gts of each image. num_classes (int): Number of classes. num_queries (int): Number of queries. class_embed (torch.Tensor): Embedding weights to map class labels to embedding space. num_dn (int, optional): Number of denoising. Defaults to 100. cls_noise_ratio (float, optional): Noise ratio for class labels. Defaults to 0.5. box_noise_scale (float, optional): Noise scale for bounding box coordinates. Defaults to 1.0. training (bool, optional): If it's in training mode. Defaults to False. Returns: (Tuple[Optional[Tensor], Optional[Tensor], Optional[Tensor], Optional[Dict]]): The modified class embeddings, bounding boxes, attention mask and meta information for denoising. If not in training mode or 'num_dn' is less than or equal to 0, the function returns None for all elements in the tuple. r)NNNNr5r clsZbboxesZ batch_idxr r*)r!devicerQ.Nr r.r/)minmaxgư>)Zeps)rSr-cSs g|]}tjt|tjdqSr)r#r$r8r%)r&numrrrr(r)z!get_cdn_group..csg|]}|qSrrr&r0Z map_indicesZmax_numsrrr(r)r+csg|]}|qSrrrWrXrrr(r)r TcSsg|]}|dqS)r*)Zreshape)r&prrrr(r))Z dn_pos_idxZ dn_num_groupZ dn_num_split)r7rUlenrepeatr:r#Zaranger%rSZrandr6Znonzeror>Z randint_liker!rZ rand_likeZclip_rZlogitintZzeroscatstackr8boolrArClistto)"batchZ num_classesZ num_queriesZ class_embedZnum_dnZcls_noise_ratioZbox_noise_scaleZtrainingr5Z total_numZ num_grouprFrDZgt_bboxZb_idxZdn_clsZdn_bboxZdn_b_idxZneg_idxridxZ new_labelZ known_bboxdiffZ rand_signZ rand_partZ dn_cls_embedZ padding_clsZ padding_bboxZpos_idxZtgt_sizeZ attn_maskr0Zdn_metarrXr get_cdn_groupsz       ""    4244   re)rPrQr.F)r#Ztorch.nnnnZtorch.nn.functionalZ functionalr;Zscipy.optimizerZultralytics.utils.metricsrZultralytics.utils.opsrrModulerrerrrrs