a yfx @sddlZddlZddlZddlZddlZddlZddlZddlm m Z ddl m Z ddlmZGdddejZdNddZdOd d Zd dZdPddZdQddZddZddZdRddZd d!Zd"d#ZdSd$d%ZdTd'd(Zd)d*Zd+d,Zd-d.Z d/d0Z!d1d2Z"d3d4Z#d5d6Z$dUd8d9Z%d:d;Z&dVdd?Z(dWd@dAZ)dXdBdCZ*dDdEZ+dYdGdHZ,ej-ej.dIdJdKZ/dLdMZ0dS)ZN)LOGGER) batch_probiouc@sBeZdZdZdejdddZddZd d Zd d Z d dZ dS)Profileai YOLOv8 Profile class. Use as a decorator with @Profile() or as a context manager with 'with Profile():'. Example: ```python from ultralytics.utils.ops import Profile with Profile(device=device) as dt: pass # slow operation here print(dt) # prints "Elapsed time is 9.5367431640625e-07 s" ``` NdevicecCs(||_||_t|ot|d|_dS)z Initialize the Profile class. Args: t (float): Initial time. Defaults to 0.0. device (torch.device): Devices used for model inference. Defaults to None (cpu). cudaN)trboolstr startswithr)selfr rrQ/var/www/html/django/DPS/env/lib/python3.9/site-packages/ultralytics/utils/ops.py__init__ szProfile.__init__cCs||_|S)z Start timing.)timestartr rrr __enter__,s zProfile.__enter__cCs$||j|_|j|j7_dS)z Stop timing.N)rrdtr )r typevalue tracebackrrr__exit__1szProfile.__exit__cCsd|jdS)zZReturns a human-readable string representing the accumulated elapsed time in the profiler.zElapsed time is z s)r rrrr__str__6szProfile.__str__cCs|jrtj|jtS)zGet current time.)rtorchZ synchronizerrrrrrr:sz Profile.time)rN) __name__ __module__ __qualname____doc__rrrrrrrrrrrrs  rcCsz|j\}}|dk|dk@||k@||k@}||}||}t|rjtj||||g|jdStjd|jdS)a Convert 1 segment label to 1 box label, applying inside-image constraint, i.e. (xy1, xy2, ...) to (xyxy). Args: segment (torch.Tensor): the segment label width (int): the width of the image. Defaults to 640 height (int): The height of the image. Defaults to 640 Returns: (np.ndarray): the minimum and maximum x and y values of the segment. rdtype)Tanynparrayminmaxr"zeros)segmentwidthheightxyZinsiderrr segment2boxAs  *r0TFcCs|durlt|d|d|d|d}t|d|d|ddt|d|d|ddf}n|dd}|d}|r|d|d8<|d|d8<|s|d|d8<|d |d8<|d dd f|<t||S) a  Rescales bounding boxes (in the format of xyxy by default) from the shape of the image they were originally specified in (img1_shape) to the shape of a different image (img0_shape). Args: img1_shape (tuple): The shape of the image that the bounding boxes are for, in the format of (height, width). boxes (torch.Tensor): the bounding boxes of the objects in the image, in the format of (x1, y1, x2, y2) img0_shape (tuple): the shape of the target image, in the format of (height, width). ratio_pad (tuple): a tuple of (ratio, pad) for scaling the boxes. If not provided, the ratio and pad will be calculated based on the size difference between the two images. padding (bool): If True, assuming the boxes is based on image augmented by yolo style. If False then do regular rescaling. xywh (bool): The box format is xywh or not, default=False. Returns: boxes (torch.Tensor): The scaled bounding boxes, in the format of (x1, y1, x2, y2) Nrg?.r.r1.r2..r#)r(round clip_boxes) img1_shapeboxes img0_shape ratio_padpaddingZxywhgainpadrrr scale_boxesXs" rAcCs*t|tjrt|}t|||S)a Returns the nearest number that is divisible by the given divisor. Args: x (int): The number to make divisible. divisor (int | torch.Tensor): The divisor. Returns: (int): The nearest number divisible by the divisor. ) isinstancerTensorintr)mathceil)r.Zdivisorrrrmake_divisible~s  rG?cCslt|dkrtjdtjdStj|dd}||}t||jdd}t|j ddd|k d }||S) ap NMS for oriented bounding boxes using probiou and fast-nms. Args: boxes (torch.Tensor): Rotated bounding boxes, shape (N, 5), format xywhr. scores (torch.Tensor): Confidence scores, shape (N,). threshold (float, optional): IoU threshold. Defaults to 0.45. Returns: (torch.Tensor): Indices of boxes to keep after NMS. r)rr!TZ descendingr1)Zdiagonaldim) lenr&emptyZint8rargsortrZtriu_Znonzeror)Zsqueeze_)r;scores thresholdZ sorted_idxZiouspickrrr nms_rotateds  rS?r,皙?0uc$sddl}dkrdks.nJddd|krBdksTnJd|dt|ttfrj|d}durtj|jd|jdd kr‡fd d |D}durfd d |D}|S|jd}|p|jdd }|jd|d }d |}|ddd |fdk}d| |}||dkM}| dd}| s| rht |ddd f|ddd f<n.tj t |ddd f|dd dffdd}t }tj dd |f|jdg|}t|D]\}}|||}|r|t||r|| s|||}tj t|||d f|jd}t |ddddf|dddd f<d|tt||dddfd f<t ||fd}|jdsq|d ||fd\}}}|rt|k\}}t ||||d |df|dddf||fd}n8|jddd\}}t ||||fd|dk}dur\||dddd fkd}|jd} | spq| | kr||ddd fjddd| }|dddd f|rdn| }!|ddd f}"| r,tj |ddddf|!|dddd f|ddddffdd}#t|#|"|}n(|dddd f|!}#|j|#|"|}|d}||||<t ||krtd|ddqq|S)aO Perform non-maximum suppression (NMS) on a set of boxes, with support for masks and multiple labels per box. Args: prediction (torch.Tensor): A tensor of shape (batch_size, num_classes + 4 + num_masks, num_boxes) containing the predicted boxes, classes, and masks. The tensor should be in the format output by a model, such as YOLO. conf_thres (float): The confidence threshold below which boxes will be filtered out. Valid values are between 0.0 and 1.0. iou_thres (float): The IoU threshold below which boxes will be filtered out during NMS. Valid values are between 0.0 and 1.0. classes (List[int]): A list of class indices to consider. If None, all classes will be considered. agnostic (bool): If True, the model is agnostic to the number of classes, and all classes will be considered as one. multi_label (bool): If True, each box may have multiple labels. labels (List[List[Union[int, float, torch.Tensor]]]): A list of lists, where each inner list contains the apriori labels for a given image. The list should be in the format output by a dataloader, with each label being a tuple of (class_index, x1, y1, x2, y2). max_det (int): The maximum number of boxes to keep after NMS. nc (int, optional): The number of classes output by the model. Any indices after this will be considered masks. max_time_img (float): The maximum time (seconds) for processing one image. max_nms (int): The maximum number of boxes into torchvision.ops.nms(). max_wh (int): The maximum box width and height in pixels. in_place (bool): If True, the input prediction tensor will be modified in place. rotated (bool): If Oriented Bounding Boxes (OBB) are being passed for NMS. Returns: (List[torch.Tensor]): A list of length batch_size, where each element is a tensor of shape (num_boxes, 6 + num_masks) containing the kept boxes, with columns (x1, y1, x2, y2, confidence, class, mask1, mask2, ...). rNr1zInvalid Confidence threshold z&, valid values are between 0.0 and 1.0z Invalid IoU rrLcs,g|]$}||dddfkdqS)Nr#r.0pred) conf_thresmax_detrr z'non_max_suppression..cs.g|]&}||ddddfkdqS)NrYr1)r%rZ)classesrrr_r`r#g@.rJrag?T)ZkeepdimrIr2uWARNING ⚠️ NMS time limit z.3fz s exceeded) torchvisionrBlisttuplertensorrshapeZamaxZ transpose xywh2xyxycatrr* enumeraterMrangelongsplitwherefloatr)viewr%rOrSopsZnmsrwarning)$Z predictionr]Z iou_thresrbZagnosticZ multi_labellabelsr^ncZ max_time_imgZmax_nmsZmax_whZin_placeZrotatedrdoutputbsnmmiZxcZ time_limitr xir.ZlbvboxclsmaskijconfncrPr;r)rbr]r^rnon_max_suppressions|/&&    ".  (( <& "  $"H   rcCst|tjrn|dd|d|d<|dd|d|d<|dd|d|d<|dd|d|d<nP|dddgfd|d|dddgf<|ddd gfd|d|ddd gf<|S) a% Takes a list of bounding boxes and a shape (height, width) and clips the bounding boxes to the shape. Args: boxes (torch.Tensor): the bounding boxes to clip shape (tuple): the shape of the image Returns: (torch.Tensor | numpy.ndarray): Clipped boxes r3rr1r4r5r6.r2r7rBrrCclampclip)r;rhrrrr9;s ((r9cCsrt|tjr>|dd|d|d<|dd|d|d<n0|dd|d|d<|dd|d|d<|S)aE Clip line coordinates to the image boundaries. Args: coords (torch.Tensor | numpy.ndarray): A list of line coordinates. shape (tuple): A tuple of integers representing the size of the image in the format (height, width). Returns: (torch.Tensor | numpy.ndarray): Clipped coordinates r3rr1r4r)coordsrhrrr clip_coordsQs rc CsB|j}|dd|ddkr"|S|dur~t|d|d|d|d}|d|d|d|d|d|df}n|d}t|dt|d}}t|d|dt|d|d}} t|jdkrtdt|j||||| f}t||d|df}t|jdkr>|dddddf}|S)a Takes a mask, and resizes it to the original image size. Args: masks (np.ndarray): resized and padded masks/images, [h, w, num]/[h, w, 3]. im0_shape (tuple): the original image shape ratio_pad (tuple): the ratio of the padding to the original image. Returns: masks (np.ndarray): The masks that are being returned with shape [h, w, num]. Nr2rr1z/"len of masks shape" should be 2 or 3, but got )rhr(rDrM ValueErrorcv2resize) masksZ im0_shaper=Z im1_shaper?r@topleftbottomrightrrr scale_imagees "2*rcCs|jddksJd|jt|tjr4t|nt|}|d|dd|d<|d|dd|d<|d|d|d<|d|d|d<|S) a Convert bounding box coordinates from (x1, y1, x2, y2) format to (x, y, width, height) format where (x1, y1) is the top-left corner and (x2, y2) is the bottom-right corner. Args: x (np.ndarray | torch.Tensor): The input bounding box coordinates in (x1, y1, x2, y2) format. Returns: y (np.ndarray | torch.Tensor): The bounding box coordinates in (x, y, width, height) format. rLr#9input shape last dimension expected 4 but input shape is r3r5r2r4r6rhrBrrC empty_liker&r.r/rrr xyxy2xywhs  rcCs|jddksJd|jt|tjr4t|nt|}|dddf}|dddfd}|||dddf<|||dddf<|S)a Convert bounding box coordinates from (x, y, width, height) format to (x1, y1, x2, y2) format where (x1, y1) is the top-left corner and (x2, y2) is the bottom-right corner. Note: ops per 2 channels faster than per channel. Args: x (np.ndarray | torch.Tensor): The input bounding box coordinates in (x, y, width, height) format. Returns: y (np.ndarray | torch.Tensor): The bounding box coordinates in (x1, y1, x2, y2) format. rLr#r.Nr2r)r.r/ZxyZwhrrrris  ricCs|jddksJd|jt|tjr4t|nt|}||d|dd||d<||d|dd||d<||d|dd||d<||d|dd||d<|S) aF Convert normalized bounding box coordinates to pixel coordinates. Args: x (np.ndarray | torch.Tensor): The bounding box coordinates. w (int): Width of the image. Defaults to 640 h (int): Height of the image. Defaults to 640 padw (int): Padding width. Defaults to 0 padh (int): Padding height. Defaults to 0 Returns: y (np.ndarray | torch.Tensor): The coordinates of the bounding box in the format [x1, y1, x2, y2] where x1,y1 is the top-left corner, x2,y2 is the bottom-right corner of the bounding box. rLr#rr3r5r2r4r6r)r.whZpadwZpadhr/rrr xywhn2xyxys     rrcCs|rt|||||f}|jddks8Jd|jt|tjrNt|nt|}|d|dd||d<|d|dd||d<|d|d||d<|d|d||d<|S) a Convert bounding box coordinates from (x1, y1, x2, y2) format to (x, y, width, height, normalized) format. x, y, width and height are normalized to image dimensions. Args: x (np.ndarray | torch.Tensor): The input bounding box coordinates in (x1, y1, x2, y2) format. w (int): The width of the image. Defaults to 640 h (int): The height of the image. Defaults to 640 clip (bool): If True, the boxes will be clipped to the image boundaries. Defaults to False eps (float): The minimum value of the box's width and height. Defaults to 0.0 Returns: y (np.ndarray | torch.Tensor): The bounding box coordinates in (x, y, width, height, normalized) format rLr#rr3r5r2r4r6)r9rhrBrrCrr&)r.rrrZepsr/rrr xyxy2xywhns rcCsRt|tjr|nt|}|d|dd|d<|d|dd|d<|S)aX Convert the bounding box format from [x, y, w, h] to [x1, y1, w, h], where x1, y1 are the top-left coordinates. Args: x (np.ndarray | torch.Tensor): The input tensor with the bounding box coordinates in the xywh format Returns: y (np.ndarray | torch.Tensor): The bounding box coordinates in the xyltwh format r3r5r2r4r6rBrrCcloner&copyrrrr xywh2ltwhs rcCsJt|tjr|nt|}|d|d|d<|d|d|d<|S)aU Convert nx4 bounding boxes from [x1, y1, x2, y2] to [x1, y1, w, h], where xy1=top-left, xy2=bottom-right. Args: x (np.ndarray | torch.Tensor): The input tensor with the bounding boxes coordinates in the xyxy format Returns: y (np.ndarray | torch.Tensor): The bounding box coordinates in the xyltwh format. r5r3r6r4rrrrr xyxy2ltwhs rcCsRt|tjr|nt|}|d|dd|d<|d|dd|d<|S)z Convert nx4 boxes from [x1, y1, w, h] to [x, y, w, h] where xy1=top-left, xy=center. Args: x (torch.Tensor): the input tensor Returns: y (np.ndarray | torch.Tensor): The bounding box coordinates in the xywh format. r3r5r2r4r6rrrrr ltwh2xywhs rc Cst|tj}|r|n|}|t|dd}g}|D]:}t|\\}}\}}} | ||||| dt j gq:|rtj ||j |jdSt |S)a Convert batched Oriented Bounding Boxes (OBB) from [xy1, xy2, xy3, xy4] to [xywh, rotation]. Rotation values are returned in radians from 0 to pi/2. Args: x (numpy.ndarray | torch.Tensor): Input box corners [xy1, xy2, xy3, xy4] of shape (n, 8). Returns: (numpy.ndarray | torch.Tensor): Converted data in [cx, cy, w, h, rotation] format of shape (n, 5). rLr2rr")rBrrCcpunumpyreshaperMrZ minAreaRectappendr&pirgrr"Zasarray) r.Zis_torchZpointsrboxesZptsZcxcyrranglerrrxyxyxyxy2xywhrs  rcsttjr tjtjtjtjfntjtjtjtjf\}}}}dddf}fddt ddD\}}}||||} } |d| |d| g} | d| |d| g} || d} || d} || | } || | }|| | }|| | }|| |||gdS) a Convert batched Oriented Bounding Boxes (OBB) from [xywh, rotation] to [xy1, xy2, xy3, xy4]. Rotation values should be in radians from 0 to pi/2. Args: x (numpy.ndarray | torch.Tensor): Boxes in [cx, cy, w, h, rotation] format of shape (n, 5) or (b, n, 5). Returns: (numpy.ndarray | torch.Tensor): Converted corner points of shape (n, 4, 2) or (b, n, 4, 2). .Nr2c3s"|]}d||dfVqdS).r1Nrr[rr.rr :r`z!xywhr2xyxyxyxy..rarLrc) rBrrCcossinrjstackr& concatenaterl)r.rrrjrZctrrrrZ cos_valueZ sin_valueZvec1Zvec2Zpt1Zpt2Zpt3Zpt4rrrxywhr2xyxyxyxy(s        rcCsJt|tjr|nt|}|d|d|d<|d|d|d<|S)a It converts the bounding box from [x1, y1, w, h] to [x1, y1, x2, y2] where xy1=top-left, xy2=bottom-right. Args: x (np.ndarray | torch.Tensor): the input image Returns: y (np.ndarray | torch.Tensor): the xyxy coordinates of the bounding boxes. r5r3r6r4rrrrr ltwh2xyxyGs rcCsHg}|D]0}|j\}}|||||gqtt|S)a. It converts segment labels to box labels, i.e. (cls, xy1, xy2, ...) to (cls, xywh). Args: segments (list): list of segments, each segment is a list of points, each point is a list of x, y coordinates Returns: (np.ndarray): the xywh coordinates of the bounding boxes. )r$rr(r)rr&r')segmentsr;sr.r/rrrsegments2boxesWs  $rcst|D]\}tjddddffddtdtd|tttjfddtdDtjddd j ||<q|S) ae Inputs a list of segments (n,2) and returns a list of segments (n,2) up-sampled to n points each. Args: segments (list): a list of (n,2) arrays, where n is the number of points in the segment. n (int): number of points to resample the segment to. Defaults to 1000 Returns: segments (list): the resampled segments. rr1N)Zaxisc s&g|]}tdd|fqS)N)r&Zinterprrr.Zxprrr_xr`z%resample_segments..r2r!rL) rkr&rZlinspacerMarangerlfloat32rr$)rrrrrrresample_segmentshs ".rc Cs|j\}}}t|dddddfdd\}}}}tj||j|jdddddf} tj||j|jdddddf} || |k| |k| |k| |kS)aV It takes a mask and a bounding box, and returns a mask that is cropped to the bounding box. Args: masks (torch.Tensor): [n, h, w] tensor of masks boxes (torch.Tensor): [n, 4] tensor of bbox coordinates in relative point form Returns: (torch.Tensor): The masks are being cropped to the bounding box. Nr#r1r)rhrchunkrrr") rr;_rrx1y1Zx2y2rrrrr crop_mask}s (""rcCs|j\}}}|\}} |||dd||} || } ||} |} | dddf| 9<| dddf| 9<| dddf| 9<| dddf| 9<t| | } |rtj| d|ddd d} | d S) a; Apply masks to bounding boxes using the output of the mask head. Args: protos (torch.Tensor): A tensor of shape [mask_dim, mask_h, mask_w]. masks_in (torch.Tensor): A tensor of shape [n, mask_dim], where n is the number of masks after NMS. bboxes (torch.Tensor): A tensor of shape [n, 4], where n is the number of masks after NMS. shape (tuple): A tuple of integers representing the size of the input image in the format (h, w). upsample (bool): A flag to indicate whether to upsample the mask to the original image size. Default is False. Returns: (torch.Tensor): A binary mask tensor of shape [n, h, w], where n is the number of masks after NMS, and h and w are the height and width of the input image. The mask is applied to the bounding boxes. rLNrr2r7r1bilinearFmodeZ align_cornersr)rhrprqrrF interpolategt_)protosmasks_inbboxesrhZupsamplermhmwZihiwrZ width_ratioZ height_ratioZdownsampled_bboxesrrr process_masks  rcCsP|j\}}}|||dd||}t|d|d}t||}|dS)a It takes the output of the mask head, and crops it after upsampling to the bounding boxes. Args: protos (torch.Tensor): [mask_dim, mask_h, mask_w] masks_in (torch.Tensor): [n, mask_dim], n is number of masks after nms bboxes (torch.Tensor): [n, 4], n is number of masks after nms shape (tuple): the size of the input image (h,w) Returns: masks (torch.Tensor): The returned masks with dimensions [h, w, n] rLNrr)rhrprq scale_masksrr)rrrrhrrrrrrrprocess_mask_natives  rc Cs|jdd\}}t||d||d}||d|||d|g}|rp|dd<|dd<|rt|dt|dfnd\}}t||dt||d} } |d|| || f}tj||ddd }|S) a Rescale segment masks to shape. Args: masks (torch.Tensor): (N, C, H, W). shape (tuple): Height and width. padding (bool): If True, assuming the boxes is based on image augmented by yolo style. If False then do regular rescaling. r2Nrr1)rr.rFr)rhr(rDrr) rrhr>rrr?r@rrrrrrrrs  $"rcCs|dur\t|d|d|d|d}|d|d|d|d|d|df}n|dd}|d}|r|d|d8<|d|d8<|d|<|d|<t||}|r|d|d<|d|d<|S)a Rescale segment coordinates (xy) from img1_shape to img0_shape. Args: img1_shape (tuple): The shape of the image that the coords are from. coords (torch.Tensor): the coords to be scaled of shape n,2. img0_shape (tuple): the shape of the image that the segmentation is being applied to. ratio_pad (tuple): the ratio of the image size to the padded image size. normalize (bool): If True, the coordinates will be normalized to the range [0, 1]. Defaults to False. padding (bool): If True, assuming the boxes is based on image augmented by yolo style. If False then do regular rescaling. Returns: coords (torch.Tensor): The scaled coordinates. Nrr1r2r3r4)r(r)r:rr<r= normalizer>r?r@rrr scale_coordss"2  rcCst|jdd\}}}}}t||k||}t||k||}t||k||tjdtj}tj|||||gddS)z Regularize rotated boxes in range [0, pi/2]. Args: rboxes (torch.Tensor): Input boxes of shape(N, 5) in xywhr format. Returns: (torch.Tensor): The regularized boxes. rLrJr2)ZunbindrrorErr)rr.r/rrr Zw_Zh_rrrregularize_rboxess "rlargestcCsg}|dD]}t|tjtjd}|r|dkrVt dd|D}q|dkrt |t dd|D  dd }n t d }||d q|S) aC It takes a list of masks(n,h,w) and returns a list of segments(n,xy). Args: masks (torch.Tensor): the output of the model, which is a tensor of shape (batch_size, 160, 160) strategy (str): 'concat' or 'largest'. Defaults to largest Returns: segments (List): list of segment masks uint8rconcatcSsg|]}|ddqS)rLr2)rr[r.rrrr_"r`z"masks2segments..rcSsg|] }t|qSr)rMrrrrr_$r`rLr2)rr2r)rDrrZastyperZ findContoursZ RETR_EXTERNALZCHAIN_APPROX_SIMPLEr&rr'Zargmaxrr*r)rZstrategyrr.rrrrmasks2segmentss , r)batchreturncCs0|dddddddtjS)as Convert a batch of FP32 torch tensors (0.0-1.0) to a NumPy uint8 array (0-255), changing from BCHW to BHWC layout. Args: batch (torch.Tensor): Input tensor batch of shape (Batch, Channels, Height, Width) and dtype torch.float32. Returns: (np.ndarray): Output NumPy array batch of shape (Batch, Height, Width, Channels) and dtype uint8. rr2r7r1)Zpermute contiguousrtorrrr)rrrrconvert_torch2numpy_batch+s rcCstjdd|dS)z Cleans a string by replacing special characters with '_' character. Args: s (str): a string needing special characters replaced Returns: (str): a string with special characters replaced by an underscore _ u"[|@#!¡·$€%&()=?¿^*;:,¨´><+]r)patternreplstring)resub)rrrr clean_str8s r)r r )NTF)rH) rTrHNFFrrUrrVrWrXTF)N)r r rr)r r Fr)r)F)T)NFT)r)1 contextlibrErrrrr&rZtorch.nn.functionalnnZ functionalrZultralytics.utilsrZultralytics.utils.metricsrContextDecoratorrr0rArGrSrr9rrrrirrrrrrrrrrrrrrrrrrCZndarrayrrrrrrsj  0  &   #    !  #