a yf@shddlZddlmZddlmZddlmZmZddlm Z ddl m Z ddl m Z Gd d d eZdS) N)Image)SegmentationPredictor) DEFAULT_CFGchecks)box_iou) scale_masks)adjust_bboxes_to_image_bordercsNeZdZdZeddffdd ZfddZd ddZd d Zd d Z Z S)FastSAMPredictora FastSAMPredictor is specialized for fast SAM (Segment Anything Model) segmentation prediction tasks in Ultralytics YOLO framework. This class extends the SegmentationPredictor, customizing the prediction pipeline specifically for fast SAM. It adjusts post-processing steps to incorporate mask prediction and non-max suppression while optimizing for single- class segmentation. Ncst|||i|_dS)z]Initializes a FastSAMPredictor for fast SAM segmentation tasks in Ultralytics YOLO framework.N)super__init__prompts)selfcfgZ overrides _callbacks __class__^/var/www/html/django/DPS/env/lib/python3.9/site-packages/ultralytics/models/fastsam/predict.pyr szFastSAMPredictor.__init__c s|jdd}|jdd}|jdd}|jdd}t|||}|D]t} tjdd| jd| jdg|djtjd} t | j j | j} t t | d| d k} | dkrL| | j j | <qL|j|||||d S) z0Applies box postprocess for FastSAM predictions.bboxesNpointslabelstextsrr)devicedtypeg?)rrrr)r popr postprocesstorchtensor orig_shaperZfloat32r boxesxyxynonzerorflattenZnumelprompt) rpredsZimgZ orig_imgsrrrrresultsresultZfull_boxr idxrrrrs" zFastSAMPredictor.postprocessc sp|dur|dur|dur|Sg}t|ts0|g}|D]4}|jjjdd|jkrjtd|jdtjt |tj |j d}|durJtj |tj |j d}|jdkr|dn|}|dddf|dddf|dddf|dddf} tfdd|D} tjd d } | dddf| | } d |tj| | dd <|durhtj |tj |j d}|jdkr||dn|}|durt|jd}tj |tj |j d}t |t |ksJd t |d t ||dkrtjt |tj |j dntjt |tj |j d} t||D]:\}}t || tjdd|d|dfd dd<q$|| O}|dur\t|tr|g}gg}}t|jjD]j\}}dd|D\}}}}|dkr||q|t|j||||dddfq|||}tj|dd }t |rT|tj||j ddt |kd7}d ||<|||q4|S)ad Internal function for image segmentation inference based on cues like bounding boxes, points, and masks. Leverages SAM's specialized architecture for prompt-based, real-time segmentation. Args: results (Results | List[Results]): The original inference results from FastSAM models without any prompts. bboxes (np.ndarray | List, optional): Bounding boxes with shape (N, 4), in XYXY format. points (np.ndarray | List, optional): Points indicating object locations with shape (N, 2), in pixels. labels (np.ndarray | List, optional): Labels for point prompts, shape (N, ). 1 = foreground, 0 = background. texts (str | List[str], optional): Textual prompts, a list contains string objects. Returns: (List[Results]): The output results determined by prompts. Nrr)rrcs>g|]6}dd|d|d|d|dfjddqS)Nrr)rr*rr*dim)sum).0bmasksrr Lz+FastSAMPredictor.prompt..r+r,Tz4Excepted `labels` got same size as `point`, but got z and )as_tuplecss|]}t|VqdS)N)int)r/xrrr gr4z*FastSAMPredictor.prompt..dr)! isinstancelistr2datashaperrrZzeroslenboolrZ as_tensorZint32ndimstackr.ZargmaxZoneszipr"str enumerater r!tolistappendrZ fromarrayZorig_img_clip_inferencerr6)rr&rrrrZprompt_resultsr'r(Z bbox_areasZ mask_areasZfull_mask_areasunionZ point_idxpointlabelZcrop_imsZ filter_idxir0x1y1Zx2y2Z similarityZtext_idxrr1rr$.sj   @    4    .  &zFastSAMPredictor.promptcsz ddl}Wn$ty0tdddl}Yn0tdrFtds^|jdjd\__t fdd |D}| | j}j |}j|}||jd d d }||jd d d }||dddfd S) au CLIP Inference process. Args: images (List[PIL.Image]): A list of source images and each of them should be PIL.Image type with RGB channel order. texts (List[str]): A list of prompt texts and each of them should be string object. Returns: (torch.Tensor): The similarity between given images and texts. rNz+git+https://github.com/ultralytics/CLIP.git clip_modelclip_preprocesszViT-B/32r;csg|]}|jqSr)rRtor)r/imagerrrr3r4z4FastSAMPredictor._clip_inference..r:T)r-Zkeepdim)clip ImportErrorrcheck_requirementshasattrloadrrQrRrrCtokenizerSZ encode_imageZ encode_textZnormr.)rZimagesrrVZtokenized_textZimage_featuresZ text_featuresrrUrrIvs      z FastSAMPredictor._clip_inferencecCs ||_dS)zSet prompts in advance.N)r )rr rrr set_promptsszFastSAMPredictor.set_prompts)NNNN) __name__ __module__ __qualname____doc__rr rr$rIr\ __classcell__rrrrr s    Hr )rZPILrZultralytics.models.yolo.segmentrZultralytics.utilsrrZultralytics.utils.metricsrZultralytics.utils.opsrutilsr r rrrrs