a yf0@sddlmZddlZddlmmZddZd ddZe e dd d Z d!e e e e d d dZ ej ej dddZd"ej ej ej edddZddZddZe e ej ej dddZej ej ej ej ee e fee e fej dddZdS)#)TupleNcs|dkst|kri}n|dks.JditfddDdd}|dur`||<tfddDdd}|dur||<|t}tfd dDfd d d d|}fd d|DfddD}|fS)a Selects the closest conditioning frames to a given frame index. Args: frame_idx (int): Current frame index. cond_frame_outputs (Dict[int, Any]): Dictionary of conditioning frame outputs keyed by frame indices. max_cond_frame_num (int): Maximum number of conditioning frames to select. Returns: (Tuple[Dict[int, Any], Dict[int, Any]]): A tuple containing two dictionaries: - selected_outputs: Selected items from cond_frame_outputs. - unselected_outputs: Items not selected from cond_frame_outputs. Examples: >>> frame_idx = 5 >>> cond_frame_outputs = {1: "a", 3: "b", 7: "c", 9: "d"} >>> max_cond_frame_num = 2 >>> selected, unselected = select_closest_cond_frames(frame_idx, cond_frame_outputs, max_cond_frame_num) >>> print(selected) {3: 'b', 7: 'c'} >>> print(unselected) {1: 'a', 9: 'd'} z,we should allow using 2+ conditioning framesc3s|]}|kr|VqdSN.0t frame_idxr`/var/www/html/django/DPS/env/lib/python3.9/site-packages/ultralytics/models/sam/modules/utils.py )z-select_closest_cond_frames..N)defaultc3s|]}|kr|VqdSrrrr rr r .rc3s|]}|vr|VqdSrrrselected_outputsrr r 6rcs t|Sr)abs)xr rr 7rz,select_closest_cond_frames..)keyc3s|]}||fVqdSrrr)cond_frame_outputsrr r 9rcsi|]\}}|vr||qSrr)rr vrrr :rz.select_closest_cond_frames..)lenmaxminsortedupdateitems)r rZmax_cond_frame_numZunselected_outputsZ idx_beforeZ idx_afterZ num_remainZ inds_remainr)rr rr select_closest_cond_frames s*    r'cCs\|d}tj|tj|jd}|d|d|}|d|}tj||gdd}|S)zQGenerates 1D sinusoidal positional embeddings for given positions and dimensions.r)dtypedevicerdim)torcharangefloat32r"Z unsqueezecatsincos)Zpos_indsr$Z temperatureZpe_dimZdim_tZ pos_embedrrr get_1d_sine_pe?s r+)end_xend_ycCs<tj||tjd}||}tj||dd}||fS)zLInitializes 1D and 2D coordinate tensors for a grid of specified dimensions.)r!floor)Z rounding_mode)r%r&r'floatdiv)r,r-r t_xt_yrrr init_t_xyJs r3@)r$r,r-thetac Csd|td|dd|d|}d|td|dd|d|}t||\}}t||}t||}tt||}tt||} tj|| gddS)z[Computes axial complex exponential positional encodings for 2D spatial positions in a grid.?rNrr#)r%r&r/r3outerZpolarZ ones_liker() r$r,r-r5Zfreqs_xZfreqs_yr1r2Z freqs_cis_xZ freqs_cis_yrrr compute_axial_cisRs**  r9) freqs_cisrcs`|jddkrks nJ|j|jd|jdfks>Jfddt|jD}|j|S)zaReshapes frequency tensor for broadcasting with input tensor, ensuring dimensional compatibility.rrcs$g|]\}}|dkr|ndqS)rr;r)ridndimrr drz)reshape_for_broadcast..)r@shape enumerateview)r:rrBrr?r reshape_for_broadcast_s rEF)xqxkr:repeat_freqs_kc Cst|jg|jddddR}|jddkrft|jg|jddddRnd}t||}t||d}|dur|| |j |fS|r|jd|jd}|j gdg|j d|dR}t||d}|| |j || |j fS)zfApplies rotary positional encoding to query and key tensors using complex-valued frequency components.Nrrr<rr;) r%Zview_as_complexr/reshaperBrEZ view_as_realflattenZtype_astor"repeatr@) rFrGr:rHZxq_Zxk_Zxq_outrZxk_outrrr apply_rotary_enchs,> $rOc Cs|j\}}}}||||}||||}|dks>|dkrVt|ddd|d|f}||||}} |||||| |||}|ddddddd|||} | || ffS)a Partitions input tensor into non-overlapping windows with padding if needed. Args: x (torch.Tensor): Input tensor with shape (B, H, W, C). window_size (int): Size of each window. Returns: (Tuple[torch.Tensor, Tuple[int, int]]): A tuple containing: - windows (torch.Tensor): Partitioned windows with shape (B * num_windows, window_size, window_size, C). - (Hp, Wp) (Tuple[int, int]): Padded height and width before partition. Examples: >>> x = torch.randn(1, 16, 16, 3) >>> windows, (Hp, Wp) = window_partition(x, window_size=4) >>> print(windows.shape, Hp, Wp) torch.Size([16, 4, 4, 3]) 16 16 rr;rIrr7r)rBFpadrDpermute contiguous) r window_sizeBHWCZpad_hZpad_wHpWpwindowsrrr window_partition~s$r]c Cs|\}}|\}}|jd||||}||||||||d} | dddddd|||d} ||ksz||kr| ddd|d|ddf} | S) af Unpartitions windowed sequences into original sequences and removes padding. This function reverses the windowing process, reconstructing the original input from windowed segments and removing any padding that was added during the windowing process. Args: windows (torch.Tensor): Input tensor of windowed sequences with shape (B * num_windows, window_size, window_size, C), where B is the batch size, num_windows is the number of windows, window_size is the size of each window, and C is the number of channels. window_size (int): Size of each window. pad_hw (Tuple[int, int]): Padded height and width (Hp, Wp) of the input before windowing. hw (Tuple[int, int]): Original height and width (H, W) of the input before padding and windowing. Returns: (torch.Tensor): Unpartitioned sequences with shape (B, H, W, C), where B is the batch size, H and W are the original height and width, and C is the number of channels. Examples: >>> windows = torch.rand(32, 8, 8, 64) # 32 windows of size 8x8 with 64 channels >>> pad_hw = (16, 16) # Padded height and width >>> hw = (15, 14) # Original height and width >>> x = window_unpartition(windows, window_size=8, pad_hw=pad_hw, hw=hw) >>> print(x.shape) torch.Size([1, 15, 14, 64]) rrr;rIrr7rPN)rBrDrSrT) r\rUZpad_hwZhwrZr[rWrXrVrrrr window_unpartitions$$r^)q_sizek_sizerel_posreturncCstdt||d}|jd|krdtj|d|jddddd|dd}|d|dd}n|}t|dddft||d}t|dddft||d}|||dt||d}|| S) a Extracts relative positional embeddings based on query and key sizes. Args: q_size (int): Size of the query. k_size (int): Size of the key. rel_pos (torch.Tensor): Relative position embeddings with shape (L, C), where L is the maximum relative distance and C is the embedding dimension. Returns: (torch.Tensor): Extracted positional embeddings according to relative positions, with shape (q_size, k_size, C). Examples: >>> q_size, k_size = 8, 16 >>> rel_pos = torch.randn(31, 64) # 31 = 2 * max(8, 16) - 1 >>> extracted_pos = get_rel_pos(q_size, k_size, rel_pos) >>> print(extracted_pos.shape) torch.Size([8, 16, 64]) rr;rrZlinear)sizemodeNr6) intrrBrQZ interpolaterJrSr%r&long)r_r`raZ max_rel_distZrel_pos_resizedZq_coordsZk_coordsZrelative_coordsrrr get_rel_poss$$rg)attnq rel_pos_h rel_pos_wr_r`rbcCs|\}}|\}} t|||} t|| |} |j\} } }|| |||}td|| }td|| }|| |||| |dddddddddf|dddddddddf| |||| }|S)aP Adds decomposed Relative Positional Embeddings to the attention map. This function calculates and applies decomposed Relative Positional Embeddings as described in the MVITv2 paper. It enhances the attention mechanism by incorporating spatial relationships between query and key positions. Args: attn (torch.Tensor): Attention map with shape (B, q_h * q_w, k_h * k_w). q (torch.Tensor): Query tensor in the attention layer with shape (B, q_h * q_w, C). rel_pos_h (torch.Tensor): Relative position embeddings for height axis with shape (Lh, C). rel_pos_w (torch.Tensor): Relative position embeddings for width axis with shape (Lw, C). q_size (Tuple[int, int]): Spatial sequence size of query q as (q_h, q_w). k_size (Tuple[int, int]): Spatial sequence size of key k as (k_h, k_w). Returns: (torch.Tensor): Updated attention map with added relative positional embeddings, shape (B, q_h * q_w, k_h * k_w). Examples: >>> B, C, q_h, q_w, k_h, k_w = 1, 64, 8, 8, 8, 8 >>> attn = torch.rand(B, q_h * q_w, k_h * k_w) >>> q = torch.rand(B, q_h * q_w, C) >>> rel_pos_h = torch.rand(2 * max(q_h, k_h) - 1, C) >>> rel_pos_w = torch.rand(2 * max(q_w, k_w) - 1, C) >>> q_size, k_size = (q_h, q_w), (k_h, k_w) >>> updated_attn = add_decomposed_rel_pos(attn, q, rel_pos_h, rel_pos_w, q_size, k_size) >>> print(updated_attn.shape) torch.Size([1, 64, 64]) References: https://github.com/facebookresearch/mvit/blob/main/mvit/models/attention.py zbhwc,hkc->bhwkzbhwc,wkc->bhwkN)rgrBrJr%ZeinsumrD)rhrirjrkr_r`Zq_hZq_wZk_hZk_wZRhZRwrV_r$Zr_qZrel_hZrel_wrrr add_decomposed_rel_poss)   Vrm)r )r4)F)typingrr%Ztorch.nn.functionalnnZ functionalrQrr+rer3r/r9ZTensorrEboolrOr]r^rgrmrrrr s2 6     &+