a Sic @sdZddlZddlZddlZddlZddlmZmZmZddl Z ddl m Z gdZ ddedhZGd d d eZd d Zd dZddZddZdWddZddZddZeedZddZddZddZeed ZGd!d"d"eZdXd#d$Zej edd%Z!ej ed&d%Z"ej edd%Z#d'd(Z$d)d*Z%d+d,Z&d-d.Z'dYd0d1Z(dZd2d3Z)d4d5Z*d6d7Z+d8d9Z,d:d;Z-dd?Z/d@dAZ0dBdCZ1GdDdEdEeZ2d[dFdGZ3iZ4e5ddHD]Z6ee4e6<qe5dHdID]Z6e!e4e6<qe5dIdJD]Z6e"e4e6<qe5dJdKD]Z6e#e4e6<qd\dLdMZ7iZ8e5ddND]Z6ee8e6<q.e5dNdOD]Z6e3e8e6<qHd]dPdQZ9e7e9ee!e"e#e)e)e)e3e3dR Z:dSdTZ;dUdVZ}|tfdd|D|D]}|dd8<qJq&|S)z Convert a path with static single assignment ids to a path with recycled linear ids. For example:: >>> ssa_to_linear([(0, 3), (2, 4), (1, 5)]) [(0, 3), (1, 2), (0, 1)] r)dtypec3s|]}t|VqdSr)int).0ssa_ididsrr Jz ssa_to_linear..N)nparangemaxmapint32appendtuple)ssa_pathpathssa_idsr*rr+r ssa_to_linear?sr9csttt|t|d}tt|t|}g}|D]F}|tfdd|Dt |ddD] }|=qdt |q8|S)z Convert a path with recycled linear ids to a path with static single assignment ids. For example:: >>> linear_to_ssa([(0, 3), (1, 2), (0, 1)]) [(0, 3), (2, 4), (1, 5)] rc3s|]}|VqdSrr)r)id_ linear_to_ssarrr-]r.z linear_to_ssa..T)reverse) sumr2lenlistrange itertoolscountr4r5sortednext)r7 num_inputsZnew_idsr6r,r:rr;rr<Ps  r<c Csh||||}}||B}||@} tj|gt|j|||hR} || @} t|| | d|} | | fS)a Calculate the resulting indices and flops for a potential pairwise contraction - used in the recursive (optimal/branch) algorithms. Parameters ---------- inputs : tuple[frozenset[str]] The indices of each tensor in this contraction, note this includes tensors unavaiable to contract as static single assignment is used -> contracted tensors are not removed from the list. output : frozenset[str] The set of output indices for the whole contraction. remaining : frozenset[int] The set of indices (corresponding to ``inputs``) of tensors still available to contract. i : int Index of potential tensor to contract. j : int Index of potential tensor to contract. size_dict dict[str, int] Size mapping of all the indices. Returns ------- k12 : frozenset The resulting indices of the potential tensor. cost : int Estimated flop count of operation. ) frozensetunionr2 __getitem__r flop_count) rr remainingijrk1k2eithersharedkeepk12costrrrcalc_k12_flopsds"rVcCs2tjt|j|}||}t|}t||||S)z Compute the flop count for a contraction of all remaining arguments. This is used when a memory limit means that no pairwise contractions can be made. )rHrIr2rJr?rrK)rrLrridx_contractioninner num_termsrrr_compute_oversize_flopssrZcszttt|}ttdttt|fdiifddd|ttt|ddtdS) a Computes all possible pair contractions in a depth-first recursive manner, sieving results based on ``memory_limit`` and the best path found so far. Returns the lowest cost path. This algorithm scales factoriallly with respect to the elements in the list ``input_sets``. Parameters ---------- inputs : list List of sets that represent the lhs side of the einsum subscript. output : set Set that represents the rhs side of the overall einsum subscript. size_dict : dictionary Dictionary of index sizes. memory_limit : int The maximum number of elements in a temporary array. Returns ------- path : list The optimal contraction order within the memory limit constraint. Examples -------- >>> isets = [set('abd'), set('ac'), set('bdc')] >>> oset = set('') >>> idx_sizes = {'a': 1, 'b':2, 'c':3, 'd':4} >>> optimal(isets, oset, idx_sizes, 5000) [(0, 2), (0, 1)] r)flopsr6c sjt|dkr |d<|d<dSt|dD]6\}}||krH||}}||||f}z|\}}Wn0tyt||||\}}|<Yn0||} | dkrq,tvr2z |} Wn&tyt|} |<Yn0| kr2|t||} | dkr,| d<|t |fd<q,|||ff||f|||ht|hB| dq,dS)Nrr[r6rGr7rrLr[) r?rB combinationsKeyErrorrV_UNLIMITED_MEMrcompute_size_by_dictrZr5) r7rLrr[rMrNkeyrTflops12 new_flopssize12_optimal_iteratebestr!r result_cache size_cacherrrrfs>   $      z!optimal.._optimal_iteraterrr\r6)r5r2rHfloatrAr?setr9)rrrr!rrerrs,rcCs||f||fkSrrr[sizeZ best_flopsZ best_sizerrrbetter_flops_firstsrncCs||f||fkSrrrlrrrbetter_size_firstsror[rmcCst|Sr) _BETTER_FNSrarrr get_better_fnsrscCs |||S)zpThe default heuristic cost, corresponding to the total reduction in memory of performing a contraction. rrdsize1size2rTrOrPrrrcost_memory_removedsrwcCstdd|||S)zyLike memory-removed, but with a slight amount of noise that breaks ties and thus jumbles the contractions a bit. g?g{Gz?)randomgaussrtrrrcost_memory_removed_jittersrz)memory-removedzmemory-removed-jitterc@s0eZdZdZd ddZedd Zd d d ZdS)ra Explores possible pair contractions in a depth-first recursive manner like the ``optimal`` approach, but with extra heuristic early pruning of branches as well sieving by ``memory_limit`` and the best path found so far. Returns the lowest cost path. This algorithm still scales factorially with respect to the elements in the list ``input_sets`` if ``nbranch`` is not set, but it scales exponentially like ``nbranch**len(input_sets)`` otherwise. Parameters ---------- nbranch : None or int, optional How many branches to explore at each contraction step. If None, explore all possible branches. If an integer, branch into this many paths at each step. Defaults to None. cutoff_flops_factor : float, optional If at any point, a path is doing this much worse than the best path found so far was, terminate it. The larger this is made, the more paths will be fully explored and the slower the algorithm. Defaults to 4. minimize : {'flops', 'size'}, optional Whether to optimize the path with regard primarily to the total estimated flop-count, or the size of the largest intermediate. The option not chosen will still be used as a secondary criterion. cost_fn : callable, optional A function that returns a heuristic 'cost' of a potential contraction with which to sort candidates. Should have signature ``cost_fn(size12, size1, size2, k12, k1, k2)``. Nr[r{cCsP||_||_||_t|||_t||_tdtdd|_ t dd|_ dS)NrrpcSstdS)Nr)rjrrrr?r.z&BranchBound.__init__..) nbranchcutoff_flops_factorminimize _COST_FNSgetcost_fnrsbetterrjrgr best_progress)rr~rrrrrr__init__7s zBranchBound.__init__cCst|jdS)Nr6)r9rg)rrrrr7AszBranchBound.pathcsv|ttt|}tfdd|Difddd|ttt|dddjS)a Parameters ---------- input_sets : list List of sets that represent the lhs side of the einsum subscript output_set : set Set that represents the rhs side of the overall einsum subscript idx_dict : dictionary Dictionary of index sizes memory_limit : int The maximum number of elements in a temporary array Returns ------- path : list The contraction order within the memory limit constraint. Examples -------- >>> isets = [set('abd'), set('ac'), set('bdc')] >>> oset = set('') >>> idx_sizes = {'a': 1, 'b':2, 'c':3, 'd':4} >>> optimal(isets, oset, idx_sizes, 5000) [(0, 2), (0, 1)] csi|]}|t|qSrrr`r)krrr er.z(BranchBound.__call__..c stdkr. jd< jd< jd<dS  f dd}g}tdD]V\}}||krx||}}||} } | | rq^|| | ||} | r^t|| q^|stdD]J\}}||kr||}}||} } || | ||} | rt|| qd} jdus0| jkr|rt|\} } }}\}}}||ff|f||hthB||d | d7} qdS) Nrrmr[r6c sz||f\}}Wn4tyHt|| \}}||f<Yn0z |}Wn&ty|t| } |<Yn0|}t|}||jdjdsdS|jtkr|jt<n|j jtkrdSt vrF|krFt  }|jdkrB|jd<t fjd<dS | |} }  || | |||} | |||||f|fS)Nr[rmr6)r^rVrr`r1rrgrr?rr_rZr5r) rOrPrMrNrTrbrdrcnew_sizerurvrU) r[rr!rr7rLrhrrmrirrr_assess_candidateqs2 (    zHBranchBound.__call__.._branch_iterate.._assess_candidaterGrr7rrLr[rm) r?rgrBr] isdisjointheapqheappushr~heappop)r7rrLr[rmr candidatesrMrNrOrP candidatebi_rcrrT_branch_iterater!rrhrrir)r[rr7rLrmrrhsD     *   z-BranchBound.__call__.._branch_iteraterrr)rr5r2rHrkrAr?r7r rrrr"EsVzBranchBound.__call__)Nr|r[r{)N)r#r$r%r&rpropertyr7r"rrrrrs   rcKstfi|}|||||Sr)r)rrrr!Zoptimizer_kwargs optimizerrrrr sr )r~rGcCs||B}||@} || } ||@| |d@B| |d@B} |t| |||||| ||} ||} ||}| |kr|||| f\}} }}| || f} | ||| fS)NrGr)rsizesrL footprintsdim_ref_countsrOrPrrQtwoonerTrUZid1Zid2rrr_get_candidates " rc sNfdd|D} |r:| D]} t|| q&nt|t| dS)Nc 3s$|]}t|VqdSr)r)r)rPrrrrOrrLrrrr-r.z"_push_candidate..)rrmin) rrrLrrrOk2squeuepush_allrrrrrr_push_candidates rcCs|D]x}t||}|dkr:|d||d|q|dkr`|d||d|q|d||d|qdS)NrrGr)r?discardadd) dim_to_keysrdimsdimrCrrr_update_ref_countss rcCs2t|\}}}}||vs"||vr&dS||||fS)zKDefault contraction chooser that simply takes the minimum cost option. N)rr)rrLrUrOrPrTrrr_simple_choosersrr{c sNt|dkrdgSt||}|dur0t}d}nd}ttt|}ttj|Bi}t t|}g}t |D]8\} } | |vr| || | ft ||| <qr| || <qrt t|D] } | D]} | | qqfdddD} fd d|D} g}D]^\} }t||jd }t |dd D]4\}}|d|d}t|| | ||||| q2q |r|||}|durql|\}}}}||}||}|D]} | |q|D]} | |q| ||f||vr| ||t |fn|D]} | |qt |||<t| ||Bt|| |<|}tfd d |D}|||rlt|| | ||||| qlfdd|D}t|t|\}}}|rJt|\}}}| t||t||f||B@}t|}t |}t||||f\}}}q|S)z This is the core function for :func:`greedy` but produces a path with static single assignment ids rather than recycled linear ids. SSA ids are cheaper to work with and easier to reason about. rrNFTcs,i|]$tfddDqS)c3s"|]\}}t|kr|VqdSrr?)r)rkeysrCrrr--r.z1ssa_greedy_optimize...)rkitems)r))rrrrr,sz'ssa_greedy_optimize..)rGrcsi|]}|t|qSrr)r)ra)rrrr2r.rrrc3s |]}|D] }|VqqdSrr)r)rrP)rrrr-Vr.z&ssa_greedy_optimize..cs&g|]\}}t|@||fqSrr)r)rar*)rrrr \r.z'ssa_greedy_optimize..) r?rrrr@r2rH intersectionrBrC enumerater4rErrkrrrDrJrpopremoverrr`rrheapifyrrr1 heappushpop)rrr choose_fnrrrLr8r6r*rarrrrrrMrOrconrUrPrTZssa_id1Zssa_id2rZssa_id12r)rrrrssa_greedy_optimizes     "              rcCs6|tvrt||||d|dSt|||||d}t|S)a Finds the path by a three stage algorithm: 1. Eagerly compute Hadamard products. 2. Greedily compute contractions to maximize ``removed_size`` 3. Greedily compute outer products. This algorithm scales quadratically with respect to the maximum number of elements sharing a common dim. Parameters ---------- inputs : list List of sets that represent the lhs side of the einsum subscript output : set Set that represents the rhs side of the overall einsum subscript size_dict : dictionary Dictionary of index sizes memory_limit : int The maximum number of elements in a temporary array choose_fn : callable, optional A function that chooses which contraction to perform from the queu cost_fn : callable, optional A function that assigns a potential contraction a cost. Returns ------- path : list The contraction order (a list of tuples of ints). Examples -------- >>> isets = [set('abd'), set('ac'), set('bdc')] >>> oset = set('') >>> idx_sizes = {'a': 1, 'b':2, 'c':3, 'd':4} >>> greedy(isets, oset, idx_sizes) [(0, 2), (0, 1)] r)r~r)rr)r_r rr9)rrrr!rrr6rrrr js'r cst|tkrgS|g}g}g}t|dkr|d}|dttdd|DD]<|dtfdd|Df7<||ddqTdd|DD],|dt|t|f7<|qq|S)a Converts a contraction tree to a contraction path as it has to be returned by path optimizers. A contraction tree can either be an int (=no contraction) or a tuple containing the terms to be contracted. An arbitrary number (>= 1) of terms can be contracted at once. Note that contractions are commutative, e.g. (j, k, l) = (k, l, j). Note that in general, solutions are not unique. Parameters ---------- c : tuple or int Contraction tree Returns ------- path : list[set[int]] Contraction path Examples -------- >>> _tree_to_sequence(((1,2),(0,(4,5,3)))) [(1, 2), (1, 2, 3), (0, 2), (0, 1)] rrcSsg|]}t|tkr|qSrtyper(r)rMrrrrr.z%_tree_to_sequence..c3s|]}|krdVqdSrNr)r)qrMrrr-r.z$_tree_to_sequence..cSsg|]}t|tkr|qSrrrrrrrr.) rr(r?rinsertr5rDr>r4)ctsrNrrr_tree_to_sequences$   $rc sg}ttt}tj|}t|dkrt}|g}t|dkr|}||||@fdd|D}||||q>||q"|S)a Finds disconnected subgraphs in the given list of inputs. Inputs are connected if they share summation indices. Note: Disconnected subgraphs can be contracted independently before forming outer products. Parameters ---------- inputs : list[set] List of sets that represent the lhs side of the einsum subscript output : set Set that represents the rhs side of the overall einsum subscript Returns ------- subgraphs : list[set[int]] List containing sets of indices for each subgraph Examples -------- >>> _find_disconnected_subgraphs([set("ab"), set("c"), set("ad")], set("bd")) [{0, 2}, {1}] >>> _find_disconnected_subgraphs([set("ab"), set("c"), set("ad")], set("abd")) [{0}, {1}, {2}] rcs$h|]}t|@dkr|qSrrrZi_tmprrr r.z/_find_disconnected_subgraphs..) rkrAr?rIrrextenddifference_updater4) rr subgraphsZ unused_inputsZi_sumgrrNnrrr_find_disconnected_subgraphss        rcCs"ddt|t|dddDS)zSelect elements of ``seq`` which are marked by the bitmap set ``s``. E.g.: >>> list(_bitmap_select(0b11010, ['A', 'B', 'C', 'D', 'E'])) ['B', 'D', 'E'] css|]\}}|dkr|VqdS)1Nr)r)xbrrrr- r.z!_bitmap_select..Nrr)zipbin)rseqrrr_bitmap_selectsrc Cs8|||A@}|r"tjt||}nt}||}||S)zoCalculates the effective outer indices of the intermediate tensor corresponding to the subgraph ``s``. )rkrIr) r all_tensorsrri1_cut_i2_wo_output i1_union_i2rZi_rZ i_contractrrr _dp_calc_legs s  rcCs||t||}||kr|||B}||vs<|||dkr|t|| || | |}t||}| dusj|| kr|||| |ff||<dS)aPerforms the inner comparison of whether the two subgraphs (the bitmaps ``s1`` and ``s2``) should be merged and added to the dynamic programming search. Will skip for a number of reasons: 1. If the number of operations to form ``s = s1 | s2`` including previous contractions is above the cost-cap. 2. If we've already found a better way of making ``s``. 3. If the intermediate tensor corresponding to ``s`` is going to break the memory limit. rN)rr`r)cost1cost2rrcost_caps1s2xnrrrrr!cntrct1cntrct2rUrrMmemrrr_dp_compare_flopss  rcCsx||B}t|| || | |}t||}t|||}||krt||vsR|||dkrt| dusb|| krt||| |ff||<dS)zLike ``_dp_compare_flops`` but sieves the potential contraction based on the size of the intermediate tensor created, rather than the number of operations, and so calculates that first. rN)rrr`r1)rrrrrrrrrrrrr!rrrrMrrUrrr_dp_compare_size3s  rcCstdd|S)zMake a simple left to right binary tree out of iterable ``seq``. >>> tuple_nest([1, 2, 3, 4]) (((1, 2), 3), 4) cSs||fSrrryrrrr}Jr.z#simple_tree_tuple..) functoolsreduce)rrrrsimple_tree_tupleCsrc s~fddt|D}ggg}}}t|D]D\}}||} | sP||fq.|| || |krl|fn|q.|||fS)aUTake ``inputs`` and parse for single term index operations, i.e. where an index appears on one tensor and nowhere else. If a term is completely reduced to a scalar in this way it can be removed to ``inputs_done``. If only some indices can be summed then add a 'single term contraction' that will perform this summation. cs h|]\}}|dkr|qS)rr)r)rMr ind_countsrrrUr.z0_dp_parse_out_single_term_ops..)rr4) rall_indsrZi_singleZ inputs_parsed inputs_doneinputs_contractionsrNrMZ i_reducedrrr_dp_parse_out_single_term_opsMs rc@s$eZdZdZd ddZd dd ZdS) ra Finds the optimal path of pairwise contractions without intermediate outer products based a dynamic programming approach presented in Phys. Rev. E 90, 033315 (2014) (the corresponding preprint is publically available at https://arxiv.org/abs/1304.6112). This method is especially well-suited in the area of tensor network states, where it usually outperforms all the other optimization strategies. This algorithm shows exponential scaling with the number of inputs in the worst case scenario (see example below). If the graph to be contracted consists of disconnected subgraphs, the algorithm scales linearly in the number of disconnected subgraphs and only exponentially with the number of inputs per subgraph. Parameters ---------- minimize : {'flops', 'size'}, optional Whether to find the contraction that minimizes the number of operations or the size of the largest intermediate tensor. cost_cap : {True, False, int}, optional How to implement cost-capping: * True - iteratively increase the cost-cap * False - implement no cost-cap at all * int - use explicit cost cap search_outer : bool, optional In rare circumstances the optimal contraction may involve an outer product, this option allows searching such contractions but may well slow down the path finding considerably on all but very small graphs. r[TFcCsB||_ttd|j|_||_ddddd|j|_||_dS)NrpcSs|Srrrrrrr}r.z-DynamicProgramming.__init__..cSsdS)NTrrrrrr}r.)FT)rrr_check_contraction search_outer _check_outerr)rrrrrrrrszDynamicProgramming.__init__Nc!s(ttjg|R}t|}ddt|DfddDtfdd|D}fddDfddttDt ||\}st t |S|d gt|}|j rtttg} n t |} d t>d } | D]} d gd d dtt| d D} tfd d| D| d <tdddd| D} tjt| } |jdurt| |@}n|jdurtd}n|j}tttj| d }t| ddkrtd t| d d D]}| |}td |d d D]}| |D]\}\}}}| ||D]x\}\}}}||@s*|||ksZ||kr*||@|}||r*||B}||||||||| | ||||q*q qq||}qt| dd\}}}||t|qfddt tt||jdDt } t | S)a Parameters ---------- inputs : list List of sets that represent the lhs side of the einsum subscript output : set Set that represents the rhs side of the overall einsum subscript size_dict : dictionary Dictionary of index sizes memory_limit : int The maximum number of elements in a temporary array Returns ------- path : list The contraction order (a list of tuples of ints). Examples -------- >>> n_in = 3 # exponential scaling >>> n_out = 2 # linear scaling >>> s = dict() >>> i_all = [] >>> for _ in range(n_out): >>> i = [set() for _ in range(n_in)] >>> for j in range(n_in): >>> for k in range(j+1, n_in): >>> c = oe.get_symbol(len(s)) >>> i[j].add(c) >>> i[k].add(c) >>> s[c] = 2 >>> i_all.extend(i) >>> o = DynamicProgramming() >>> o(i_all, set(), s) [(1, 2), (0, 4), (1, 2), (0, 2), (0, 1)] cSsi|]\}}||qSrr)r)rNrrrrrr.z/DynamicProgramming.__call__..cs"g|]}tfdd|DqS)c3s|]}|VqdSrrr)r symbol2intrrr-r.z9DynamicProgramming.__call__...)rkrrrrrr.z/DynamicProgramming.__call__..c3s|]}|VqdSrrrrrrr-r.z.DynamicProgramming.__call__..cs"i|]\}}|vr||qSrr)r)rvrrrrr.csg|] }|qSrrr)rNrrrrr.rNrGcSsg|] }tqSr)dictrrrrrr.c3s(|] }d|>|d|ffVqdS)rrNrr)rrrrr-r.cSs||BSrrrrrrr}r.z-DynamicProgramming.__call__..css|]}d|>VqdSrrrrrrr-r.TFrrrcsg|] }|qSrrr)subgraph_contractionsrrrsrr)!rrBchainr5rrkrrAr?rrrrrrrrrIrrrr`rjr1rr2rJrrr@valuesr4rD)!rrrrr!rrrZsubgraph_contractions_sizerrrrZ subgraph_indsrZcost_incrementrrmri1rrri2rrrrrMrU contractiontreer)rrrrrrr"sd%   $   "      zDynamicProgramming.__call__)r[TF)N)r#r$r%r&rr"rrrrrds rcKstfi|}|||||Sr)r)rrrr!kwargsrrrrr sr cCst|}t|t||||S)zuFinds the contraction path by automatically choosing the method based on how many input arguments there are. )r? _AUTO_CHOICESrr )rrrr!Nrrrr 0sr cCs*ddlm}t|}t||||||S)zFinds the contraction path by automatically choosing the method based on how many input arguments there are, but targeting a more generous amount of search time than ``'auto'``. r)random_greedy_128) path_randomrr?_AUTO_HQ_CHOICESr)rrrr!rr rrrr ?s r ) r zauto-hqrz branch-allzbranch-2zbranch-1r eagerZ opportunisticdpzdynamic-programmingcCs&|tvrtd||t|<dS)zAAdd path finding function ``fn`` as an option with ``name``. z#Path optimizer '{}' already exists.N) _PATH_OPTIONSr^formatlower)namefnrrrregister_path_fnYsrcCs(|tvr td|ttt|S)zBGet the correct path finding function from str ``path_type``. z4Path optimizer '{}' not found, valid options are {}.)rr^rrkr) path_typerrrr bs  r )N)N)Nr{)NNr{)N)N)N)=r&rrrBrx collectionsrrrnumpyr/r__all__rjr_objectrr9r<rVrZrrnrorqrsrwrzrrr partialZ branch_allZbranch_2Zbranch_1rrrrrr rrrrrrrrrrr rArMr rr rrr rrrrs () Z)   n .:0  =