a Sics@sdZddlZddlZddlZddlZddlZddlmm Z ddl m Z ddl mZeddddZdd d Zd d Zdd dZdddZd ddZddZddZddZGdddZdS)!z/Utilities related to layer/model functionality.N)io_utils) keras_exportzkeras.utils.get_source_inputscst|ds|S|dus|r&|j\}}}|js2|gS|j|}|jrPtj|jSg}|D]B\}}}}t |||}|D]$t fdd|Drx| qxq\|SdS)aReturns the list of input tensors necessary to compute `tensor`. Output will always be a list of tensors (potentially with 1 element). Args: tensor: The tensor to start from. layer: Origin layer of the tensor. Will be determined via tensor._keras_history if not provided. node_index: Origin node index of the tensor. Returns: List of input tensors. _keras_historyNc3s|]}|uVqdSN).0txrS/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/utils/layer_utils.py Bz$get_source_inputs..) hasattrr_inbound_nodesis_inputtfnestflatten input_tensorsiterate_inboundget_source_inputsallappend)tensorlayer node_index_nodesource_tensorsprevious_sourcesrr r r s      rFc Cs|r|durdS|r t|r dSt|tr6||vr6dS|r>dnd}||rLdnd7}|d|f7}|rrd|d}nd}td|d |d |d |d | dS) z0Validates the correctness of a string-based arg.Nz`None`, za `Callable`, z"or one of the following values: %szIf restoring a model and `z` is a custom callable, please ensure the callable is registered as a custom object. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details. zUnkown value for `z` argument of layer z. zAllowed values are: z . Received: )callable isinstancestr ValueError) input_dataallowable_strings layer_namearg_name allow_noneallow_callables allowed_argsZ callable_noterrr validate_string_argGs0    r,cCsRdd|D}dd|D}dd|D}dd|D}ttdd|DS) zCount the total number of scalars composing the weights. Args: weights: An iterable containing the weights on which to compute params Returns: The total number of scalars composing the weights cSsi|]}t||qSr)idrwrrr ur z count_params..cSsg|]}t|dr|qS)shape)rr.rrr wr z count_params..cSsg|]}|jqSr)r1as_listr.rrr r2xr cSsg|]}dd|DqS)cSsg|]}|durdn|qS)Nrr)rw_irrr r2zr z+count_params...rr.rrr r2yscss|]}t|VqdSr)npprodrprrr r |r zcount_params..)valuesintsum)weightsunique_weights weight_shapesstandardized_weight_shapesrrr count_paramsls r@csdurXtdkr,tddttdtrHtdtsftdndt|jgSfdd t|jD}fd d t|jD}|r|std t|t|krt|t|dgSt|t|dgS) a3Get the layer indexes from the model based on layer names. The layer indexes can be used to slice the model into sub models for display. Args: model: `tf.keras.Model` instance. layer_names: a list or tuple of 2 strings, the starting layer name and ending layer name (both inclusive) for the result. All layers will be included when `None` is provided. Returns: The index value of layer based on its unique name (layer_names). Output will be [first_layer_index, last_layer_index + 1]. NzIlayer_range must be a list or tuple of length 2. Received: layer_range = z of length rz7layer_range should contain string type only. Received: cs&g|]\}}td|jr|qSrrematchnameridxr layer_rangerr r2sz7get_layer_index_bound_by_layer_name..cs&g|]\}}td|jr|qS)rBrDrHrJrr r2szHPassed layer_names do not match the layer names in the model. Received: )lenr$r"r#layers enumerateminmax)modelrKZ lower_indexZ upper_indexrrJr #get_layer_index_bound_by_layer_namesB   rRcsdurtj|jjdkr d n|js,d nd |j}g}|D]F} t| dksxt| dkrttj | dj dkrd q|| 7}qB r|j D]6} d} | j D]} | |vr| rd qqd} q sqq rpdpgdd dkrfd d Dgd } n^pd p&gdd dkrHfdd Dgd} g|jD]} | 7q^ rd7| dt||}d"fdd d|jd| dd# fdd d$ fdd d% fdd |j |d|dD]} | q0dt|drdt|j}n t|j}t|j}d||d |d!|ddS)&aPrints a summary of a model. Args: model: Keras model instance. line_length: Total length of printed lines (e.g. set this to adapt the display to different terminal window sizes). positions: Relative or absolute positions of log elements in each line. If not provided, defaults to `[.33, .55, .67, 1.]`. print_fn: Print function to use. It will be called on each line of the summary. You can set it to a custom function in order to capture the string summary. It defaults to `print` (prints to stdout). expand_nested: Whether to expand the nested models. If not provided, defaults to `False`. show_trainable: Whether to show if a layer is trainable. If not provided, defaults to `False`. layer_range: List or tuple containing two strings, the starting layer name and ending layer name (both inclusive), indicating the range of layers to be printed in the summary. The strings could also be regexes instead of an exact name. In this case, the starting layer will be the first layer that matches `layer_range[0]` and the ending layer will be the last element that matches `layer_range[1]`. By default (`None`) all layers in the model are included in the summary. N SequentialTrBrFA)g?g333333??csg|]}t|qSrr:r7 line_lengthrr r2r z!print_summary..) Layer (type) Output ShapeParam #b)gQ?g?gq= ףp?rUcsg|]}t|qSrrWr7rXrr r2 r )rZr[r\z Connected to Z Trainablec sRdd|D}t|rNd}tt|D] }|dkrD||d}nd}||}|t|dkrddnd}|||} ||d| d} fdd| D} | rt| } d| |dkr|d |d 7}|7}||rd |nd7}||| d||<|r|d ||t||7}q(|d ||t|7}q(|d |7}|qdS) NcSsg|] }t|qSr)r#rr rrr r2r z4print_summary..print_row..r rrBrA)z),z},z],z',cs,g|]$}|dkr|t|qSrC)findrLr_Z fit_into_linerr r2)s| )anyrangerLrO) fields positions nested_levelZ left_to_printlinecol start_posend_posspacecutoffZline_break_conditionsZcandidate_cutoffs)print_fnrar print_rows8     z print_summary..print_rowz Model: "{}"r=csz |j}Wn*ty d}Ynty4d}Yn0|j}|jj}|js\t|dds\d}n|}|d|d||g}r| |j rdnd ||d S) zPrints a summary for a single layer. Args: layer: target layer. nested_level: level of nesting of the layer inside its parent layer (e.g. 0 for a top-level layer, 1 for a nested layer). multiple?_is_graph_networkFz 0 (unused) ()YNN) output_shapeAttributeError RuntimeErrorrG __class____name__builtgetattrr@r trainable)rrhryrGcls_nameparamsrf)rgrpshow_trainablerr print_layer_summaryEs    z*print_summary..print_layer_summaryc sz |j}Wnty d}Yn0g}|jD]>}r>|vr>q,|D]"\}}}}|d|j||qFq,|j} |jj} | d| d|| |g} r| |j rdnd| |dS)aPrints a summary for a single layer (including topological connections). Args: layer: target layer. nested_level: level of nesting of the layer inside its parent layer (e.g. 0 for a top-level layer, 1 for a nested layer). rrz {}[{}][{}]rurvrwrxN) ryrzrrrformatrGr|r}r@r) rrhry connectionsr inbound_layerr tensor_indexrrGrrf)rgrprelevant_nodesrrr $print_layer_summary_with_connectionscs:     z;print_summary..print_layer_summary_with_connectionscsr||n ||rt|dr|jrƈd|ddd|dd|d|j}d}tt|D],}|t|dkrd}|||d|qtd|dd|d||sd|dd|d|dS) NrMrbrB¯rAFTrc)rrMrerL)rrhZis_nested_lastZ nested_layeri) expand_nestedrYro print_layerrrsequential_likerr rs@    z"print_summary..print_layer_collected_trainable_weightszTotal params: {:,}zTrainable params: {:,}zNon-trainable params: {:,})r)r)r)rF)r print_msgr|r}rt_nodes_by_depthr9rLrrr keras_inputsrMrrrRrrGrr@rtrainable_weightsnon_trainable_weights)rQrYrgrorrrKnodes_by_depthnodesvrflagr to_displaytrainable_countnon_trainable_countr) rrYrgrorrrrprrrr print_summarys$              )   ,!      rchannels_firstc Cs|dvs J|\}}t|jdD]}|dkrj|\}}}|||f} |dd|f| } t| d} n6|\}}}|||f} |dd|f| } t| d} t| t|f|dd|f<q&|||gdS)a]Utility useful when changing a convnet's `data_format`. When porting the weights of a convnet from one data format to the other, if the convnet includes a `Flatten` layer (applied to the last convolutional feature map) followed by a `Dense` layer, the weights of that `Dense` layer should be updated to reflect the new dimension ordering. Args: dense: The target `Dense` layer. previous_feature_map_shape: A shape tuple of 3 integers, e.g. `(512, 7, 7)`. The shape of the convolutional feature map right before the `Flatten` layer that came before the target `Dense` layer. target_data_format: One of "channels_last", "channels_first". Set it "channels_last" if converting a "channels_first" model to "channels_last", or reciprocally. >r channels_lastrBrN)rArrB)rBrAr) get_weightsrer1reshaper5 transposer6 set_weights) denseprevious_feature_map_shapetarget_data_formatkernelbiasrchr/original_fm_shapekirrr !convert_dense_weights_data_formats       "rcCs$t|ddsdS|jdko"|jdkS)N_keras_api_namesF)zkeras.layers.Layer)rr_keras_api_names_v1)rrrr is_builtin_layers   rcs*ttfdd}|_|S)a Lightweight decorator for caching lazily constructed properties. When to use: This decorator provides simple caching with minimal overhead. It is designed for properties which are expensive to compute and static over the life of a class instance, and provides no mechanism for cache invalidation. Thus it is best suited for lazily exposing derived properties of other static data. For classes with custom getattr / setattr behavior (such as trackable objects), storing cache results as object attributes is not performant. Instead, a specialized cache can significantly reduce property lookup overhead. (While still allowing the decorated property to be lazily computed.) Consider the following class: ``` class MyClass: def __setattr__(self, key, value): # Some expensive class specific code # ... # ... super(MyClass, self).__setattr__(key, value) @property def thing(self): # `thing` is expensive to compute (and may not even be requested), so we # want to lazily compute it and then cache it. output = getattr(self, '_thing', None) if output is None: self._thing = output = compute_thing(self) return output ``` It's also worth noting that ANY overriding of __setattr__, even something as simple as: ``` def __setattr__(self, key, value): super(MyClass, self).__setattr__(key, value) ``` Slows down attribute assignment by nearly 10x. By contrast, replacing the definition of `thing` with the following sidesteps the expensive __setattr__ altogether: ''' @property @tracking.cached_per_instance def thing(self): # `thing` is expensive to compute (and may not even be requested), so we # want to lazily compute it and then cache it. return compute_thing(self) ''' Performance: The overhead for this decorator is ~0.4 us / call. A much lower overhead implementation (~0.085 us / call) can be achieved by using a custom dict type: ``` def dict_based_cache(f): class Cache(dict): __slots__ = () def __missing__(self, key): self[key] = output = f(key) return output return property(Cache().__getitem__) ``` However, that implementation holds class instances as keys, and as a result blocks garbage collection. (And modifying it to use weakref's as keys raises the lookup overhead to ~0.4 us) As a result, the WeakKeyDictionary implementation below turns out to be more prudent. Args: f: The function to cache. Returns: f decorated with simple caching behavior. cs&|}|dur"||<}|Sr)get)itemoutputcachefrr wrappedHs z$cached_per_instance..wrapped)weakrefWeakKeyDictionary functoolswrapsr)rrrrr cached_per_instances Srccst}|ddd}|r~|}t||vr.q|t|t|drXt|tsX|Vqt|ddpfg}||dddqdS)z4Filter out empty Layer-like containers and uniquify.NrV _is_layerrM) setpopr-addrr"typerextend) layer_listexistingto_visitobj sub_layersrrr filter_empty_layer_containersSs rc@seZdZdZddZeddZeddZejddZee d d Z ed d Z e jd d Z eddZ e jddZ eddZ dddZdddZdddZddZdS) CallFunctionSpeczKCaches the spec and provides utilities for handling call function args.cCs||_t|jj|_|jr8|jddkr8|jdd|_|j|jjpHg7_|jjdu}d|jvpf||_d|jvpv||_|jjpg}t }t dt |dD]}||||j|<q| |jj pi|d|_dS)zInitialies a `CallFunctionSpec`. Args: full_argspec: the FullArgSpec of a call function of a layer. rselfrBNtrainingmaskrV) _full_argspeclistargs _arg_names kwonlyargsvarkw_expects_training_arg_expects_mask_argdefaultsdictrerLupdatekwonlydefaultsr_default_training_arg)r full_argspecZcall_accepts_kwargscall_fn_defaultsrrrrr __init__ks     zCallFunctionSpec.__init__cCs|jS)z-Returns the FullArgSpec of the call function.)rrrrr rszCallFunctionSpec.full_argspeccCs|jS)z%List of names of args and kwonlyargs.rrrrr arg_namesszCallFunctionSpec.arg_namescCs ||_dSrrrvaluerrr rscCs&t}t|jD]\}}|||<q|S)z:Returns a dict mapping arg names to their index positions.)rrNr)rcall_fn_arg_positionsposargrrr arg_positionss zCallFunctionSpec.arg_positionscCs|jS)z9Whether the call function uses 'training' as a parameter.rrrrr expects_training_argsz%CallFunctionSpec.expects_training_argcCs ||_dSrrrrrr rscCs|jS)z5Whether the call function uses `mask` as a parameter.rrrrr expects_mask_argsz!CallFunctionSpec.expects_mask_argcCs ||_dSrrrrrr rscCs|jS)z3The default value given to the "training" argument.)rrrrr default_training_argsz%CallFunctionSpec.default_training_argFcCs@|s |s dS||vrdS|j}|s.|dd}|tt||vS)aReturns true if argument is present in `args` or `kwargs`. Args: arg_name: String name of the argument to find. args: Tuple of args passed to the call function. kwargs: Dictionary of kwargs passed to the call function. inputs_in_args: Whether the input argument (the first argument in the call function) is included in `args`. Defaults to `False`. Returns: True if argument with `arg_name` is present in `args` or `kwargs`. FTrBNrrzip)rr(rkwargsinputs_in_args call_fn_argsrrr arg_was_passeds zCallFunctionSpec.arg_was_passedcCs<||vr||S|j}|s&|dd}tt||}||S)akRetrieves the value for the argument with name `arg_name`. Args: arg_name: String name of the argument to find. args: Tuple of args passed to the call function. kwargs: Dictionary of kwargs passed to the call function. inputs_in_args: Whether the input argument (the first argument in the call function) is included in `args`. Defaults to `False`. Returns: The value of the argument with name `arg_name`, extracted from `args` or `kwargs`. Raises: KeyError if the value of `arg_name` cannot be found. rBNr)rr(rrrr args_dictrrr get_arg_values zCallFunctionSpec.get_arg_valuecCs|jjrLz*|jj|}|jjddkr0|d8}WqZtyHd}YqZ0n|j|d}|dur|sn|d}t||krt|}|||<t ||fS|dur|r| |dn|||<||fS)aSets the value of an argument into the given args/kwargs. Args: arg_name: String name of the argument to find. new_value: New value to give to the argument. args: Tuple of args passed to the call function. kwargs: Dictionary of kwargs passed to the call function. inputs_in_args: Whether the input argument (the first argument in the call function) is included in `args`. Defaults to `False`. pop_kwarg_if_none: If the new value is `None`, and this is `True`, then the argument is deleted from `kwargs`. Returns: The updated `(args, kwargs)`. rrrBN) rvarargsrindexr$rrrLrtupler)rr( new_valuerrrpop_kwarg_if_nonearg_posrrr set_arg_values&      zCallFunctionSpec.set_arg_valuecCsV|r|d}|dd}n2|jd|vrDt|}||jd}ntd|||fS)z2Splits (args, kwargs) into (inputs, args, kwargs).rrBNz9The first argument to `Layer.call` must always be passed.)rcopyrr$)rrrinputsrrr split_out_first_arg s z$CallFunctionSpec.split_out_first_argN)F)F)FF)r} __module__ __qualname____doc__rpropertyrrsetterrrrrrrrrrrrrr rgs6!            0r)NN)FF)N)NNNFFN)r)rrrrErnumpyr5tensorflow.compat.v2compatv2r keras.utilsr tensorflow.python.util.tf_exportrrr,r@rRrrrrrrrrrr s<   + % 8  ' `