a Sic@sdZddlZddlZddlZddlZddlZddlZddlZddl m m Z ddl mZddl mZddl mZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lm Z!ddl"m#Z#ddl"m$Z$ddl%m&Z'ddl(m)Z)ddZ*ddZ+daddZ,e)dgde'j-dbddZ.e/Z0de0_1de0_2de0_3de0_4e#j5ddZ6e#j5dd Z7d!d"Z8e#j5d#d$Z9d%d&Z:d'd(Z;d)d*ZGd/d0d0ej?Z@Gd1d2d2ej?ZAd3d4ZBd5d6ZCd7d8ZDGd9d:d:ej?ZEGd;d<dZGGd?d@d@ejHZIGdAdBdBej?ZJGdCdDdDej?ZKdedEdFZLejMejNejOejPejQe!jRejSdGZTdHdIZUdfdKdLZVdMdNZWdOdPZXdQdRZYej5dSdTZZej5dUdVZ[dWdXZ\dYdZZ]d[d\Z^dgd]d^Z_d_d`Z`dS)hz!Utilities for unit-testing Keras.N)backend)layers)models)base_layer_utils)adadelta)adagrad)adam)adamax)gradient_descent)nadam)rmsprop) tf_contextlib) tf_inspect) test_util) keras_exportcCstj||dSN)nptestingassert_array_equalactualexpectedrZ/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/testing_infra/test_utils.py string_test3srcCstjj||ddddS)NgMbP?gư>)rtolatol)rrassert_allcloserrrr numeric_test7src Cs|durtj|||}d|tj|f|}tjjd||fd}tj|f|tjd}t|D]&} ||| tjjdd|d|| <qh|d||d|f||d||dffS)aGenerates test data to train a model on. Args: train_samples: Integer, how many training samples to generate. test_samples: Integer, how many test samples to generate. input_shape: Tuple of integers, shape of the inputs. num_classes: Integer, number of classes for the data and targets. random_seed: Integer, random seed used by numpy to generate data. Returns: A tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`. Nr)size)dtypeg?)locscaler )rrandomseedrandintzerosfloat32rangenormal) Z train_samplesZ test_samples input_shape num_classes random_seedZ num_sample templatesyxirrr get_test_data;s  r2z#keras.__internal__.utils.layer_test)v1Tc !sB|dur|durtd|s d}t|} t| D]"\}}|dur0tjdd| |<q0dtj| }|dddkr||d 8}||}n|dur|j}|dur|j}|dur|}t |t j kr| r| j }qt }n| r| j}nt}pifi}| dur&|j| kr&td j|j| f| dur:|| |}||d tjvrt|d <fi}tj|dd|d |}t||krtd jt||ffdd}|dur|t ||jt|}t| t |!}|"t j#||d }|$|}|j}|||||j||j|jkrttdj|j|jf|dur||||%}tj&|| }|j'r|}|||$|}||||}|r,t|}t(j)dur|j*dddgt+dn|j*dddgd|,|||%}||d<|j-&|}| dur\|| t.}|/tj|dd|d |/||||$|}|j}t0||D]6\}} |dur|| krtdj||fq|dur||||%}tj.&|| }|j'r>|}|||$|}||||S)aTest routine for a layer with a single input and single output. Args: layer_cls: Layer class object. kwargs: Optional dictionary of keyword arguments for instantiating the layer. input_shape: Input shape tuple. input_dtype: Data type of the input data. input_data: Numpy array of input data. expected_output: Numpy array of the expected output. expected_output_dtype: Data type expected for the output. expected_output_shape: Shape tuple for the expected shape of the output. validate_training: Whether to attempt to validate training on this layer. This might be set to False for non-differentiable layers that output string or integer values. adapt_data: Optional data for an 'adapt' call. If None, adapt() will not be tested for this layer. This is only relevant for PreprocessingLayers. custom_objects: Optional dictionary mapping name strings to custom objects in the layer class. This is helpful for testing custom layers. test_harness: The Tensorflow test, if any, that this function is being called in. supports_masking: Optional boolean to check the `supports_masking` property of the layer. If None, the check will not be performed. Returns: The output data (Numpy array) returned by the layer, for additional checks to be done by the calling code. Raises: ValueError: if `input_shape is None`. Nzinput_shape is Noner( float?zbWhen testing layer %s, the `supports_masking` property is %rbut expected to be %r. Full kwargs: %sweights)shaper!zcWhen testing layer %s, for input %s, found output dtype=%s but expected to find %s. Full kwargs: %scst|t|kr(tdj||ft||D]\\}}t|tjjjrP|j }t|tjjjrf|j }|dur2||kr2tdj||fq2dS)zNAsserts that the output shape from the layer matches the actual shape.zcWhen testing layer %s, for input %s, found output_shape=%s but expected to find %s. Full kwargs: %sN) lenAssertionError__name__zip isinstancetfcompatr3 Dimensionvalue)rr expected_dim actual_dimkwargs layer_clsr0rrassert_shapes_equals"z'layer_test..assert_shapes_equalzcWhen testing layer %s, for input %s, found output_dtype=%s but expected to find %s. Full kwargs: %sr mseacc)weighted_metrics run_eagerly)rMbatch_input_shapezWhen testing layer %s **after deserialization**, for input %s, found output_shape=%s but expected to find inferred shape %s. Full kwargs: %s)1 ValueErrorlist enumeraterr$r&astyper;r!rAas_dtypestringassertAllEqualrassertAllClosersupports_maskingr=r>adapt get_weights set_weightsr getargspec__init__rInputr TensorShaperModeltuplecompute_output_shapeas_listcompute_output_signature TensorSpecpredict get_config from_configr:_thread_local_datarNcompileshould_run_eagerlytrain_on_batch __class__ Sequentialaddr?)!rIrHr+ input_dtype input_dataZexpected_outputZexpected_output_dtypeZexpected_output_shapeZvalidate_trainingZ adapt_datacustom_objectsZ test_harnessrXZinput_data_shaper1e assert_equallayerr:r/rJmodelZcomputed_output_shapeZcomputed_output_signatureZ actual_outputZactual_output_shape model_configZrecovered_modeloutput layer_weights layer_configrErFrrGr layer_testZs$0                                r{ccs*tj}z|t_|VW|t_n|t_0dS)zProvides a scope within which the model type to test is equal to `value`. The model type gets restored to its original value upon exiting the scope. Args: value: model type value Yields: The provided value. N)ri model_typerDprevious_valuerrrmodel_type_scopeds rccs*tj}z|t_|VW|t_n|t_0dS)a>Provides a scope within which we compile models to run eagerly or not. The boolean gets restored to its original value upon exiting the scope. Args: value: Bool specifying if we should run models eagerly in the active test. Should be True or False. Yields: The provided value. N)rirNr}rrrrun_eagerly_scopeys rcCs tjdurtdtjotS)z@Returns whether the models we are testing should be run eagerly.NziCannot call `should_run_eagerly()` outside of a `run_eagerly_scope()` or `run_all_keras_modes` decorator.)rirNrPrAexecuting_eagerlyrrrrrks  rkcksBtj}tj}z"|t_|t_dVW|t_|t_n|t_|t_0dS)aAProvides a scope within which the savde model format to test is `value`. The saved model format gets restored to its original value upon exiting the scope. Args: value: saved model format value **kwargs: optional kwargs to pass to the save function. Yields: The provided value. N)risaved_model_format save_kwargs)rDrHZprevious_formatZprevious_kwargsrrrsaved_model_format_scopesrcCstjdurtdtjS)NzzCannot call `get_save_format()` outside of a `saved_model_format_scope()` or `run_with_all_saved_model_formats` decorator.)rirrPrrrrget_save_formats  rcCstjdurtdtjpiS)NzzCannot call `get_save_kwargs()` outside of a `saved_model_format_scope()` or `run_with_all_saved_model_formats` decorator.)rirrPrrrrget_save_kwargss  rcCstjdurtdtjS)z*Gets the model type that should be tested.NziCannot call `get_model_type()` outside of a `model_type_scope()` or `run_with_all_model_types` decorator.)rir|rPrrrrget_model_types  rcCs`t}|r$|tj|d|dn|tj|dd|dkrDdnd}|tj||d|S)Nrelu) activation input_dimrr4sigmoidsoftmax)rrnrorDense) num_hiddenr,rrvrrrrget_small_sequential_mlpsrcCsNtj|fd}tj|dd|}|dkr,dnd}tj||d|}t||S)N)r;rrr4rr)rr^rrr`)rr,rinputsoutputsrrrrget_small_functional_mlps rcs*eZdZdZdfdd ZddZZS)SmallSubclassMLPz!A subclass model based small MLP.Fc s|tjfddi|||_||_tj|dd|_|dkr>dnd}tj||d|_|jrdtd|_ |jrxtj d d |_ dS) NnameZ test_modelrrr4rrr9)axis) superr]use_bnuse_dprrlayer_alayer_bDropoutdpBatchNormalizationbn)selfrr,rrrHrrmrrr]s zSmallSubclassMLP.__init__cKs4||}|jr||}|jr*||}||Sr)rrrrrrrrrHr0rrrcalls    zSmallSubclassMLP.call)FFr> __module__ __qualname____doc__r]r __classcell__rrrrrsrcs0eZdZdZfddZddZddZZS)_SmallSubclassMLPCustomBuildz;A subclass model small MLP that uses a custom build method.cs&td|_d|_||_||_dSr)rr]rrrr,)rrr,rrrr]s  z%_SmallSubclassMLPCustomBuild.__init__cCs:tj|jdd|_|jdkr dnd}tj|j|d|_dS)Nrrr4rr)rrrrr,r)rr+rrrrbuildsz"_SmallSubclassMLPCustomBuild.buildcKs||}||Sr)rrrrrrrs z!_SmallSubclassMLPCustomBuild.callr>rrrr]rrrrrrrrs rcCs t||Sr)rrr,rrrget_small_subclass_mlpsrcCs t||Sr)rrrrr(get_small_subclass_mlp_with_custom_buildsrcCsdt}|dkrt||S|dkr*t||S|dkr>t|||S|dkrRt|||Std|dS)z@Get a small mlp of the model type specified by `get_model_type`.subclasssubclass_custom_build sequential functionalUnknown model type {}N)rrrrrrPformat)rr,rr|rrr get_small_mlp!s    rcs8eZdZdZfddZddZddZdd ZZS) _SubclassModelzA Keras subclass model.csb|dd}tj|i|t|D]\}}t||||q&t||_|dur^||dS)aInstantiate a model. Args: model_layers: a list of layers to be added to the model. *args: Model's args **kwargs: Model's keyword args, at most one of input_tensor -> the input tensor required for ragged/sparse input. input_tensorN) poprr]rRsetattr_layer_name_for_ir< num_layers _set_inputs)r model_layersargsrHrr1rurrrr]2s  z_SubclassModel.__init__cCs d|S)Nzlayer{})r)rr1rrrrHsz _SubclassModel._layer_name_for_icKs0|}t|jD]}t|||}||}q|Sr)r)rgetattrr)rrrHr0r1rurrrrKs  z_SubclassModel.callcCstdSr)NotImplementedError)rrrrrgRsz_SubclassModel.get_config) r>rrrr]rrrgrrrrrr/s  rcs0eZdZdZfddZddZddZZS)_SubclassModelCustomBuildz7A Keras subclass model that uses a custom build method.cs"tj|i|d|_||_dSr)rr] all_layers_layer_generating_func)rlayer_generating_funcrrHrrrr][sz"_SubclassModelCustomBuild.__init__cCs&g}|D]}||q ||_dSr)rappendr)rr+rrurrrr`s  z_SubclassModelCustomBuild.buildcKs|}|jD] }||}q |Sr)r)rrrHr0rurrrrfs  z_SubclassModelCustomBuild.callrrrrrrXs rc s|durt}|dkrBd}|s"|r4tj||||d}t||dS|dkrbfdd}t||dS|d krtj|d} |r| tj||||d D]} | | q| S|d kr|st d tj||||d}|} D] } | | } qtj || |dSt d |dS)aEBuilds a model from a sequence of layers. Args: model_layers: The layers used to build the network. input_shape: Shape tuple of the input or 'TensorShape' instance. input_dtype: Datatype of the input. name: Name for the model. input_ragged: Boolean, whether the input data is a ragged tensor. input_sparse: Boolean, whether the input data is a sparse tensor. model_type: One of "subclass", "subclass_custom_build", "sequential", or "functional". When None, defaults to `get_model_type`. Returns: A Keras model. Nr)r;r!raggedsparse)rrrcsSrrrrrrz'get_model_from_layers..rr)r+r!rrrzACannot create a functional model from layers with no input shape.r) rrr^rrrrnro InputLayerrPr`r) rr+rprZ input_raggedZ input_sparser|rrrvrurrrrget_model_from_layersmsZ     rc@seZdZddZddZdS)BiascCs|jdddd|_dS)Nbias)r4r') initializer) add_weightrrr+rrrrsz Bias.buildcCs ||jSr)r)rrrrrrsz Bias.callN)r>rrrrrrrrrsrcs*eZdZdZdfdd ZddZZS)_MultiIOSubclassModelzMulti IO Keras subclass model.Ncs*tj|d||_||_||_||_dS)Nr)rr]_shared_input_branch _branch_a _branch_b_shared_output_branch)rbranch_abranch_bshared_input_branchshared_output_branchrrrrr]s z_MultiIOSubclassModel.__init__cKs|jr$|jD] }||}q |}|}n$t|tr@|d}|d}n|\}}|jD] }||}qN|jD] }||}qb||g}|jr|jD] }||}q|S)NZinput_1Zinput_2)rr@dictrrrrrrHruaboutsrrrrs$          z_MultiIOSubclassModel.call)NNNrrrrrrs rcs2eZdZdZd fdd ZddZddZZS) _MultiIOSubclassModelCustomBuildz>Multi IO Keras subclass model that uses a custom build method.Ncs>t||_||_||_||_d|_d|_d|_d|_ dSr) rr]_shared_input_branch_func_branch_a_func_branch_b_func_shared_output_branch_funcrrrr)rZ branch_a_funcZ branch_b_funcZshared_input_branch_funcZshared_output_branch_funcrrrr]s z)_MultiIOSubclassModelCustomBuild.__init__cCs<|r||_||_||_|r8||_dSr)rrrrrrrrrrrrrs    z&_MultiIOSubclassModelCustomBuild.buildcKsz|jr$|jD] }||}q |}|}n|\}}|jD] }||}q2|jD] }||}qF||f}|jrv|jD] }||}qh|Sr)rrrrrrrrr s        z%_MultiIOSubclassModelCustomBuild.call)NNrrrrrrs  rc s>rd}ddn(ddf}ddddt}|dkr^tS|dkrtfddfddfd dfd dS|d krtd |d kr,r|}D] }||}q|}|} n|\}} D] }||}qވD] }|| } q|| f} r D]}|| } qt|| Std|dS)a] Builds a multi-io model that contains two branches. The produced model will be of the type specified by `get_model_type`. To build a two-input, two-output model: Specify a list of layers for branch a and branch b, but do not specify any shared input branch or shared output branch. The resulting model will apply each branch to a different input, to produce two outputs. The first value in branch_a must be the Keras 'Input' layer for branch a, and the first value in branch_b must be the Keras 'Input' layer for branch b. example usage: ``` branch_a = [Input(shape=(2,), name='a'), Dense(), Dense()] branch_b = [Input(shape=(3,), name='b'), Dense(), Dense()] model = get_multi_io_model(branch_a, branch_b) ``` To build a two-input, one-output model: Specify a list of layers for branch a and branch b, and specify a shared output branch. The resulting model will apply each branch to a different input. It will then apply the shared output branch to a tuple containing the intermediate outputs of each branch, to produce a single output. The first layer in the shared_output_branch must be able to merge a tuple of two tensors. The first value in branch_a must be the Keras 'Input' layer for branch a, and the first value in branch_b must be the Keras 'Input' layer for branch b. example usage: ``` input_branch_a = [Input(shape=(2,), name='a'), Dense(), Dense()] input_branch_b = [Input(shape=(3,), name='b'), Dense(), Dense()] shared_output_branch = [Concatenate(), Dense(), Dense()] model = get_multi_io_model(input_branch_a, input_branch_b, shared_output_branch=shared_output_branch) ``` To build a one-input, two-output model: Specify a list of layers for branch a and branch b, and specify a shared input branch. The resulting model will take one input, and apply the shared input branch to it. It will then respectively apply each branch to that intermediate result in parallel, to produce two outputs. The first value in the shared_input_branch must be the Keras 'Input' layer for the whole model. Branch a and branch b should not contain any Input layers. example usage: ``` shared_input_branch = [Input(shape=(2,), name='in'), Dense(), Dense()] output_branch_a = [Dense(), Dense()] output_branch_b = [Dense(), Dense()] model = get_multi_io_model(output__branch_a, output_branch_b, shared_input_branch=shared_input_branch) ``` Args: branch_a: A sequence of layers for branch a of the model. branch_b: A sequence of layers for branch b of the model. shared_input_branch: An optional sequence of layers to apply to a single input, before applying both branches to that intermediate result. If set, the model will take only one input instead of two. Defaults to None. shared_output_branch: An optional sequence of layers to merge the intermediate results produced by branch a and branch b. If set, the model will produce only one output instead of two. Defaults to None. Returns: A multi-io model of the type specified by `get_model_type`, specified by the different branches. rr4NrrcsSrrr)rrrrrz$get_multi_io_model..csSrrr)rrrrrcsSrrr)rrrrrcsSrrr)rrrrrrz>Cannot use `get_multi_io_model` to construct sequential modelsrr)rrrrPrr`r) rrrrrr|Za_and_brurrrr)rrrrrget_multi_io_modelsPR            r)rrrr r r sgdc KsDzt|fi|WSty>td|ttYn0dS)aGet the v2 optimizer requested. This is only necessary until v2 are the default, as we are testing in Eager, and Eager + v1 optimizers fail tests. When we are in v2, the strings alone should be sufficient, and this mapping can theoretically be removed. Args: name: string name of Keras v2 optimizer. **kwargs: any kwargs to pass to the optimizer constructor. Returns: Initialized Keras v2 optimizer. Raises: ValueError: if an unknown name was passed. z;Could not find requested v2 optimizer: {} Valid choices: {}N)_V2_OPTIMIZER_MAPKeyErrorrPrrQkeys)rrHrrrget_v2_optimizers  rcs4tjjstr"dd|DSfdd|DS)zEReturns expected metric variable names given names and prefix/suffix.cSsg|] }|dqSz:0r.0nrrr rz6get_expected_metric_variable_names..csg|]}|dqSrrr name_suffixrrrr)rA __internal__tf2enabledr) var_namesrrrr"get_expected_metric_variable_namessrcCs t|dS)z=Decorator for enabling the layer V2 dtype behavior on a test.T_set_v2_dtype_behaviorfnrrrenable_v2_dtype_behaviorsrcCs t|dS)z>Decorator for disabling the layer V2 dtype behavior on a test.Frrrrrdisable_v2_dtype_behaviorsrcs(tfdd}tjj|S)zCReturns version of 'fn' that runs with v2 dtype behavior on or off.cs0tj}t_z|i|W|t_S|t_0dSr)rV2_DTYPE_BEHAVIOR)rrHZv2_dtype_behaviorrrrrwrappersz'_set_v2_dtype_behavior..wrapper) functoolswrapsrAr decoratormake_decorator)rrrrrrrsrccsL|rtjrd}nd}t|dVWdn1s>0YdS)&Uses gpu when requested and available.z /device:GPU:0z /device:CPU:0N)rAtestis_gpu_availabledevice)should_use_gpudevrrrr s  r ccs4tdddVWdn1s&0YdS)rT)r N)r rrrruse_gpus r csfdd}|S)aGenerate class-level decorator from given method-level decorator. It is expected for the given decorator to take some arguments and return a method that is then called on the test method to produce a decorated method. Args: decorator: The decorator to apply. *args: Positional arguments **kwargs: Keyword arguments Returns: Function that will decorate a given classes test methods with the decorator. csPt|D]B}t||}t|r|dr|dkrt||i|q|S)z-Apply decorator to all test methods in class.r test_session)dirrcallable startswithr)clsrrDrrrHrrall_test_methods_impl s  z3for_all_test_methods..all_test_methods_implr)rrrHrrrrfor_all_test_methodss rcCs dd}|S)aExecute test with TensorFloat-32 disabled. While almost every real-world deep learning model runs fine with TensorFloat-32, many tests use assertAllClose or similar methods. TensorFloat-32 matmuls typically will cause such methods to fail with the default tolerances. Args: description: A description used for documentation purposes, describing why the test requires TensorFloat-32 to be disabled. Returns: Decorator which runs a test with TensorFloat-32 disabled. cstfdd}|S)Nc sXtjj}z6tjjd|g|Ri|Wtjj|ntjj|0dS)NF)rAconfig experimental!tensor_float_32_execution_enabled enable_tensor_float_32_execution)rrrHallowedfrr decorated.s  zArun_without_tensor_float_32..decorator..decorated)rrrrrrrr-sz.run_without_tensor_float_32..decoratorr) descriptionrrrrrun_without_tensor_float_32s r cCs tt|S)z:Execute all tests in a class with TensorFloat-32 disabled.)rr )rrrrrun_all_without_tensor_float_32=sr!cs6tjj dfdd}dur2|S|S)aqExecute the decorated test only if running in v2 mode. This function is intended to be applied to tests that exercise v2 only functionality. If the test is run in v1 mode it will simply be skipped. See go/tf-test-decorator-cheatsheet for the decorators to use in different v1/v2/eager/graph combinations. Args: obj: function to be annotated. If None, return a decorator the can be applied to a function or class. If `obj` is not None, return the decorator applied to `obj`. Returns: Returns a decorator that will conditionally skip the decorated test method. z#Test is only compatible with TF v2.cs0trtjdSfdd}|S)N) conditionreasoncs$r||g|Ri|Sr)skipTest)rrrH)r"rr#rrr]s z1run_v2_only..decorator..decorated)risclassunittestskipIfrr"objr#rrrYs zrun_v2_only..decoratorN)rArrr)r)rrr(r run_v2_onlyDs  r*c sdd}g}t||dD]0\}t|ts4|g}|fdd|Dqddtj|D}g}|D]T}t|tjs~Jd dd|D}|tt|d d |fgqj|S) aKGenerate combinations based on its keyword arguments using combine(). This function calls combine() and appends a testcase name to the list of dictionaries returned. The 'testcase_name' key is a required for named parameterized tests. Args: **kwargs: keyword arguments of form `option=[possibilities, ...]` or `option=the_only_possibility`. Returns: a list of dictionaries for each combination. Keys in the dictionaries are the keyword argument names. Each key has one value - one of the corresponding keyword argument values. cSs|dS)Nrr)krrrrzrz:generate_combinations_with_testcase_name..keycsg|] }|fqSrr)rrDr,rrrrz.cSsg|]}t|qSr) collections OrderedDict)rresultrrrrsrc Ss<g|]4\}}ddttj|dttjt|qS)z_{}_{}r)rjoinfilterstrisalnum)rr-rDrrrrs  testcase_namez_test{}) sorteditemsr@rQr itertoolsproductr.r/r1r)rH sort_by_key combinationsvaluesnamed_combinations combinationrrr,r(generate_combinations_with_testcase_namejs2   r?)N) NNNNNNNTNNNN)N)NNNNNN)NN)r)N)arr. contextlibrr8 threadingr&numpyrZtensorflow.compat.v2rBv2rAkerasrrr keras.enginerkeras.optimizers.optimizer_v2r adadelta_v2r adagrad_v2radam_v2r adamax_v2r gradient_descent_v2r nadam_v2r rmsprop_v2 keras.utilsr rtensorflow.python.frameworkrZ tf_test_utils tensorflow.python.util.tf_exportrrrr2disable_cudnn_autotuner{localrir|rNrrcontextmanagerrrrkrrrrrrr`rrrrrrrrLayerrrrrAdadeltaAdagradAdamAdamaxNadamRMSpropSGDrrrrrrr r rr r!r*r?rrrrs                        ) L*5       &