a Sic{@s.dZddlmZddlmZddlmZddlZddlZddlmm Z ddl m Z ddl m Z ddl mZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZdZdZddZd=ddZ ddZ!edgdedgdGdddej"Z#Gddde#Z$ed gded!gdGd"d#d#e$Z%ed$gded%gdGd&d'd'e$Z&e'd(d)Z(ed*gded+gdGd,d(d(e(Z)ed-gded.gdGd/d0d0e$Z*ed1gded2gdGd3d4d4e$Z+ed5gded6gdGd7d8d8e#Z,d9d:Z-d;d<Z.dS)>a}Legacy module implementing RNN Cells. This module provides a number of basic commonly used RNN cells, such as LSTM (Long Short Term Memory) or GRU (Gated Recurrent Unit), and a number of operators that allow adding dropouts, projections, or embeddings for inputs. Constructing multi-layer cells is supported by the class `MultiRNNCell`, or by calling the `rnn` ops several times. )absolute_import)division)print_functionN) activations)backend) initializers)base_layer_utils) input_spec)base)tf_utils) tf_logging) keras_export) tf_exportbiaskernelcCs,zt||Wnty"YdS0dSdS)NFT)getattrAttributeError)obj attr_namerY/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/layers/rnn/legacy_cells.py_hasattr3s  rFcCst|tjrT|}t|}|jjdkr8tjj|d}q|jjdkrt d|n@t |}|jdurp| nd}| rtj | tjdnd}t|tjr|}t|}|jjdkrtjj|d}n|jjdkrt d|nDt |}|jdur| nd}| r(tj | tjdnd}|r\t ||}|jdurV| nd}n8|dusp|durt d|d|t||fd}|S) a#Concat that enables int, Tensor, or TensorShape values. This function takes a size specification, which can be an integer, a TensorShape, or a Tensor, and converts it into a concatenated Tensor (if static = False) or a list of integers (if static = True). Args: prefix: The prefix; usually the batch size (and/or time step size). (TensorShape, int, or Tensor.) suffix: TensorShape, int, or Tensor. static: If `True`, return a python list with possibly unknown dimensions. Otherwise return a `Tensor`. Returns: shape: the concatenation of prefix and suffix. Raises: ValueError: if `suffix` is not a scalar or vector (or TensorShape). ValueError: if prefix or suffix was `None` and asked for dynamic Tensors out. rzFPrefix tensor must be either a scalar or vector, but received tensor: NdtypezFsuffix tensor must be either a scalar or vector, but received tensor: z2Prefix or suffix can't be None. Received prefix = z and suffix = ) isinstancetfTensorget_static_valueshapendimscompatv1 expand_dims ValueError TensorShapeas_listis_fully_definedconstantint32 concatenateconcat)prefixsuffixstaticpp_staticss_staticrrrr_concat<s^          r3csfdd}tj||S)zCCreate tensors of zeros based on state_size, batch_size, and dtype.cs<t|}tj|d}ts8t|dd}|||S)z7Combine s with batch_size to get a proper tensor shape.rT)r.)r3rzerosexecuting_eagerly set_shape)r1csizec_static batch_sizerrrget_state_shapes   z,_zero_state_tensors..get_state_shape)rnest map_structure) state_sizer;rr<rr:r_zero_state_tensorss r@z*keras.__internal__.legacy.rnn_cell.RNNCell)r"znn.rnn_cell.RNNCellcseZdZdZdfdd Zdfdd Zdd Zed d Zed d Z ddZ dddZ ddZ fddZ eddZZS)RNNCellacAbstract object representing an RNN cell. Every `RNNCell` must have the properties below and implement `call` with the signature `(output, next_state) = call(input, state)`. The optional third input argument, `scope`, is allowed for backwards compatibility purposes; but should be left off for new subclasses. This definition of cell differs from the definition used in the literature. In the literature, 'cell' refers to an object with a single scalar output. This definition refers to a horizontal array of such units. An RNN cell, in the most abstract setting, is anything that has a state and performs some operation that takes a matrix of inputs. This operation results in an output matrix with `self.output_size` columns. If `self.state_size` is an integer, this operation also results in a new state matrix with `self.state_size` columns. If `self.state_size` is a (possibly nested tuple of) TensorShape object(s), then it should return a matching structure of Tensors having shape `[batch_size].concatenate(s)` for each `s` in `self.batch_size`. TNc s$tjf|||d|d|_dS)N) trainablenamerT)super__init___is_tf_rnn_cell)selfrBrCrkwargs __class__rrrEszRNNCell.__init__cs|durPtjjj||jd"}tj|||dWdS1sD0Ynrd}t||d}|durtjjjtjj|jd}t ||||t||WdS1s0YdS)aRun this RNN cell on inputs, starting from the given state. Args: inputs: `2-D` tensor with shape `[batch_size, input_size]`. state: if `self.state_size` is an integer, this should be a `2-D Tensor` with shape `[batch_size, self.state_size]`. Otherwise, if `self.state_size` is a tuple of integers, this should be a tuple with shapes `[batch_size, s] for s in self.state_size`. scope: VariableScope for the created subgraph; defaults to class name. Returns: A pair containing: - Output: A `2-D` tensor with shape `[batch_size, self.output_size]`. - New state: Either a single `2-D` tensor, or a tuple of tensors matching the arity and shapes of `state`. N) custom_getter)scope rnncell_scope) rr!r"variable_scope_rnn_get_variablerD__call__rget_variable_scopesetattr)rGinputsstaterLscope_attrnamerIrrrPs 2   zRNNCell.__call__cs||i|tjjr"j}n2tjjvpRtoRtdtjjv}|r~t fdd|j Dr~|j n(|st fdd|j Dr|j S)Nrc3s|]}|uVqdSNr.0vvariablerr sz,RNNCell._rnn_get_variable..c3s|]}|uVqdSrVrrWrZrrr\s) rr!r"#executing_eagerly_outside_functionsrBtrainable_variablesris_split_variablelistall_trainable_weightsappend_non_trainable_weights)rGgetterargsrHrBrrZrrOs    zRNNCell._rnn_get_variablecCs tddS)zsize(s) of state(s) used by this cell. It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes. Abstract methodNNotImplementedErrorrGrrrr?szRNNCell.state_sizecCs tddS)z>Integer or TensorShape: size of outputs produced by this cell.rgNrhrjrrr output_sizeszRNNCell.output_sizecCsdSrVr)rG_rrrbuildsz RNNCell.buildcCs|durtj|dd}|durlt|r8tj|dd}n|}|jjdj|krltd|jjdjd||dur|j|krtd|jd ||jjdjptj j |d}|j}|dus|durtd |d || ||S) NrSrCT)partialrzTbatch size from input tensor is different from the input param. Input tensor batch: z, batch_size: zOdtype from input tensor is different from the input param. Input tensor dtype: z , dtype: zQbatch_size and dtype cannot be None while constructing initial state: batch_size=z, dtype=) rconvert_to_tensor is_tensorrrdimsvaluer$rr!r" zero_state)rGrSr;rstatic_batch_sizerrrget_initial_statesH  zRNNCell.get_initial_statec Cs|j}t}|rJt|drJt|d\}}}}||krJ||krJ||krJ|Stt|jdt |||} Wdn1s0Y|r|||| f|_ | S)aNReturn zero-filled state tensor(s). Args: batch_size: int, float, or unit Tensor representing the batch size. dtype: the data type to use for the state. Returns: If `state_size` is an int or TensorShape, then the return value is a `N-D` tensor of shape `[batch_size, state_size]` filled with zeros. If `state_size` is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of `2-D` tensors with the shapes `[batch_size, s]` for each s in `state_size`. _last_zero_state ZeroStateN) r?rr5rrr name_scopetype__name__r@rw) rGr;rr?is_eagerlast_state_sizelast_batch_size last_dtype last_outputoutputrrrrt#s**zRNNCell.zero_statecs tSrV)rD get_configrjrIrrrJszRNNCell.get_configcCsdS)NFrrjrrr!_use_input_spec_as_call_signatureMsz)RNNCell._use_input_spec_as_call_signature)TNN)N)NNN)r{ __module__ __qualname____doc__rErPrOpropertyr?rkrmrvrtrr __classcell__rrrIrrAs #   &' rAc@seZdZdZdddZdS) LayerRNNCellaSubclass of RNNCells that act like proper `tf.Layer` objects. For backwards compatibility purposes, most `RNNCell` instances allow their `call` methods to instantiate variables via `tf.compat.v1.get_variable`. The underlying variable scope thus keeps track of any variables, and returning cached versions. This is atypical of `tf.layer` objects, which separate this part of layer building into a `build` method that is only called once. Here we provide a subclass for `RNNCell` objects that act exactly as `Layer` objects do. They must provide a `build` method and their `call` methods do not access Variables `tf.compat.v1.get_variable`. NcOs"tjj|||g|Rd|i|S)aORun this RNN cell on inputs, starting from the given state. Args: inputs: `2-D` tensor with shape `[batch_size, input_size]`. state: if `self.state_size` is an integer, this should be a `2-D Tensor` with shape `[batch_size, self.state_size]`. Otherwise, if `self.state_size` is a tuple of integers, this should be a tuple with shapes `[batch_size, s] for s in self.state_size`. scope: optional cell scope. *args: Additional positional arguments. **kwargs: Additional keyword arguments. Returns: A pair containing: - Output: A `2-D` tensor with shape `[batch_size, self.output_size]`. - New state: Either a single `2-D` tensor, or a tuple of tensors matching the arity and shapes of `state`. rL) base_layerLayerrP)rGrSrTrLrfrHrrrrPdszLayerRNNCell.__call__)N)r{rrrrPrrrrrUsrz/keras.__internal__.legacy.rnn_cell.BasicRNNCellznn.rnn_cell.BasicRNNCellcs\eZdZdZdfdd ZeddZeddZej d d Z d d Z fd dZ Z S) BasicRNNCellaThe most basic RNN cell. Note that this cell is not optimized for performance. Please use `tf.contrib.cudnn_rnn.CudnnRNNTanh` for better performance on GPU. Args: num_units: int, The number of units in the RNN cell. activation: Nonlinearity to use. Default: `tanh`. It could also be string that is within Keras activation function names. reuse: (optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised. name: String, the name of the layer. Layers with the same name will share weights, but to avoid mistakes we require reuse=True in such cases. dtype: Default dtype of the layer (default of `None` means use the type of the first input). Required when `build` is called before `call`. **kwargs: Dict, keyword named properties for common layer attributes, like `trainable` etc when constructing the cell from configs of get_config(). Nc stjdddtjf|||d|t|jtrRtj drRt d|t j dd|_ ||_|rxt||_ntj|_dS)Nz`tf.nn.rnn_cell.BasicRNNCell` is deprecated and will be removed in a future version. This class is equivalent as `tf.keras.layers.SimpleRNNCell`, and will be replaced by that in Tensorflow 2.0. stacklevel_reuserCrGPUz%s: Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnRNNTanh for better performance on GPU.ndim)warningswarnrDrE_check_supported_dtypesrrr5configlist_logical_devicesloggingwarningr InputSpec _num_unitsrget _activationtanh)rG num_units activationreuserCrrHrIrrrEs  zBasicRNNCell.__init__cCs|jSrVrrjrrrr?szBasicRNNCell.state_sizecCs|jSrVrrjrrrrkszBasicRNNCell.output_sizecCsv|ddurtd|t|j|d}|jt||j|jgd|_|jt|jgtj j j |jdd|_ d|_ dS)N7Expected inputs.shape[-1] to be known, received shape: rrr initializerT)r$rr add_weight_WEIGHTS_VARIABLE_NAMEr_kernel_BIAS_VARIABLE_NAMErr!r"zeros_initializer_biasbuiltrG inputs_shape input_depthrrrrms$  zBasicRNNCell.buildcCsHt||gtt||gd|j}tj||j}||}||fS)zLMost basic RNN: output = new_state = act(W * input + U * state + B).r) _check_rnn_cell_input_dtypesrmatmulr+rnnbias_addrr)rGrSrT gate_inputsrrrrcalls   zBasicRNNCell.callcs>|jt|j|jd}t}tt| t| S)N)rrr) rr serializerrrDrdictr`itemsrGr base_configrIrrrs   zBasicRNNCell.get_config)NNNNr{rrrrErr?rkr shape_type_conversionrmrrrrrrIrrs#    rz*keras.__internal__.legacy.rnn_cell.GRUCellznn.rnn_cell.GRUCellcs\eZdZdZdfdd ZeddZeddZej d d Z d d Z fd dZ Z S)GRUCellaGated Recurrent Unit cell. Note that this cell is not optimized for performance. Please use `tf.contrib.cudnn_rnn.CudnnGRU` for better performance on GPU, or `tf.contrib.rnn.GRUBlockCellV2` for better performance on CPU. Args: num_units: int, The number of units in the GRU cell. activation: Nonlinearity to use. Default: `tanh`. reuse: (optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised. kernel_initializer: (optional) The initializer to use for the weight and projection matrices. bias_initializer: (optional) The initializer to use for the bias. name: String, the name of the layer. Layers with the same name will share weights, but to avoid mistakes we require reuse=True in such cases. dtype: Default dtype of the layer (default of `None` means use the type of the first input). Required when `build` is called before `call`. **kwargs: Dict, keyword named properties for common layer attributes, like `trainable` etc when constructing the cell from configs of get_config(). References: Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation: [Cho et al., 2014] (https://aclanthology.coli.uni-saarland.de/papers/D14-1179/d14-1179) ([pdf](http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf)) Nc  stjdddtjf|||d|t|jtrRtj drRt d|t j dd|_ ||_|rxt||_ntj|_t||_t||_dS)Nz`tf.nn.rnn_cell.GRUCell` is deprecated and will be removed in a future version. This class is equivalent as `tf.keras.layers.GRUCell`, and will be replaced by that in Tensorflow 2.0.rrrrz%s: Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnGRU for better performance on GPU.r)rrrDrErrrr5rrrrr rrrrrrr_kernel_initializer_bias_initializer) rGrrrkernel_initializerbias_initializerrCrrHrIrrrE s$   zGRUCell.__init__cCs|jSrVrrjrrrr?0szGRUCell.state_sizecCs|jSrVrrjrrrrk4szGRUCell.output_sizecCs|ddurtd|t|j|d}|jdt||jd|jg|jd|_|jdtd|jg|j durx|j nt j j j d|jdd|_|jdt||j|jg|jd|_|jdt|jg|j dur|j nt j j j|jdd|_d |_dS) Nrrzgates/%srr?rz candidate/%sT)r$rrrrrr _gate_kernelrrrr!r"constant_initializer _gate_bias_candidate_kernelr_candidate_biasrrrrrrm8sB     z GRUCell.buildc Cst||gtt||gd|j}tj||j}t|}tj |ddd\}}||}tt||gd|j }tj||j }| |} ||d|| } | | fS)z-Gated recurrent unit (GRU) with nunits cells.rrrsnum_or_size_splitsaxis) rrrr+rrrrsigmoidsplitrrr) rGrSrTrrsrur_state candidater7new_hrrrr`s   z GRUCell.callcsR|jt|jt|jt|j|jd}t }t t | t | S)N)rrrrr) rrrrrrrrrDrrr`rrrIrrrws   zGRUCell.get_config)NNNNNNrrrrIrrs '   'rLSTMStateTuple)r7hz1keras.__internal__.legacy.rnn_cell.LSTMStateTupleznn.rnn_cell.LSTMStateTuplec@s eZdZdZdZeddZdS)rzTuple used by LSTM Cells for `state_size`, `zero_state`, and output state. Stores two elements: `(c, h)`, in that order. Where `c` is the hidden state and `h` is the output. Only used when `state_is_tuple=True`. rcCs2|\}}|j|jkr,td|jd|j|jS)Nz(Inconsistent dtypes for internal state: z vs )r TypeError)rGr7rrrrrs zLSTMStateTuple.dtypeN)r{rrr __slots__rrrrrrrsz0keras.__internal__.legacy.rnn_cell.BasicLSTMCellznn.rnn_cell.BasicLSTMCellcs\eZdZdZdfdd ZeddZed d Zej d d Z d dZ fddZ Z S) BasicLSTMCella DEPRECATED: Please use `tf.compat.v1.nn.rnn_cell.LSTMCell` instead. Basic LSTM recurrent network cell. The implementation is based on We add forget_bias (default: 1) to the biases of the forget gate in order to reduce the scale of forgetting in the beginning of the training. It does not allow cell clipping, a projection layer, and does not use peep-hole connections: it is the basic baseline. For advanced models, please use the full `tf.compat.v1.nn.rnn_cell.LSTMCell` that follows. Note that this cell is not optimized for performance. Please use `tf.contrib.cudnn_rnn.CudnnLSTM` for better performance on GPU, or `tf.contrib.rnn.LSTMBlockCell` and `tf.contrib.rnn.LSTMBlockFusedCell` for better performance on CPU. rTNc  stjdddtjf|||d|t|j|sBtd|t rbtj drbtd|t j dd|_ ||_||_||_|rt||_ntj|_d S) aInitialize the basic LSTM cell. Args: num_units: int, The number of units in the LSTM cell. forget_bias: float, The bias added to forget gates (see above). Must set to `0.0` manually when restoring from CudnnLSTM-trained checkpoints. state_is_tuple: If True, accepted and returned states are 2-tuples of the `c_state` and `m_state`. If False, they are concatenated along the column axis. The latter behavior will soon be deprecated. activation: Activation function of the inner states. Default: `tanh`. It could also be string that is within Keras activation function names. reuse: (optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised. name: String, the name of the layer. Layers with the same name will share weights, but to avoid mistakes we require reuse=True in such cases. dtype: Default dtype of the layer (default of `None` means use the type of the first input). Required when `build` is called before `call`. **kwargs: Dict, keyword named properties for common layer attributes, like `trainable` etc when constructing the cell from configs of get_config(). When restoring from CudnnLSTM-trained checkpoints, must use `CudnnCompatibleLSTMCell` instead. z`tf.nn.rnn_cell.BasicLSTMCell` is deprecated and will be removed in a future version. This class is equivalent as `tf.keras.layers.LSTMCell`, and will be replaced by that in Tensorflow 2.0.rrr_%s: Using a concatenated state is slower and will soon be deprecated. Use state_is_tuple=True.r%s: Note that this cell is not optimized for performance. Please use tf.contrib.cudnn_rnn.CudnnLSTM for better performance on GPU.rN)rrrDrErrrrrr5rrr rr _forget_bias_state_is_tuplerrrr) rGr forget_biasstate_is_tuplerrrCrrHrIrrrEs.& zBasicLSTMCell.__init__cCs|jrt|j|jSd|jS)Nr)rrrrjrrrr?szBasicLSTMCell.state_sizecCs|jSrVrrjrrrrkszBasicLSTMCell.output_sizecCs|ddurtd|t|j|d}|j}|jt||d|jgd|_|jtd|jgtj j j |jdd|_ d|_ dS)NrrrrrT)r$rrrrrrrrr!r"rrr)rGrrh_depthrrrrm s&   zBasicLSTMCell.buildcCst||gtj}tjdtjd}|jr2|\}}ntj|d|d\}}tt||gd|j }tj ||j }tj|d|d\}} } } tj|j | jd} tj} tj}| |||| | | ||||| }||||| }|jrt||}nt||gd}||fS)a"Long short-term memory cell (LSTM). Args: inputs: `2-D` tensor with shape `[batch_size, input_size]`. state: An `LSTMStateTuple` of state tensors, each shaped `[batch_size, num_units]`, if `state_is_tuple` has been set to `True`. Otherwise, a `Tensor` shaped `[batch_size, 2 * num_units]`. Returns: A pair containing the new hidden state, and the new state (either a `LSTMStateTuple` or a concatenated state, depending on `state_is_tuple`). rrrrr)rrrr(r)rrrr+rrrrrraddmultiplyrr)rGrSrTroner7rrijfoforget_bias_tensorrrnew_cr new_staterrrr"s*   zBasicLSTMCell.callcsF|j|j|jt|j|jd}t}t t | t | S)N)rrrrr) rrrrrrrrDrrr`rrrIrrrQs  zBasicLSTMCell.get_config)rTNNNNrrrrIrrs H   /rz+keras.__internal__.legacy.rnn_cell.LSTMCellznn.rnn_cell.LSTMCellc s\eZdZdZdfdd Zedd Zed d Zej d d Z ddZ fddZ Z S)LSTMCellaLong short-term memory unit (LSTM) recurrent network cell. The default non-peephole implementation is based on (Gers et al., 1999). The peephole implementation is based on (Sak et al., 2014). The class uses optional peep-hole connections, optional cell clipping, and an optional projection layer. Note that this cell is not optimized for performance. Please use `tf.contrib.cudnn_rnn.CudnnLSTM` for better performance on GPU, or `tf.contrib.rnn.LSTMBlockCell` and `tf.contrib.rnn.LSTMBlockFusedCell` for better performance on CPU. References: Long short-term memory recurrent neural network architectures for large scale acoustic modeling: [Sak et al., 2014] (https://www.isca-speech.org/archive/interspeech_2014/i14_0338.html) ([pdf] (https://www.isca-speech.org/archive/archive_papers/interspeech_2014/i14_0338.pdf)) Learning to forget: [Gers et al., 1999] (http://digital-library.theiet.org/content/conferences/10.1049/cp_19991218) ([pdf](https://arxiv.org/pdf/1409.2329.pdf)) Long Short-Term Memory: [Hochreiter et al., 1997] (https://www.mitpressjournals.org/doi/abs/10.1162/neco.1997.9.8.1735) ([pdf](http://ml.jku.at/publications/older/3504.pdf)) FNrTc s2tjdddtjf| | |d|t|j| sBtd||dusR|dur^td|t r~tj dr~td |t j dd |_ ||_||_||_t||_||_||_||_||_| |_| |_| rt| |_ntj|_|r| rt||n|||_||_n | rt||nd||_||_dS) a, Initialize the parameters for an LSTM cell. Args: num_units: int, The number of units in the LSTM cell. use_peepholes: bool, set True to enable diagonal/peephole connections. cell_clip: (optional) A float value, if provided the cell state is clipped by this value prior to the cell output activation. initializer: (optional) The initializer to use for the weight and projection matrices. num_proj: (optional) int, The output dimensionality for the projection matrices. If None, no projection is performed. proj_clip: (optional) A float value. If `num_proj > 0` and `proj_clip` is provided, then the projected values are clipped elementwise to within `[-proj_clip, proj_clip]`. num_unit_shards: Deprecated, will be removed by Jan. 2017. Use a variable_scope partitioner instead. num_proj_shards: Deprecated, will be removed by Jan. 2017. Use a variable_scope partitioner instead. forget_bias: Biases of the forget gate are initialized by default to 1 in order to reduce the scale of forgetting at the beginning of the training. Must set it manually to `0.0` when restoring from CudnnLSTM trained checkpoints. state_is_tuple: If True, accepted and returned states are 2-tuples of the `c_state` and `m_state`. If False, they are concatenated along the column axis. This latter behavior will soon be deprecated. activation: Activation function of the inner states. Default: `tanh`. It could also be string that is within Keras activation function names. reuse: (optional) Python boolean describing whether to reuse variables in an existing scope. If not `True`, and the existing scope already has the given variables, an error is raised. name: String, the name of the layer. Layers with the same name will share weights, but to avoid mistakes we require reuse=True in such cases. dtype: Default dtype of the layer (default of `None` means use the type of the first input). Required when `build` is called before `call`. **kwargs: Dict, keyword named properties for common layer attributes, like `trainable` etc when constructing the cell from configs of get_config(). When restoring from CudnnLSTM-trained checkpoints, use `CudnnCompatibleLSTMCell` instead. z`tf.nn.rnn_cell.LSTMCell` is deprecated and will be removed in a future version. This class is equivalent as `tf.keras.layers.LSTMCell`, and will be replaced by that in Tensorflow 2.0.rrrrNz%s: The num_unit_shards and proj_unit_shards parameters are deprecated and will be removed in Jan 2017. Use a variable scope with a partitioner instead.rrr) rrrDrErrrrrr5rrr rr_use_peepholes _cell_cliprr _initializer _num_proj _proj_clip_num_unit_shards_num_proj_shardsrrrrrr _state_size _output_size)rGr use_peepholes cell_cliprnum_proj proj_clipnum_unit_shardsnum_proj_shardsrrrrrCrrHrIrrrE}s\<   zLSTMCell.__init__cCs|jSrV)rrjrrrr?szLSTMCell.state_sizecCs|jSrV)rrjrrrrkszLSTMCell.output_sizecCsn|ddurtd|t|j|d}|jdur<|jn|j}|jdur\tjj |jnd}|j t ||d|jg|j |d|_ |jdurtjjj}ntjjj|jd}|j td|jg|d|_|jr|j d|jg|j d|_|j d|jg|j d|_|j d |jg|j d|_|jdurd|jdur>tjj |jnd}|j d t |j|jg|j |d|_d |_dS) Nrrr)rr partitionerrrw_f_diagw_i_diagw_o_diagz projection/%sT)r$rrrrrrr!r"fixed_size_partitionerrrrrrrrr _w_f_diag _w_i_diag _w_o_diagr _proj_kernelr)rGrrrmaybe_partitionerrmaybe_proj_partitionerrrrrmsn       zLSTMCell.buildcCst||g|jdur|jn|j}tj}|jr8|\}}n0t|ddgd|jg}t|d|jgd|g}|dj dj }|durt d|t t ||gd|j}tj||j}tj|ddd\} } } } |jr|| |j|j|||| |j||| } n$|| |j||| || } |jdur`t| |j |j} |jr|| |j| || }n|| || }|jdurt ||j}|jdurt||j |j}|jrt| |nt | |gd}||fS) aRun one step of LSTM. Args: inputs: input Tensor, must be 2-D, `[batch, input_size]`. state: if `state_is_tuple` is False, this must be a state Tensor, `2-D, [batch, state_size]`. If `state_is_tuple` is True, this must be a tuple of state Tensors, both `2-D`, with column sizes `c_state` and `m_state`. Returns: A tuple containing: - A `2-D, [batch, output_dim]`, Tensor representing the output of the LSTM after reading `inputs` when previous state was `state`. Here output_dim is: num_proj if num_proj was set, num_units otherwise. - Tensor(s) representing the new state of LSTM after reading `inputs` when the previous state was `state`. Same type and shape(s) as `state`. Raises: ValueError: If input size cannot be inferred from inputs via static shape inference. NrrrrzMCould not infer input size from inputs.get_shape()[-1].Received input shape: rr)rrrrrrslice get_shape with_rankrrrsr$rr+rrrrrrrrrrr clip_by_valuerrrr)rGrSrTrrc_prevm_prev input_size lstm_matrixrrrrr7mrrrrr>sd      z LSTMCell.callc sh|j|j|jt|j|j|j|j|j |j |j t |j |jd }t}tt|t|S)N) rrrrrrrrrrrr)rrrrrrrrrrrrrrrrDrrr`rrrIrrrs   zLSTMCell.get_config) FNNNNNNrTNNNNrrrrIrr]s. {   =Vrz/keras.__internal__.legacy.rnn_cell.MultiRNNCellznn.rnn_cell.MultiRNNCellcsfeZdZdZdfdd ZeddZeddZfd d Zed d Z ed dZ ddZ Z S) MultiRNNCellzRNN cell composed sequentially of multiple simple cells. Example: ```python num_units = [128, 64] cells = [BasicLSTMCell(num_units=n) for n in num_units] stacked_rnn_cell = MultiRNNCell(cells) ``` Tcstdt|s tdtj|s.zWAt least two cells provided to MultiRNNCell are the same object and will share weights.rzcell-%drncss|]}tj|jVqdSrV)rr= is_nestedr?rXr7rrrr\rz]Some cells return tuples of states, but the flag state_is_tuple is not set. State sizes are: cSsg|] }|jqSrr?rrrr rz)MultiRNNCell.__init__..N)rrrDrEr$rr=rrlenset log_first_nWARN_cells enumerater __internal__tracking Trackable_track_trackablerany)rGcellsr cell_numberrrIrrrEs8   zMultiRNNCell.__init__cCs2|jrtdd|jDStdd|jDSdS)Ncss|] }|jVqdSrVrrrrrr\rz*MultiRNNCell.state_size..css|] }|jVqdSrVrrrrrr\r)rtuplersumrjrrrr?szMultiRNNCell.state_sizecCs |jdjS)Nr)rrkrjrrrrkszMultiRNNCell.output_sizecstt|jdZ|jrDtfdd|jDWdStWdSWdn1st0YdS)Nrxc3s|]}|VqdSrV)rtrr:rrr\sz*MultiRNNCell.zero_state..) rryrzr{rr(rrDrt)rGr;rrIr:rrts zMultiRNNCell.zero_statecCs4|js gSg}|jD]}t|tjr||j7}q|SrV)rBrrrrtrainable_weights)rGweightsrrrrr*s   zMultiRNNCell.trainable_weightscCs^g}|jD]}t|tjr ||j7}q |jsZg}|jD]}t|tjr6||j7}q6||S|SrV)rrrrnon_trainable_weightsrBr*)rGr+rr*rrrr, s      z"MultiRNNCell.non_trainable_weightsc Csd}|}g}t|jD]\}}tjjd||jrhtj|s^t dt |j d|||}n"t |d|gd|j g}||j 7}|||\}} | | Wdq1s0Yq|jrt|n t|d}||fS)z9Run this multi-layer cell on inputs, starting from state.rzcell_%dz'Expected state to be a tuple of length z, but received: rNr)r rrr!r"rNrr=rr$rr?rrcr(r+) rGrSrT cur_state_poscur_inp new_statesrr cur_staterrrrrs4   *  zMultiRNNCell.call)T) r{rrrrErr?rkrtr*r,rrrrrIrrs 4     rcCs tj|D]}t|jq dS)aCheck whether the input tensors are with supported dtypes. Default RNN cells only support floats and complex as its dtypes since the activation function (tanh and sigmoid) only allow those types. This function will throw a proper error message if the inputs is not in a supported type. Args: inputs: tensor or nested structure of tensors that are feed to RNN cell as input or state. Raises: ValueError: if any of the input tensor are not having dtypes of float or complex. N)rr=flattenrr)rStrrrr8srcCs4|dur dSt|}|js0|js0td|dS)NzBRNN cell only supports floating point inputs, but received dtype: )ras_dtype is_floating is_complexr$rrrrrKs  r)F)/r __future__rrr collectionsrtensorflow.compat.v2r!v2rkerasrrr keras.enginerr Zkeras.legacy_tf_layersr r keras.utilsr tensorflow.python.platformr r tensorflow.python.util.tf_exportr rrrrr3r@rrArrr namedtuple_LSTMStateTuplerrrrrrrrrrsh              H  A+  i       <  J