a Sic,@sdZddlmmZddlmZddlmZddlm Z ddl m Z ddl mZddlmZe jZe jZe jZe jZed d Gd d d ejZdS) z"Keras hashing preprocessing layer.N)backend) base_layer)base_preprocessing_layer)preprocessing_utils) layer_utils) keras_exportzkeras.layers.Hashingz/keras.layers.experimental.preprocessing.HashingcsNeZdZdZdfdd ZddZd d Zd d Zd dZfddZ Z S)HashingaA preprocessing layer which hashes and bins categorical features. This layer transforms categorical inputs to hashed output. It element-wise converts a ints or strings to ints in a fixed range. The stable hash function uses `tensorflow::ops::Fingerprint` to produce the same output consistently across all platforms. This layer uses [FarmHash64](https://github.com/google/farmhash) by default, which provides a consistent hashed output across different platforms and is stable across invocations, regardless of device and context, by mixing the input bits thoroughly. If you want to obfuscate the hashed output, you can also pass a random `salt` argument in the constructor. In that case, the layer will use the [SipHash64](https://github.com/google/highwayhash) hash function, with the `salt` value serving as additional input to the hash function. For an overview and full list of preprocessing layers, see the preprocessing [guide](https://www.tensorflow.org/guide/keras/preprocessing_layers). **Example (FarmHash64)** >>> layer = tf.keras.layers.Hashing(num_bins=3) >>> inp = [['A'], ['B'], ['C'], ['D'], ['E']] >>> layer(inp) **Example (FarmHash64) with a mask value** >>> layer = tf.keras.layers.Hashing(num_bins=3, mask_value='') >>> inp = [['A'], ['B'], [''], ['C'], ['D']] >>> layer(inp) **Example (SipHash64)** >>> layer = tf.keras.layers.Hashing(num_bins=3, salt=[133, 137]) >>> inp = [['A'], ['B'], ['C'], ['D'], ['E']] >>> layer(inp) **Example (Siphash64 with a single integer, same as `salt=[133, 133]`)** >>> layer = tf.keras.layers.Hashing(num_bins=3, salt=133) >>> inp = [['A'], ['B'], ['C'], ['D'], ['E']] >>> layer(inp) Args: num_bins: Number of hash bins. Note that this includes the `mask_value` bin, so the effective number of bins is `(num_bins - 1)` if `mask_value` is set. mask_value: A value that represents masked inputs, which are mapped to index 0. Defaults to None, meaning no mask term will be added and the hashing will start at index 0. salt: A single unsigned integer or None. If passed, the hash function used will be SipHash64, with these values used as an additional input (known as a "salt" in cryptography). These should be non-zero. Defaults to `None` (in that case, the FarmHash64 hash function is used). It also supports tuple/list of 2 unsigned integer numbers, see reference paper for details. output_mode: Specification for the output of the layer. Defaults to `"int"`. Values can be `"int"`, `"one_hot"`, `"multi_hot"`, or `"count"` configuring the layer as follows: - `"int"`: Return the integer bin indices directly. - `"one_hot"`: Encodes each individual element in the input into an array the same size as `num_bins`, containing a 1 at the input's bin index. If the last dimension is size 1, will encode on that dimension. If the last dimension is not size 1, will append a new dimension for the encoded output. - `"multi_hot"`: Encodes each sample in the input into a single array the same size as `num_bins`, containing a 1 for each bin index index present in the sample. Treats the last dimension as the sample dimension, if input shape is `(..., sample_length)`, output shape will be `(..., num_tokens)`. - `"count"`: As `"multi_hot"`, but the int array contains a count of the number of times the bin index appeared in the sample. sparse: Boolean. Only applicable to `"one_hot"`, `"multi_hot"`, and `"count"` output modes. If True, returns a `SparseTensor` instead of a dense `Tensor`. Defaults to False. **kwargs: Keyword arguments to construct a layer. Input shape: A single or list of string, int32 or int64 `Tensor`, `SparseTensor` or `RaggedTensor` of shape `(batch_size, ...,)` Output shape: An int64 `Tensor`, `SparseTensor` or `RaggedTensor` of shape `(batch_size, ...)`. If any input is `RaggedTensor` then output is `RaggedTensor`, otherwise if any input is `SparseTensor` then output is `SparseTensor`, otherwise the output is `Tensor`. Reference: - [SipHash with salt](https://www.131002.net/siphash/siphash.pdf) NintFc s|dus|dkr td|dd|vs4|ddurP|tkrBtjnt|d<n"|dkrrt|djsrtj|d<tj fi|t j d d|tkrt|jjs|d}td|tj|ttttf|jjd d |r|tkrtd |d |||_||_|durdnd |_||_||_d|_|durt|ttfrft|dkrf||_n(t|t r~||g|_ntd|ddS)NrzYThe `num_bins` for `Hashing` cannot be `None` or non-positive values. Received: num_bins=.dtyper rTzMWhen `output_mode="int"`, `dtype` should be an integer type. Received: dtype= output_mode)allowable_strings layer_namearg_namezi`sparse` may only be true if `output_mode` is `"one_hot"`, `"multi_hot"`, or `"count"`. Received: sparse=z and output_mode=FznThe `salt` argument for `Hashing` can only be a tuple of size 2 integers, or a single integer. Received: salt=)! ValueErrorINTtfint64rfloatxas_dtype is_integersuper__init__rkeras_kpl_gaugeget_cellset compute_dtypervalidate_string_argONE_HOT MULTI_HOTCOUNT __class____name__num_bins mask_value strong_hashr sparsesalt isinstancetuplelistlenr )selfr$r%r(r r'kwargs input_dtyper"^/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/layers/preprocessing/hashing.pyrsp       zHashing.__init__cCsZt|}t|tjr4tj|j||j|jd}n ||}tj ||j |j |j |j dS)N)indicesvalues dense_shape)r depthr'r )utils ensure_tensorr)r SparseTensorr3_hash_values_to_binsr4r5encode_categorical_inputsr r$r'r)r-inputsr3r1r1r2calls    z Hashing.callcCs|j}d}|jdur2|dkr2|d8}t||j}|jjrDt|}|jrbtjj ||d|j d}ntjj ||dd}|durt |t |}t|t||}|S)z6Converts a non-sparse tensor of values to bin indices.Nhash)namekey)r@)r$r%requalr r as_stringr&stringsto_hash_bucket_strongr(to_hash_bucket_fastadd ones_likewhere zeros_like)r-r4Z hash_binsmaskr1r1r2r:s$  zHashing._hash_values_to_binscCs|S)Nr1)r- input_shaper1r1r2compute_output_shapeszHashing.compute_output_shapecCs<||j}t|tjr(tj||jdStj||jdSdS)N)shaper )rMrNr)rSparseTensorSpecr TensorSpec)r- input_spec output_shaper1r1r2compute_output_signatures   z Hashing.compute_output_signaturecs.t}||j|j|j|j|jd|S)N)r$r(r%r r')r get_configupdater$r(r%r r')r-configr0r1r2rTs  zHashing.get_config)NNr F) r# __module__ __qualname____doc__rr=r:rMrSrT __classcell__r1r1r0r2r#syK r)rYtensorflow.compat.v2compatv2rkerasr keras.enginerrZkeras.layers.preprocessingrr7 keras.utilsr tensorflow.python.util.tf_exportrrr rr!Layerrr1r1r1r2s