a Sic(@sNdZddlZddlmmZddlmZeddgddd d Z d d Z dS)z#Keras timeseries dataset utilities.N) keras_exportz)keras.utils.timeseries_dataset_from_arrayz1keras.preprocessing.timeseries_dataset_from_array)v1Fc s|r>|dkrtd||t|kr>td|dt|| r|rd| |krdtd|d| d| t|krtd| dt|| dkrtd | dkrtd t|krtd dt||dkrtd ||t|krtd |dt||dur"d}| dur4t|} | |d} |dur`t| t|} | dkrpd} nd} tjd| || d} |r|durtjd}tj|} | | t j | dt j | dt j j | }t j j t j j t| |fjfddt j jd}t|||| }|durt j j t j j t| |fjddt j jd}t|||| }t j j ||f}|t j j}|dur|r|j|d|d}||}n|r|jd|d}|S)aCreates a dataset of sliding windows over a timeseries provided as array. This function takes in a sequence of data-points gathered at equal intervals, along with time series parameters such as length of the sequences/windows, spacing between two sequence/windows, etc., to produce batches of timeseries inputs and targets. Args: data: Numpy array or eager tensor containing consecutive data points (timesteps). Axis 0 is expected to be the time dimension. targets: Targets corresponding to timesteps in `data`. `targets[i]` should be the target corresponding to the window that starts at index `i` (see example 2 below). Pass None if you don't have target data (in this case the dataset will only yield the input data). sequence_length: Length of the output sequences (in number of timesteps). sequence_stride: Period between successive output sequences. For stride `s`, output samples would start at index `data[i]`, `data[i + s]`, `data[i + 2 * s]`, etc. sampling_rate: Period between successive individual timesteps within sequences. For rate `r`, timesteps `data[i], data[i + r], ... data[i + sequence_length]` are used for creating a sample sequence. batch_size: Number of timeseries samples in each batch (except maybe the last one). If `None`, the data will not be batched (the dataset will yield individual samples). shuffle: Whether to shuffle output samples, or instead draw them in chronological order. seed: Optional int; random seed for shuffling. start_index: Optional int; data points earlier (exclusive) than `start_index` will not be used in the output sequences. This is useful to reserve part of the data for test or validation. end_index: Optional int; data points later (exclusive) than `end_index` will not be used in the output sequences. This is useful to reserve part of the data for test or validation. Returns: A tf.data.Dataset instance. If `targets` was passed, the dataset yields tuple `(batch_of_sequences, batch_of_targets)`. If not, the dataset yields only `batch_of_sequences`. Example 1: Consider indices `[0, 1, ... 99]`. With `sequence_length=10, sampling_rate=2, sequence_stride=3`, `shuffle=False`, the dataset will yield batches of sequences composed of the following indices: ``` First sequence: [0 2 4 6 8 10 12 14 16 18] Second sequence: [3 5 7 9 11 13 15 17 19 21] Third sequence: [6 8 10 12 14 16 18 20 22 24] ... Last sequence: [78 80 82 84 86 88 90 92 94 96] ``` In this case the last 3 data points are discarded since no full sequence can be generated to include them (the next sequence would have started at index 81, and thus its last step would have gone over 99). Example 2: Temporal regression. Consider an array `data` of scalar values, of shape `(steps,)`. To generate a dataset that uses the past 10 timesteps to predict the next timestep, you would use: ```python input_data = data[:-10] targets = data[10:] dataset = tf.keras.utils.timeseries_dataset_from_array( input_data, targets, sequence_length=10) for batch in dataset: inputs, targets = batch assert np.array_equal(inputs[0], data[:10]) # First sequence: steps [0-9] # Corresponding target: step 10 assert np.array_equal(targets[0], data[10]) break ``` Example 3: Temporal regression for many-to-many architectures. Consider two arrays of scalar values `X` and `Y`, both of shape `(100,)`. The resulting dataset should consist samples with 20 timestamps each. The samples should not overlap. To generate a dataset that uses the current timestamp to predict the corresponding target timestep, you would use: ```python X = np.arange(100) Y = X*2 sample_length = 20 input_dataset = tf.keras.utils.timeseries_dataset_from_array( X, None, sequence_length=sample_length, sequence_stride=sample_length) target_dataset = tf.keras.utils.timeseries_dataset_from_array( Y, None, sequence_length=sample_length, sequence_stride=sample_length) for batch in zip(input_dataset, target_dataset): inputs, targets = batch assert np.array_equal(inputs[0], X[:sample_length]) # second sample equals output timestamps 20-40 assert np.array_equal(targets[1], Y[sample_length:2*sample_length]) break ``` rz:`start_index` must be 0 or greater. Received: start_index=zO`start_index` must be lower than the length of the data. Received: start_index=z, for data of length zE`end_index` must be higher than `start_index`. Received: start_index=z, and end_index= zK`end_index` must be lower than the length of the data. Received: end_index=z7`end_index` must be higher than 0. Received: end_index=z?`sampling_rate` must be higher than 0. Received: sampling_rate=zS`sampling_rate` must be lower than the length of the data. Received: sampling_rate=zC`sequence_stride` must be higher than 0. Received: sequence_stride=zW`sequence_stride` must be lower than the length of the data. Received: sequence_stride=Nriint32int64)dtypeg.Acst||||SN)tfrangei positions sampling_ratesequence_lengthZ/var/www/html/django/DPS/env/lib/python3.9/site-packages/keras/utils/timeseries_dataset.pysz/timeseries_dataset_from_array..num_parallel_callscSs||Sr rr rrrr) buffer_sizeseedi) ValueErrorlenminnparangerandomrandint RandomStateshuffler castdataDataset from_tensorsrepeatzipr mapAUTOTUNEsequences_from_indicesprefetchbatch)r&targetsrZsequence_strider batch_sizer$r start_index end_indexZnum_seqs index_dtypeZstart_positionsrngZ positions_dsindicesdatasetZ target_dsrrrtimeseries_dataset_from_arrays~                 r8cCsBtjj|||}tjj||fjddtjjd}|S)NcSs t||Sr )r gather)stepsindsrrrrrz(sequences_from_indices..r)r r&r'r(r*r)r+r,)arrayZ indices_dsr2r3r7rrrr- s r-)rrrFNNN) __doc__numpyrtensorflow.compat.v2compatv2r tensorflow.python.util.tf_exportrr8r-rrrrs$   q