Tensorflow Layers
MCLSTM
- class ai4water.models.tensorflow.MCLSTM(*args, **kwargs)[source]
Bases:
keras.engine.base_layer.Layer
Mass-Conserving LSTM model from Hoedt et al. [1]_.
This implementation follows of NeuralHydrology’s implementation of MCLSTM with some changes: 1) reduced sum is not performed for over the units 2) time_major argument is added 3) no implementation of Embedding
Examples
>>> inputs = tf.range(150, dtype=tf.float32) >>> inputs = tf.reshape(inputs, (10, 5, 3)) >>> mc = MCLSTM(1, 2, 8, 1) >>> h = mc(inputs) # (batch, units) ... >>> mc = MCLSTM(1, 2, 8, 1, return_sequences=True) >>> h = mc(inputs) # (batch, lookback, units) ... >>> mc = MCLSTM(1, 2, 8, 1, return_state=True) >>> _h, _o, _c = mc(inputs) # (batch, lookback, units) ... >>> mc = MCLSTM(1, 2, 8, 1, return_state=True, return_sequences=True) >>> _h, _o, _c = mc(inputs) # (batch, lookback, units) ... ... # with time_major as True >>> inputs = tf.range(150, dtype=tf.float32) >>> inputs = tf.reshape(inputs, (5, 10, 3)) >>> mc = MCLSTM(1, 2, 8, 1, time_major=True) >>> _h = mc(inputs) # (batch, units) ... >>> mc = MCLSTM(1, 2, 8, 1, time_major=True, return_sequences=True) >>> _h = mc(inputs) # (lookback, batch, units) ... >>> mc = MCLSTM(1, 2, 8, 1, time_major=True, return_state=True) >>> _h, _o, _c = mc(inputs) # (batch, units), ..., (lookback, batch, units) ... ... # end to end keras Model >>> from tensorflow.keras.layers import Dense, Input >>> from tensorflow.keras.models import Model >>> import numpy as np ... >>> inp = Input(batch_shape=(32, 10, 3)) >>> lstm = MCLSTM(1, 2, 8)(inp) >>> out = Dense(1)(lstm) ... >>> model = Model(inputs=inp, outputs=out) >>> model.compile(loss='mse') ... >>> x = np.random.random((320, 10, 3)) >>> y = np.random.random((320, 1)) >>> y = model.fit(x=x, y=y)
References
- __init__(num_mass_inputs, dynamic_inputs, units, num_targets=1, time_major: bool = False, return_sequences: bool = False, return_state: bool = False, name='MCLSTM', **kwargs)[source]
- Parameters
num_targets (int) – number of inputs for which mass balance is to be reserved.
dynamic_inputs – number of inpts other than mass_targets
units – hidden size, determines the size of weight matrix
time_major (bool, optional (default=True)) – if True, the data is expected to be of shape (lookback, batch_size, input_features) otherwise, data is expected of shape (batch_size, lookback, input_features)
- call(inputs)[source]
This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
- Parameters
inputs –
Input tensor, or dict/list/tuple of input tensors. The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method) using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
*args – Additional positional arguments. May contain tensors, although this is not recommended, for the reasons above.
**kwargs –
Additional keyword arguments. May contain tensors, although this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).
- Returns
A tensor or list/tuple of tensors.
Conditionalize
- class ai4water.models.tensorflow.Conditionalize(*args, **kwargs)[source]
Bases:
keras.engine.base_layer.Layer
Mimics the behaviour of cond_rnn of Philipperemy but puts the logic of condition in a separate layer so that it becomes easier to use it.
Example
>>> from ai4water.models.tensorflow import Conditionalize >>> from tensorflow.keras.layers import Input, LSTM >>> i = Input(shape=(10, 3)) >>> raw_conditions = Input(shape=(14,)) >>> processed_conds = Conditionalize(32)([raw_conditions, raw_conditions, raw_conditions]) >>> rnn = LSTM(32)(i, initial_state=[processed_conds, processed_conds])
EALSTM
- class ai4water.models.tensorflow.EALSTM(*args, **kwargs)[source]
Bases:
keras.engine.base_layer.Layer
Entity Aware LSTM as proposed by Kratzert et al., 2019 [1]_
The difference here is that a Dense layer is not applied on cell state as done in original implementation in NeuralHydrology 2. This is left to user’s discretion.
Examples
>>> import tensorflow as tf >>> batch, lookback, num_dyn_inputs, num_static_inputs, units = 10, 5, 3, 2, 8 >>> inputs = tf.range(batch*lookback*num_dyn_inputs, dtype=tf.float32) >>> inputs = tf.reshape(inputs, (batch, lookback, num_dyn_inputs)) >>> stat_inputs = tf.range(batch*num_static_inputs, dtype=tf.float32) >>> stat_inputs = tf.reshape(stat_inputs, (batch, num_static_inputs)) >>> lstm = EALSTM(units, num_static_inputs) >>> h_n = lstm(inputs, stat_inputs) # -> (batch, units) ... ... # with return sequences >>> lstm = EALSTM(units, num_static_inputs, return_sequences=True) >>> h_n = lstm(inputs, stat_inputs) # -> (batch, lookback, units) ... ... # with return sequences and return_state >>> lstm = EALSTM(units, num_static_inputs, return_sequences=True) >>> h_n, [c_n, y_hat] = lstm(inputs, stat_inputs) # -> (batch, lookback, units), [(), ()] ... ... # end to end Keras model >>> from tensorflow.keras.models import Model >>> from tensorflow.keras.layers import Input, Dense >>> import numpy as np >>> inp_dyn = Input(batch_shape=(batch, lookback, num_dyn_inputs)) >>> inp_static = Input(batch_shape=(batch, num_static_inputs)) >>> lstm = EALSTM(units, num_static_inputs)(inp_dyn, inp_static) >>> out = Dense(1)(lstm) >>> model = Model(inputs=[inp_dyn, inp_static], outputs=out) >>> model.compile(loss='mse') >>> print(model.summary()) ... # generate hypothetical data and train it >>> dyn_x = np.random.random((100, lookback, num_dyn_inputs)) >>> static_x = np.random.random((100, num_static_inputs)) >>> y = np.random.random((100, 1)) >>> h = model.fit(x=[dyn_x, static_x], y=y, batch_size=batch)
References
- __init__(units, num_static_inputs: int, use_bias=True, activation='tanh', recurrent_activation='sigmoid', static_activation='sigmoid', kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', static_initializer='glorot_uniform', kernel_constraint=None, recurrent_constraint=None, bias_constraint=None, static_constraint=None, kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None, static_regularizer=None, return_state=False, return_sequences=False, time_major=False, **kwargs)[source]
- Parameters
units (int) – number of units
num_static_inputs (int) – number of static features
static_activation – activation function for static input gate
static_regularizer –
static_constraint –
static_initializer –
- build(input_shape)[source]
kernel, recurrent_kernel and bias are initiated for 3 gates instead of 4 gates as in original LSTM