Tensorflow Layers

MCLSTM

class ai4water.models.tensorflow.MCLSTM(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Mass-Conserving LSTM model from Hoedt et al. [1]_.

This implementation follows of NeuralHydrology’s implementation of MCLSTM with some changes: 1) reduced sum is not performed for over the units 2) time_major argument is added 3) no implementation of Embedding

Examples

>>> inputs = tf.range(150, dtype=tf.float32)
>>> inputs = tf.reshape(inputs, (10, 5, 3))
>>> mc = MCLSTM(1, 2, 8, 1)
>>> h = mc(inputs)  # (batch, units)
...
>>> mc = MCLSTM(1, 2, 8, 1, return_sequences=True)
>>> h = mc(inputs)  # (batch, lookback, units)
...
>>> mc = MCLSTM(1, 2, 8, 1, return_state=True)
>>> _h, _o, _c = mc(inputs)  # (batch, lookback, units)
...
>>> mc = MCLSTM(1, 2, 8, 1, return_state=True, return_sequences=True)
>>> _h, _o, _c = mc(inputs)  # (batch, lookback, units)
...
... # with time_major as True
>>> inputs = tf.range(150, dtype=tf.float32)
>>> inputs = tf.reshape(inputs, (5, 10, 3))
>>> mc = MCLSTM(1, 2, 8, 1, time_major=True)
>>> _h = mc(inputs)  # (batch, units)
...
>>> mc = MCLSTM(1, 2, 8, 1, time_major=True, return_sequences=True)
>>> _h = mc(inputs)  # (lookback, batch, units)
...
>>> mc = MCLSTM(1, 2, 8, 1, time_major=True, return_state=True)
>>> _h, _o, _c = mc(inputs)  # (batch, units), ..., (lookback, batch, units)
...
... # end to end keras Model
>>> from tensorflow.keras.layers import Dense, Input
>>> from tensorflow.keras.models import Model
>>> import numpy as np
...
>>> inp = Input(batch_shape=(32, 10, 3))
>>> lstm = MCLSTM(1, 2, 8)(inp)
>>> out = Dense(1)(lstm)
...
>>> model = Model(inputs=inp, outputs=out)
>>> model.compile(loss='mse')
...
>>> x = np.random.random((320, 10, 3))
>>> y = np.random.random((320, 1))
>>> y = model.fit(x=x, y=y)

References

1

https://arxiv.org/abs/2101.05186

__init__(num_mass_inputs, dynamic_inputs, units, num_targets=1, time_major: bool = False, return_sequences: bool = False, return_state: bool = False, name='MCLSTM', **kwargs)[source]
Parameters
  • num_targets (int) – number of inputs for which mass balance is to be reserved.

  • dynamic_inputs – number of inpts other than mass_targets

  • units – hidden size, determines the size of weight matrix

  • time_major (bool, optional (default=True)) – if True, the data is expected to be of shape (lookback, batch_size, input_features) otherwise, data is expected of shape (batch_size, lookback, input_features)

call(inputs)[source]

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters
  • inputs

    Input tensor, or dict/list/tuple of input tensors. The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

    arguments, and inputs cannot be provided via the default value of a keyword argument.

    • NumPy array or Python scalar values in inputs get cast as tensors.

    • Keras mask metadata is only collected from inputs.

    • Layers are built (build(input_shape) method) using shape info from inputs only.

    • input_spec compatibility is only checked against inputs.

    • Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

    • The SavedModel input specification is generated using inputs only.

    • Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

  • *args – Additional positional arguments. May contain tensors, although this is not recommended, for the reasons above.

  • **kwargs

    Additional keyword arguments. May contain tensors, although this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

    whether the call is meant for training or inference.

    • mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns

A tensor or list/tuple of tensors.

Conditionalize

class ai4water.models.tensorflow.Conditionalize(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Mimics the behaviour of cond_rnn of Philipperemy but puts the logic of condition in a separate layer so that it becomes easier to use it.

Example

>>> from ai4water.models.tensorflow import Conditionalize
>>> from tensorflow.keras.layers import Input, LSTM
>>> i = Input(shape=(10, 3))
>>> raw_conditions = Input(shape=(14,))
>>> processed_conds = Conditionalize(32)([raw_conditions, raw_conditions, raw_conditions])
>>> rnn = LSTM(32)(i, initial_state=[processed_conds, processed_conds])
__init__(units, max_num_cond=10, **kwargs)[source]

EALSTM

class ai4water.models.tensorflow.EALSTM(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Entity Aware LSTM as proposed by Kratzert et al., 2019 [1]_

The difference here is that a Dense layer is not applied on cell state as done in original implementation in NeuralHydrology 2. This is left to user’s discretion.

Examples

>>> import tensorflow as tf
>>> batch, lookback, num_dyn_inputs, num_static_inputs, units = 10, 5, 3, 2, 8
>>> inputs = tf.range(batch*lookback*num_dyn_inputs, dtype=tf.float32)
>>> inputs = tf.reshape(inputs, (batch, lookback, num_dyn_inputs))
>>> stat_inputs = tf.range(batch*num_static_inputs, dtype=tf.float32)
>>> stat_inputs = tf.reshape(stat_inputs, (batch, num_static_inputs))
>>> lstm = EALSTM(units, num_static_inputs)
>>> h_n = lstm(inputs, stat_inputs)  # -> (batch, units)
...
... # with return sequences
>>> lstm = EALSTM(units, num_static_inputs, return_sequences=True)
>>> h_n = lstm(inputs, stat_inputs)  # -> (batch, lookback, units)
...
... # with return sequences and return_state
>>> lstm = EALSTM(units, num_static_inputs, return_sequences=True)
>>> h_n, [c_n, y_hat] = lstm(inputs, stat_inputs)  # -> (batch, lookback, units), [(), ()]
...
... # end to end Keras model
>>> from tensorflow.keras.models import Model
>>> from tensorflow.keras.layers import Input, Dense
>>> import numpy as np
>>> inp_dyn = Input(batch_shape=(batch, lookback, num_dyn_inputs))
>>> inp_static = Input(batch_shape=(batch, num_static_inputs))
>>> lstm = EALSTM(units, num_static_inputs)(inp_dyn, inp_static)
>>> out = Dense(1)(lstm)
>>> model = Model(inputs=[inp_dyn, inp_static], outputs=out)
>>> model.compile(loss='mse')
>>> print(model.summary())
... # generate hypothetical data and train it
>>> dyn_x = np.random.random((100, lookback, num_dyn_inputs))
>>> static_x = np.random.random((100, num_static_inputs))
>>> y = np.random.random((100, 1))
>>> h = model.fit(x=[dyn_x, static_x], y=y, batch_size=batch)

References

1

https://doi.org/10.5194/hess-23-5089-2019

2

https://github.com/neuralhydrology/neuralhydrology

__init__(units, num_static_inputs: int, use_bias=True, activation='tanh', recurrent_activation='sigmoid', static_activation='sigmoid', kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', static_initializer='glorot_uniform', kernel_constraint=None, recurrent_constraint=None, bias_constraint=None, static_constraint=None, kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None, static_regularizer=None, return_state=False, return_sequences=False, time_major=False, **kwargs)[source]
Parameters
  • units (int) – number of units

  • num_static_inputs (int) – number of static features

  • static_activation – activation function for static input gate

  • static_regularizer

  • static_constraint

  • static_initializer

build(input_shape)[source]

kernel, recurrent_kernel and bias are initiated for 3 gates instead of 4 gates as in original LSTM

call(inputs, static_inputs, initial_state, **kwargs)[source]
static_inputs :

of shape (batch, num_static_inputs)

cell(inputs, i, states)[source]