Mass-Conserving LSTM model from Hoedt et al. [1]_.
This implementation follows of NeuralHydrology’s implementation of MCLSTM
with some changes:
1) reduced sum is not performed for over the units
2) time_major argument is added
3) no implementation of Embedding
num_targets (int) – number of inputs for which mass balance is to be reserved.
dynamic_inputs – number of inpts other than mass_targets
units – hidden size, determines the size of weight matrix
time_major (bool, optional (default=True)) – if True, the data is expected to be of shape (lookback, batch_size, input_features)
otherwise, data is expected of shape (batch_size, lookback, input_features)
Note here that call() method in tf.keras is little bit different
from keras API. In keras API, you can pass support masking for
layers as additional arguments. Whereas tf.keras has compute_mask()
method to support masking.
Parameters:
inputs –
Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules:
- inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value
of a keyword argument.
NumPy array or Python scalar values in inputs get cast as tensors.
Keras mask metadata is only collected from inputs.
Layers are built (build(input_shape) method)
using shape info from inputs only.
input_spec compatibility is only checked against inputs.
Mixed precision input casting is only applied to inputs.
If a layer has tensor arguments in *args or **kwargs, their
casting behavior in mixed precision should be handled manually.
The SavedModel input specification is generated using inputs only.
Integration with various ecosystem packages like TFMOT, TFLite,
TF.js, etc is only supported for inputs and not for tensors in
positional and keyword arguments.
*args – Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
**kwargs –
Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above.
The following optional keyword arguments are reserved:
- training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer’s call() method takes a
mask argument, its default value will be set to the mask generated
for inputs by the previous layer (if input did come from a layer
that generated a corresponding mask, i.e. if it came from a Keras
layer with masking support).
Entity Aware LSTM as proposed by Kratzert et al., 2019 [1]_
The difference here is that a Dense layer is not applied on cell state as done in
original implementation in NeuralHydrology [2]. This is left to user’s discretion.
Examples
>>> fromai4water.models._tensorflowimportEALSTM>>> importtensorflowastf>>> batch_size,lookback,num_dyn_inputs,num_static_inputs,units=10,5,3,2,8>>> inputs=tf.range(batch_size*lookback*num_dyn_inputs,dtype=tf.float32)>>> inputs=tf.reshape(inputs,(batch_size,lookback,num_dyn_inputs))>>> stat_inputs=tf.range(batch_size*num_static_inputs,dtype=tf.float32)>>> stat_inputs=tf.reshape(stat_inputs,(batch_size,num_static_inputs))>>> lstm=EALSTM(units,num_static_inputs)>>> h_n=lstm(inputs,stat_inputs)# -> (batch_size, units)...... # with return sequences>>> lstm=EALSTM(units,num_static_inputs,return_sequences=True)>>> h_n=lstm(inputs,stat_inputs)# -> (batch, lookback, units)...... # with return sequences and return_state>>> lstm=EALSTM(units,num_static_inputs,return_sequences=True,return_state=True)>>> h_n,[c_n,y_hat]=lstm(inputs,stat_inputs)# -> (batch, lookback, units), [(), ()]...... # end to end Keras model>>> fromtensorflow.keras.modelsimportModel>>> fromtensorflow.keras.layersimportInput,Dense>>> importnumpyasnp>>> inp_dyn=Input(batch_shape=(batch_size,lookback,num_dyn_inputs))>>> inp_static=Input(batch_shape=(batch_size,num_static_inputs))>>> lstm=EALSTM(units,num_static_inputs)(inp_dyn,inp_static)>>> out=Dense(1)(lstm)>>> model=Model(inputs=[inp_dyn,inp_static],outputs=out)>>> model.compile(loss='mse')>>> print(model.summary())... # generate hypothetical data and train it>>> dyn_x=np.random.random((100,lookback,num_dyn_inputs))>>> static_x=np.random.random((100,num_static_inputs))>>> y=np.random.random((100,1))>>> h=model.fit(x=[dyn_x,static_x],y=y,batch_size=batch_size)
A layer config is a Python dictionary (serializable)
containing the configuration of a layer.
The same layer can be reinstantiated later
(without its trained weights) from this configuration.
The config of a layer does not include connectivity
information, nor the layer class name. These are handled
by Network (one layer of abstraction above).
Note that get_config() does not guarantee to return a fresh copy of dict
every time it is called. The callers should make a copy of the returned dict
if they want to modify it.
A layer config is a Python dictionary (serializable)
containing the configuration of a layer.
The same layer can be reinstantiated later
(without its trained weights) from this configuration.
The config of a layer does not include connectivity
information, nor the layer class name. These are handled
by Network (one layer of abstraction above).
Note that get_config() does not guarantee to return a fresh copy of dict
every time it is called. The callers should make a copy of the returned dict
if they want to modify it.
tensorflow/keras layer which implements logic of TabTransformer model.
The TabTransformer layer converts categorical features into contextual embeddings
by passing them into Transformer block. The output of Transformer block is
concatenated with numerical features and passed through an MLP to
get the final model output.
num_numeric_features (int) – number of numeric features to be used as input.
cat_vocabulary (dict) – a dictionary whose keys are names of categorical features and values
are lists which consist of unique values of categorical features.
You can use the function ai4water.models.utils.gen_cat_vocab()
to create this for your own data. The length of dictionary should be
equal to number of categorical features. If it is None, then this
layer expects only numeri features
hidden_units (int, optional (default=32)) – number of hidden units
num_heads (int, optional (default=4)) – number of attention heads
depth (int (default=4)) – number of transformer blocks to be stacked on top of each other
dropout (int, optional (default=0.1)) – droput rate in transformer
tensorflow/keras layer which implements logic of FTTransformer model.
In FTTransformer, both categorical and numerical features are passed
through transformer block and then passed through MLP layer to get
the final model prediction.
num_numeric_features (int) – number of numeric features to be used as input.
cat_vocabulary (dict/None) – a dictionary whose keys are names of categorical features and values
are lists which consist of unique values of categorical features.
You can use the function ai4water.models.utils.gen_cat_vocab()
to create this for your own data. The length of dictionary should be
equal to number of categorical features. If it is None, then this
layer expects only numeri features
hidden_units (int, optional (default=32)) – number of hidden units
num_heads (int, optional (default=4)) – number of attention heads
depth (int (default=4)) – number of transformer blocks to be stacked on top of each other
dropout (float, optional (default=0.1)) – droput rate in transformer
lookup_kws (dict) – keyword arguments for lookup layer
Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model
can override if they need a state-creation step in-between
layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
Parameters:
input_shape – Instance of TensorShape, or list of instances of
TensorShape if the layer expects a list of inputs
(one instance per input).