Loading an existing model from config
This notebook describes how to load an existing model from its config file, update the state (weights) of the model and make predictions from it.
One of the core philosophies of ai4water is that, whatever the underlying algorithm we use the front-end API should be same. One of its manifestation is loading the model from config. Whether you are loading sklearn type models, or tensorflow based models or pytorch based models, the method to load the pretrained models from config and then update the weights is same.
[ ]:
[1]:
import site
site.addsitedir("D:\\mytools\\AI4Water")
import os
from ai4water import Model
from ai4water import InputAttentionModel
from ai4water.datasets import CAMELS_AUS
from ai4water.utils.utils import get_version_info
WARNING:tensorflow:From D:\mytools\AI4Water\ai4water\backend.py:80: The name tf.keras.losses.cosine is deprecated. Please use tf.keras.losses.cosine_similarity instead.
WARNING:tensorflow:From D:\mytools\AI4Water\ai4water\backend.py:80: The name tf.keras.losses.cosine_proximity is deprecated. Please use tf.keras.losses.cosine_similarity instead.
WARNING:tensorflow:From D:\mytools\AI4Water\ai4water\backend.py:80: The name tf.keras.layers.CuDNNGRU is deprecated. Please use tf.compat.v1.keras.layers.CuDNNGRU instead.
WARNING:tensorflow:From D:\mytools\AI4Water\ai4water\backend.py:80: The name tf.keras.layers.CuDNNLSTM is deprecated. Please use tf.compat.v1.keras.layers.CuDNNLSTM instead.
D:\C\Anaconda3\envs\tfgpu115_py37\lib\site-packages\sklearn\experimental\enable_hist_gradient_boosting.py:17: UserWarning: Since version 1.0, it is not needed to import enable_hist_gradient_boosting anymore. HistGradientBoostingClassifier and HistGradientBoostingRegressor are now stable and can be normally imported from sklearn.ensemble.
"Since version 1.0, "
[2]:
for lib,ver in get_version_info().items():
print(lib, ver)
python 3.7.7 (default, Apr 15 2020, 05:09:04) [MSC v.1916 64 bit (AMD64)]
os nt
ai4water 1.07
lightgbm 3.2.1
catboost 0.26
xgboost 1.2.1
easy_mpl 0.21.3
SeqMetrics 1.3.4
tensorflow 1.15.0
tensorflow.python.keras.api._v1.keras 2.2.4-tf
torch 1.8.1+cpu
numpy 1.18.1
pandas 1.3.5
matplotlib 3.1.3
h5py 2.10.0
sklearn 1.0.1
shapefile 2.1.3
fiona 1.8.11
xarray 0.19.0
netCDF4 1.5.6
optuna 2.7.0
skopt 0.9.0
plotly 4.14.1
seaborn 0.10.0
Random Forest
XGBoost
ANN
Input Attention Model
[3]:
config_path = os.path.join(os.getcwd(), "results", "20230222_191908", "config.json")
model = InputAttentionModel.from_config_file(config_path)
model.update_weights(os.path.join(model.path, "weights", "weights_038_0.32582.hdf5"))
building input attention model
WARNING:tensorflow:From D:\C\Anaconda3\envs\tfgpu115_py37\lib\site-packages\tensorflow_core\python\ops\resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From D:\C\Anaconda3\envs\tfgpu115_py37\lib\site-packages\tensorflow_core\python\keras\initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
enc_input1 (InputLayer) [(None, 15, 7)] 0
__________________________________________________________________________________________________
lambda (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
enc_first_hidden_state_1 (Input [(None, 62)] 0
__________________________________________________________________________________________________
enc_first_cell_state_1 (InputLa [(None, 62)] 0
__________________________________________________________________________________________________
reshape (Reshape) (None, 1, 7) 0 lambda[0][0]
__________________________________________________________________________________________________
lambda_1 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
encoder_LSTM_1 (LSTM) [(None, 62), (None, 17360 reshape[0][0]
enc_first_hidden_state_1[0][0]
enc_first_cell_state_1[0][0]
reshape_1[0][0]
encoder_LSTM_1[0][0]
encoder_LSTM_1[0][2]
reshape_2[0][0]
encoder_LSTM_1[1][0]
encoder_LSTM_1[1][2]
reshape_3[0][0]
encoder_LSTM_1[2][0]
encoder_LSTM_1[2][2]
reshape_4[0][0]
encoder_LSTM_1[3][0]
encoder_LSTM_1[3][2]
reshape_5[0][0]
encoder_LSTM_1[4][0]
encoder_LSTM_1[4][2]
reshape_6[0][0]
encoder_LSTM_1[5][0]
encoder_LSTM_1[5][2]
reshape_7[0][0]
encoder_LSTM_1[6][0]
encoder_LSTM_1[6][2]
reshape_8[0][0]
encoder_LSTM_1[7][0]
encoder_LSTM_1[7][2]
reshape_9[0][0]
encoder_LSTM_1[8][0]
encoder_LSTM_1[8][2]
reshape_10[0][0]
encoder_LSTM_1[9][0]
encoder_LSTM_1[9][2]
reshape_11[0][0]
encoder_LSTM_1[10][0]
encoder_LSTM_1[10][2]
reshape_12[0][0]
encoder_LSTM_1[11][0]
encoder_LSTM_1[11][2]
reshape_13[0][0]
encoder_LSTM_1[12][0]
encoder_LSTM_1[12][2]
__________________________________________________________________________________________________
reshape_1 (Reshape) (None, 1, 7) 0 lambda_1[0][0]
__________________________________________________________________________________________________
lambda_2 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 124) 0 enc_first_hidden_state_1[0][0]
enc_first_cell_state_1[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 124) 0 encoder_LSTM_1[0][0]
encoder_LSTM_1[0][2]
__________________________________________________________________________________________________
reshape_2 (Reshape) (None, 1, 7) 0 lambda_2[0][0]
__________________________________________________________________________________________________
lambda_3 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
enc_We_1 (Dense) (None, 15) 1875 concatenate[0][0]
concatenate_1[0][0]
concatenate_2[0][0]
concatenate_3[0][0]
concatenate_4[0][0]
concatenate_5[0][0]
concatenate_6[0][0]
concatenate_7[0][0]
concatenate_8[0][0]
concatenate_9[0][0]
concatenate_10[0][0]
concatenate_11[0][0]
concatenate_12[0][0]
concatenate_13[0][0]
concatenate_14[0][0]
__________________________________________________________________________________________________
my_transpose (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
my_transpose_1 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 124) 0 encoder_LSTM_1[1][0]
encoder_LSTM_1[1][2]
__________________________________________________________________________________________________
reshape_3 (Reshape) (None, 1, 7) 0 lambda_3[0][0]
__________________________________________________________________________________________________
lambda_4 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
repeat_vector (RepeatVector) (None, 7, 15) 0 enc_We_1[0][0]
__________________________________________________________________________________________________
eq_8_mul_0_1 (MyDot) (None, 7, 15) 225 my_transpose[0][0]
__________________________________________________________________________________________________
repeat_vector_1 (RepeatVector) (None, 7, 15) 0 enc_We_1[1][0]
__________________________________________________________________________________________________
eq_8_mul_1_1 (MyDot) (None, 7, 15) 225 my_transpose_1[0][0]
__________________________________________________________________________________________________
my_transpose_2 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 124) 0 encoder_LSTM_1[2][0]
encoder_LSTM_1[2][2]
__________________________________________________________________________________________________
reshape_4 (Reshape) (None, 1, 7) 0 lambda_4[0][0]
__________________________________________________________________________________________________
lambda_5 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
add (Add) (None, 7, 15) 0 repeat_vector[0][0]
eq_8_mul_0_1[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 7, 15) 0 repeat_vector_1[0][0]
eq_8_mul_1_1[0][0]
__________________________________________________________________________________________________
repeat_vector_2 (RepeatVector) (None, 7, 15) 0 enc_We_1[2][0]
__________________________________________________________________________________________________
eq_8_mul_2_1 (MyDot) (None, 7, 15) 225 my_transpose_2[0][0]
__________________________________________________________________________________________________
my_transpose_3 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 124) 0 encoder_LSTM_1[3][0]
encoder_LSTM_1[3][2]
__________________________________________________________________________________________________
reshape_5 (Reshape) (None, 1, 7) 0 lambda_5[0][0]
__________________________________________________________________________________________________
lambda_6 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, 7, 15) 0 add[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 7, 15) 0 add_1[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 7, 15) 0 repeat_vector_2[0][0]
eq_8_mul_2_1[0][0]
__________________________________________________________________________________________________
repeat_vector_3 (RepeatVector) (None, 7, 15) 0 enc_We_1[3][0]
__________________________________________________________________________________________________
eq_8_mul_3_1 (MyDot) (None, 7, 15) 225 my_transpose_3[0][0]
__________________________________________________________________________________________________
my_transpose_4 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 124) 0 encoder_LSTM_1[4][0]
encoder_LSTM_1[4][2]
__________________________________________________________________________________________________
reshape_6 (Reshape) (None, 1, 7) 0 lambda_6[0][0]
__________________________________________________________________________________________________
lambda_7 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
my_dot (MyDot) (None, 7, 1) 15 activation[0][0]
__________________________________________________________________________________________________
my_dot_1 (MyDot) (None, 7, 1) 15 activation_1[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, 7, 15) 0 add_2[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 7, 15) 0 repeat_vector_3[0][0]
eq_8_mul_3_1[0][0]
__________________________________________________________________________________________________
repeat_vector_4 (RepeatVector) (None, 7, 15) 0 enc_We_1[4][0]
__________________________________________________________________________________________________
eq_8_mul_4_1 (MyDot) (None, 7, 15) 225 my_transpose_4[0][0]
__________________________________________________________________________________________________
my_transpose_5 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 124) 0 encoder_LSTM_1[5][0]
encoder_LSTM_1[5][2]
__________________________________________________________________________________________________
reshape_7 (Reshape) (None, 1, 7) 0 lambda_7[0][0]
__________________________________________________________________________________________________
lambda_8 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
eq_8_0_1 (MyTranspose) (None, 1, 7) 0 my_dot[0][0]
__________________________________________________________________________________________________
eq_8_1_1 (MyTranspose) (None, 1, 7) 0 my_dot_1[0][0]
__________________________________________________________________________________________________
my_dot_2 (MyDot) (None, 7, 1) 15 activation_2[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, 7, 15) 0 add_3[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 7, 15) 0 repeat_vector_4[0][0]
eq_8_mul_4_1[0][0]
__________________________________________________________________________________________________
repeat_vector_5 (RepeatVector) (None, 7, 15) 0 enc_We_1[5][0]
__________________________________________________________________________________________________
eq_8_mul_5_1 (MyDot) (None, 7, 15) 225 my_transpose_5[0][0]
__________________________________________________________________________________________________
my_transpose_6 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 124) 0 encoder_LSTM_1[6][0]
encoder_LSTM_1[6][2]
__________________________________________________________________________________________________
reshape_8 (Reshape) (None, 1, 7) 0 lambda_8[0][0]
__________________________________________________________________________________________________
lambda_9 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
eq_9_0_1 (Activation) (None, 1, 7) 0 eq_8_0_1[0][0]
__________________________________________________________________________________________________
eq_9_1_1 (Activation) (None, 1, 7) 0 eq_8_1_1[0][0]
__________________________________________________________________________________________________
eq_8_2_1 (MyTranspose) (None, 1, 7) 0 my_dot_2[0][0]
__________________________________________________________________________________________________
my_dot_3 (MyDot) (None, 7, 1) 15 activation_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 7, 15) 0 add_4[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 7, 15) 0 repeat_vector_5[0][0]
eq_8_mul_5_1[0][0]
__________________________________________________________________________________________________
repeat_vector_6 (RepeatVector) (None, 7, 15) 0 enc_We_1[6][0]
__________________________________________________________________________________________________
eq_8_mul_6_1 (MyDot) (None, 7, 15) 225 my_transpose_6[0][0]
__________________________________________________________________________________________________
my_transpose_7 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 124) 0 encoder_LSTM_1[7][0]
encoder_LSTM_1[7][2]
__________________________________________________________________________________________________
reshape_9 (Reshape) (None, 1, 7) 0 lambda_9[0][0]
__________________________________________________________________________________________________
lambda_10 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
attn_weight_1_1 (Concatenate) (None, 2, 7) 0 eq_9_0_1[0][0]
eq_9_1_1[0][0]
__________________________________________________________________________________________________
eq_9_2_1 (Activation) (None, 1, 7) 0 eq_8_2_1[0][0]
__________________________________________________________________________________________________
eq_8_3_1 (MyTranspose) (None, 1, 7) 0 my_dot_3[0][0]
__________________________________________________________________________________________________
my_dot_4 (MyDot) (None, 7, 1) 15 activation_4[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 7, 15) 0 add_5[0][0]
__________________________________________________________________________________________________
add_6 (Add) (None, 7, 15) 0 repeat_vector_6[0][0]
eq_8_mul_6_1[0][0]
__________________________________________________________________________________________________
repeat_vector_7 (RepeatVector) (None, 7, 15) 0 enc_We_1[7][0]
__________________________________________________________________________________________________
eq_8_mul_7_1 (MyDot) (None, 7, 15) 225 my_transpose_7[0][0]
__________________________________________________________________________________________________
my_transpose_8 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_9 (Concatenate) (None, 124) 0 encoder_LSTM_1[8][0]
encoder_LSTM_1[8][2]
__________________________________________________________________________________________________
reshape_10 (Reshape) (None, 1, 7) 0 lambda_10[0][0]
__________________________________________________________________________________________________
lambda_11 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
attn_weight_2_1 (Concatenate) (None, 3, 7) 0 attn_weight_1_1[0][0]
eq_9_2_1[0][0]
__________________________________________________________________________________________________
eq_9_3_1 (Activation) (None, 1, 7) 0 eq_8_3_1[0][0]
__________________________________________________________________________________________________
eq_8_4_1 (MyTranspose) (None, 1, 7) 0 my_dot_4[0][0]
__________________________________________________________________________________________________
my_dot_5 (MyDot) (None, 7, 1) 15 activation_5[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, 7, 15) 0 add_6[0][0]
__________________________________________________________________________________________________
add_7 (Add) (None, 7, 15) 0 repeat_vector_7[0][0]
eq_8_mul_7_1[0][0]
__________________________________________________________________________________________________
repeat_vector_8 (RepeatVector) (None, 7, 15) 0 enc_We_1[8][0]
__________________________________________________________________________________________________
eq_8_mul_8_1 (MyDot) (None, 7, 15) 225 my_transpose_8[0][0]
__________________________________________________________________________________________________
my_transpose_9 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_10 (Concatenate) (None, 124) 0 encoder_LSTM_1[9][0]
encoder_LSTM_1[9][2]
__________________________________________________________________________________________________
reshape_11 (Reshape) (None, 1, 7) 0 lambda_11[0][0]
__________________________________________________________________________________________________
lambda_12 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
attn_weight_3_1 (Concatenate) (None, 4, 7) 0 attn_weight_2_1[0][0]
eq_9_3_1[0][0]
__________________________________________________________________________________________________
eq_9_4_1 (Activation) (None, 1, 7) 0 eq_8_4_1[0][0]
__________________________________________________________________________________________________
eq_8_5_1 (MyTranspose) (None, 1, 7) 0 my_dot_5[0][0]
__________________________________________________________________________________________________
my_dot_6 (MyDot) (None, 7, 1) 15 activation_6[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, 7, 15) 0 add_7[0][0]
__________________________________________________________________________________________________
add_8 (Add) (None, 7, 15) 0 repeat_vector_8[0][0]
eq_8_mul_8_1[0][0]
__________________________________________________________________________________________________
repeat_vector_9 (RepeatVector) (None, 7, 15) 0 enc_We_1[9][0]
__________________________________________________________________________________________________
eq_8_mul_9_1 (MyDot) (None, 7, 15) 225 my_transpose_9[0][0]
__________________________________________________________________________________________________
my_transpose_10 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_11 (Concatenate) (None, 124) 0 encoder_LSTM_1[10][0]
encoder_LSTM_1[10][2]
__________________________________________________________________________________________________
reshape_12 (Reshape) (None, 1, 7) 0 lambda_12[0][0]
__________________________________________________________________________________________________
lambda_13 (Lambda) (None, 7) 0 enc_input1[0][0]
__________________________________________________________________________________________________
attn_weight_4_1 (Concatenate) (None, 5, 7) 0 attn_weight_3_1[0][0]
eq_9_4_1[0][0]
__________________________________________________________________________________________________
eq_9_5_1 (Activation) (None, 1, 7) 0 eq_8_5_1[0][0]
__________________________________________________________________________________________________
eq_8_6_1 (MyTranspose) (None, 1, 7) 0 my_dot_6[0][0]
__________________________________________________________________________________________________
my_dot_7 (MyDot) (None, 7, 1) 15 activation_7[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, 7, 15) 0 add_8[0][0]
__________________________________________________________________________________________________
add_9 (Add) (None, 7, 15) 0 repeat_vector_9[0][0]
eq_8_mul_9_1[0][0]
__________________________________________________________________________________________________
repeat_vector_10 (RepeatVector) (None, 7, 15) 0 enc_We_1[10][0]
__________________________________________________________________________________________________
eq_8_mul_10_1 (MyDot) (None, 7, 15) 225 my_transpose_10[0][0]
__________________________________________________________________________________________________
my_transpose_11 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_12 (Concatenate) (None, 124) 0 encoder_LSTM_1[11][0]
encoder_LSTM_1[11][2]
__________________________________________________________________________________________________
reshape_13 (Reshape) (None, 1, 7) 0 lambda_13[0][0]
__________________________________________________________________________________________________
attn_weight_5_1 (Concatenate) (None, 6, 7) 0 attn_weight_4_1[0][0]
eq_9_5_1[0][0]
__________________________________________________________________________________________________
eq_9_6_1 (Activation) (None, 1, 7) 0 eq_8_6_1[0][0]
__________________________________________________________________________________________________
eq_8_7_1 (MyTranspose) (None, 1, 7) 0 my_dot_7[0][0]
__________________________________________________________________________________________________
my_dot_8 (MyDot) (None, 7, 1) 15 activation_8[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 7, 15) 0 add_9[0][0]
__________________________________________________________________________________________________
add_10 (Add) (None, 7, 15) 0 repeat_vector_10[0][0]
eq_8_mul_10_1[0][0]
__________________________________________________________________________________________________
repeat_vector_11 (RepeatVector) (None, 7, 15) 0 enc_We_1[11][0]
__________________________________________________________________________________________________
eq_8_mul_11_1 (MyDot) (None, 7, 15) 225 my_transpose_11[0][0]
__________________________________________________________________________________________________
my_transpose_12 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_13 (Concatenate) (None, 124) 0 encoder_LSTM_1[12][0]
encoder_LSTM_1[12][2]
__________________________________________________________________________________________________
attn_weight_6_1 (Concatenate) (None, 7, 7) 0 attn_weight_5_1[0][0]
eq_9_6_1[0][0]
__________________________________________________________________________________________________
eq_9_7_1 (Activation) (None, 1, 7) 0 eq_8_7_1[0][0]
__________________________________________________________________________________________________
eq_8_8_1 (MyTranspose) (None, 1, 7) 0 my_dot_8[0][0]
__________________________________________________________________________________________________
my_dot_9 (MyDot) (None, 7, 1) 15 activation_9[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, 7, 15) 0 add_10[0][0]
__________________________________________________________________________________________________
add_11 (Add) (None, 7, 15) 0 repeat_vector_11[0][0]
eq_8_mul_11_1[0][0]
__________________________________________________________________________________________________
repeat_vector_12 (RepeatVector) (None, 7, 15) 0 enc_We_1[12][0]
__________________________________________________________________________________________________
eq_8_mul_12_1 (MyDot) (None, 7, 15) 225 my_transpose_12[0][0]
__________________________________________________________________________________________________
my_transpose_13 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
concatenate_14 (Concatenate) (None, 124) 0 encoder_LSTM_1[13][0]
encoder_LSTM_1[13][2]
__________________________________________________________________________________________________
attn_weight_7_1 (Concatenate) (None, 8, 7) 0 attn_weight_6_1[0][0]
eq_9_7_1[0][0]
__________________________________________________________________________________________________
eq_9_8_1 (Activation) (None, 1, 7) 0 eq_8_8_1[0][0]
__________________________________________________________________________________________________
eq_8_9_1 (MyTranspose) (None, 1, 7) 0 my_dot_9[0][0]
__________________________________________________________________________________________________
my_dot_10 (MyDot) (None, 7, 1) 15 activation_10[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, 7, 15) 0 add_11[0][0]
__________________________________________________________________________________________________
add_12 (Add) (None, 7, 15) 0 repeat_vector_12[0][0]
eq_8_mul_12_1[0][0]
__________________________________________________________________________________________________
repeat_vector_13 (RepeatVector) (None, 7, 15) 0 enc_We_1[13][0]
__________________________________________________________________________________________________
eq_8_mul_13_1 (MyDot) (None, 7, 15) 225 my_transpose_13[0][0]
__________________________________________________________________________________________________
my_transpose_14 (MyTranspose) (None, 7, 15) 0 enc_input1[0][0]
__________________________________________________________________________________________________
attn_weight_8_1 (Concatenate) (None, 9, 7) 0 attn_weight_7_1[0][0]
eq_9_8_1[0][0]
__________________________________________________________________________________________________
eq_9_9_1 (Activation) (None, 1, 7) 0 eq_8_9_1[0][0]
__________________________________________________________________________________________________
eq_8_10_1 (MyTranspose) (None, 1, 7) 0 my_dot_10[0][0]
__________________________________________________________________________________________________
my_dot_11 (MyDot) (None, 7, 1) 15 activation_11[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, 7, 15) 0 add_12[0][0]
__________________________________________________________________________________________________
add_13 (Add) (None, 7, 15) 0 repeat_vector_13[0][0]
eq_8_mul_13_1[0][0]
__________________________________________________________________________________________________
repeat_vector_14 (RepeatVector) (None, 7, 15) 0 enc_We_1[14][0]
__________________________________________________________________________________________________
eq_8_mul_14_1 (MyDot) (None, 7, 15) 225 my_transpose_14[0][0]
__________________________________________________________________________________________________
attn_weight_9_1 (Concatenate) (None, 10, 7) 0 attn_weight_8_1[0][0]
eq_9_9_1[0][0]
__________________________________________________________________________________________________
eq_9_10_1 (Activation) (None, 1, 7) 0 eq_8_10_1[0][0]
__________________________________________________________________________________________________
eq_8_11_1 (MyTranspose) (None, 1, 7) 0 my_dot_11[0][0]
__________________________________________________________________________________________________
my_dot_12 (MyDot) (None, 7, 1) 15 activation_12[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, 7, 15) 0 add_13[0][0]
__________________________________________________________________________________________________
add_14 (Add) (None, 7, 15) 0 repeat_vector_14[0][0]
eq_8_mul_14_1[0][0]
__________________________________________________________________________________________________
attn_weight_10_1 (Concatenate) (None, 11, 7) 0 attn_weight_9_1[0][0]
eq_9_10_1[0][0]
__________________________________________________________________________________________________
eq_9_11_1 (Activation) (None, 1, 7) 0 eq_8_11_1[0][0]
__________________________________________________________________________________________________
eq_8_12_1 (MyTranspose) (None, 1, 7) 0 my_dot_12[0][0]
__________________________________________________________________________________________________
my_dot_13 (MyDot) (None, 7, 1) 15 activation_13[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, 7, 15) 0 add_14[0][0]
__________________________________________________________________________________________________
attn_weight_11_1 (Concatenate) (None, 12, 7) 0 attn_weight_10_1[0][0]
eq_9_11_1[0][0]
__________________________________________________________________________________________________
eq_9_12_1 (Activation) (None, 1, 7) 0 eq_8_12_1[0][0]
__________________________________________________________________________________________________
eq_8_13_1 (MyTranspose) (None, 1, 7) 0 my_dot_13[0][0]
__________________________________________________________________________________________________
my_dot_14 (MyDot) (None, 7, 1) 15 activation_14[0][0]
__________________________________________________________________________________________________
attn_weight_12_1 (Concatenate) (None, 13, 7) 0 attn_weight_11_1[0][0]
eq_9_12_1[0][0]
__________________________________________________________________________________________________
eq_9_13_1 (Activation) (None, 1, 7) 0 eq_8_13_1[0][0]
__________________________________________________________________________________________________
eq_8_14_1 (MyTranspose) (None, 1, 7) 0 my_dot_14[0][0]
__________________________________________________________________________________________________
attn_weight_13_1 (Concatenate) (None, 14, 7) 0 attn_weight_12_1[0][0]
eq_9_13_1[0][0]
__________________________________________________________________________________________________
eq_9_14_1 (Activation) (None, 1, 7) 0 eq_8_14_1[0][0]
__________________________________________________________________________________________________
attn_weight_14_1 (Concatenate) (None, 15, 7) 0 attn_weight_13_1[0][0]
eq_9_14_1[0][0]
__________________________________________________________________________________________________
enc_output_1 (Multiply) (None, 15, 7) 0 attn_weight_14_1[0][0]
enc_input1[0][0]
__________________________________________________________________________________________________
enc_lstm_input_1 (Reshape) (None, 15, 7) 0 enc_output_1[0][0]
__________________________________________________________________________________________________
LSTM_after_encoder_1 (LSTM) (None, 62) 17360 enc_lstm_input_1[0][0]
__________________________________________________________________________________________________
leaky_re_lu (LeakyReLU) (None, 62) 0 LSTM_after_encoder_1[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 1) 63 leaky_re_lu[0][0]
==================================================================================================
Total params: 40,258
Trainable params: 40,258
Non-trainable params: 0
__________________________________________________________________________________________________
********** Successfully loaded weights from weights_038_0.32582.hdf5 file **********
[4]:
dataset = CAMELS_AUS(path="F:\\data\\CAMELS\\CAMELS_AUS")
inputs = ['et_morton_point_SILO',
'precipitation_AWAP',
'tmax_AWAP',
'tmin_AWAP',
'vprp_AWAP',
'rh_tmax_SILO',
'rh_tmin_SILO'
]
outputs = ['streamflow_MLd']
data = dataset.fetch('401203', dynamic_features=inputs+outputs, as_dataframe=True)
data = data.unstack()
data.columns = [a[1] for a in data.columns.to_flat_index()]
data.shape
[4]:
(21184, 8)
[5]:
model.evaluate_on_test_data(data=data, metrics=["r2", "nse", "rmse"])
***** Test *****
input_x shape: (6352, 15, 7)
target shape: (6352, 1)
input_x shape: [(6352, 15, 7), (6352, 62), (6352, 62)]
target shape: (6352, 1)
D:\mytools\AI4Water\ai4water\_main.py:2083: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
6352/6352 [==============================] - 3s 432us/sample
[5]:
{'r2': 0.6044269510974395, 'nse': 0.5990882468044689, 'rmse': 795.323322617915}
TFT
[6]:
config_path = os.path.join(os.getcwd(), "results", "20230228_094829", "config.json")
model = Model.from_config_file(config_path)
model.update_weights(os.path.join(model.path, "weights", "weights_695_0.01510.hdf5"))
building DL model for
regression problem using Model
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Input (InputLayer) [(None, 15, 7)] 0
__________________________________________________________________________________________________
tf_op_layer_strided_slice (Tens [(None, 15, 7)] 0 Input[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_1 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_2 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_3 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_4 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_5 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_6 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_7 (Te [(None, 15, 1)] 0 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
time_distributed (TimeDistribut (None, 15, 80) 160 tf_op_layer_strided_slice_1[0][0]
__________________________________________________________________________________________________
time_distributed_1 (TimeDistrib (None, 15, 80) 160 tf_op_layer_strided_slice_2[0][0]
__________________________________________________________________________________________________
time_distributed_2 (TimeDistrib (None, 15, 80) 160 tf_op_layer_strided_slice_3[0][0]
__________________________________________________________________________________________________
time_distributed_3 (TimeDistrib (None, 15, 80) 160 tf_op_layer_strided_slice_4[0][0]
__________________________________________________________________________________________________
time_distributed_4 (TimeDistrib (None, 15, 80) 160 tf_op_layer_strided_slice_5[0][0]
__________________________________________________________________________________________________
time_distributed_5 (TimeDistrib (None, 15, 80) 160 tf_op_layer_strided_slice_6[0][0]
__________________________________________________________________________________________________
time_distributed_6 (TimeDistrib (None, 15, 80) 160 tf_op_layer_strided_slice_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_stack (TensorFlowOp [(None, 15, 80, 7)] 0 time_distributed[0][0]
time_distributed_1[0][0]
time_distributed_2[0][0]
time_distributed_3[0][0]
time_distributed_4[0][0]
time_distributed_5[0][0]
time_distributed_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_8 (Te [(None, 15, 80, 7)] 0 tf_op_layer_stack[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape (TensorFlow [(None, 15, 560)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
ff_GRN_with_history (Dense) (None, 15, 80) 44880 tf_op_layer_Reshape[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_9 (Te [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_10 (T [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_11 (T [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_12 (T [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_13 (T [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_14 (T [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_15 (T [(None, 15, 80)] 0 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
GRN_with_history_elu (Activatio (None, 15, 80) 0 ff_GRN_with_history[0][0]
__________________________________________________________________________________________________
ff_GRN_with_history_for_0 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_9[0][0]
__________________________________________________________________________________________________
ff_GRN_with_history_for_1 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_10[0][0
__________________________________________________________________________________________________
ff_GRN_with_history_for_2 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_11[0][0
__________________________________________________________________________________________________
ff_GRN_with_history_for_3 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_12[0][0
__________________________________________________________________________________________________
ff_GRN_with_history_for_4 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_13[0][0
__________________________________________________________________________________________________
ff_GRN_with_history_for_5 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_14[0][0
__________________________________________________________________________________________________
ff_GRN_with_history_for_6 (Dens (None, 15, 80) 6480 tf_op_layer_strided_slice_15[0][0
__________________________________________________________________________________________________
GRN_with_history_LastDense (Den (None, 15, 80) 6480 GRN_with_history_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_0_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_0[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_1_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_1[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_2_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_2[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_3_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_3[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_4_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_4[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_5_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_5[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_6_elu (Act (None, 15, 80) 0 ff_GRN_with_history_for_6[0][0]
__________________________________________________________________________________________________
Dropout_GRN_with_history (Dropo (None, 15, 80) 0 GRN_with_history_LastDense[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_0_LastDens (None, 15, 80) 6480 GRN_with_history_for_0_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_1_LastDens (None, 15, 80) 6480 GRN_with_history_for_1_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_2_LastDens (None, 15, 80) 6480 GRN_with_history_for_2_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_3_LastDens (None, 15, 80) 6480 GRN_with_history_for_3_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_4_LastDens (None, 15, 80) 6480 GRN_with_history_for_4_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_5_LastDens (None, 15, 80) 6480 GRN_with_history_for_5_elu[0][0]
__________________________________________________________________________________________________
GRN_with_history_for_6_LastDens (None, 15, 80) 6480 GRN_with_history_for_6_elu[0][0]
__________________________________________________________________________________________________
gating_act_GRN_with_history (De (None, 15, 7) 567 Dropout_GRN_with_history[0][0]
__________________________________________________________________________________________________
gating_GRN_with_history (Dense) (None, 15, 7) 567 Dropout_GRN_with_history[0][0]
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_0 (None, 15, 80) 0 GRN_with_history_for_0_LastDense[
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_1 (None, 15, 80) 0 GRN_with_history_for_1_LastDense[
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_2 (None, 15, 80) 0 GRN_with_history_for_2_LastDense[
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_3 (None, 15, 80) 0 GRN_with_history_for_3_LastDense[
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_4 (None, 15, 80) 0 GRN_with_history_for_4_LastDense[
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_5 (None, 15, 80) 0 GRN_with_history_for_5_LastDense[
__________________________________________________________________________________________________
Dropout_GRN_with_history_for_6 (None, 15, 80) 0 GRN_with_history_for_6_LastDense[
__________________________________________________________________________________________________
skip_connection_GRN_with_histor (None, 15, 7) 3927 tf_op_layer_Reshape[0][0]
__________________________________________________________________________________________________
MulGating_GRN_with_history (Mul (None, 15, 7) 0 gating_act_GRN_with_history[0][0]
gating_GRN_with_history[0][0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_0[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_0 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_0[0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_1[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_1 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_1[0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_2[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_2 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_2[0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_3[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_3 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_3[0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_4[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_4 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_4[0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_5[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_5 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_5[0]
__________________________________________________________________________________________________
gating_act_GRN_with_history_for (None, 15, 80) 6480 Dropout_GRN_with_history_for_6[0]
__________________________________________________________________________________________________
gating_GRN_with_history_for_6 ( (None, 15, 80) 6480 Dropout_GRN_with_history_for_6[0]
__________________________________________________________________________________________________
add_GRN_with_history (Add) (None, 15, 7) 0 skip_connection_GRN_with_history[
MulGating_GRN_with_history[0][0]
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_0
gating_GRN_with_history_for_0[0][
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_1
gating_GRN_with_history_for_1[0][
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_2
gating_GRN_with_history_for_2[0][
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_3
gating_GRN_with_history_for_3[0][
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_4
gating_GRN_with_history_for_4[0][
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_5
gating_GRN_with_history_for_5[0][
__________________________________________________________________________________________________
MulGating_GRN_with_history_for_ (None, 15, 80) 0 gating_act_GRN_with_history_for_6
gating_GRN_with_history_for_6[0][
__________________________________________________________________________________________________
norm_GRN_with_history (LayerNor (None, 15, 7) 14 add_GRN_with_history[0][0]
__________________________________________________________________________________________________
add_GRN_with_history_for_0 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_9[0][0]
MulGating_GRN_with_history_for_0[
__________________________________________________________________________________________________
add_GRN_with_history_for_1 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_10[0][0
MulGating_GRN_with_history_for_1[
__________________________________________________________________________________________________
add_GRN_with_history_for_2 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_11[0][0
MulGating_GRN_with_history_for_2[
__________________________________________________________________________________________________
add_GRN_with_history_for_3 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_12[0][0
MulGating_GRN_with_history_for_3[
__________________________________________________________________________________________________
add_GRN_with_history_for_4 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_13[0][0
MulGating_GRN_with_history_for_4[
__________________________________________________________________________________________________
add_GRN_with_history_for_5 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_14[0][0
MulGating_GRN_with_history_for_5[
__________________________________________________________________________________________________
add_GRN_with_history_for_6 (Add (None, 15, 80) 0 tf_op_layer_strided_slice_15[0][0
MulGating_GRN_with_history_for_6[
__________________________________________________________________________________________________
sparse_history_weights_softmax (None, 15, 7) 0 norm_GRN_with_history[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_0 (La (None, 15, 80) 160 add_GRN_with_history_for_0[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_1 (La (None, 15, 80) 160 add_GRN_with_history_for_1[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_2 (La (None, 15, 80) 160 add_GRN_with_history_for_2[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_3 (La (None, 15, 80) 160 add_GRN_with_history_for_3[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_4 (La (None, 15, 80) 160 add_GRN_with_history_for_4[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_5 (La (None, 15, 80) 160 add_GRN_with_history_for_5[0][0]
__________________________________________________________________________________________________
norm_GRN_with_history_for_6 (La (None, 15, 80) 160 add_GRN_with_history_for_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_ExpandDims (TensorF [(None, 15, 1, 7)] 0 sparse_history_weights_softmax[0]
__________________________________________________________________________________________________
tf_op_layer_stack_1 (TensorFlow [(None, 15, 80, 7)] 0 norm_GRN_with_history_for_0[0][0]
norm_GRN_with_history_for_1[0][0]
norm_GRN_with_history_for_2[0][0]
norm_GRN_with_history_for_3[0][0]
norm_GRN_with_history_for_4[0][0]
norm_GRN_with_history_for_5[0][0]
norm_GRN_with_history_for_6[0][0]
__________________________________________________________________________________________________
sparse_and_transform_history (M (None, 15, 80, 7) 0 tf_op_layer_ExpandDims[0][0]
tf_op_layer_stack_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sum (TensorFlowOpLa [(None, 15, 80)] 0 sparse_and_transform_history[0][0
__________________________________________________________________________________________________
history (LSTM) [(None, 15, 80), (No 51520 tf_op_layer_Sum[0][0]
__________________________________________________________________________________________________
Dropout_GatingOnLSTM (Dropout) (None, 15, 80) 0 history[0][0]
__________________________________________________________________________________________________
gating_act_GatingOnLSTM (Dense) (None, 15, 80) 6480 Dropout_GatingOnLSTM[0][0]
__________________________________________________________________________________________________
gating_GatingOnLSTM (Dense) (None, 15, 80) 6480 Dropout_GatingOnLSTM[0][0]
__________________________________________________________________________________________________
MulGating_GatingOnLSTM (Multipl (None, 15, 80) 0 gating_act_GatingOnLSTM[0][0]
gating_GatingOnLSTM[0][0]
__________________________________________________________________________________________________
add_AfterLSTM (Add) (None, 15, 80) 0 MulGating_GatingOnLSTM[0][0]
tf_op_layer_Sum[0][0]
__________________________________________________________________________________________________
norm_AfterLSTM (LayerNormalizat (None, 15, 80) 160 add_AfterLSTM[0][0]
__________________________________________________________________________________________________
ff_GRN_temp_feature (Dense) (None, 15, 80) 6480 norm_AfterLSTM[0][0]
__________________________________________________________________________________________________
GRN_temp_feature_elu (Activatio (None, 15, 80) 0 ff_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
GRN_temp_feature_LastDense (Den (None, 15, 80) 6480 GRN_temp_feature_elu[0][0]
__________________________________________________________________________________________________
Dropout_GRN_temp_feature (Dropo (None, 15, 80) 0 GRN_temp_feature_LastDense[0][0]
__________________________________________________________________________________________________
gating_act_GRN_temp_feature (De (None, 15, 80) 6480 Dropout_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
gating_GRN_temp_feature (Dense) (None, 15, 80) 6480 Dropout_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
MulGating_GRN_temp_feature (Mul (None, 15, 80) 0 gating_act_GRN_temp_feature[0][0]
gating_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
add_GRN_temp_feature (Add) (None, 15, 80) 0 norm_AfterLSTM[0][0]
MulGating_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
norm_GRN_temp_feature (LayerNor (None, 15, 80) 160 add_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape (TensorFlowOp [(3,)] 0 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_16 (T [()] 0 tf_op_layer_Shape[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape_1 (TensorFlow [(3,)] 0 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_eye/Minimum (Tensor [()] 0 tf_op_layer_strided_slice_16[0][0
tf_op_layer_strided_slice_16[0][0
__________________________________________________________________________________________________
dense_2 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_17 (T [(1,)] 0 tf_op_layer_Shape_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_eye/concat/values_1 [(1,)] 0 tf_op_layer_eye/Minimum[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
dense_8 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
dense_10 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
dense_12 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape_2 (TensorFlow [(3,)] 0 dense_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_eye/concat (TensorF [(2,)] 0 tf_op_layer_strided_slice_17[0][0
tf_op_layer_eye/concat/values_1[0
__________________________________________________________________________________________________
tf_op_layer_Shape_3 (TensorFlow [(3,)] 0 dense_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape_4 (TensorFlow [(3,)] 0 dense_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape_5 (TensorFlow [(3,)] 0 dense_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape_6 (TensorFlow [(3,)] 0 dense_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_Shape_7 (TensorFlow [(3,)] 0 dense_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_18 (T [()] 0 tf_op_layer_Shape_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_eye/ones (TensorFlo [(None, None)] 0 tf_op_layer_eye/concat[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_19 (T [()] 0 tf_op_layer_Shape_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_20 (T [()] 0 tf_op_layer_Shape_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_21 (T [()] 0 tf_op_layer_Shape_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_22 (T [()] 0 tf_op_layer_Shape_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_23 (T [()] 0 tf_op_layer_Shape_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_Cast (TensorFlowOpL [()] 0 tf_op_layer_strided_slice_18[0][0
__________________________________________________________________________________________________
tf_op_layer_eye/diag (TensorFlo [(None, None, None)] 0 tf_op_layer_eye/ones[0][0]
__________________________________________________________________________________________________
tf_op_layer_Cast_1 (TensorFlowO [()] 0 tf_op_layer_strided_slice_19[0][0
__________________________________________________________________________________________________
tf_op_layer_Cast_2 (TensorFlowO [()] 0 tf_op_layer_strided_slice_20[0][0
__________________________________________________________________________________________________
tf_op_layer_Cast_3 (TensorFlowO [()] 0 tf_op_layer_strided_slice_21[0][0
__________________________________________________________________________________________________
tf_op_layer_Cast_4 (TensorFlowO [()] 0 tf_op_layer_strided_slice_22[0][0
__________________________________________________________________________________________________
tf_op_layer_Cast_5 (TensorFlowO [()] 0 tf_op_layer_strided_slice_23[0][0
__________________________________________________________________________________________________
dense_1 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sqrt (TensorFlowOpL [()] 0 tf_op_layer_Cast[0][0]
__________________________________________________________________________________________________
tf_op_layer_Cumsum (TensorFlowO [(None, None, None)] 0 tf_op_layer_eye/diag[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sqrt_1 (TensorFlowO [()] 0 tf_op_layer_Cast_1[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sqrt_2 (TensorFlowO [()] 0 tf_op_layer_Cast_2[0][0]
__________________________________________________________________________________________________
dense_7 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sqrt_3 (TensorFlowO [()] 0 tf_op_layer_Cast_3[0][0]
__________________________________________________________________________________________________
dense_9 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sqrt_4 (TensorFlowO [()] 0 tf_op_layer_Cast_4[0][0]
__________________________________________________________________________________________________
dense_11 (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sqrt_5 (TensorFlowO [()] 0 tf_op_layer_Cast_5[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambda0 (MyLa (None, 15, 15) 0 dense_1[0][0]
dense_2[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambdaMask0 ( (None, None, None) 0 tf_op_layer_Cumsum[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambda1 (MyLa (None, 15, 15) 0 dense_3[0][0]
dense_4[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambdaMask1 ( (None, None, None) 0 tf_op_layer_Cumsum[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambda2 (MyLa (None, 15, 15) 0 dense_5[0][0]
dense_6[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambdaMask2 ( (None, None, None) 0 tf_op_layer_Cumsum[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambda3 (MyLa (None, 15, 15) 0 dense_7[0][0]
dense_8[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambdaMask3 ( (None, None, None) 0 tf_op_layer_Cumsum[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambda4 (MyLa (None, 15, 15) 0 dense_9[0][0]
dense_10[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambdaMask4 ( (None, None, None) 0 tf_op_layer_Cumsum[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambda5 (MyLa (None, 15, 15) 0 dense_11[0][0]
dense_12[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenLambdaMask5 ( (None, None, None) 0 tf_op_layer_Cumsum[0][0]
__________________________________________________________________________________________________
SDPA_ADD_0 (Add) (None, 15, 15) 0 ScaledDotProdAttenLambda0[0][0]
ScaledDotProdAttenLambdaMask0[0][
__________________________________________________________________________________________________
SDPA_ADD_1 (Add) (None, 15, 15) 0 ScaledDotProdAttenLambda1[0][0]
ScaledDotProdAttenLambdaMask1[0][
__________________________________________________________________________________________________
SDPA_ADD_2 (Add) (None, 15, 15) 0 ScaledDotProdAttenLambda2[0][0]
ScaledDotProdAttenLambdaMask2[0][
__________________________________________________________________________________________________
SDPA_ADD_3 (Add) (None, 15, 15) 0 ScaledDotProdAttenLambda3[0][0]
ScaledDotProdAttenLambdaMask3[0][
__________________________________________________________________________________________________
SDPA_ADD_4 (Add) (None, 15, 15) 0 ScaledDotProdAttenLambda4[0][0]
ScaledDotProdAttenLambdaMask4[0][
__________________________________________________________________________________________________
SDPA_ADD_5 (Add) (None, 15, 15) 0 ScaledDotProdAttenLambda5[0][0]
ScaledDotProdAttenLambdaMask5[0][
__________________________________________________________________________________________________
ScaledDotProdAtten_softmax (Act (None, 15, 15) 0 SDPA_ADD_0[0][0]
SDPA_ADD_1[0][0]
SDPA_ADD_2[0][0]
SDPA_ADD_3[0][0]
SDPA_ADD_4[0][0]
SDPA_ADD_5[0][0]
__________________________________________________________________________________________________
ScaledDotProdAtten_dropout (Dro (None, 15, 15) 0 ScaledDotProdAtten_softmax[0][0]
ScaledDotProdAtten_softmax[1][0]
ScaledDotProdAtten_softmax[2][0]
ScaledDotProdAtten_softmax[3][0]
ScaledDotProdAtten_softmax[4][0]
ScaledDotProdAtten_softmax[5][0]
__________________________________________________________________________________________________
dense (Dense) (None, 15, 13) 1040 norm_GRN_temp_feature[0][0]
norm_GRN_temp_feature[0][0]
norm_GRN_temp_feature[0][0]
norm_GRN_temp_feature[0][0]
norm_GRN_temp_feature[0][0]
norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenOutput0 (Lamb (None, 15, 13) 0 ScaledDotProdAtten_dropout[0][0]
dense[0][0]
__________________________________________________________________________________________________
ScaledDotProdAttenOutput1 (Lamb (None, 15, 13) 0 ScaledDotProdAtten_dropout[1][0]
dense[1][0]
__________________________________________________________________________________________________
ScaledDotProdAttenOutput2 (Lamb (None, 15, 13) 0 ScaledDotProdAtten_dropout[2][0]
dense[2][0]
__________________________________________________________________________________________________
ScaledDotProdAttenOutput3 (Lamb (None, 15, 13) 0 ScaledDotProdAtten_dropout[3][0]
dense[3][0]
__________________________________________________________________________________________________
ScaledDotProdAttenOutput4 (Lamb (None, 15, 13) 0 ScaledDotProdAtten_dropout[4][0]
dense[4][0]
__________________________________________________________________________________________________
ScaledDotProdAttenOutput5 (Lamb (None, 15, 13) 0 ScaledDotProdAtten_dropout[5][0]
dense[5][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 15, 13) 0 ScaledDotProdAttenOutput0[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 15, 13) 0 ScaledDotProdAttenOutput1[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 15, 13) 0 ScaledDotProdAttenOutput2[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 15, 13) 0 ScaledDotProdAttenOutput3[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, 15, 13) 0 ScaledDotProdAttenOutput4[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, 15, 13) 0 ScaledDotProdAttenOutput5[0][0]
__________________________________________________________________________________________________
tf_op_layer_MultiHeadAtten_head [(6, None, 15, 13)] 0 dropout[0][0]
dropout_1[0][0]
dropout_2[0][0]
dropout_3[0][0]
dropout_4[0][0]
dropout_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [(None, 15, 13)] 0 tf_op_layer_MultiHeadAtten_heads[
__________________________________________________________________________________________________
MH_atten_output (Dense) (None, 15, 80) 1040 tf_op_layer_Mean[0][0]
__________________________________________________________________________________________________
MHA_output_do (Dropout) (None, 15, 80) 0 MH_atten_output[0][0]
__________________________________________________________________________________________________
Dropout_GatingOnX (Dropout) (None, 15, 80) 0 MHA_output_do[0][0]
__________________________________________________________________________________________________
gating_act_GatingOnX (Dense) (None, 15, 80) 6480 Dropout_GatingOnX[0][0]
__________________________________________________________________________________________________
gating_GatingOnX (Dense) (None, 15, 80) 6480 Dropout_GatingOnX[0][0]
__________________________________________________________________________________________________
MulGating_GatingOnX (Multiply) (None, 15, 80) 0 gating_act_GatingOnX[0][0]
gating_GatingOnX[0][0]
__________________________________________________________________________________________________
add_XAndEnriched (Add) (None, 15, 80) 0 MulGating_GatingOnX[0][0]
norm_GRN_temp_feature[0][0]
__________________________________________________________________________________________________
norm_XAndEnriched (LayerNormali (None, 15, 80) 160 add_XAndEnriched[0][0]
__________________________________________________________________________________________________
ff_NonLinearityOnOut (Dense) (None, 15, 80) 6480 norm_XAndEnriched[0][0]
__________________________________________________________________________________________________
NonLinearityOnOut_elu (Activati (None, 15, 80) 0 ff_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
NonLinearityOnOut_LastDense (De (None, 15, 80) 6480 NonLinearityOnOut_elu[0][0]
__________________________________________________________________________________________________
Dropout_NonLinearityOnOut (Drop (None, 15, 80) 0 NonLinearityOnOut_LastDense[0][0]
__________________________________________________________________________________________________
gating_act_NonLinearityOnOut (D (None, 15, 80) 6480 Dropout_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
gating_NonLinearityOnOut (Dense (None, 15, 80) 6480 Dropout_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
MulGating_NonLinearityOnOut (Mu (None, 15, 80) 0 gating_act_NonLinearityOnOut[0][0
gating_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
add_NonLinearityOnOut (Add) (None, 15, 80) 0 norm_XAndEnriched[0][0]
MulGating_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
norm_NonLinearityOnOut (LayerNo (None, 15, 80) 160 add_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
gating_act_FinalSkip (Dense) (None, 15, 80) 6480 norm_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
gating_FinalSkip (Dense) (None, 15, 80) 6480 norm_NonLinearityOnOut[0][0]
__________________________________________________________________________________________________
MulGating_FinalSkip (Multiply) (None, 15, 80) 0 gating_act_FinalSkip[0][0]
gating_FinalSkip[0][0]
__________________________________________________________________________________________________
add_DecoderAndTempFeature (Add) (None, 15, 80) 0 MulGating_FinalSkip[0][0]
norm_AfterLSTM[0][0]
__________________________________________________________________________________________________
norm_DecoderAndTempFeature (Lay (None, 15, 80) 160 add_DecoderAndTempFeature[0][0]
__________________________________________________________________________________________________
lambda (Lambda) (None, 80) 0 norm_DecoderAndTempFeature[0][0]
__________________________________________________________________________________________________
Dense_out (Dense) (None, 1) 81 lambda[0][0]
==================================================================================================
Total params: 397,796
Trainable params: 397,796
Non-trainable params: 0
__________________________________________________________________________________________________
"dot" with args ['-Tpng', 'C:\\Users\\USER\\AppData\\Local\\Temp\\tmpxnpzzce_'] returned code: 3221225477
stdout, stderr:
b''
b''
dot plot of model could not be plotted due to "dot" with args ['-Tpng', 'C:\\Users\\USER\\AppData\\Local\\Temp\\tmpxnpzzce_'] returned code: 3221225477
********** Successfully loaded weights from weights_695_0.01510.hdf5 file **********
[7]:
model.evaluate_on_test_data(data=data, metrics=["r2", "nse", "rmse"])
***** Test *****
input_x shape: (6352, 15, 7)
target shape: (6352, 1)
6352/6352 [==============================] - 4s 649us/sample
[7]:
{'r2': 0.6147775425572483,
'nse': 0.6143773584544838,
'rmse': 780.0106970420342}
[ ]: