LSTM encoder-decoder model

I'd like to make LSTM encoder-decoder model with deep learning toolbox, whichbased on this link(this is for making same model with Keras). I'm trying to make the timeseries prediction(seq2seq).

https://machinelearningmastery.com/how-to-develop-lstm-models-for-multi-step-time-series-forecasting-of-household-power-consumption/

However, the corresponded warper layer fucvtions(ex TimeDistributed, RepeatVector) are not found in the deep learnig tool box.

Is there any solutions to make LSTM encoder-decoder model with Matlab.

[Lisa Huber]

Since 2021a the TimeDistributed Layer is available.

But still I cannot find any RepeatVector layer. I mean the layer is really really simple it just repeats a vector like repmat without any weights or anything to train...

Dear Tomohiro, have you found any solution?

Question to the staff: Is there a way to implement a RepeatVector layer by myself?

[David Willingham]

Hi Lisa, our equivalent to the RepeatVector layer behaviour is to use the fullyconnectedLayer.

David

[angeline yap]

Hi David, i dont see the fullyconnectedLayer is equivalent to RepeatVector as RepeatVector is simple duplicate the produced vector. Can you provide more information of how fullyconnectedlayer can be implemted as repeatVector. Thank you

[Abhishek Shivayya]

Hi Tomohiro, did you find any solution to this problem? even I am finding difficult to develop LSTM encoder-decoder model for sequence2sequence modeling. If you have any refernece code in matlab related to this can you please share me? Thanks in advance.

[Hend ElMohandes]

I've been attempting to do the same , wanted to create an encoder -decoder (seq to seq) for regression using matlab and its been very difficult.

Did you reach any solution or source of help?

Answer 1

[David Willingham]

Hi Tomohiro,

See this example of how to perform Time Series Forecasting using LSTM in MATLAB.

David

[Abhishek Shivayya]

Hello Mr.David, thanks for the reference to perform time series forecasting. I am working on similar concept to reconstruct the time series data using LSTM encoder - decoder based for anomaly detection. I developed different blocks for encoder and decoder and then connected layers with a bridge of fullyconnectedlayer(repeatvector) and output layer with regressionlayer. Once the model is trained on normal data, I am trying to reconstruct for faulty data and I would also like to access the output of an Encoder which encoded into latent space vector (repeatvector). I didnt find any reference how to get the encoder output once the model has been trained. Can you please let me know how to perform these kind of problems, it would be great help.

[David Willingham]

Hi Abhishek,

Check out this example:

Anomaly Detection in Industrial Machinery Using Three-Axis Vibration Data

It uses an autoencoders.

Answer 2

[Marcello Ferro]

Hi Tomohiro,

check this seq-to-seq encoder-decoder model with attention (translation task): https://it.mathworks.com/help/deeplearning/ug/sequence-to-sequence-translation-using-attention.html