Factorized Self Attention Explained for Time Series AI

In this video, I break down how transformer models can be adapted for time series data through factorized self attention. Learn how this specialized architecture processes financial time series and multivariate data. Timestamps: [00:56] Turning time series into embeddings through patching [01:42] Embedding techniques and projecting patches into higher dimensions [02:47] Processing multivariate time series with multiple symbols [04:12] Understanding self attention fundamentals [04:58] Query, key, and value vectors explained [05:42] Using dot products and softmax to calculate attention weights [06:45] Time-wise vs. space-wise attention in factorized self attention [07:52] Computing time-wise attention with code walkthrough [08:56] Reshaping tensors for different attention mechanisms [09:38] Space-wise attention implementation [10:47] Reshaping context vectors between attention blocks All code demonstrated in this video is available on GitHub (https://github.com/bitsofchris/deep-learning/tree/main/code/09_time-series-ai-learning). For more content on time series machine learning and AI research, visit https://bitsofchris.com.