Variational Inference: Simple Example (+ Python Demo)

Variational Bayesian Methods can be difficult to understand. In this video, we will look at the simple Exponential-Normal model for which the posterior is intractable. We will show why and then propose a surrogate and perform VI. Here are the notes: https://github.com/Ceyron/machine-learning-and-simulation/blob/main/english/probabilistic_machine_learning/vi_simple_example_exponential_normal_model.pdf Find the visualization here: https://share.streamlit.io/ceyron/machine-learning-and-simulation/main/english/probabilistic_machine_learning/vi_exponential_normal_model_visualization.py Variational Inference is a powerful technique in Machine Learning that is used to find approximate posteriors for generative models. In particular, it is being used extensively for Variational Autoencoders. ------- 📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation 📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler 💸 : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim ------- Timestamps: 00:00 Introduction 00:38 Agenda 01:30 Joint distribution 04:49 Trying to find the true posterior (and fail) 14:45 Visualization (Joint, Posterior & Surrogate) 19:05 Recap: Variational Inference & ELBO 21:56 Introducing a parametric surrogate posterior 24:45 Remark: Approximating the ELBO by sampling 27:24 Performing Variational Inference (Optimizing ELBO) 38:38 Python example with TensorFlow Probability 47:09 Outro