Contrastive Learning on Text Embeddings | Full Code with Sarcasm Dataset

We will explore contrastive learning on BERT embeddings in this video. Specifically, we will take a look at what is contrastive learning and use triplet lossand random sampling for our code demonstration.

Optional CC is available.

00:00 - What is Contrastive Learning
5:10 - Start of code, create train test sets
6:35 - Load BERT and how it works
10:27 - Create test embeddings
15:33 - t-SNE on test embeddings
18:13 - Create custom torch dataset
24:32 - Create wrapper BERT model
26:30 - train function
27:20 - Get post fine tuning embeddings
28:25 - t-SNE on fine turned test embeddings
30:04 - Logistic Regression on before and after embeddings




Dataset: https://www.kaggle.com/datasets/rmisra/news-headlines-dataset-for-sarcasm-detection
Notebook: https://colab.research.google.com/drive/1oifY1pLwJMZeMHU0cbtE8YBDfkxDobUS?usp=sharing

Thanks for watching!