Rust Web Development (Search Engine Ep.02)

References: - Source Code: https://github.com/tsoding/seroost Timestamps: 00:00:00 - Intro 00:00:42 - TF-IDF Recap 00:02:41 - What we are developing 00:03:10 - Web UI 00:03:29 - Syncthing 00:04:18 - About Electron 00:04:41 - What I wanted to do today 00:05:03 - RuSt Is ThE mOsT lIkEd LaNgUaGe 00:05:34 - Subs 00:06:21 - Picking the Web Framework 00:07:46 - tiny-http 00:09:54 - Adding tiny-http to the dependencies 00:11:04 - Introducing subcommands 00:13:08 - Recursive indexing 00:15:19 - Subs 00:15:29 - Plans to replace JSON with Sqlite 00:16:14 - Creating a new subcommand 00:17:41 - English is weird 00:18:53 - Starting up the Server 00:21:08 - Handling incoming Requests 00:24:39 - The Power of Simplicity 00:25:32 - Serving HTML 00:26:07 - Google y r u so bad? 00:27:21 - Setting up correct Content-Type 00:31:03 - First Try God Cooder *dab-dab-dab-dab-dab* 00:31:48 - Please don't take me seriously 00:32:50 - Designing the query form 00:33:38 - Unbaking the HTML 00:36:46 - How I handle errors these days 00:40:36 - tiny-http Response ownership approach 00:41:14 - Finishing HTML unbaking and fixing compilation errors 00:43:25 - Changed the shirt and ready to implement the router 00:47:09 - Rust developers are Java developers in disguise 00:47:52 - Continue implementing router 00:49:26 - Easy route aliases 00:49:51 - Adding index.js 00:53:06 - Factoring out static files serving 00:55:41 - Why do I copy-paste code so much 00:56:04 - Separation of different method 00:56:48 - Factoring out serving 404 00:57:22 - Rambling about code aesthetics 00:57:47 - Going through compilation errors 00:58:29 - Our small Web Framework 00:58:58 - Subs 00:59:26 - REST API for Search 01:00:05 - Getting the Body of HTTP Request 01:01:54 - Reading everything from Reader 01:03:03 - Struggling to convert bytes to string 'cause Rust 01:06:33 - Fixing compilation errors 01:07:36 - JavaScript fetch() 01:10:27 - Got some data from the Client! 01:10:52 - Refactoring JavaScript code 01:11:17 - Designing the search query format 01:12:33 - Being Web Developer is hard! 01:13:18 - Tokenizing Search Query 01:14:48 - Tokenizer must convert characters to upper case 01:15:46 - Subs 01:16:01 - Compiler Assisted Refactoring 01:18:44 - Tokenizer can handle punctuation 01:19:12 - Loading up the Document Index 01:21:26 - TF-IDF recap 01:23:04 - Implementing tf computation 01:29:11 - Constructing a smaller set of documents for testing 01:31:11 - Computing tf for each document 01:35:31 - Sorting documents by TF 01:37:12 - Rustaceans are scared of floats lol 01:39:32 - TF reflects the relevancy! 01:40:08 - Studying IDF 01:42:03 - Rediscovering logarithmic scale 01:44:20 - Math is like Programming from alternative universe 01:44:56 - Implementing IDF computation 01:46:37 - Suskell 01:49:30 - Combining TF and IDF 01:50:39 - Rustaceans were right! Floats are scary! 01:51:51 - Negative IDF bug 01:56:23 - Is that a mistake in Wikipedia?! 01:57:56 - Fixing the denominator adjustment 01:59:37 - TF-IDF works! 02:00:43 - Computing final ranking 02:02:26 - Testing on bigger data 02:06:37 - Trying other queries 02:08:23 - We need stemming 02:09:18 - The importance of owning your data 02:10:02 - What is stemming 02:11:02 - Performance sucks and I don't know why 02:12:14 - UI/UX improvements ideas 02:13:39 - --release 02:15:23 - Outro 02:15:33 - Smooch