MENU

Fun & Interesting

The Google File System - Paper Explained

Arpit Bhayani 42,240 1 year ago
Video Not Working? Fix It Now

System Design for SDE-2 and above: https://arpitbhayani.me/masterclass System Design for Beginners: https://arpitbhayani.me/sys-design Redis Internals: https://arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - https://app.codecrafters.io/join?via=arpitbbhayani In the video, I dissected the Google File System research paper from 2003, highlighting its key aspects. The system is designed to run on commodity hardware, making it scalable and fault-tolerant. Google optimized for storing large files, focusing on append operations rather than overwrites. They also emphasized fault detection, fault tolerance, and automated recovery. The system divided files into chunks for parallelism, load balancing, fault tolerance, and easy storage allocation. Replication factor of 3 ensured fault tolerance, with a master node managing metadata. # Recommended videos and playlists If you liked this video, you will find the following videos and playlists helpful System Design: https://www.youtube.com/watch?v=o7qLKfILuD8&list=PLsdq-3Z1EPT27BuTnJ_trF7BsaTpYLqst Designing Microservices: https://www.youtube.com/watch?v=JPj6mhVLQN0&list=PLsdq-3Z1EPT0ug8eizS71G6LZb6-4FAFt Database Engineering: https://www.youtube.com/watch?v=-htbah3eCYg&list=PLsdq-3Z1EPT2C-Da7Jscr7NptGcIZgQ2l&pp=gAQBiAQB Concurrency In-depth: https://www.youtube.com/watch?v=2PjlaUnrAMQ&list=PLsdq-3Z1EPT3VjDhjMb5yBsgn0wn2-fjp Research paper dissections: https://www.youtube.com/watch?v=LXhgFAZroG8&list=PLsdq-3Z1EPT2XEJ0AmF02LBK1RFNd-jK8 Outage Dissections: https://www.youtube.com/watch?v=LeT_s-UFw-U&list=PLsdq-3Z1EPT3_Z97svMs6S2y7tv1PcUmc Hash Table Internals: https://www.youtube.com/watch?v=jjW8w8ED3Ns&list=PLsdq-3Z1EPT2UnueESBLReaVSLIo_BuAc Bittorrent Internals: https://www.youtube.com/watch?v=v7cR0ZolaUA&list=PLsdq-3Z1EPT1rNeq2GXpnivaWINnOaCd0 # Things you will find amusing Knowledge Base: https://arpitbhayani.me/knowledge-base Bookshelf: https://arpitbhayani.me/bookshelf Papershelf: https://arpitbhayani.me/papershelf # Other socials I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff. LinkedIn: https://linkedin.com/in/arpitbhayani Twitter: https://twitter.com/arpit_bhayani Weekly Newsletter: https://arpit.substack.com Thank you for watching and supporting! it means a ton. I am on a mission to bring out the best engineering stories from around the world and make you all fall in love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

Comment