MENU

Fun & Interesting

How One Line of Code Almost Blew Up the Internet

Kevin Fang 2,157,177 2 years ago
Video Not Working? Fix It Now

Sources: https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/ https://blog.cloudflare.com/quantifying-the-impact-of-cloudbleed/ https://bugs.chromium.org/p/project-zero/issues/detail?id=1139 https://asamborski.github.io/cs558_s17_blog/2017/04/08/cloudbleed.html https://www.colm.net/open-source/ragel/ "[CloudFlare] A Day at the CloudFlare Office" https://www.youtube.com/watch?v=_ttI4eWuQU4 Assumptions: - The graph for "email obfuscation" vs. "bug occurrence" at 2:51. This was added to illustrate that the bug was being triggered by this feature. They did not have a convenient graph that told them when the bug was being triggered. - The "crossroads" mentioned at 3:55 probably did not happen. Just to add drama/plot. - Explanation of why fhold is called within the finishing action of script_consume_attr is my best guess 7:50 - The history behind the empty last buffer was never explained. But I assume that some existing Module A would originally feed data to the Ragel parser. Module A still existed, and still continued to output this empty last buffer, but now cf-html can stand between Module A and the existing Ragel parser. Here, cf-html would consume Module A's data + the empty last buffer with no issues, but it's output would no longer include the dummy buffer. This output can then be taken in by the Ragel parser. - Whether or not Cloudflare modified the compiled C code is unknown/never mentioned. There must be a reason that Ragel chooses to use == for the buffer end check rather than ≥, and semantically, == makes more sense if it checks for the buffer end with every iteration, which should make buffer overrunning impossible. - Technically in the strictest sense this is a "buffer over-read" as opposed to an "overflow" or "overrun" but the Wikipedia page for Cloudbleed says "overflow" so w/e - Whether or not this bug going unnoticed/discovered by hackers first would've "blown up the internet" is arguable Error corrections: - 13:13, the correct number is 0.06% (what is shown), but I say 0.6% - 13:28, the bug was possible since September (what is shown) Chapters: 0:00 Exposition/useless story building stuff 0:50 Explanation of Cloudflare and CDNs 1:44 Implications of the bug 2:40 Mitigation timeline 4:46 Root cause 10:43 Lessons learned 12:41 Resolution Music by LEMMiNO: Nocturnal - https://www.youtube.com/watch?v=epmoV2HRs9U Encounters - https://www.youtube.com/watch?v=xdwWCl_5x2s Cipher - https://www.youtube.com/watch?v=b0q5PR1xpA0 Twitter: https://twitter.com/kevinfaang/ Instagram: https://instagram.com/kevinfaang_yt/

Comment