In the 7th episode, we speak with Nathan Marz, founder of Red Planet Labs and creator of Apache Storm, about his 15-year programming journey and the challenges of building scalable systems in Clojure. Nathan shares his experiences transitioning from C++ and Java to Clojure and his innovative work on Rama, a platform designed to simplify software development.
Our conversation covers testing techniques like with-redefs for debugging, the challenges of hiring Clojure developers, and the architecture's approach to event sourcing and materialized views. Nathan explains why Clojure was the ideal choice for Rama and highlights his team's learning curve and the need for strong engineering expertise.
Worth your attention:
https://paulgraham.com/avg.html
https://storm.apache.org/releases/2.2.1/Clojure-DSL.html
https://redplanetlabs.com/
https://blog.redplanetlabs.com/2021/06/03/tour-of-our-250k-line-clojure-codebase/
Blog posts:
Deep dive into how we reduced the line count for building the Twitter consumer product at scale by 100x: https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/
Introduction to Rama's Clojure API: https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-clojure-api/
Post focused on Rama's new programming paradigm https://blog.redplanetlabs.com/2024/10/10/rama-on-clojures-terms-and-the-magic-of-continuation-passing-style/
Introduction to Rama's instant migrations and how they work https://blog.redplanetlabs.com/2024/09/30/migrating-terabytes-of-data-instantly-can-your-alter-table-do-this/
0:00 - Teaser
0:55 - Intro
2:14 - Lisp, Cascalog and coming into Clojure
4:50 - Red Planet Labs. Choosing and Sticking to Clojure
6:36 - Rama. New paradigm. New Platform.
9:22 - Clojure in Rama
15:48 - Rama in production
23:32 - Rama vs Datomic
25:18 - Clojure deficiencies for high performance and the workarounds
31:00 - Clojure alternative core library
31:12 - Lessons learned building a high-performance platform in Clojure
35:42 - Schema
38:09 - Testing coverage. Mutation Testing. Chaos testing.
40:50 - Strength of Clojure for testing
43:07 - Over-engineering and how to avoid it
46:27 - Building and leading a team
49:11 - Hiring Clojure Engineers
57:17 - State of Clojrue ecosystem
01:00:55 - Question from the previous guest
01:04:05 - Question from Nathan
#clojure #productdevelopment #java #c++ #softwaredevelopment #freshcode
Tune in for valuable insights into the complexities of building scalable applications with Clojure and its evolving ecosystem. Subscribe to our podcast for more!