Learn how to harness the power of cluster computing from your local R session! Specifically, we'll discuss how to transform your regular R code into turbocharged computations by distributing work across multiple computers through future topologies.
Using Stanford's FarmShare 2 cluster as our example, we'll walk through:
- Understanding cluster computing basics
- Setting up the 'future' R package for cluster work
- Running parallel computations across multiple machines
- Moving data between your computer and the cluster
- Live demonstrations of job submission and monitoring
Timeline
0:00 - What's a cluster and why use one?
1:00 - Meet Stanford's FarmShare
2:24 - The three future topologies to run your R code on a cluster
4:04 - Setting up your computing resources (memory, time, etc.)
5:50 - Making your code run faster with parallel processing
8:30 - Success! Seeing your cluster results in action
10:41 - Moving data between your computer and the cluster
13:38 - Getting your system ready for cluster computing
💻 Code & Resources:
FarmShare 2 Documentation: https://docs.farmshare.stanford.edu/
GitHub Repository: https://github.com/coatless-videos/future-topology-demo
🔗 Connect with me:
GitHub: https://github.com/coatless
Website: https://thecoatlessprofessor.com
LinkedIn: https://www.linkedin.com/in/jamesbalamuta/
BlueSky: https://bsky.app/profile/coatless.bsky.social
Mastodon: https://mastodon.social/@coatless
Twitter/X: https://twitter.com/axiomsofxyz
#rstats #parallelprocessing #distributedcomputing #futureverse #future #rprogramming #programming #cluster #farmshare #stanford #rstudio #tutorial
❓ Have questions? Leave them in the comments below!