Chapter #9 - Designing data pipeline solution on GCP
#datapipeline #googlecloud #gcp
What is Data Pipeline? - https://youtu.be/PjBhIDOdLM0
***Google Cloud Platform Beginner Series***
Google Cloud Platform Beginner Series (2020) - https://www.youtube.com/watch?v=OzwSBbuHY-0&list=PLcnJIHtHiTA2Jp3klIkeHiwqGrxjXBzh6
***Networking Basics Playlist***
Networking and Infra Concepts - https://www.youtube.com/playlist?list=PLcnJIHtHiTA0C9UXgU2C6eTQMVFzSlIdP
*** Other Popular Playlists***
Crunching Data Series (2020) - https://www.youtube.com/playlist?list=PLcnJIHtHiTA2HmIleev4scARSFwrQ0bIy
Latest technology tutorial (2020) -
https://www.youtube.com/watch?v=l5UcUEt1IzM&list=PLcnJIHtHiTA2Hdgon51_-G3Kt43hcU6kI
Hi Friends, I am Anshul Tiwari, and welcome to our youtube channel IT k Funde.
More about this video -
In this video, we will learn something different, instead of discussing any specific product on google cloud platform, we will learn how to design a data pipeline solution on google cloud platform.
This will help us in 2 ways -
1. This will helps us improve our design thinking
2. We will get an overview of all google cloud products that can be used in developing a data pipeline.
We will start with understanding a typical on premise data pipeline and enterprise data warehouse (edw) design. We will then take this same solution and try to implement it on google cloud platform.
On-Prem solution design contains below products -
#Streaming data - Apache Kafka
#ETL - Informatica, IBM Datastage, SAP Data Services
#EDW - Teradata
#Dashbaords & reporting - Tableau and SAP Business Objects
GCP cloud design contains below products -
#Streaming data - Cloud PubSub
#ETL - Cloud Data Fusion
#Storage - Google Cloud Storage
#EDW - Google Big Query
#Dashboards - Looker, Google Data Studio
#ML - AutoML
#Metadata Management - Data Catalog
PLEASE WATCH ALL THE VIDEOS IN ANY PLAYLIST FOR BETTER UNDERSTANDING !!
Credits & Resources for further study -
GCP resources documentation for detailed study -
#Cloud PubSub - https://cloud.google.com/pubsub/docs/overview
#Cloud Data Fusion - https://cloud.google.com/data-fusion/docs/concepts/overview
#Google Cloud Storage - https://cloud.google.com/storage/docs/quickstart-console
#Google Big Query- https://cloud.google.com/bigquery/docs/quickstarts
#Data Studio - https://cloud.google.com/bi-engine/docs/getting-started-data-studio
#Looker - https://docs.looker.com/
#AutoML - https://cloud.google.com/automl/docs
#Data Catalog - https://cloud.google.com/data-catalog/docs
images - pixabay.com
***Social Links***
Youtube - https://www.youtube.com/c/itkfunde
Facebook - https://www.facebook.com/ITkFUNDE/
Linkedin - https://www.linkedin.com/in/ansh9685
Twitter - https://twitter.com/ansh9685
Instagram -https://www.instagram.com/itkfunde/
***About This Channel***
Friends ITkFUNDE channel wants to bring I.T. industry knowledge, information, career advice, and much more to every individual regardless of whether he or she belongs to I.T or not. This channel is for everyone interested in learning something new!