Discover how to create dynamic pipelines in Microsoft Fabric! In this video, we demonstrate how to parameterize pipelines for flexible data movement, turning a single pipeline into a powerful tool capable of handling multiple tables. Watch as we build a child pipeline, configure table parameters, and invoke it dynamically from a parent pipeline. Along the way, we explore essential activities like Lookup and For Each, and validate results in a Lakehouse.
👉 On-Demand Learning - Start With The FREE Community Plan: https://prag.works/odl-trial-yt
🔗Pragmatic Works On-Demand Learning Packages: https://pragmaticworks.com/pricing/
🔗Pragmatic Works Boot Camps: https://pragmaticworks.com/boot-camps/
🔗Pragmatic Works Hackathons: https://pragmaticworks.com/private-training/hackathons/
🔗Pragmatic Works Virtual Mentoring: https://pragmaticworks.com/virtual-mentoring/
🔗Pragmatic Works Enterprise Private Training: https://pragmaticworks.com/private-training/customized-enterprise-training/
🔗Pragmatic Works Blog: http://blog.pragmaticworks.com/
00:00 - Introduction to Microsoft Fabric Pipelines
00:31 - Overview of the Dynamic Pipeline Example
01:26 - Setting Up a Dynamic Child Pipeline
02:26 - Configuring Table Parameters for Data Movement
03:48 - Preparing Dynamic Mappings in the Copy Activity
04:41 - Parameterizing Destination Tables
05:14 - Creating the Parent Pipeline
05:43 - Using Lookup Activities for Table Lists
08:00 - Iterating Through Tables with For Each
09:00 - Invoking Child Pipelines from the Parent
10:31 - Testing the Parent-Child Pipeline Setup
11:06 - Verifying Results in the Lakehouse
11:35 - Exploring the Final Dynamic Design
12:05 - Benefits of Dynamic Pipelines for Developers
Let's connect:
✔️Discord: https://prag.works/yt-discord
✔️Facebook: https://prag.works/yt-fb
✔️Instagram: https://prag.works/yt-insta
✔️LinkedIn: https://prag.works/yt-li
Pragmatic Works
7175 Hwy 17, Suite 2 Fleming Island, FL 32003
Phone: (904) 638-5743
Email: [email protected]
#PragmaticWorks