| Название | : | 3. Building the First Data Pipeline with ETL in Azure Synapse Analytics tutorial |
| Продолжительность | : | 40.01 |
| Дата публикации | : | |
| Просмотров | : | 37 rb |
 |
Thanks for this video Great Tutorial! This really helped me alot in my POC work as well as learning Comment from : Prashant Jadhav |
 |
Thank u so much for sharing your knowledge it helped me alott Comment from : Prachi Bhoj |
 |
Your demo is good, but in my humble opinion you probably need to work on your presentation skills; the accent sounds like you're trying to mimic US English accents Indian accents sound better when they're clear and not imitated by the US accent, but when you trying to sound like us, things become a bit complicated since the focus would be on what you are trying to say rather than what's on the screen Comment from : V K |
 |
Technically you can do the same in Databricks with SQL, Python or Scala Even ADF can create a pipeline with tagging Synapse studio can do pipelines & queries but bring in spark notebook from synapse brbrThis is great work from you thou Comment from : Dennis |
 |
Excellent work on explaining the concepts Comment from : rajesh naidu |
 |
This was awesome Only ran into a few problems with role assignments and firewall rules but worked it out pretty easily The rest just worked :) thank you very much Comment from : Spagneto |
 |
The video seems nice but the sound quality didn't help Too much echoing Comment from : Kevin Odibeli |
 |
HELP Comment from : mark cuello |
 |
Thanks, but instead of using a csv file and uploading it to Azure Data Lake Storage Gen2, Can I use a SQL Server table in pyspark as a source, making a connection to SQL? Comment from : Nancy Silva Moreno |
 |
Highly appreciate the superb explanation!! Nice Video!! But this much effort to load a flat file to a DB? Need a separate Spark and SQL pool for this?? Microsoft should learn to simplify stuffs :-) :-) Comment from : Shiv Subramaniam K N |
 |
What a detailed stuff …kudos Comment from : Manoj Katasani |
 |
Thank you for the video, but it is hard to hear you you whispered :D Comment from : Maryam Alizadeh |
 |
Hi and thanks for the video ! I have a question please: for "repartition(4)" this repartitions is done using the SQL pool ? and does the SQL pool uses the same nodes of the Spark pool or they use other nodes?? thanks Comment from : thouraya sboui |
 |
How to schedule the sql script in a pipeline ? Just like you would use notebook activity I saw you have used store procedure activity but that store procedure creation you ran manually What if I have a script that creates few tables , add some rows in it as well as does some transformations How to schedule that as an activity in synapse pipeline ? How does this scenario will be in production if I was to have no databases, no tables and blank workspace in the production How would you automatically create tables Couldn't see any sql script activity that can be added in the pipeline So is sql script only for manual use ? Comment from : KJ |
 |
Thanks for this demo I am looking for suggestions for one my requirements It would be great if you have any input on thisbrbrI need to do upsert in synapse dedicated sql pool from data in adls using pyspark Could you please help me with any suggestions?I know merge is not possible now So I was just wondering how can we achieve it Comment from : Wasim Akram |
 |
Really good walkthrough! One question though, which I'm not sure about 31:55 pipeline has the name 'ETLPipeline'; wouldn't it be ELT, since we are extracting the data from source using the notebook , loading the data onto the source using the CopySpark_SQL spark pipeline and transforming the data using the Stored Proc? Please don't take it as criticism, it's a genuine question I'm having Thank you Comment from : Andrei Dumitru |
 |
You tried to explain well but you still need some more practice with that UI You could have simply opened the dataset to see the name of the folder which you wantedbrThanks for the video though Happy learning! Comment from : Sam Talks Tech |
 |
Can we use an already existing adls storage and sql database for this purpose? Won't we experience permission issues? Comment from : Saurab Rao |
 |
Spark jobs generally takes 3-4 minutes to start, if cluster is already not running, and I think we can refresh the output, instead of refreshing the entire windows Comment from : Manish Patel |
|
|
Extremely thoughtful explanation for anyone who has just begun to understand azure synapse I look forward for more videos Comment from : Manish Patel |
 |
Hi Team, Can you share us the GutHub link so that we can do play around with the scripts to run Azure Synapse for our understanding Thank You in anticipation Comment from : Mohammed Khan |
 |
Is there any difference between doing the Pyspark transformations in Az Databricks(Notebook activity) Vs doing the same in Az Synapse Comment from : IamRam |
 |
Thanks, this is what I was looking for I am new in Azure Very well explained Comment from : Dhruv Arora |
 |
#10. Azure Synapse Analytics - Use copy activity in Synapse Pipeline to load data РѕС‚ : All About BI ! Download Full Episodes | The Most Watched videos of all time |
 |
Part 8 - Data Loading (Azure Synapse Analytics) | End to End Azure Data Engineering Project РѕС‚ : Mr. K Talks Tech Download Full Episodes | The Most Watched videos of all time |
 |
Azure Devops YAML CI Pipeline | Learn YAML for Azure devops pipeline | classic pipeline to YAML РѕС‚ : Avin Techno Download Full Episodes | The Most Watched videos of all time |
 |
37. Run Synapse notebook from pipeline | Pass values to Notebook parameters from pipeline in Synapse РѕС‚ : WafaStudies Download Full Episodes | The Most Watched videos of all time |
 |
For Each Activity in Azure Data Factory and Azure Synapse Analytics Pipelines | Iteration Concepts РѕС‚ : Cloud Knowledge Download Full Episodes | The Most Watched videos of all time |
 |
Until Activity in Azure Data Factory and Azure Synapse Analytics Pipelines | Do Until Concept РѕС‚ : Cloud Knowledge Download Full Episodes | The Most Watched videos of all time |
 |
Build CI and CD Pipeline using Azure DevOps - Step by Step | AZ-400:Azure DevOps | CICD Pipeline РѕС‚ : BestDotNetTraining Download Full Episodes | The Most Watched videos of all time |
 |
Real-Time CI CD Pipeline Project | CI CD Pipeline | Jenkins CI CD Pipeline РѕС‚ : DevOps Shack Download Full Episodes | The Most Watched videos of all time |
 |
91. Script Activity in Azure Data Factory or Azure Synapse РѕС‚ : WafaStudies Download Full Episodes | The Most Watched videos of all time |
 |
Azure DevOps - Release Pipeline in Azure DevOps | Azure DevOps CICD РѕС‚ : Saravanan Seenivasan Download Full Episodes | The Most Watched videos of all time |
