Udemy - Data Engineering using Kafka and Spark Structured Streaming 收录时间:2025-04-23 01:46:17 文件大小:4GB 下载次数:1 最近下载:2025-04-23 01:46:17 磁力链接: magnet:?xt=urn:btih:c0d2ffb3700b64ccf72acb192a618c30354a6847 立即下载 复制链接 文件列表 11. Incremental Loads using Spark Structured Streaming/14. Using maxFilerPerTrigger and latestFirst.mp4 98MB 11. Incremental Loads using Spark Structured Streaming/5. Upload GHArchive Files to HDFS.mp4 93MB 11. Incremental Loads using Spark Structured Streaming/17. Incremental Load using Archival Process.mp4 92MB 11. Incremental Loads using Spark Structured Streaming/4. Logic to Upload GHArchive Files.mp4 82MB 10. Kafka and Spark Structured Streaming Integration/5. Transform Data using Spark APIs.mp4 82MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/7. Setup Jupyter Lab.mp4 79MB 10. Kafka and Spark Structured Streaming Integration/6. Write Data to HDFS using Spark.mp4 77MB 9. Overview of Spark Structured Streaming/5. Reading Web Server logs using Spark Structured Streaming.mp4 75MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/5. Setup Docker.mp4 75MB 8. Data Ingestion using Kafka Connect/2. Define Kafka Connect to Produce Messages.mp4 69MB 11. Incremental Loads using Spark Structured Streaming/11. Load Data Incrementally to Target Table.mp4 69MB 5. Setup Hive and Spark/10. Validate Jupyter Lab Setup.mp4 68MB 10. Kafka and Spark Structured Streaming Integration/7. Validate Data in HDFS using Spark.mp4 67MB 5. Setup Hive and Spark/8. Configure Spark 2.mp4 67MB 5. Setup Hive and Spark/9. Validate Spark 2 using CLIs.mp4 66MB 10. Kafka and Spark Structured Streaming Integration/8. Write Data to HDFS using Spark using Header.mp4 65MB 9. Overview of Spark Structured Streaming/7. Using append as Output Mode.mp4 65MB 8. Data Ingestion using Kafka Connect/6. Setup HDFS 3 Sink Connector Plugin.mp4 64MB 8. Data Ingestion using Kafka Connect/3. Validate Kafka Connect to produce messages.mp4 64MB 6. Setup Single Node Kafka Cluster/5. Overview of Kafka CLI.mp4 63MB 9. Overview of Spark Structured Streaming/12. Previewing the Streaming Data.mp4 63MB 5. Setup Hive and Spark/14. Validate Spark 3 using CLIs.mp4 63MB 7. Getting Started with Kafka/10. Validate Kafka Connect to produce.mp4 60MB 7. Getting Started with Kafka/9. Define Kafka Connect to produce.mp4 60MB 5. Setup Hive and Spark/13. Configure Spark 3.mp4 59MB 8. Data Ingestion using Kafka Connect/7. Overview of Kafka Consumer Groups.mp4 59MB 8. Data Ingestion using Kafka Connect/9. Run and Validate HDFS 3 Sink.mp4 56MB 5. Setup Hive and Spark/4. Configure and Setup Hive Metastore.mp4 54MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/4. Provision Ubuntu VM from GCP.mp4 54MB 5. Setup Hive and Spark/3. Setup Database for Hive Metastore.mp4 53MB 9. Overview of Spark Structured Streaming/8. Using complete as Output Mode.mp4 53MB 8. Data Ingestion using Kafka Connect/8. Configure HDFS 3 Sink Properties.mp4 53MB 11. Incremental Loads using Spark Structured Streaming/9. Analyze GHArchive Data in Parquet files using Spark.mp4 50MB 6. Setup Single Node Kafka Cluster/7. Redirecting logs to Kafka.mp4 48MB 7. Getting Started with Kafka/2. Managing Topics using Kafka CLI.mp4 48MB 4. Setup Single Node Hadoop Cluster/5. Configure Hadoop HDFS.mp4 47MB 11. Incremental Loads using Spark Structured Streaming/7. Read JSON Data using Spark Structured streaming.mp4 46MB 4. Setup Single Node Hadoop Cluster/6. Start and Validate HDFS.mp4 46MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/3. Overview of GCP Pricing.mp4 44MB 10. Kafka and Spark Structured Streaming Integration/2. Read Data from Kafka Topic.mp4 42MB 2. Setting up Environment using AWS Cloud9/8. Setup Jupyter Lab on Cloud9.mp4 42MB 6. Setup Single Node Kafka Cluster/3. Configure and Start Kafka Broker.mp4 42MB 7. Getting Started with Kafka/1. Overview of Kafka.mp4 41MB 9. Overview of Spark Structured Streaming/10. Overview of Triggers in Spark Structured Streaming.mp4 39MB 5. Setup Hive and Spark/1. Setup Data Sets for Practice.mp4 39MB 5. Setup Hive and Spark/11. Integrate Spark 2 with Jupyter Lab.mp4 39MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/8. Setup Jupyter Lab locally on Mac.mp4 39MB 9. Overview of Spark Structured Streaming/1. Understanding Streaming Context.mp4 39MB 10. Kafka and Spark Structured Streaming Integration/3. Preview data using console.mp4 37MB 5. Setup Hive and Spark/5. Launch and Validate Hive.mp4 36MB 6. Setup Single Node Kafka Cluster/1. Download and Install Kafka.mp4 36MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/6. Validating Python.mp4 34MB 11. Incremental Loads using Spark Structured Streaming/6. Add new GHActivity JSON Files.mp4 33MB 10. Kafka and Spark Structured Streaming Integration/9. Cleanup Kafka Connect and Files in HDFS.mp4 33MB 4. Setup Single Node Hadoop Cluster/2. Setup Prerequisites.mp4 32MB 6. Setup Single Node Kafka Cluster/6. Setup Retail log Generator.mp4 31MB 1. Introduction/1. Introduction.mp4 31MB 4. Setup Single Node Hadoop Cluster/4. Download and Install Hadoop.mp4 31MB 6. Setup Single Node Kafka Cluster/2. Configure and Start Zookeeper.mp4 29MB 2. Setting up Environment using AWS Cloud9/6. Associating Elastic IPs to Cloud9 Instance.mp4 29MB 10. Kafka and Spark Structured Streaming Integration/4. Preview data using memory.mp4 28MB 9. Overview of Spark Structured Streaming/3. Push log messages to Netcat Webserver.mp4 28MB 2. Setting up Environment using AWS Cloud9/2. Creating Cloud9 Environment.mp4 28MB 7. Getting Started with Kafka/3. Produce and Consume Messages using CLI.mp4 27MB 6. Setup Single Node Kafka Cluster/4. Scripts to manage single node cluster.mp4 26MB 5. Setup Hive and Spark/7. Download and Install Spark 2.mp4 25MB 5. Setup Hive and Spark/15. Integrate Spark 3 with Jupyter Lab.mp4 25MB 8. Data Ingestion using Kafka Connect/10. Cleanup Kafka Connect to consume messages.mp4 25MB 9. Overview of Spark Structured Streaming/9. Using update as Output Mode.mp4 24MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/1. Signing up for GCP.mp4 24MB 11. Incremental Loads using Spark Structured Streaming/2. Steps for Incremental Data Processing.mp4 23MB 11. Incremental Loads using Spark Structured Streaming/18. Validate Incremental Load.mp4 23MB 5. Setup Hive and Spark/6. Scripts to Manage Single Node Cluster.mp4 22MB 11. Incremental Loads using Spark Structured Streaming/15. Validate Incremental Load.mp4 22MB 1. Introduction/2. Important Note for first time Data Engineering Customers.mp4 22MB 1. Introduction/3. Important Note for Data Engineering Essentials (Python and Spark) Customers.mp4 22MB 4. Setup Single Node Hadoop Cluster/9. Managing Single Node Hadoop.mp4 20MB 8. Data Ingestion using Kafka Connect/4. Cleanup Kafka Connect to produce messages.mp4 20MB 11. Incremental Loads using Spark Structured Streaming/12. Validate Incremental Load.mp4 20MB 2. Setting up Environment using AWS Cloud9/1. Getting Started with Cloud9.mp4 20MB 7. Getting Started with Kafka/4. Validate Generation of Web Server Logs.mp4 19MB 9. Overview of Spark Structured Streaming/11. Overview of built-in Output Sinks.mp4 19MB 1. Introduction/4. How to get 30 days complementary lab access.mp4 19MB 2. Setting up Environment using AWS Cloud9/5. Opening ports for Cloud9 Instance.mp4 18MB 2. Setting up Environment using AWS Cloud9/7. Increase EBS Volume Size of Cloud9 Instance.mp4 18MB 9. Overview of Spark Structured Streaming/4. Overview of built-in Input Sources.mp4 18MB 10. Kafka and Spark Structured Streaming Integration/1. Create Kafka Topic.mp4 18MB 2. Setting up Environment using AWS Cloud9/3. Warming up with Cloud9 IDE.mp4 17MB 4. Setup Single Node Hadoop Cluster/3. Setup Password less login.mp4 17MB 4. Setup Single Node Hadoop Cluster/1. Introduction to Single Node Hadoop Cluster.mp4 16MB 11. Incremental Loads using Spark Structured Streaming/3. Create Working Directory in HDFS.mp4 16MB 7. Getting Started with Kafka/5. Create Web Server using nc.mp4 16MB 9. Overview of Spark Structured Streaming/2. Validate Log Data for Streaming.mp4 16MB 7. Getting Started with Kafka/6. Produce retail logs to Kafka Topic.mp4 15MB 5. Setup Hive and Spark/12. Download and Install Spark 3.mp4 15MB 4. Setup Single Node Hadoop Cluster/8. Start and Validate YARN.mp4 14MB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/2. Overview of GCP Web Console.mp4 14MB 7. Getting Started with Kafka/7. Consume retail logs from Kafka Topic.mp4 14MB 5. Setup Hive and Spark/2. Download and Install Hive.mp4 13MB 11. Incremental Loads using Spark Structured Streaming/10. Add New GHActivity JSON files.mp4 13MB 11. Incremental Loads using Spark Structured Streaming/8. Write in Parquet File Format.mp4 12MB 11. Incremental Loads using Spark Structured Streaming/16. Add New GHActivity JSON files.mp4 10MB 7. Getting Started with Kafka/8. Clean up Kafka CLI Sessions to produce and consume messages.mp4 10MB 8. Data Ingestion using Kafka Connect/1. Overview of Kafka Connect.mp4 10MB 2. Setting up Environment using AWS Cloud9/4. Overview of EC2 related to Cloud9.mp4 9MB 11. Incremental Loads using Spark Structured Streaming/1. Overview of Spark Structured Streaming Triggers.mp4 9MB 9. Overview of Spark Structured Streaming/6. Overview of Output Modes.mp4 9MB 8. Data Ingestion using Kafka Connect/5. Write Data to HDFS using Kafka Connect.mp4 8MB 11. Incremental Loads using Spark Structured Streaming/13. Add New GHActivity JSON files.mp4 8MB 4. Setup Single Node Hadoop Cluster/7. Configure Hadoop YARN.mp4 8MB 11. Incremental Loads using Spark Structured Streaming/5. Upload GHArchive Files to HDFS.srt 21KB 11. Incremental Loads using Spark Structured Streaming/4. Logic to Upload GHArchive Files.srt 20KB 9. Overview of Spark Structured Streaming/5. Reading Web Server logs using Spark Structured Streaming.srt 19KB 11. Incremental Loads using Spark Structured Streaming/14. Using maxFilerPerTrigger and latestFirst.srt 17KB 11. Incremental Loads using Spark Structured Streaming/17. Incremental Load using Archival Process.srt 17KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/7. Setup Jupyter Lab.srt 16KB 5. Setup Hive and Spark/10. Validate Jupyter Lab Setup.srt 16KB 8. Data Ingestion using Kafka Connect/2. Define Kafka Connect to Produce Messages.srt 15KB 10. Kafka and Spark Structured Streaming Integration/5. Transform Data using Spark APIs.srt 15KB 6. Setup Single Node Kafka Cluster/5. Overview of Kafka CLI.srt 14KB 5. Setup Hive and Spark/8. Configure Spark 2.srt 14KB 7. Getting Started with Kafka/9. Define Kafka Connect to produce.srt 13KB 5. Setup Hive and Spark/9. Validate Spark 2 using CLIs.srt 13KB 5. Setup Hive and Spark/14. Validate Spark 3 using CLIs.srt 13KB 8. Data Ingestion using Kafka Connect/7. Overview of Kafka Consumer Groups.srt 13KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/4. Provision Ubuntu VM from GCP.srt 13KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/5. Setup Docker.srt 13KB 10. Kafka and Spark Structured Streaming Integration/6. Write Data to HDFS using Spark.srt 13KB 8. Data Ingestion using Kafka Connect/3. Validate Kafka Connect to produce messages.srt 12KB 11. Incremental Loads using Spark Structured Streaming/11. Load Data Incrementally to Target Table.srt 12KB 11. Incremental Loads using Spark Structured Streaming/9. Analyze GHArchive Data in Parquet files using Spark.srt 12KB 9. Overview of Spark Structured Streaming/7. Using append as Output Mode.srt 12KB 5. Setup Hive and Spark/4. Configure and Setup Hive Metastore.srt 12KB 5. Setup Hive and Spark/3. Setup Database for Hive Metastore.srt 12KB 7. Getting Started with Kafka/10. Validate Kafka Connect to produce.srt 12KB 8. Data Ingestion using Kafka Connect/9. Run and Validate HDFS 3 Sink.srt 12KB 9. Overview of Spark Structured Streaming/12. Previewing the Streaming Data.srt 12KB 2. Setting up Environment using AWS Cloud9/8. Setup Jupyter Lab on Cloud9.srt 12KB 7. Getting Started with Kafka/2. Managing Topics using Kafka CLI.srt 12KB 4. Setup Single Node Hadoop Cluster/5. Configure Hadoop HDFS.srt 12KB 10. Kafka and Spark Structured Streaming Integration/7. Validate Data in HDFS using Spark.srt 11KB 10. Kafka and Spark Structured Streaming Integration/8. Write Data to HDFS using Spark using Header.srt 11KB 11. Incremental Loads using Spark Structured Streaming/7. Read JSON Data using Spark Structured streaming.srt 11KB 8. Data Ingestion using Kafka Connect/8. Configure HDFS 3 Sink Properties.srt 11KB 9. Overview of Spark Structured Streaming/8. Using complete as Output Mode.srt 11KB 8. Data Ingestion using Kafka Connect/6. Setup HDFS 3 Sink Connector Plugin.srt 11KB 5. Setup Hive and Spark/13. Configure Spark 3.srt 10KB 4. Setup Single Node Hadoop Cluster/6. Start and Validate HDFS.srt 10KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/8. Setup Jupyter Lab locally on Mac.srt 10KB 5. Setup Hive and Spark/11. Integrate Spark 2 with Jupyter Lab.srt 10KB 7. Getting Started with Kafka/1. Overview of Kafka.srt 9KB 10. Kafka and Spark Structured Streaming Integration/2. Read Data from Kafka Topic.srt 9KB 9. Overview of Spark Structured Streaming/10. Overview of Triggers in Spark Structured Streaming.srt 9KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/3. Overview of GCP Pricing.srt 9KB 6. Setup Single Node Kafka Cluster/7. Redirecting logs to Kafka.srt 9KB 9. Overview of Spark Structured Streaming/1. Understanding Streaming Context.srt 9KB 6. Setup Single Node Kafka Cluster/3. Configure and Start Kafka Broker.srt 9KB 4. Setup Single Node Hadoop Cluster/2. Setup Prerequisites.srt 8KB 6. Setup Single Node Kafka Cluster/1. Download and Install Kafka.srt 8KB 10. Kafka and Spark Structured Streaming Integration/3. Preview data using console.srt 8KB 5. Setup Hive and Spark/5. Launch and Validate Hive.srt 8KB 2. Setting up Environment using AWS Cloud9/2. Creating Cloud9 Environment.srt 8KB 5. Setup Hive and Spark/6. Scripts to Manage Single Node Cluster.srt 8KB 6. Setup Single Node Kafka Cluster/4. Scripts to manage single node cluster.srt 8KB 1. Introduction/1. Introduction.srt 8KB 4. Setup Single Node Hadoop Cluster/4. Download and Install Hadoop.srt 7KB 9. Overview of Spark Structured Streaming/3. Push log messages to Netcat Webserver.srt 7KB 5. Setup Hive and Spark/1. Setup Data Sets for Practice.srt 7KB 4. Setup Single Node Hadoop Cluster/9. Managing Single Node Hadoop.srt 7KB 5. Setup Hive and Spark/15. Integrate Spark 3 with Jupyter Lab.srt 7KB 9. Overview of Spark Structured Streaming/9. Using update as Output Mode.srt 7KB 6. Setup Single Node Kafka Cluster/6. Setup Retail log Generator.srt 7KB 7. Getting Started with Kafka/3. Produce and Consume Messages using CLI.srt 7KB 2. Setting up Environment using AWS Cloud9/6. Associating Elastic IPs to Cloud9 Instance.srt 7KB 4. Setup Single Node Hadoop Cluster/3. Setup Password less login.srt 6KB 11. Incremental Loads using Spark Structured Streaming/6. Add new GHActivity JSON Files.srt 6KB 8. Data Ingestion using Kafka Connect/10. Cleanup Kafka Connect to consume messages.srt 6KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/6. Validating Python.srt 6KB 11. Incremental Loads using Spark Structured Streaming/2. Steps for Incremental Data Processing.srt 6KB 2. Setting up Environment using AWS Cloud9/3. Warming up with Cloud9 IDE.srt 6KB 2. Setting up Environment using AWS Cloud9/7. Increase EBS Volume Size of Cloud9 Instance.srt 6KB 6. Setup Single Node Kafka Cluster/2. Configure and Start Zookeeper.srt 6KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/1. Signing up for GCP.srt 5KB 11. Incremental Loads using Spark Structured Streaming/15. Validate Incremental Load.srt 5KB 5. Setup Hive and Spark/7. Download and Install Spark 2.srt 5KB 1. Introduction/2. Important Note for first time Data Engineering Customers.srt 5KB 10. Kafka and Spark Structured Streaming Integration/4. Preview data using memory.srt 5KB 2. Setting up Environment using AWS Cloud9/5. Opening ports for Cloud9 Instance.srt 5KB 7. Getting Started with Kafka/5. Create Web Server using nc.srt 5KB 11. Incremental Loads using Spark Structured Streaming/3. Create Working Directory in HDFS.srt 5KB 1. Introduction/3. Important Note for Data Engineering Essentials (Python and Spark) Customers.srt 5KB 4. Setup Single Node Hadoop Cluster/1. Introduction to Single Node Hadoop Cluster.srt 5KB 11. Incremental Loads using Spark Structured Streaming/18. Validate Incremental Load.srt 5KB 2. Setting up Environment using AWS Cloud9/1. Getting Started with Cloud9.srt 5KB 10. Kafka and Spark Structured Streaming Integration/1. Create Kafka Topic.srt 5KB 1. Introduction/4. How to get 30 days complementary lab access.srt 5KB 10. Kafka and Spark Structured Streaming Integration/9. Cleanup Kafka Connect and Files in HDFS.srt 5KB 11. Incremental Loads using Spark Structured Streaming/12. Validate Incremental Load.srt 5KB 8. Data Ingestion using Kafka Connect/4. Cleanup Kafka Connect to produce messages.srt 4KB 9. Overview of Spark Structured Streaming/11. Overview of built-in Output Sinks.srt 4KB 7. Getting Started with Kafka/4. Validate Generation of Web Server Logs.srt 4KB 5. Setup Hive and Spark/2. Download and Install Hive.srt 4KB 9. Overview of Spark Structured Streaming/4. Overview of built-in Input Sources.srt 4KB 7. Getting Started with Kafka/6. Produce retail logs to Kafka Topic.srt 4KB 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/2. Overview of GCP Web Console.srt 3KB 9. Overview of Spark Structured Streaming/2. Validate Log Data for Streaming.srt 3KB 4. Setup Single Node Hadoop Cluster/8. Start and Validate YARN.srt 3KB 11. Incremental Loads using Spark Structured Streaming/8. Write in Parquet File Format.srt 3KB 5. Setup Hive and Spark/12. Download and Install Spark 3.srt 3KB 8. Data Ingestion using Kafka Connect/1. Overview of Kafka Connect.srt 3KB 2. Setting up Environment using AWS Cloud9/4. Overview of EC2 related to Cloud9.srt 3KB 7. Getting Started with Kafka/7. Consume retail logs from Kafka Topic.srt 3KB 11. Incremental Loads using Spark Structured Streaming/1. Overview of Spark Structured Streaming Triggers.srt 3KB 8. Data Ingestion using Kafka Connect/5. Write Data to HDFS using Kafka Connect.srt 2KB 11. Incremental Loads using Spark Structured Streaming/16. Add New GHActivity JSON files.srt 2KB 11. Incremental Loads using Spark Structured Streaming/10. Add New GHActivity JSON files.srt 2KB 7. Getting Started with Kafka/8. Clean up Kafka CLI Sessions to produce and consume messages.srt 2KB 4. Setup Single Node Hadoop Cluster/7. Configure Hadoop YARN.srt 2KB 11. Incremental Loads using Spark Structured Streaming/13. Add New GHActivity JSON files.srt 2KB 9. Overview of Spark Structured Streaming/6. Overview of Output Modes.srt 2KB 2. Setting up Environment using AWS Cloud9/9. [Commands] Setup Jupyter Lab on Cloud9.html 615B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/2.1 Understanding GCP Web Console.html 158B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/4.1 Provision Ubuntu 18.04 Virtual Machine.html 153B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/3.1 Overview of GCP Pricing.html 151B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/1.1 Signing up for GCP.html 146B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/6.1 Validating Python.html 145B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/7.1 Setup Jupyter Lab.html 145B 3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/5.1 Setup Docker.html 140B