Google Data Engineer Course E-Learning

In this online Google Data Engineer training we cover all facets of the certificate: Professional Data Engineer. A professional data engineer makes data-driven decision making possible by collecting, transforming and visualizing data. The Data Engineer analyzes data to gain insight into business results, builds statistical models to support decision-making and creates machine learning models for automating and simplifying important business processes.

The training demonstrates the following:

Construction and maintenance of data structures and data Basics
Design data processing systems
Analyze data and make machine learning possible
Modeling business processes for analysis and optimization
Reliability designing
Visualize data data to support policy
Design for security and compliance.

Your result

You are familiar with the work of a Data Engineer on the Google Cloud Platform. You are also prepared for the Google Professional Data Engineer exam.

Target audience

Data analyst

Course content

GCP Fundamentals

Course: 24 Minutes

• Course Introduction
• Google Cloud Platform Concepts
• Navigating Google Cloud Platform Services
• Benefits of Google Cloud Platform
• Comparing GCP and Other Models

GCP Account Management

Course: 21 Minutes

• Creating a GCP Account
• Navigating the GCP Console
• Creating a Project
• Using GCP BigQuery

Practice: Fundamentals of GCP

Course: 4 Minutes

• Exercise: Using GCP Data Products and Services

GCP Compute and Storage Fundamentals

Course: 19 Minutes

• Course Introduction
• Compute and Storage Introduction
• CPUs on Demand with Google Compute Engine
• Cloud Storage
• Creating and Starting an Instance

Transforming Data with GCP

Course: 28 Minutes

• Google Cloud Shell Introduction
• Activating and Using Google Cloud Shell
• Creating and Starting a VM Instance in Cloud Shell
• Creating a Cloud Storage Bucket in Cloud Shell

Exploring GCP Data Analytics

Course: 21 Minutes

• Data Analysis with GCP
• Introduction to Cloud SQL
• Creating a MySQL Database with Cloud SQL
• Creating a PostgreSQL Database with Cloud SQL

Practice: Navigating Google Cloud Platform

Course: 6 Minutes

• Exercise: Use GCP Storage, Compute and Analytics

Further Analysis Concepts

Course: 7 Minutes

• Course Introduction
• Importing Data with Cloud SQL

Hadoop and GCP

Course: 10 Minutes

• Hadoop in GCP
• Spark on Dataproc
• SparkML

Analysis at Scale

Course: 23 Minutes

• Fast Random Access with GCP
• GCP Cloud Data Lab
• BigQuery Introduction
• Machine Learning and Building Datasets
• TensorFlow Framework and Machine Learning

Practice: Analysis and Machine Learning

Course: 6 Minutes

• Exercise: Combining Analysis with Machine Learning

GCP Network Modeling

Course: 10 Minutes

• Course Introduction
• Introduction to GCP Networks
• Using GCP Models to Create a Network

Machine Learning with TensorFlow GPU

Course: 8 Minutes

• Using TensorFlow for Machine Learning
• Using Common Machine Learning APIs

Data Processing Architecture

Course: 19 Minutes

• Processing Data with GCP
• Messaging Middleware Pub and Sub
• Working with GCP Pipelines
• Creating Dataflow Pipelines
• Using Real-time and Batch Data Processing

Practice: Data Analysis and Processing

Course: 6 Minutes

• Exercise: Analyzing Data Processing

Cloud Dataproc

Course: 19 Minutes

• Course Introduction
• Overview of Data Management in GCP
• Exploring Cloud Dataproc
• Accessing Cloud Dataproc
• Using the Cloud Dataproc Dashboard

Cluster Management

Course: 19 Minutes

• Creating Clusters in GCP
• Deleting Clusters in GCP
• Defining Cluster Master and Worker Nodes
• Using Custom Machine Types and Preemptible Workers
• Using Dataproc Identity and Access Management

Practice: Cloud Data and Cluster Management

Course: 6 Minutes

• Exercise: Cluster Management and Dataproc

Continued Study of Cluster Management

Course: 9 Minutes

• Course Introduction
• Clusters with the CLI
• Dataproc REST API

Architecture and Machine Types

Course: 15 Minutes

• Dataproc Architecture
• Machine Types
• Custom Machine Types

Dataproc Jobs

Course: 7 Minutes

• Dataproc Job Execution
• HDFS and GCS

Pig and Hive

Course: 8 Minutes

• Introduction to Pig and Hive
• Pig and Hive Jobs

Practice: Dataproc Implementations

Course: 8 Minutes

• Exercise: Concepts in Dataproc Implementations

Implementation using Dataproc

Course: 24 Minutes

• Course Introduction
• Spark and Hadoop Processes
• Benefits of Cloud Dataproc
• Job Monitoring and Logging
• SSH into Master and Worker Nodes
• Spark REPL

Implementation using Cloud Shell

Course: 24 Minutes

• Separation of Compute and Storage
• BigQuery Features and Capabilities
• MapReduce with Big Data
• Job Submission with Cloud Shell

Practice: Dataproc Implementations

Course: 11 Minutes

• Exercise: Dataproc and Cloud Shell Implementations

Complex GCP Implementations with Dataproc

Course: 34 Minutes

• Course Introduction
• Implementing Initialization Actions with GCP
• Using Functions and Operators with BigQuery
• Configuring Connectors with BigQuery
• Loading Data with BigQuery
• Updating and Extracting Data with BigQuery

Jupyter and Datalab

Course: 35 Minutes

• Programming Notebooks with Jupyter and Datalab
• Using Jupyter Notebooks with Apache Spark
• Using Google Cloud Storage and SQL
• Querying Data with Google Bigtable
• Using Google APIs

Practice: Complex GCP with Big Data

Course: 11 Minutes

• Exercise: Using Dataproc with Bigtable and Datalab

Exploring BigQuery

Course: 34 Minutes

• Course Introduction
• Introduction to BigQuery
• BigQuery Exploring Datasets, Tables, and Views
• Writing Queries with BigQuery
• Big Query - Loading Data
• Exporting with BigQuery

BigQuery Operations

Course: 33 Minutes

• Using Nested and Repeated Fields
• Using Complex Queries
• Multiple Table Queries
• Building BigQuery Queries

Practice: BigQuery Fundamentals

Course: 13 Minutes

• Exercise: BigQuery Concepts and Operations

Cloud Machine Learning Engine

Course: 15 Minutes

• Course Introduction
• GCP Machine Learning Engine
• Machine Learning APIs

Exploring ML APIs

Course: 29 Minutes

• Using the Vision API
• Using the Natural Language API
• Using the Translation and Speech API
• Invoking ML APIs
• Using ML REST APIs

Implementing Vision API

Course: 10 Minutes

• Cloud Vision API and Kubernetes Cluster

Practice: Overview of GCP ML APIs

Course: 12 Minutes

• Exercise: Implementing ML APIs with GCP

Expressing Data Pipelines

Course: 32 Minutes

• Course Introduction
• Apache Beam SDK
• Data Pipelines Using Beam Python SDK
• Data Pipelines Using Beam Java SDK
• Cloud Dataprep Initialization
• Ingestion of Data into a Pipeline
• Recipe Configuration with Cloud Dataprep
• Import/Export Flows and Dataflow Jobs

Big Data Processing

Course: 7 Minutes

• MapReduce Process versus Cloud Dataflow
• Serverless Architecture and Data Analytics

Practice: Create and Manage Pipelines

Course: 8 Minutes

• Exercise: Using Beam, Dataflow, and Dataprep

Machine Learning Fundamentals

Course: 15 Minutes

• Course Introduction
• Exploring Machine Learning
• Machine Learning Datasets
• Loading Datasets for Cloud ML in GCP

Machine Learning and TensorFlow

Course: 50 Minutes

• Understanding TensorFlow
• Running a TensorFlow Python Program
• Using TensorFlow to Run a Local Trainer
• Using TensorBoard to Inspect Logs
• Running Local Trainers in Distributed Mode
• Running a Single-instance Trainer in the Cloud
• Inspecting Stackdriver Logs

Scaling with Cloud ML

Course: 14 Minutes

• Understanding Cloud ML Scaling
• Running Distributed Training in the Cloud
• Using Hyperparameter Tuning

Practice: TensorFlow and Cloud ML

Course: 10 Minutes

• Exercise: Using TensorFlow and Cloud ML

GCP Feature Engineering

Course: 19 Minutes

• Course Introduction
• Exploring Features
• Benefits of Quality Features
• Input Operations
• Feature Engineering Operations

Streaming Architecture

Course: 52 Minutes

• Exploring Stream Data
• Exploring Dataflow Triggers and Late Data
• Installing Java JDK on Windows 10
• Installing Apache Maven on Windows 10
• Installing Google Cloud SDK on Windows 10
• Implementing Streaming Pipelines with Java
• Implementing Streaming Pipelines with Python

Practice: Feature Engineering and Streaming

Course: 6 Minutes

• Exercise: Features and Streaming Data Models

Data Stream Operations

Course: 36 Minutes

• Course Introduction
• Handling Late Data Recap
• Understanding Watermarks, Triggers, and Accumulation
• Configuring Storage, BigQuery Data, and Pub/Sub
• Configuring Cloud Functions
• Building a Streaming Data Pipeline
• Understanding Batch and Stream Operations

Streaming Analytics

Course: 9 Minutes

• Understanding Streaming Analytics
• Working with Big Data Streaming and BigQuery

Streaming Dashboard

Course: 13 Minutes

• Using Google Data Studio
• Building a Dashboard with Google Data Studio

Stream Processing Pipelines

Course: 6 Minutes

• Using Pipelines, PCollections, and Transforms

Practice: Exploring Streams and Analytics

Course: 7 Minutes

• Exercise: Using Streaming and Analytics Operations

Handling Throughput Operations

Course: 42 Minutes

• Course Introduction
• Introduction to Cloud Spanner
• Cloud Spanner Replication and Instances
• Cloud Spanner Schema, Datatypes and Best Practices
• Bigtable Operations
• Designing a Schema using Bigtable
• Creating a Bigtable Instance

Security Considerations

Course: 25 Minutes

• Secure Data with Security Control
• Identify and Access Management
• GCP Security Layers
• Data and Security Compliance on GCP

Practice: Big Data and Security

Course: 7 Minutes

• Exercise: Overview of Big Data and Security