Introduction to Data Engineering
Chapter 1: What is Data Engineering?
Defining Data Engineering
The Role of a Data Engineer
Data Engineering vs Data Science vs Data Analysis
The Data Lifecycle
Common Data Engineering Tasks
Why Data Engineering Matters for AI
Quiz for Chapter 1
Chapter 2: Foundational Concepts
Understanding Data Types
Data Sources and Collection Methods
Introduction to Databases
Data Warehouses Explained
Data Lakes Explained
Introduction to APIs for Data Retrieval
Hands-on Practical: Identifying Data Types
Quiz for Chapter 2
Chapter 3: Building Your First Data Pipeline
What is a Data Pipeline?
ETL Process Explained
ELT Process Explained
Data Extraction Techniques
Basic Data Transformation Operations
Loading Data into Storage
Simple Pipeline Orchestration Concepts
Practice: Sketching a Basic Pipeline
Quiz for Chapter 3
Chapter 4: Data Storage Fundamentals
Choosing the Right Data Storage
Working with Relational Databases (SQL Basics)
Introduction to NoSQL Databases
Understanding File Storage Systems
Object Storage Basics
Common Data Formats
Practice: Setting up a Simple Database Table
Quiz for Chapter 4
Chapter 5: Introduction to Data Processing
Batch Processing Explained
Stream Processing Explained
Processing Frameworks Overview
Understanding Compute Resources
Data Cleaning Basics
Data Validation Techniques
Practice: Simple Data Cleaning Script
Quiz for Chapter 5
Chapter 6: Essential Tools for Data Engineers
Introduction to SQL for Data Manipulation
Version Control with Git for Code
Command-Line Interface (CLI) Basics
Overview of Cloud Platforms
Introduction to Workflow Schedulers
Practice: Basic Git Commands
Quiz for Chapter 6
Chapter 7: Next Steps in Data Engineering
Areas for Further Learning
Building a Portfolio Project Idea
Contributing to Open Source
Keeping Up with New Tools
Recap of Course Concepts
Quiz for Chapter 7
Understanding Compute Resources
Was this section helpful?
- Computer Organization and Design RISC-V Edition: The Hardware/Software Interface, David A. Patterson and John L. Hennessy, 2021 (Morgan Kaufmann) DOI: 10.1016/C2019-0-02555-5 - This classic textbook provides a comprehensive understanding of CPU, memory, and computer architecture fundamentals, essential for grasping the core compute resources discussed.
- Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann, 2017 (O'Reilly Media) - This highly regarded book offers in-depth explanations of scaling strategies, distributed systems, and the architectural principles behind data processing, directly relevant to horizontal scaling and efficient resource usage.
- Overview of Amazon Web Services, AWS, 2025 (Amazon Web Services) - This official whitepaper introduces fundamental cloud computing concepts, including the benefits of elasticity, scalability, and various service models, which are crucial for understanding how modern data processing resources are provisioned and managed.