Introduction to Data Engineering

In this course, you will be introduced to the data engineering lifecycle, from data generation in source systems, data ingestion, data transformation, storage, all the way to serving data to downstream stakeholders. You'll take note of the key undercurrents that affect all stages of the lifecycle, and start developing a framework for how to think like a data engineer.

To gain hands-on practice, you'll gather stakeholder needs, translate those needs into system requirements, and choose tools and technologies to build systems that provide business value. You'll also build an end-to-end data system that encompasses all stages of the data engineering lifecycle, dive into some of the details of batch and streaming pipelines to serve data for a product recommendation system, and apply principles of good data architecture to assess the security, performance, reliability, and scalability of a web app hosted on AWS.