Data Engineering & Analytics

The foundation everything else builds on. You'll learn to navigate a Linux system, create and manage files, control who can access what, and chain commands together to automate tasks. By the end, you'll be able to set up users, connect to servers over SSH, install software, and run your first Docker container.

Master the data-engineering fundamentals that power every analytics job. You'll work with CSV, JSON, TSV, and SQLite — the formats you'll meet every day. You'll learn regex patterns, sed cleanup, awk aggregation, cut/paste/join column ops, jq for JSON, and SQL basics. By the end, you can inspect, clean, extract from, and query any tabular data file at the command line.

Turn raw files into answers. You'll aggregate with awk arrays, reshape JSON with jq, inspect CSVs with csvkit, join SQL tables with sqlite3, chain ETL pipelines with tee and xargs, compute log analytics, generate formatted reports, and wield Python's csv/json/statistics/sqlite3 modules from one-liners. By the end, you can turn any CSV, JSON, or log file into a summary report.

Data engineering proper. You'll query CSV, JSON, and Parquet directly with duckdb, reshape DataFrames in pandas, author advanced awk and jq pipelines, write production-grade Python ETL scripts with argparse and logging, and orchestrate end-to-end pipelines with bash, cron, systemd timers, and make. By the end, you can ship an analytics pipeline from raw file to scheduled report.

Data analytics from first principles. You'll write SQL window functions for rankings and running totals, clean dirty datasets by finding nulls / duplicates / type mismatches / outliers, compute descriptive statistics and correlations, extract structured data from text with regex, pull and paginate API responses with curl and jq, produce terminal and matplotlib charts, automate report pipelines with sed templates and cron, and tie it all together in an e-commerce analytics capstone.

Database administration for production PostgreSQL. You'll move from sqlite3 to psql, design schemas with proper types (JSONB, TIMESTAMPTZ, NUMERIC), bulk-load data with \COPY and upserts, diagnose slow queries with EXPLAIN and index them, manage users/roles/permissions with GRANT and pg_hba.conf, use transactions and savepoints for ACID integrity, run pg_dump / pg_restore / VACUUM / ANALYZE for maintenance, and monitor a live database via pg_stat_activity, pg_stat_user_tables, pg_locks, and pg_stat_database.

Data pipeline engineering from first principles. You'll design idempotent pipelines with state management, extract from paginated APIs with retries and backoff, transform with validation and deduplication, load via staging → upsert transactions, handle errors with structured logs and dead letter queues, emit JSON-lines observability, assert data quality between stages, and recover from failures using precise error messages and dependency graphs.

Machine learning from the command line with scikit-learn. You'll build features from CSV with pandas, apply one-hot and ordinal encoding, split train/test with random_state, train decision-tree and linear models via the sklearn fit/predict API, save models with joblib, evaluate with MAE/RMSE/R² for regression and accuracy/precision/recall/F1 for classification, read confusion matrices and feature importances, build batch prediction pipelines with feature-name reindexing, train TF-IDF + Naive Bayes text classifiers with Pipeline, and monitor production models for data and concept drift.

Python for data engineering from first principles. You'll write modular multi-file Python projects with functions and docstrings, connect to databases with DB-API 2.0 (sqlite3/psycopg2), create matplotlib and seaborn visualizations with the Agg backend, ingest data from Excel with skiprows, handle encoding issues with Latin-1, paginate APIs with requests, scrape HTML with BeautifulSoup, write advanced pandas with method chains/pipe/chunks/memory optimization/reshaping, build automated analysis pipelines with argparse and logging, and generate self-contained HTML reports with base64-embedded PNG charts.

Cross-level master labs that use skills from all 9 levels. Four bonus labs — no tutorials, no quizzes, no exam. Messy Data Challenge (clean a vendor dump end-to-end), Pipeline Debugging (fix a broken 3-stage ETL), Automated Intelligence Report (build a daily executive briefing from 90 days of data), and ML Model Showdown (train and compare three classifiers on loan default prediction). Progress tracked independently.