Building & Deploying

Chapter 13 APIs and HTTP

What Is an API?

An API (Application Programming Interface) is a contract between two programs. One program says "send me a request in this format, and I'll send you a response in that format." The API defines the rules—what you can ask for, how to ask, and what you'll get back.

You've already used APIs without knowing it. When you call len([1, 2, 3]) in Python, you're using a function API—you pass a list, it returns the length. Web APIs work the same way, except the function lives on a remote server and the call travels over the internet.

Think of a restaurant analogy. You (the client) read the menu (the API documentation), place an order with the waiter (send a request), the kitchen (the server) prepares your food, and the waiter brings it back (the response). You never walk into the kitchen. The menu is the contract.

HTTP: The Language of the Web

HTTP (HyperText Transfer Protocol) is the protocol that clients and servers use to communicate. Every time you visit a website, your browser sends an HTTP request and receives an HTTP response. APIs use the same protocol.

An HTTP exchange has two parts:

• Request—sent by the client. Contains a method, a URL, headers, and optionally a body.
• Response—sent by the server. Contains a status code, headers, and a body.

HTTP Methods

The method tells the server what you want to do. There are four you need to know:

GET and DELETE typically don't send a body. POST and PUT send data in the request body (usually JSON).

Status Codes

Every HTTP response includes a status code—a three-digit number that tells the client what happened. You don't need to memorize all of them, just these five groups:

When something goes wrong, the status code is the first thing to check. A 404 means the URL is wrong. A 401 means you're not authenticated. A 500 means the server is broken—not your fault.

JSON: The Data Format

APIs need a shared format for sending data. The standard is JSON (JavaScript Object Notation). You already know JSON—it looks exactly like Python dictionaries and lists:

{
    "id": 1,
    "title": "Buy groceries",
    "done": false
}

Python's json module converts between JSON strings and Python objects. But when using the requests library (below), this conversion happens automatically.

Headers

Headers are metadata attached to requests and responses. They carry information about the message itself, not the data. Two headers you'll see constantly:

• Content-Type: application/json — tells the server "I'm sending you JSON."
• Authorization: Bearer <token> — proves your identity (like a password).

You rarely set headers manually—tools like the requests library handle most of them for you.

Using Python's requests Library

Python's requests library makes HTTP calls simple. It's not part of the standard library, so you need to install it:

Making a GET Request

import requests

response = requests.get("https://api.github.com/users/octocat")

print(response.status_code)  # 200
print(response.json())       # Python dict with user data

requests.get() sends a GET request to the URL. The response object gives you the status code with .status_code and the parsed JSON body with .json().

Making a POST Request

import requests

data = {"title": "Buy groceries", "done": False}
response = requests.post("http://localhost:5000/tasks", json=data)

print(response.status_code)  # 201
print(response.json())       # {"id": 1, "title": "Buy groceries", "done": false}

The json=data parameter automatically converts the Python dict to JSON and sets the Content-Type header for you.

All Four Methods

import requests

BASE = "http://localhost:5000"

# GET    - read
r = requests.get(f"{BASE}/tasks")

# POST   - create
r = requests.post(f"{BASE}/tasks", json={"title": "New task"})

# PUT    - update
r = requests.put(f"{BASE}/tasks/1/done")

# DELETE - remove
r = requests.delete(f"{BASE}/tasks/1")

Consuming a Public API

Let's try a real public API. The GitHub API is free and requires no authentication for basic requests:

import requests

# Get public info about a GitHub user
response = requests.get("https://api.github.com/users/octocat")

if response.status_code == 200:
    user = response.json()
    print(f"Name: {user['name']}")
    print(f"Public repos: {user['public_repos']}")
    print(f"Followers: {user['followers']}")
else:
    print(f"Error: {response.status_code}")

Always check the status code before using the response data. A 200 means success. Anything else means something went wrong, and response.json() might not contain what you expect.

Common Misconceptions

TaskForge Connection

Right now TaskForge is a CLI tool—you interact with it from the terminal. But what if a web frontend, a mobile app, or an AI agent wants to manage tasks? They would need an API. In Chapter 14, you'll build exactly that: a Flask web API for TaskForge. But first, you'll practice consuming an API with Python's requests library so you understand both sides of the conversation.

Micro-Exercises

import requests

response = requests.get("https://api.github.com/users/octocat")
data = response.json()
print(f"Name: {data['name']}")
print(f"Repos: {data['public_repos']}")

import requests

response = requests.get("https://api.github.com/users/this-user-does-not-exist-999999")
print(response.status_code)  # 404

Interactive Exercises

Optional: Web Scraping with Python

Web scraping means making HTTP requests to web pages and extracting data from the HTML. The difference from APIs: APIs return structured JSON, web pages return HTML that you parse.

When to Scrape vs. When to Use an API

If the site has an API, use the API — it's faster, more reliable, and more polite. Scrape only when there's no API and the data is publicly accessible.

Basics with BeautifulSoup

import requests
from bs4 import BeautifulSoup

# Fetch the page
response = requests.get("https://example.com/data")
soup = BeautifulSoup(response.text, "html.parser")

# Find elements
title = soup.find("h1").text
links = [a["href"] for a in soup.find_all("a")]
rows = soup.select("table tr")  # CSS selector

Common patterns:

• soup.find("tag") — first matching element
• soup.find_all("tag") — all matching elements
• soup.select(".class") — CSS selector
• tag.text or tag.get_text(strip=True) — extract text
• tag["href"] — extract attribute

Chapter 14 Building Web APIs with Flask

What Is Flask?

Flask is a minimal Python web framework. It lets you turn Python functions into API endpoints with just a few lines of code. Unlike larger frameworks (Django), Flask gives you only what you need and gets out of the way.

Your First Flask App

A complete Flask app in 5 lines:

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello, World!"

if __name__ == "__main__":
    app.run(debug=True)

Save as app.py and run with python3 app.py. Open http://localhost:5000 in your browser. You just built a web server.

@app.route("/") is a decorator—it tells Flask which URL triggers which function. debug=True auto-reloads on code changes and shows helpful error pages.

Routes and Methods

Each endpoint is a URL + HTTP method combination. Flask defaults to GET only. To accept other methods, specify them explicitly:

@app.route("/tasks", methods=["GET", "POST"])
def handle_tasks():
    if request.method == "GET":
        # return all tasks
        pass
    elif request.method == "POST":
        # create a new task
        pass

You can also use separate functions for each method, which is cleaner:

@app.route("/tasks", methods=["GET"])
def get_tasks():
    # return all tasks
    pass

@app.route("/tasks", methods=["POST"])
def create_task():
    # create a new task
    pass

Returning JSON

APIs return JSON, not HTML. Flask provides jsonify() to convert Python dicts to proper JSON responses with the correct Content-Type header:

from flask import Flask, jsonify

@app.route("/status")
def status():
    return jsonify({"status": "running", "version": "1.0"}), 200

The second value (200) is the HTTP status code. Common codes:

Reading Request Data

When a client sends data to your API, you need to read it. Flask provides two main ways:

from flask import Flask, request, jsonify

# JSON body (for POST/PUT requests)
@app.route("/tasks", methods=["POST"])
def create_task():
    data = request.json          # parse JSON body
    title = data.get("title")   # safely get a field
    return jsonify({"title": title}), 201

# Query parameters (for GET requests)
# GET /tasks?status=pending
@app.route("/tasks", methods=["GET"])
def get_tasks():
    status = request.args.get("status", "all")  # default to "all"
    return jsonify({"filter": status})

request.json parses the JSON body sent by the client. request.args reads URL query parameters (everything after the ?).

Building a TaskForge API

Let's build a complete REST API for TaskForge, step by step. We'll use an in-memory list to store tasks—no database needed yet.

Step 1: Setup and Data Store

from flask import Flask, jsonify, request

app = Flask(__name__)

# In-memory task storage
tasks = []
next_id = 1

Step 2: GET /tasks — List All Tasks

@app.route("/tasks", methods=["GET"])
def get_tasks():
    return jsonify(tasks), 200

This returns the entire task list as a JSON array. Simple and direct.

Step 3: POST /tasks — Add a Task

@app.route("/tasks", methods=["POST"])
def create_task():
    global next_id
    data = request.json

    if not data or "title" not in data:
        return jsonify({"error": "title is required"}), 400

    task = {
        "id": next_id,
        "title": data["title"],
        "done": False
    }
    tasks.append(task)
    next_id += 1
    return jsonify(task), 201

Notice the validation: if the client doesn't send a title, we return a 400 Bad Request with a clear error message. Never trust client input.

Step 4: PUT /tasks/<id>/done — Mark Task Complete

@app.route("/tasks/<int:task_id>/done", methods=["PUT"])
def complete_task(task_id):
    for task in tasks:
        if task["id"] == task_id:
            task["done"] = True
            return jsonify(task), 200

    return jsonify({"error": "task not found"}), 404

<int:task_id> is a URL variable—Flask extracts the number from the URL and passes it as a function parameter. If no task matches, we return 404.

Step 5: Run It

if __name__ == "__main__":
    app.run(debug=True)

Testing Your API

With your Flask app running in one terminal, open another terminal and test with curl:

You can also test with Python requests (from Chapter 13):

import requests

# Add a task
r = requests.post("http://localhost:5000/tasks",
                   json={"title": "Test from Python"})
print(r.status_code)  # 201
print(r.json())       # {"id": 1, "title": "Test from Python", "done": false}

# List all tasks
r = requests.get("http://localhost:5000/tasks")
print(r.json())       # [{"id": 1, ...}]

Error Handling in APIs

A good API returns clear error responses so the client knows what went wrong and how to fix it:

# Bad request: missing required field
@app.route("/tasks", methods=["POST"])
def create_task():
    data = request.json
    if not data or "title" not in data:
        return jsonify({"error": "title is required"}), 400

    if not isinstance(data["title"], str) or len(data["title"].strip()) == 0:
        return jsonify({"error": "title must be a non-empty string"}), 400

    # ... create the task

# Not found
@app.route("/tasks/<int:task_id>/done", methods=["PUT"])
def complete_task(task_id):
    for task in tasks:
        if task["id"] == task_id:
            task["done"] = True
            return jsonify(task), 200
    return jsonify({"error": f"task {task_id} not found"}), 404

# Catch unexpected errors
@app.errorhandler(500)
def internal_error(e):
    return jsonify({"error": "internal server error"}), 500

The pattern: always return JSON with an "error" key and an appropriate status code. Never return a bare string or an HTML error page from an API.

Common Misconceptions

TaskForge Connection

You just turned TaskForge from a CLI tool into a web API. Any program—a web frontend, a mobile app, another Python script, or an AI agent—can now create and manage tasks by making HTTP requests. In the Phase 4 Gate, this API is part of the required artifact.

Micro-Exercises

from flask import Flask, jsonify

app = Flask(__name__)

@app.route("/")
def hello():
    return jsonify({"message": "Hello, World!"})

if __name__ == "__main__":
    app.run(debug=True)

from flask import Flask, jsonify, request

app = Flask(__name__)

@app.route("/greet", methods=["POST"])
def greet():
    data = request.json
    name = data.get("name", "stranger")
    return jsonify({"greeting": f"Hello, {name}!"}), 200

if __name__ == "__main__":
    app.run(debug=True)

Interactive Exercises

Connecting a Frontend to Your API

You built an API. Now let's connect it to a web page. You need just enough HTML, CSS, and JavaScript to consume your own API — this is minimum viable frontend, not a full web development course.

HTML in 5 Minutes

HTML is a tree of nested tags that describe a document's structure:

<!DOCTYPE html>
<html>
<head>
    <title>TaskForge</title>
</head>
<body>
    <h1>My Tasks</h1>
    <ul id="task-list"></ul>
    <input id="new-task" type="text" placeholder="New task...">
    <button id="add-btn">Add Task</button>
    <script src="app.js"></script>
</body>
</html>

CSS in 2 Minutes

CSS controls how your HTML looks — colors, layout, spacing, fonts. AI generates CSS well, so you mainly need to read it rather than write it from scratch. The key concept: a <link rel='stylesheet' href='style.css'> tag in your HTML imports a CSS file that styles elements by matching selectors to tags, classes, and IDs.

JavaScript fetch() in 5 Minutes

fetch() calls your API from the browser. Here's a complete example that talks to a TaskForge Flask API:

// GET: Fetch all tasks and display them
fetch("http://localhost:5000/api/tasks")
  .then(response => response.json())
  .then(tasks => {
    const list = document.getElementById("task-list");
    list.innerHTML = "";
    tasks.forEach(task => {
      const li = document.createElement("li");
      li.textContent = task.title;
      list.appendChild(li);
    });
  });

// POST: Add a new task
function addTask(title) {
  fetch("http://localhost:5000/api/tasks", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ title: title, status: "todo" })
  })
  .then(response => response.json())
  .then(task => console.log("Created:", task));
}

Chapter 15 Database Basics

What Is a Database?

A database is structured storage that persists beyond program execution. When you close your Python script, variables disappear. When you close a program backed by a database, the data stays. Every real application—from TaskForge to GitHub to the Anthropic API—stores its data in a database.

SQL vs NoSQL

There are two major families of databases. For this course, we focus on SQL.

SQLite — Your First Database

We start with SQLite because it removes every barrier to getting started:

• No server—it's a library, not a service. No installation, no configuration, no passwords.
• Built into Python—import sqlite3 works out of the box. No pip install required.
• File-based—your entire database is a single file (tasks.db). Copy it, back it up, delete it—just like any file.

SQLite is not a toy. It's used in production by every iPhone, every Android phone, every web browser, and every copy of Python. For learning and for single-user applications like TaskForge, it's the right choice.

Basic SQL

SQL is a language for talking to databases. The core is just 5 commands:

CREATE TABLE — Define Your Structure

CREATE TABLE tasks (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    title TEXT NOT NULL,
    done BOOLEAN DEFAULT 0
);

This creates a table with three columns. PRIMARY KEY AUTOINCREMENT means each task gets a unique, auto-assigned ID. NOT NULL means the title is required. DEFAULT 0 means new tasks start as not done.

INSERT — Add Data

INSERT INTO tasks (title, done) VALUES ('Buy groceries', 0);
INSERT INTO tasks (title, done) VALUES ('Write tests', 0);
INSERT INTO tasks (title, done) VALUES ('Deploy API', 1);

SELECT — Read Data

-- Get all tasks
SELECT * FROM tasks;

-- Get only incomplete tasks
SELECT * FROM tasks WHERE done = 0;

-- Get just titles
SELECT title FROM tasks WHERE done = 0;

UPDATE — Modify Data

UPDATE tasks SET done = 1 WHERE id = 1;

Always include a WHERE clause. Without it, you update every row.

DELETE — Remove Data

DELETE FROM tasks WHERE id = 1;

Same rule: always use WHERE. DELETE FROM tasks with no WHERE deletes everything.

Python's sqlite3 Module

Python includes sqlite3 in the standard library. Here's the workflow:

import sqlite3

# 1. Connect to database (creates file if it doesn't exist)
conn = sqlite3.connect("tasks.db")

# 2. Create a cursor (the object that executes SQL)
cursor = conn.cursor()

# 3. Execute SQL
cursor.execute("""
    CREATE TABLE IF NOT EXISTS tasks (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        title TEXT NOT NULL,
        done BOOLEAN DEFAULT 0
    )
""")

# 4. Insert data
cursor.execute("INSERT INTO tasks (title, done) VALUES (?, ?)", ("Buy groceries", 0))

# 5. Commit changes (saves to disk)
conn.commit()

# 6. Query data
cursor.execute("SELECT * FROM tasks")
rows = cursor.fetchall()
for row in rows:
    print(row)  # (1, 'Buy groceries', 0)

# 7. Close the connection
conn.close()

Key functions: connect() opens the database, cursor() creates an executor, execute() runs SQL, fetchall() retrieves results, commit() saves changes, close() cleans up.

Parameterized Queries

# DANGEROUS — never do this
title = input("Task title: ")
cursor.execute(f"INSERT INTO tasks (title) VALUES ('{title}')")
# A user could type: '); DROP TABLE tasks; --
# And your entire table is deleted.

# SAFE — always do this
title = input("Task title: ")
cursor.execute("INSERT INTO tasks (title) VALUES (?)", (title,))
# The ? placeholder handles escaping automatically.

The ? placeholder tells sqlite3 to safely escape the value. This one rule prevents an entire class of security vulnerabilities.

TaskForge Connection

Replace TaskForge's JSON file storage with SQLite. Here's the before and after:

Before: JSON File Storage

import json

def load_tasks(filepath):
    try:
        with open(filepath, "r") as f:
            return json.load(f)
    except FileNotFoundError:
        return []

def save_tasks(tasks, filepath):
    with open(filepath, "w") as f:
        json.dump(tasks, f, indent=2)

def add_task(tasks, title, filepath):
    task = {"id": len(tasks) + 1, "title": title, "done": False}
    tasks.append(task)
    save_tasks(tasks, filepath)
    return task

After: SQLite Storage

import sqlite3

def get_connection():
    conn = sqlite3.connect("tasks.db")
    conn.row_factory = sqlite3.Row  # access columns by name
    return conn

def init_db():
    conn = get_connection()
    conn.execute("""
        CREATE TABLE IF NOT EXISTS tasks (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            title TEXT NOT NULL,
            done BOOLEAN DEFAULT 0
        )
    """)
    conn.commit()
    conn.close()

def add_task(title):
    conn = get_connection()
    conn.execute("INSERT INTO tasks (title) VALUES (?)", (title,))
    conn.commit()
    conn.close()

def list_tasks():
    conn = get_connection()
    rows = conn.execute("SELECT * FROM tasks").fetchall()
    conn.close()
    return [dict(row) for row in rows]

def complete_task(task_id):
    conn = get_connection()
    conn.execute("UPDATE tasks SET done = 1 WHERE id = ?", (task_id,))
    conn.commit()
    conn.close()

The biggest difference: no more loading the entire file into memory, no more rewriting the whole file on every change. The database handles reads and writes efficiently, even with thousands of tasks.

ORMs — A Preview

An ORM (Object-Relational Mapper) bridges the gap between Python objects and database rows. Instead of writing raw SQL, you define Python classes that map to tables:

# Raw SQL (what you learned above)
cursor.execute("SELECT * FROM tasks WHERE done = 0")
rows = cursor.fetchall()

# ORM style (conceptual — SQLAlchemy)
pending_tasks = Task.query.filter_by(done=False).all()

The most popular Python ORM is SQLAlchemy. We won't teach it in depth here—raw SQL gives you a better understanding of what ORMs do under the hood. But know that ORMs exist and that most production applications use them.

Common Misconceptions

Micro-Exercises

import sqlite3

conn = sqlite3.connect("practice.db")
cursor = conn.cursor()

cursor.execute("""
    CREATE TABLE IF NOT EXISTS tasks (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        title TEXT NOT NULL,
        done BOOLEAN DEFAULT 0
    )
""")

cursor.execute("INSERT INTO tasks (title) VALUES (?)", ("Buy groceries",))
cursor.execute("INSERT INTO tasks (title) VALUES (?)", ("Write tests",))
cursor.execute("INSERT INTO tasks (title) VALUES (?)", ("Deploy API",))
conn.commit()

# Verify
rows = cursor.execute("SELECT * FROM tasks").fetchall()
for row in rows:
    print(row)

conn.close()

import sqlite3

conn = sqlite3.connect("practice.db")
cursor = conn.cursor()

cursor.execute("SELECT * FROM tasks WHERE done = 0")
incomplete = cursor.fetchall()
for task in incomplete:
    print(task)

conn.close()

Interactive Exercises

Chapter 16 Docker and Containers

What Is Docker?

Docker lets you package your application and everything it needs—Python version, libraries, system tools—into a single container that runs identically on any machine. Think of a shipping container: it doesn't matter whether the ship is going to Tokyo or Rotterdam, the box is the same and the contents arrive intact. Docker containers work the same way for software.

Containers are not virtual machines. A VM emulates an entire operating system with its own kernel—heavyweight and slow to start. A container shares the host OS kernel and only isolates the application layer. This makes containers lightweight (megabytes instead of gigabytes) and fast to start (seconds instead of minutes).

Key Concepts

Installing Docker

Install Docker Desktop, which includes the Docker engine, CLI, and a GUI dashboard:

• macOS: Download from docker.com/products/docker-desktop. Drag to Applications. Launch Docker Desktop—the whale icon appears in your menu bar.
• Windows: Download the installer from the same URL. Enable WSL 2 backend during setup. Restart when prompted.

Verify the installation:

Your First Container

Run Docker's built-in test image to confirm everything works:

Docker pulls the hello-world image from Docker Hub (a public registry of images), creates a container from it, runs it, and prints a success message. That's the entire workflow: pull, create, run.

Now try something more useful—a Python REPL inside a container:

The -it flags give you an interactive terminal. You're now inside a Python 3.12 environment that's completely isolated from your host machine. Type exit() to leave.

Writing a Dockerfile

A Dockerfile is a text file that tells Docker how to build an image. Here's one for TaskForge:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "taskforge.py"]

Each line is an instruction:

Why copy requirements.txt before everything else? Docker caches each layer. If your code changes but your dependencies don't, Docker reuses the cached dependency layer—making rebuilds much faster.

Building and Running

Build the image and run a container from it:

The -t taskforge flag gives your image a name (tag). The . tells Docker to use the current directory as the build context (where to find the Dockerfile and files to copy).

Volumes: Persisting Data

Containers are ephemeral—when you stop a container, any data it created inside is lost. This is a problem for TaskForge's SQLite database. Volumes mount a host directory into the container so data survives restarts:

This maps ./data on your host to /app/data inside the container. The SQLite database file lives in this shared directory, so it persists even when the container stops.

Docker Compose Basics

When your project needs multiple services (a web API, a database, etc.), Docker Compose manages them all with a single file:

# docker-compose.yml
services:
  web:
    build: .
    ports:
      - "5000:5000"
    volumes:
      - ./data:/app/data
    command: ["python", "-m", "flask", "run", "--host=0.0.0.0"]

Start everything with one command:

The ports mapping "5000:5000" means: forward port 5000 on your host to port 5000 in the container. This is how you access the Flask API from your browser.

Essential Commands

.dockerignore

Just like .gitignore tells git which files to skip, .dockerignore tells Docker which files to exclude from the build context. This keeps images small and avoids copying sensitive data:

.venv
__pycache__
.git
.env
*.pyc
.pytest_cache

Common Misconceptions

TaskForge Connection

Containerize TaskForge. Write a Dockerfile, build an image, and run it. Verify that TaskForge works the same inside the container as it does on your host machine. This is the foundation for running Claude Code agents in Docker in Phase 6.

Micro-Exercises

Interactive Exercises

Chapter 17 CI/CD and GitHub Actions

What Is CI/CD?

CI (Continuous Integration) means automatically running tests every time code is pushed to a repository. Instead of remembering to run pytest before merging, the system does it for you—every single time, without fail.

CD (Continuous Deployment/Delivery) takes it further: after tests pass, the code is automatically deployed to production (deployment) or packaged and ready for a one-click release (delivery).

The core idea is simple: catch bugs before they reach production. If a test fails, the pipeline stops and notifies you. No broken code gets deployed. No "I forgot to run tests" disasters.

GitHub Actions Basics

GitHub Actions is GitHub's built-in CI/CD system. It's free for public repos and has generous free-tier minutes for private repos. The key concepts:

Workflows live in .github/workflows/ inside your repository. Each workflow is a YAML file (a human-readable data format, similar to JSON but with indentation instead of braces).

Common triggers:

• push — runs when code is pushed to specified branches
• pull_request — runs when a PR is opened or updated
• schedule — runs on a cron schedule (e.g., nightly tests)

Your First Workflow

Here's a complete, working GitHub Actions workflow that runs your tests on every push and pull request:

name: Run Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -r requirements.txt
      - run: python -m pytest

Let's walk through each line:

Save this as .github/workflows/test.yml in your repository. Push to GitHub, and the workflow runs automatically.

Adding More Checks

Tests alone aren't enough. Professional pipelines also check code quality:

Linting with ruff

Linting catches style issues, unused imports, and potential bugs without running the code. Add a linting step to your workflow:

      - run: pip install ruff
      - run: ruff check .

ruff is fast and catches many common Python mistakes. An alternative is flake8, which has been the standard for years.

Type Checking

If you use type hints in your Python code, tools like mypy can check them in CI. This is more advanced—for now, know that it exists and can be added as another step.

Matrix Strategy: Multiple Python Versions

Your code might work on Python 3.12 but break on 3.11. A matrix strategy tests against multiple versions in parallel:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.11', '3.12', '3.13']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -r requirements.txt
      - run: python -m pytest

This creates three parallel jobs, one for each Python version. If any fails, you know exactly which version has the problem.

Status Badges

Add a badge to your README.md that shows whether your tests are passing:

![Tests](https://github.com/YOUR-USERNAME/YOUR-REPO/actions/workflows/test.yml/badge.svg)

Replace YOUR-USERNAME and YOUR-REPO with your actual GitHub username and repository name. The badge turns green when tests pass and red when they fail—visible to anyone viewing the repo.

Secrets Management

Some workflows need API keys or tokens (e.g., for deployment or calling external services). Never hardcode credentials in your workflow files or source code.

GitHub provides Secrets—encrypted variables stored in your repository settings. To add one: go to your repo on GitHub, navigate to Settings, then Secrets and variables, then Actions, and click "New repository secret."

Reference secrets in your workflow with the ${{ secrets.NAME }} syntax:

      - run: python deploy.py
        env:
          API_KEY: ${{ secrets.API_KEY }}
          DATABASE_URL: ${{ secrets.DATABASE_URL }}

When Things Fail

Your workflow will fail. This is normal—it's doing its job by catching problems. Here's how to debug:

Reading the logs: On GitHub, go to the Actions tab, click the failed workflow run, click the failed job, and expand the failed step. The log shows the exact error and line number.

Common failures:

Common Misconceptions

TaskForge Connection

Add a GitHub Actions workflow to TaskForge that runs pytest on every push and pull request. Save it as .github/workflows/test.yml in your TaskForge repository. After pushing, visit the Actions tab on GitHub to see your tests run automatically. This is the same CI infrastructure that CI-triggered agents will use in Phase 5.

Micro-Exercises

name: Run Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -r requirements.txt
      - run: python -m pytest

      - run: pip install ruff
      - run: ruff check .

Interactive Exercises

Chapter 18 Project Architecture

Why Architecture Matters

When TaskForge was a single file, you could hold the entire program in your head. Now it has a Flask API, database functions, tests, a Dockerfile, and a CI pipeline. If all of that lived in one file, you'd spend more time scrolling than coding.

Architecture is how you organize code into files, folders, and modules so that each piece has a clear purpose. Good architecture provides three things:

• Findability—you can locate any piece of functionality quickly.
• Changeability—you can modify one part without breaking others.
• Readability—a new developer (or an AI agent reading your codebase) can understand the structure without reading every line.

Separation of Concerns

Separation of concerns means each file or module handles one responsibility. The API layer handles HTTP requests and responses. The database layer handles data storage and retrieval. The business logic handles the rules of your application. When these concerns are separated, a change to how you store data doesn't require changes to your API routes, and vice versa.

Think of it like a restaurant. The waiter takes orders (API layer), the kitchen cooks food (business logic), and the pantry stores ingredients (data layer). If you reorganize the pantry, the waiter doesn't need retraining.

Common Patterns: From Flat Scripts to Packages

Projects evolve through predictable stages as they grow:

Stage 1: Single File

Everything in one file. Fine for scripts under 100 lines.

# taskforge.py — everything in one file
tasks = []

def add_task(title):
    tasks.append({"title": title, "done": False})

def list_tasks():
    return tasks

if __name__ == "__main__":
    add_task("Buy groceries")
    print(list_tasks())

Stage 2: Multiple Modules

Split by responsibility. Each file does one thing.

# db.py — database operations
# api.py — Flask routes
# cli.py — command-line interface
# models.py — data structures

Stage 3: Package with Directory Structure

Group related modules into directories. Add configuration, tests, and infrastructure files at the top level.

Directory Structure Conventions

Here is a conventional Python project layout. You don't need to memorize it—just know the pattern exists so you can recognize it and follow it.

Key conventions:

• Source code lives in a package directory (same name as the project) with an __init__.py file.
• Tests live in a separate tests/ directory, mirroring the source structure.
• Configuration files (Dockerfile, requirements.txt, .gitignore) live at the project root.
• CI workflows live in .github/workflows/.

Configuration Management

Applications need configuration: database paths, API keys, debug flags. Hardcoding these values into your source code is fragile and insecure. Two approaches:

Environment Variables

Set configuration outside the code, in the environment where the program runs:

import os

DB_PATH = os.environ.get("TASKFORGE_DB", "taskforge.db")
DEBUG = os.environ.get("FLASK_DEBUG", "0") == "1"

os.environ.get() reads an environment variable with a fallback default. This means the same code works in development (using the default) and in production (using the environment variable set by Docker or CI).

The .env File

During development, you can store environment variables in a .env file at the project root:

# .env — local development configuration
TASKFORGE_DB=taskforge.db
FLASK_DEBUG=1

Add .env to your .gitignore so it never gets committed. Each developer has their own local .env. Production uses real environment variables set by the deployment system.

The TaskForge Evolution

Here's how TaskForge grew across the phases, and why each structural change was made:

The progression was natural: you split code when a file gets too long, when two parts change for different reasons, or when you need to test pieces independently. You never reorganize for the sake of it—you reorganize when the current structure creates friction.

When to Refactor

Refactoring means restructuring code without changing what it does. You refactor when:

• A file is too long to navigate (rough threshold: 200+ lines).
• Two parts of the code change for different reasons (e.g., API routes and database queries).
• You find yourself scrolling past large blocks of code to reach the part you need.
• Tests are hard to write because everything is tangled together.

You do not refactor when:

• The current structure works and is easy to understand.
• You're adding a feature and reorganizing at the same time (do one at a time).
• You're copying a "best practice" structure for a 50-line script. Simple code deserves simple structure.

Common Misconceptions

TaskForge Connection

Take the TaskForge code you've built across Phases 2–4 and reorganize it into the conventional structure shown in the diagram above. Create a taskforge/ package directory with __init__.py, api.py, db.py, and cli.py. Move tests into a tests/ directory. Keep Dockerfile, docker-compose.yml, requirements.txt, and .github/workflows/ at the project root. Verify that python -m pytest and docker compose up still work after the reorganization.

Micro-Exercises

# In your current taskforge files, label sections like this:

# --- API LAYER ---
@app.route("/tasks", methods=["GET"])
def get_tasks():
    ...

# --- DATABASE LAYER ---
def init_db():
    conn = sqlite3.connect(DB_PATH)
    ...

# --- CLI LAYER ---
if __name__ == "__main__":
    import sys
    command = sys.argv[1]
    ...

Interactive Exercises

Phase 4 Gate Checkpoint & TaskForge Full Stack

TaskForge Checkpoint

TaskForge is now a full-stack application: CLI, REST API, database, container, CI pipeline. It's ready for data structures and algorithms in Phase 5.

What You Can Now Do

Bridge Exercise: Human Code vs AI Code

You've been building TaskForge by hand. Now compare your approach to an AI-generated version of the same feature. This is exactly the skill you'll practice throughout Phase 5.