2.Docker Labs.md

# 🐳 Docker Lab Roadmap (DCA-Aligned, Multi-Host)

This roadmap is designed to accompany your 3-node Vagrant-based Docker lab. It covers all practical skills needed for the Docker Certified Associate (DCA) exam and for real-world Docker expertise.

---

## 🔹 Phase 1 – Core Docker Skills (Single Host)

| Lab # | Status | Task                                                                | Docker Host | Key Topics Covered                                     |
| ----- | ------ | ------------------------------------------------------------------- | ----------- | ------------------------------------------------------ |
| 1     | ☑️     | Install Docker & test with `hello-world`                            | docker01    | Docker engine basics                                   |
| 2     | ☑️     | Write a simple `Dockerfile` for a Flask app                         | docker01    | Dockerfile, image builds, `docker build`, `docker run` |
| 3     | ☑️     | Use `docker logs`, `inspect`, `ps`                                  | docker01    | Logging, metadata inspection, debugging                |
| 4     | ☑️     | Handle PID 1 & signals with shell script vs `tini`                  | docker01    | Signal handling, process management                    |
| 5     | ☑️     | Use bind mounts and named volumes                                   | docker01    | Volume types, persistence                              |
| 6     | ☑️     | Backup & restore volume with `docker cp` and `tar`                  | docker01    | Data portability and disaster recovery                 |
| 7     | ☑️     | Resource limits with `--memory` and `--cpus`                        | docker01    | Namespaces, cgroups                                    |
| 8     | ☑️     | Use `docker inspect --format` to extract image/container attributes | docker01    | Filtering CLI output for automation/debugging          |
| 9     | ☑️     | Use `.dockerignore` to optimize build contexts                      | docker01    | Efficient Dockerfile usage                             |
| 10    | ☑️     | Clean up unused images using prune/rmi/system df                    | docker01    | Image lifecycle management                             |
| 11    | ☑️     | Flatten a multi-layer image into a single layer                     | docker01    | Image optimization, layering                           |
## 🔹 Phase 2 – Container Networking & Compose

| Lab # | Status | Task                                               | Docker Host | Key Topics Covered                       |
| ----- | ------ | -------------------------------------------------- | ----------- | ---------------------------------------- |
| 12    | ☑️     | Create custom bridge network, connect 2 containers | docker01    | DNS discovery, container networking      |
| 13    | ☑️     | Deploy Flask + Redis with `docker-compose.yml`     | docker01    | Compose basics, networking, volume reuse |
| 14    | ☑️     | Use Docker healthchecks in Compose                 | docker01    | Container health & lifecycle             |
| 15    | ☑️     | Create a local private registry & push/pull image  | docker01    | Registries, image tags, publishing       |
| 16    | ☑️     | Configure external DNS for Docker containers       | docker01    | Advanced container networking            |
| 17    | ☑️     | Use macvlan and host network modes                 | docker01    | Network drivers, host isolation          |
| 18    | ☑️     | Identify container's external IP/port via inspect  | docker01    | Port discovery, metadata inspection      |
## 🔹 Phase 3 – Multi-Host Swarm Cluster

| Lab # | Status | Task                                                        | Docker Hosts | Key Topics Covered                             |
| ----- | ------ | ----------------------------------------------------------- | ------------ | ---------------------------------------------- |
| 19    | ☑️     | Initialize Swarm on `docker01`, join `docker02`, `docker03` | docker01-03  | Swarm init, node join, manager vs worker roles |
| 20    | ☑️     | Deploy replicated Nginx service with routing mesh           | docker01-03  | Swarm, scaling, ingress mode                   |
| 21    | ☑️     | Deploy global logging agent (e.g., busybox)                 | docker01-03  | Global vs replicated services                  |
| 22    | ☑️     | Create overlay network & interconnect services              | docker01-03  | Overlay networking                             |
| 23    | ☑️     | Use placement constraints to schedule on specific nodes     | docker01-03  | Swarm scheduling strategies                    |
| 24    | ☑️     | Deploy stack via `docker stack deploy`                      | docker01-03  | Stack files, Swarm orchestration               |
| 25    | ☑️     | Lock/unlock swarm and rotate join tokens                    | docker01-03  | Swarm security, token rotation                 |
| 26    | ☑️     | Demonstrate node labels and task placement templates        | docker01-03  | Targeted Swarm scheduling                      |
| 27    | ☑️     | Compare host vs ingress publishing modes                    | docker01-03  | Port publishing models                         |
| 28    | ☑️     | Drain node and observe task redistribution                  | docker01-03  | Node availability and service resilience       |

## 🔹 Phase 4 – Security, Storage, Monitoring

| Lab # | Status | Task                                                        | Docker Hosts | Key Topics Covered                   |
| ----- | ------ | ----------------------------------------------------------- | ------------ | ------------------------------------ |
| 29    | ☑️     | Configure TLS auth for Swarm join tokens                    | docker01-03  | Swarm mTLS, secure joins             |
| 30    | ☑️     | Scan images using Trivy (manual and CI-simulated)           | docker01     | Image scanning, CVEs                 |
| 31    | ☑️     | Explore storage drivers with `docker info` + layered builds | docker01     | Storage drivers, image layering      |
| 32    | ☑️     | Create & inspect signed image with `cosign`                 | docker01     | Image signing, secure supply chain   |
| 33    | ☑️     | Setup basic log forwarding to Loki/Promtail stack           | docker01-03  | Log shipping, observability          |
| 34    | ☑️     | Enable Docker Content Trust and sign/pull trusted images    | docker01     | Content Trust, verified registry use |
| 35    | ☑️     | Describe and demonstrate default engine & Swarm security    | docker01-03  | Default security posture             |
| 36    | ☑️     | Use external certs with registry or Docker daemon           | docker01     | TLS configuration                    |
| 37    | ☑️     | Configure UCP-style RBAC or simulate via labels/metadata    | docker01     | Access control simulation            |
| 38    | ☑️     | Configure non-default logging drivers (`journald`, etc.)    | docker01     | Logging architecture                 |
| 39    | ☑️     | Describe/compare CVE severities & patch process             | docker01     | Vulnerability response               |
| 40    | ☑️     | Explain layered file system and where layers reside         | docker01     | Deep image storage architecture      |
| 41    | ☑️     | Simulate Secret Injection via Bind Mount & tmpfs            | docker01     | Secrets handling                     |
| 42    | ☑️     | Serve Static HTML via Bind Mount                            | docker01     | Simple HTML server via bind mounts   |

---

# Lab Solutions

## Phase 1
### Lab 1 - Install Docker & run Hello-World

This lab ensures Docker is installed correctly and functional on your system. You'll use the `hello-world` image to verify both Docker client and daemon operation.

---

Install Docker
```bash
curl -fsSL https://get.docker.com | sudo bash && sudo usermod -aG docker $USER && newgrp docker
```
#### 🔍 curl Flags Explained
Flag - Description
`-f` **Fail silently** on HTTP errors (e.g., 404). Prevents output if download fails.|
`-s` **Silent mode**: no progress bar or error messages (useful in scripts).|
`-S` **Show error**: show error message **only if** `-s` is used and something goes wrong.|
`-L` **Follow redirects** (e.g., HTTP 301/302 to final URL).|

This part installs docker (pipes the install script into bash)
```bash
curl -fsSL https://get.docker.com | sudo bash
```

This part creates the docker group to avoid having to log-out & log-in again and adds the current user to it, allowing the user to use docker without doing sudo
```bash
sudo usermod -aG docker $USER && newgrp docker
```

Confirm Docker is installed and operational
```bash
docker version
docker info
docker ps
```
You should see version details, system info, and no running containers.

#### 🚀 Run Hello-World Container
```bash
docker run Hello-World
```

This command will pull the `hello-world` image if it's not already on your system, create a container, run it once, and print a message. The container exits immediately after running, illustrating the basic lifecycle:

- **Pull image** → **Create container** → **Run** → **Exit**
#### 🟢 Expected Output

```bash
vagrant@docker01:~$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
e6590344b1a5: Pull complete
Digest: sha256:940c619fbd418f9b2b1b63e25d8861f9cc1b46e3fc8b018ccfe8b78f19b8cc4f
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/
```

#### 📤 Clean Up (Optional)

Remove the container (if it still exists):

```
docker container rm hello-world
```

Remove the image to clean up local storage:

```
docker image rm hello-world
```

Removes the pulled image from your system to save space or re-run the image pull.

---

### Lab 2 - Simple Dockerfile for Flask app

This lab walks you through building a simple Python Flask web app inside a Docker container. You'll write a `Dockerfile`, install dependencies, and run the app locally.

Create the appropriate directory & files:

```
flask-app/
├── app.py
├── requirements.txt
└── Dockerfile
```

- Simple Python application published as a web application via Flask
- Just prints out *"Hello from your Flask app running in Docker!"*
- requirements.txt defines the necessary flask version that pip will install once invoked by the dockerfile

#### 🧪 Step 1: Create Flask App
**app.py**
```python
from flask import Flask
app = Flask(__name__)
  
@app.route('/')

def hello():

    return "Hello from your Flask app running in Docker!"

if __name__ == '__main__':

    app.run(host='0.0.0.0', port=5000)
```

**requirements.txt**
```
flask==2.3.2
```

#### 🧱 Step 2: Write the Dockerfile
**Dockerfile**
```dockerfile
# Base image
FROM python:3.11-slim

# Set work directory
WORKDIR /app

# Install dependencies
COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

# Copy app code
COPY . .

# Expose Flask port
EXPOSE 5000

# Run the app
CMD ["python", "app.py"]
```

#### 🔨 Step 3: Build the Image
From inside the directory, build the image
```bash
docker build -t my-flask-app .
```

#### 🚀 Step 4: Run the Container
```bash
docker run -p 5000:5000 my-flask-app
```

Open a browser, go to http://localhost:5000

Expected output:
```
Hello from your Flask app running in Docker!
```

#### 📤 Clean Up
```bash
docker rm -f flask-demo
docker image rm flask-lab
```

---
### Lab 3 - Using `docker logs`, `inspect`, and `ps`

This lab introduces key inspection tools for diagnosing and debugging running containers such as `docker ps`, `inspect`, and `logs`. You'll also extract specific data using `--format`.

#### 🧪 Step 1: Launch a Test Container
```bash
docker run -d --name logtest nginx
```
- Starts an Nginx container in detached mode.

#### 📋 Step 2: View Logs
```bash
docker logs logtest
docker logs -f logtest
```
- Shows container STDOUT/STDERR output.
- `-f` follows logs live.

**Inspect Container Metadata**
```bash
docker inspect logtest
```

Key fields of interest:
- IP address:  
    `docker inspect --format='{{.NetworkSettings.IPAddress}}' logtest`
- Environment variables:  
    `docker inspect --format='{{.Config.Env}}' logtest`
- Mounts:  
    `docker inspect --format='{{.Mounts}}' logtest`
#### 🧾 Step 3: View Container State
```bash
docker ps            # Running containers
docker ps -a         # All containers
docker ps --format "{{.Names}} - {{.Status}}" # Use --format to customize the output
```

Used to verify state and runtime metadata.

#### 🧪 Failure Debug Example
```bash
docker run --name failtest busybox false
docker ps -a
docker logs failtest
```

Shows exit code and failure reason.

#### 🧱 Addendum: `--restart unless-stopped`

When starting containers with `docker run`, you can ensure they automatically restart after reboots or crashes using a **restart policy**.

Add this flag to your run command:

```
--restart unless-stopped
```

🧠 **What it does:**

- Ensures the container will restart automatically unless explicitly stopped by the user.
    
- Useful for production workloads and long-running services.
    

Example:

```bash
docker run -d --name myservice --restart unless-stopped nginx
```

#### 📤 Clean Up
```bash
docker rm -f logtest
docker image rm nginx
```

---

### Lab 4 - Handling PID 1 and Signals in Docker

This lab demonstrates how to correctly trap and respond to `SIGTERM` signals inside a Docker container using a shell script, avoiding the common pitfalls of PID 1 signal swallowing. Processes in containers must correctly handle signals (like SIGTERM). This lab shows how naive shells fail and how to fix it with `Tini`.

---

#### ❌ Test Case: Improper Signal Handling (`test-bad`)

`bad-entrypoint.sh`
```sh
#!/bin/sh
trap "echo Caught SIGTERM; exit" TERM
echo "Started (bad)"
sleep infinity
```

Dockerfile:
```dockerfile
FROM alpine
COPY bad-entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
```

Procedure:
```bash
docker build -t bad-pid1 .
docker run --name test-bad -d bad-pid1
docker stop test-bad
docker logs test-bad
```

❌ Output:
```
Started (bad)
```

**Explanation:**
- Shell sets trap, but `sleep` becomes a child process
- Shell never receives `SIGTERM` because it’s replaced or bypassed
- `trap` never triggers → no `Caught SIGTERM`

---

#### ✅ Fixed Case: Proper Signal Handling with Shell + Tini (`test-good`)

`good-entrypoint.sh`
```sh
#!/bin/sh
trap "echo Caught SIGTERM; exit" TERM
echo "Started (good)"
while :; do
  sleep 1
done
```

**Explanation:**
- `trap` registers a signal handler for `SIGTERM`
- `while :; do sleep 1; done` keeps the shell alive without replacing it
- The shell remains PID 1 and receives signals directly

---

#### 🐳 Dockerfile (for `good-pid1`)
```dockerfile
FROM alpine
RUN apk add --no-cache tini
COPY good-entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/sbin/tini", "--", "/entrypoint.sh"]
```

**Explanation:**
- `tini` is used to forward signals properly to PID 1
- Script is copied and made executable
- Entrypoint ensures script runs as intended

---

#### ✅ Test Procedure
```bash
docker build -t good-pid1 .
docker run --name test-good -d good-pid1
docker stop test-good
docker logs test-good
```

#### ✅ Expected Output
```
Started (good)
Caught SIGTERM
```

---

#### 🧠 Key Takeaways
- A shell script must remain PID 1 to catch signals via `trap`
- Avoid using `exec` if your script installs signal traps — unless forwarding explicitly
- Use `tini` to ensure proper signal forwarding and zombie process reaping

This setup demonstrates the difference between naive and correct signal handling strategies in Docker.

### Lab 5 - Using Bind Mounts and Named Volumes

In this lab you’ll explore Docker’s three volume types: `bind mounts`, `named volumes`, and `tmpfs`. You’ll also compare their behavior and ideal use cases.

Docker volumes can be either **named** or **anonymous**. Named volumes are easier to manage and persist across container lifecycles.

---

#### 📦 Step 1: Named Volume (Persistent)

**Create a named volume**
```bash
docker volume create my_named_vol
```

**Run a container with the volume**
```bash
docker run -d --name voltest -v my_named_vol:/data busybox sleep 9999
```

**Write and verify persistence**
```bash
docker exec voltest sh -c 'echo hello > /data/hello.txt'
docker rm -f voltest
docker run --rm -v my_named_vol:/data busybox cat /data/hello.txt
```

- We run a container named 'voltest' which mounts 'my_named_vol' and writes "hello" to /data/hello.txt
- We then delete the container (but not the volume!)
- We recreate the container and read the contents of /data/hello.txt since the volume data persist

**Expected output:**
```
hello
```

🧠 **Why use named volumes?**
- Named volumes are created and managed by Docker.
- Ideal for persistent data that outlives containers as they survive container deletion.
- Easier to reference, back up, and inspect.

---

#### 🗂️ #### Step 2: Bind Mount (Host-coupled)

**Create test data on host**
```bash
mkdir ~/bind-test
echo "bind mount works" > ~/bind-test/test.txt
```

**Mount and verify**
```bash
docker run --rm -v ~/bind-test:/app busybox cat /app/test.txt
```

**Expected output:**
```
bind mount works
```

**Explanation:**
- Bind mounts use an exact host path.
- Great for local development, but not portable.

---

#### ⚡ tmpfs Mount (In-memory, ephemeral)

**Simulate a secret file**
```bash
echo "my-secret-value" > ./secret.txt
```

**Mount tmpfs and copy in the secret**
```bash
docker run --rm \
  --tmpfs /tmpfs-secret:rw,size=64m \
  -v $(pwd)/secret.txt:/run/secret.txt:ro \
  busybox sh -c "cat /run/secret.txt > /tmpfs-secret/secret && cat /tmpfs-secret/secret"
```

**Expected output:**
```
my-secret-value
```

**Explanation:**
- `--tmpfs` mounts an in-memory filesystem (RAM only)
- `-v` mounts a real secret as read-only
- Data copied to tmpfs is isolated in RAM and wiped on container exit

**Best Practice:**
- Use tmpfs for sensitive data (e.g. secrets) and ephemeral state
- Do not leave real secrets bind-mounted longer than needed

---

#### 📊 Volume Type Comparison

| Type             | Description                        | Use Case                            | Limitations                         |
|------------------|------------------------------------|--------------------------------------|--------------------------------------|
| **Named Volume** | Docker-managed storage             | ✅ Most stateful containers          | No control over exact path           |
| **Bind Mount**   | Host path into container           | 🟡 Dev/test with host files          | Not portable, brittle across hosts   |
| **tmpfs**        | RAM-only in-container filesystem   | 🟡 Secrets, ephemeral in-memory use | Volatile, disappears after shutdown  |
| **External**     | Plugin-based (NFS, EBS, etc.)      | ✅ Multi-host & cloud scenarios     | Requires plugin & config             |

---

#### ✅ Lab Complete When:
- You’ve used and verified:
  - [ ] Named volume with data persistence
  - [ ] Bind mount with host file access
  - [ ] tmpfs mount for secret handling

Optional:
- [ ] External volume driver like NFS (if available)

### Lab 6 - Backup & Restore Docker Volume (`docker cp` + `tar`)

This lab demonstrates how to back up and restore data from Docker named volumes using standard tools like `docker cp` and `tar`. It's useful for disaster recovery and portability scenarios.

---

#### 📦 Create and Populate Named Volume

**Create a named volume and container:**

```bash
docker volume create alex_test_vol

docker run -dit \
  --name test_container \
  -v alex_test_vol:/data \
  busybox
```

**Add data into the volume:**

```bash
docker exec -it test_container sh -c 'echo "Hello World" > /data/hello.txt'
```

**Verify the data:**

```bash
docker exec test_container cat /data/hello.txt
```

---

#### 🛑 Back Up the Volume

**Copy volume data from container to host:**

```bash
docker cp test_container:/data ./alex_test_vol_backup
```

**Create a compressed archive with `tar`:**

```bash
tar -czvf alex_test_vol_backup.tar.gz alex_test_vol_backup
```

---

#### 💥 Simulate Loss & Restore

**Delete original container and volume:**

```bash
docker rm -f test_container
docker volume rm alex_test_vol
```

**Recreate volume:**

```bash
docker volume create alex_test_vol
```

**Restore data using a temporary container:**

```bash
docker run --rm -it \
  -v alex_test_vol:/data \
  -v $PWD:/backup \
  busybox sh
```

Inside the container:

```sh
cd /data
tar -xzvf /backup/alex_test_vol_backup.tar.gz -C .
mv alex_test_vol_backup/* .
exit
```

---

#### 🔍 Validate Restore

```bash
docker run --rm -v alex_test_vol:/data busybox cat /data/hello.txt
```

**Expected output:**

```
Hello World
```

---

#### 📊 Volume Backup Summary

|Type|Method|Portability|Persistent|Notes|
|---|---|---|---|---|
|Named Volume|`docker cp` + `tar`|✅|✅|Portable backup, reusable|
|Bind Mount|OS-level copy|✅|✅|Depends on host path stability|
|External Volume|NFS, plugins|✅|✅|Needs setup, not always local|

---

#### ✅ Lab Complete When:

-  You created a named volume with test data
    
-  You backed it up with `docker cp` and `tar`
    
-  You restored the data into a new volume
    
-  You verified the restored content

### Lab 7 - Enforce Resource Limits on Containers

This lab demonstrates how to apply resource limits to Docker containers using cgroups and namespaces. This is useful for testing container behavior under constrained environments and ensuring fair resource allocation.

---

#### ⚖️ Run a Limited Container

**Run a container with explicit memory and CPU limits:**

```bash
docker run -dit \
  --name limited_container \
  --memory="100m" \
  --cpus="0.50" \
  busybox sh
```

**Explanation:**

- `--memory="100m"` → Limits the container to 100MB of RAM.
    
- `--cpus="0.50"` → Limits container to 50% of one CPU core.
    

---

#### ⚖️ Observe the Limits from Inside

**Connect to the container:**

```bash
docker exec -it limited_container sh
```

Inside:

```sh
cat /sys/fs/cgroup/memory/memory.limit_in_bytes
cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us
cat /sys/fs/cgroup/cpu/cpu.cfs_period_us
```

Use these values to calculate CPU limit: `quota / period = CPU share`

---

#### 📈 Run a Load Test to Trigger Constraints

**Generate memory pressure (should fail >100MB):**

```bash
docker exec limited_container sh -c "dd if=/dev/zero of=/tmp/bigfile bs=1M count=200"
```

**Generate CPU load:**

```bash
docker exec limited_container sh -c "yes > /dev/null"
```

In another terminal:

```bash
docker stats limited_container
```

Observe CPU throttling and memory cap.

---

#### 📊 Resource Control Summary

| Resource | Flag       | Example | Notes                                   |
| -------- | ---------- | ------- | --------------------------------------- |
| Memory   | `--memory` | `100m`  | Soft limit; kills container if exceeded |
| CPU      | `--cpus`   | `0.5`   | Uses cgroups for CPU time slicing       |

---

#### ✅ Lab Complete When:

-  You launched a container with `--memory` and `--cpus` flags
    
-  You verified resource limits inside the container
    
-  You observed constraint behavior with load tests

---

### Lab 8 - Deep Dive into Container Metadata with `docker inspect`

This lab expands on your knowledge of `docker inspect`, focusing entirely on extracting structured metadata using filters and formatting. You'll use real-world examples to retrieve information useful for automation, monitoring, and troubleshooting.

---

#### 🧪 Step 1: Run a Test Container

```
docker run -d --name inspectme nginx
```

---

#### 🔍 Step 2: Explore Full Metadata

```
docker inspect inspectme
```

Optionally, pretty-print the JSON output (requires `jq`):

```
docker inspect inspectme | jq
```

---

#### 🔎 Step 3: Extract Key Metadata Fields

**Image name:**

```
docker inspect -f '{{.Config.Image}}' inspectme
```

**Container internal IP address (default network):**

```
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' inspectme
```

**Mounts (volumes or bind):**

```
docker inspect -f '{{json .Mounts}}' inspectme | jq
```

**Restart policy (if set):**

```
docker inspect -f '{{.HostConfig.RestartPolicy.Name}}' inspectme
```

---

#### 🧰 Bonus: Extract from `docker ps` Output Using `--format`

```
docker ps --format 'table {{.Names}}	{{.Status}}	{{.Ports}}'
```

This is great for scripts or dashboards

#### 📤 Clean Up
```
docker rm -f inspectme
```

---

### Lab 9 - Using `.dockerignore` to Optimize Build Context

This lab demonstrates how ignoring unnecessary files in your build context reduces Docker image size and build time. While in small examples the image size may not differ significantly, this practice has major benefits in large real-world projects.

---

#### 🛠️ Step 1: Set Up the Project

```bash
mkdir lab9-dockerignore && cd lab9-dockerignore
```

Create the following structure:

```bash
echo -e "FROM alpine\nCOPY . /app\nCMD [\"ls\", \"/app\"]" > Dockerfile
mkdir node_modules && echo "bloat" > node_modules/bloat.js
echo "DEBUG=true" > .env
echo "This is a log file" > debug.log
echo "main content" > main.txt
```

At this point, your folder contains:

```bash
ls -1
Dockerfile
debug.log
.env
main.txt
node_modules/
```

---

#### 🧪 Step 2: Build Without `.dockerignore`

```bash
docker build -t unoptimized-image .
docker run --rm unoptimized-image
```

You should see:
```
Dockerfile
debug.log
main.txt
node_modules
```

> 🔍 Note: `.env` is present in the image filesystem but **won’t be shown** by `ls` inside the container because files starting with a dot are hidden by default unless you use `ls -a`.

---

#### ⚙️ Step 3: Create `.dockerignore`

```bash
echo -e "node_modules\n*.log\n.env" > .dockerignore
```

---

#### 🚀 Step 4: Build With `.dockerignore`

```bash
docker build -t optimized-image .
docker run --rm optimized-image
```

This time, you should only see:
```
Dockerfile
main.txt
```

---

#### 🔍 Optional: Compare Image Sizes

```bash
docker image ls | grep -E "optimized-image|unoptimized-image"
```

> 💡 Even if both images are similarly sized in this small example (~8.3MB), in real-world projects with large folders like `node_modules`, this technique greatly reduces build context and speeds up builds.

---

#### ✅ Lab Complete When:
- You’ve built two images and observed `.dockerignore` excluding files.
- You see reduced file count inside `/app` in the optimized version.
- You understand how this technique benefits large projects even if the test images are similar in size.

---

#### 🧹 Cleanup

```bash
cd .. && rm -rf lab9-dockerignore
docker image rm unoptimized-image optimized-image
```

### Lab 10 - Cleaning Up Unused Docker Resources

This lab teaches you how to reclaim disk space by removing unused Docker containers, images, volumes, and networks.

---

#### 🛠️ Commands

```bash
# Remove unused containers, networks, images (not in use), and build cache
docker system prune -f

# Remove dangling images (no tag and not in use)
docker image prune -f

# Remove **all unused images**
docker image prune -a

# List disk usage by Docker
docker system df

# Remove a specific image
docker rmi <image-id>

# Remove unused volumes (be careful!)
docker volume prune -f
```

Use `docker system df` before and after cleanup to measure impact.

#### 🧨 Optional: Full Docker Reset (Danger Zone)

To completely restore Docker to its post-install state:

```bash
sudo systemctl stop docker
sudo rm -rf /var/lib/docker
sudo systemctl start docker
```

> ⚠️ This will permanently delete **all containers, images, volumes, logs, and metadata**. Use only if you want a clean slate.

---

#### ✅ Lab Complete When:
- You’ve successfully removed all unused resources and reclaimed space.
- Output of `docker system df` shows reduced disk usage.

---

### Lab 11 - Flattening a Multi-Layer Image

Docker images are composed of multiple layers, each representing an instruction in the Dockerfile. Flattening reduces all layers into a single one, which may help simplify image storage or mitigate layer caching in some edge cases.

> ⚠️ Warning: Flattening removes Docker metadata such as environment variables, labels, and CMD — use cautiously.

---

#### 🧠 What's the point of flattening a Docker image?

A standard Docker image has **multiple layers**, each created by commands in your `Dockerfile`. Each layer adds some data and is cached, which is helpful during development but can cause issues in some edge cases:

- When pushing to custom registries with layer limits
    
- When trying to eliminate sensitive files introduced in intermediate layers
    
- When building minimal or reproducible base images

#### 🛠️ Step 1: Set Up the Project

```bash
mkdir lab11-flatten && cd lab11-flatten
```

Create a Dockerfile with multiple layers:

```bash
echo -e "FROM alpine\nRUN touch /layer1\nRUN touch /layer2" > Dockerfile
```

Build the image:

```bash
docker build -t layered-image .
```

---

#### 🔍 Step 2: View Image Layers

To view the layers created by the image:

```bash
docker history layered-image
```

Expected output:
```
IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
<image-id>     ...              RUN touch /layer2                               0B
<image-id>     ...              RUN touch /layer1                               0B
<image-id>     ...              FROM alpine                                     0B
```

> Each `RUN` creates a new layer. Our goal is to flatten these into one.

---

#### 🧪 Step 3: Flatten the Image

```bash
docker run --name flattenme layered-image

docker export flattenme | docker import - flattened-image
```

#### 🔄 What flattening actually does

By running:

```bash
docker run --name flattenme layered-image docker export flattenme | docker import - flattened-image
```

You are:

1. **Running a container** from `layered-image`
    
2. **Exporting the entire container filesystem** as a tarball (like a full snapshot)
    
3. **Re-importing** that filesystem as a **new image**, with no history, no Dockerfile, no layers — just raw filesystem contents

---

#### 🚀 Step 4: Verify Flattened Image

```bash
docker run --rm flattened-image ls /
```

Expected output includes:
```
bin
layer1
layer2
...
```

> All files from the original image are present, but layers have been merged.

#### 🔬 What changes?

Let’s compare the two images:

| Feature                 | `layered-image`           | `flattened-image`        |
| ----------------------- | ------------------------- | ------------------------ |
| Layers                  | Multiple (1 per `RUN`)    | Just 1 layer             |
| Metadata (`CMD`, `ENV`) | Preserved from Dockerfile | **Lost**                 |
| Image size              | Same (in this case)       | Same                     |
| `docker history` output | Shows all layers          | Shows **a single** layer |

Run:
```bash
docker history flattened-image
```

You’ll see something like:
```pgsql
IMAGE          CREATED          CREATED BY    SIZE      COMMENT
083df7dd694b   1 minute ago     import        8.31MB    Imported from -
```

#### 📦 So what did we achieve?

- **You collapsed multiple layers into one**, making the image simpler.
    
- **You stripped all build history and metadata** — which could be useful for security or reproducibility.

---

#### ✅ Lab Complete When:
- You’ve confirmed that `layer1` and `layer2` exist in the final flattened image.
- `docker history flattened-image` shows **only one** layer (use `docker history` to verify).

---

#### 🧹 Cleanup

```bash
cd .. && rm -rf lab11-flatten
docker rm -f flattenme
docker image rm layered-image flattened-image
```

## Phase 2
### Lab 12 - Custom Bridge Network & DNS-based Container Discovery

This lab demonstrates how to create a user-defined Docker bridge network and verify automatic DNS-based name resolution between containers on the same network. This is a fundamental concept in Docker networking, enabling service discovery without hardcoded IPs.

---

#### 🌐 Create a Custom Bridge Network

**Create a new bridge network:**
```
docker network create my_bridge_net
```

**Explanation:**

- Creates a user-defined bridge network with automatic DNS.
    
- Containers on this network can reach each other using container names.

**Verify the network:**
```
docker network ls
docker network inspect my_bridge_net
```

**Launch Containers on the Custom Network**
```
docker run -dit --name container1 --network my_bridge_net busybox sh
docker run -dit --name container2 --network my_bridge_net busybox sh
```

**Explanation:**

- Both containers are attached to `my_bridge_net`.
    
- `--name` assigns a DNS name within the network.

**Test DNS-Based Name Resolution**
```
docker exec container2 ping -c 3 container1
```

**Expected Output:**
```
PING container1 (172.18.0.2): 56 data bytes
64 bytes from container1: icmp_seq=0 ttl=64 time=0.049 ms
64 bytes from container1: icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from container1: icmp_seq=2 ttl=64 time=0.042 ms
```

**📌 Key Concepts Recap**

| Concept                     | Command / Setting             | Notes                                              |
| --------------------------- | ----------------------------- | -------------------------------------------------- |
| Create custom network       | ```docker network create```   | Enables isolated and name-resolvable container net |
| Attach container to network | ```--network my_bridge_net``` | Joins container to user-defined bridge             |
| DNS discovery               | ```ping container_name```     | Only works on user-defined bridge networks         |
**✅ Lab Complete When:**
- A custom bridge network is created
    
- Two containers are launched on that network
    
- One container can resolve and ping the other by name

### Lab 13 - Flask + Redis with Docker Compose & persistent storage

This lab demonstrates how to use Docker Compose to define and deploy a basic multi-container application: a Flask web server that uses Redis as a backend counter. This introduces Compose fundamentals like service linking, volume mounting, and dependency management.

---

#### 📦 Define Your Project Structure

Create the following structure:

```
lab-09/
├── docker-compose.yml
└── app/
    └── app.py
```

---

#### 🧠 What is Redis?

Redis is an in-memory key-value store used for fast data storage and retrieval. In this lab, it's used as a backend counter. Every time the Flask app receives a request, it increments a value in Redis. Redis is commonly used for caching, real-time analytics, session stores, and lightweight message queues.

**Note:** By default, Redis **does not persist** data across container restarts unless a volume is defined. We'll fix this below.

---

#### 📝 Create `app/app.py`

```python
from flask import Flask
import redis

app = Flask(__name__)
r = redis.Redis(host='redis', port=6379)

@app.route('/')
def hello():
    r.incr('hits')
    return f"Hello! Hits: {r.get('hits').decode()}"

app.run(host='0.0.0.0', port=5000)
```

---

#### 🛠 Create `docker-compose.yml`

```yaml
version: '3.8'
services:
  web:
    image: python:3.12-slim
    working_dir: /app
    volumes:
      - ./app:/app
    command: sh -c "pip install flask redis && python app.py"
    ports:
      - "5000:5000"
    depends_on:
      - redis

  redis:
    image: redis:alpine
    volumes:
      - redis-data:/data
    command: redis-server --save 60 1 --loglevel warning

volumes:
  redis-data:
```

---

#### 🚀 Run the Application

```bash
cd lab-09
docker compose up -d
```

**Test in your browser or curl:**

```bash
curl http://localhost:5000
```

**Expected Output:**

```
Hello! Hits: 1
```

On refresh:

```
Hello! Hits: 2
```

If you restart the containers, the counter value will be **preserved** due to the volume attached to Redis.

---

#### 📌 Key Concepts Recap

|Concept|Command / Setting|Notes|
|---|---|---|
|Compose version|`version: '3.8'`|Modern Compose file syntax|
|Volumes|`./app:/app`, `redis-data:/data`|Mounts app code and persists Redis data|
|Dependencies|`depends_on: redis`|Ensures Redis starts before Flask|
|Networking|Automatic via Compose|Services can reach each other by name|
|Redis config|`--save 60 1`|Saves data every 60 sec if at least 1 write occurs|

---

#### ✅ Lab Complete When:

- You create a Compose file with two services: `web` and `redis`
    
- You can visit the web server and see the counter increment
    
- Redis service persists data across container restarts

### Lab 14 - Docker Healthchecks in Compose

This lab demonstrates how to use Docker healthchecks within a `docker-compose.yml` file to monitor container health and control service dependencies. Healthchecks allow your containers to communicate their state to the orchestrator, and they are especially useful for coordinating startup order and recovery logic.

---

#### 🧠 What is a Healthcheck?

A healthcheck is a command that Docker runs inside a container to determine whether it is "healthy." Compose can use this to delay dependent services until the container is ready.

---

#### 📝 Update `docker-compose.yml` from Lab 9

Add a healthcheck to the Redis service and make Flask wait until Redis is healthy before starting:

```yaml
version: '3.8'
services:
  web:
    image: python:3.12-slim
    working_dir: /app
    volumes:
      - ./app:/app
    command: sh -c "pip install flask redis && python app.py"
    ports:
      - "5000:5000"
    depends_on:
      redis:
        condition: service_healthy

  redis:
    image: redis:alpine
    volumes:
      - redis-data:/data
    command: redis-server --save 60 1 --loglevel warning
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5

volumes:
  redis-data:
```

---

#### 🔎 View Healthcheck Status

Run the app:

```bash
docker compose up -d
```

Check Redis health status:

```bash
docker inspect --format='{{json .State.Health}}' <redis-container-id>
```

Or watch:

```bash
docker ps
```

Look for the `healthy` state in the `STATUS` column.

---

#### ❗ Why `depends_on.condition: service_healthy` Matters

This ensures that Flask only starts **after** Redis has successfully passed the healthcheck. Without it, Flask may attempt to connect too early and fail.

---

#### 📌 Key Concepts Recap

|Concept|Field / Setting|Notes|
|---|---|---|
|Healthcheck|`test`, `interval`, `timeout`, etc.|Defines how Docker determines container health|
|Compose dependency|`condition: service_healthy`|Delays startup until dependency is healthy|
|Monitoring|`docker ps`, `docker inspect`|Tools to observe and verify container health|

---

#### ✅ Lab Complete When:

- You define a healthcheck for Redis
    
- You verify the health status via `docker ps`
    
- You confirm that Flask does not start until Redis is healthy

### Lab 15 - Local Docker Registry

This lab demonstrates how to deploy a local private Docker registry, push a custom image to it, and pull it back to verify that the registry is functional. This is foundational for environments with no internet access, restricted image sources, or internal build pipelines.

---

#### 🧠 What is a Docker Registry?

A Docker registry is a service that stores and distributes Docker images. Docker Hub is the default public registry, but you can run your own registry locally or inside your network.

This lab uses the official `registry:2` image to deploy a local registry.

---

#### 🛠 Step 1 - Start the Local Registry

```bash
docker run -d \
  -p 5000:5000 \
  --name registry \
  registry:2
```

**Explanation:**

- Exposes the registry on port `5000`
    
- Container name is `registry`
    
- Detached mode (`-d`) so it runs in background
    

---

#### 🏷️ Step 2 - Tag an Existing Image

```bash
docker tag busybox localhost:5000/mybusybox
```

**Explanation:**

- Prepares `busybox` image for push to `localhost:5000`
    
- `localhost:5000/mybusybox` is the full registry path
    

---

#### 📤 Step 3 - Push the Image

```bash
docker push localhost:5000/mybusybox
```

If this fails with a `http: server gave HTTP response to HTTPS client` error, see the note on insecure registries below.

---

#### 🧪 Step 4 - Test the Registry

Remove the local image and pull it back from the registry:

```bash
docker image rm localhost:5000/mybusybox

docker pull localhost:5000/mybusybox
```

You should see it successfully pulled from your local registry.

---

#### ⚠️ Insecure Registry Note (Optional)

If you're using a tool that **refuses HTTP** (like Docker Desktop or some cloud images), add this to `/etc/docker/daemon.json`:

```json
{
  "insecure-registries" : ["localhost:5000"]
}
```

Then restart Docker:

```bash
sudo systemctl restart docker
```

---

#### 📌 Key Concepts Recap

|Concept|Command / Setting|Notes|
|---|---|---|
|Run registry|`docker run registry:2`|Starts official registry container|
|Tag image|`docker tag`|Prepares image for pushing to registry|
|Push/pull|`docker push`, `docker pull`|Push to and fetch from custom registry|
|Insecure access|`daemon.json` with `insecure-registries`|Required for HTTP-only local registries|

---

#### ✅ Lab Complete When:

- You can run a local Docker registry on port 5000
    
- You tag, push, and pull an image to/from it
    
- You see successful output on `docker pull localhost:5000/mybusybox`

---

### Lab 16 - Configure External DNS for Docker Containers

This lab demonstrates how to configure Docker containers to use external DNS servers for name resolution.

---

#### 🧪 Lab Goals
- Override the default Docker DNS configuration.
- Verify name resolution using public DNS (e.g. 1.1.1.1 or 8.8.8.8).
- Understand the impact of DNS settings in Docker.

---

#### 🛠️ Steps

1. **Create a container using external DNS:**
```bash
docker run -it --rm \
  --dns=1.1.1.1 \
  busybox nslookup google.com
```

2. **Inspect behavior with multiple `--dns` entries:**
```bash
docker run -it --rm \
  --dns=9.9.9.9 --dns=8.8.8.8 \
  busybox nslookup google.com
```

3. **Set default DNS in Docker daemon (optional):**
Edit or create `/etc/docker/daemon.json`:
```json
{
  "dns": ["1.1.1.1", "8.8.8.8"]
}
```
Then restart Docker:
```bash
sudo systemctl restart docker
```

4. **Verify from a running container:**
```bash
docker run -it --rm busybox cat /etc/resolv.conf
```

---

#### ✅ Lab Complete When
- A container successfully resolves `google.com` using an external DNS server.
- You understand how to set DNS at container and daemon level.

---

### Lab 17 - Use Macvlan and Host Network Modes

This lab covers two advanced Docker network modes: `macvlan` for L2 network isolation and `host` for bypassing NAT.

---

#### 🧪 Lab Goals
- Use `macvlan` to assign a container its own IP on the LAN.
- Use `host` mode to share the host’s network stack.

---

#### 🛠️ Steps

**Macvlan (Requires Bridged Network):**
1. **Create a macvlan network:**
```bash
docker network create -d macvlan \
  --subnet=172.16.0.0/22 \
  --gateway=172.16.1.1 \
  -o parent=eth0 \
  macvlan_net
```

2. **Launch a container on the macvlan network with a specific IP:**
```bash
docker run -dit --name macvlan-test \
  --network macvlan_net \
  --ip 172.16.3.211 \
  busybox sh
```

3. **Ping LAN resources from the container:**
```bash
docker exec macvlan-test ping -c 3 172.16.1.1
```

> 🔎 Ensure the IP you assign is free and within the defined subnet. Avoid overlap with DHCP pools or other statically assigned hosts.

> ⚠️ **Note for VMs and Enterprise Networks:**
> Macvlan requires your network to allow MAC spoofing and promiscuous mode.  
> - In **Hyper-V**, enable “MAC address spoofing” in the VM NIC settings.  
> - On **ESXi**, enable forged transmits.  
> - On **physical switches**, ensure the port allows multiple MACs and doesn’t block unknown sources.

**Host Mode:**

> 🧠 **What is host mode?**
> In `--network host`, the container shares the host’s network stack entirely:
> - No user-defined network is created.
> - The container uses the host’s IP address directly.
> - Ports exposed in the container are exposed directly on the host (no NAT).
>
> ✅ **Use cases:**
> - Performance-sensitive workloads (no NAT overhead).
> - Apps that require binding to `localhost` or raw socket access.
> - Legacy apps that assume host networking.
>
> ❌ **Caveats:**
> - No network isolation.
> - Port conflicts with the host.

1. **Run a web server in host mode:**
```bash
docker run -d --name hostmode-nginx \
  --network host \
  nginx
```

2. **Access the web server from your host:**
```bash
curl http://localhost
```

> ❗ **Note for VM-based Docker hosts:**  
> `--network host` binds the container to the **VM’s** network stack.  
> - From **inside the VM**, use `localhost` or `127.0.0.1`.  
> - From **your PC or another device**, use the **VM’s LAN IP** (e.g., `172.16.3.201`).  
> - The container is **not** reachable via `localhost` on your Windows host.

---

#### ✅ Lab Complete When
- macvlan container has its own IP and can ping LAN.
- `host` network container is reachable via host IP directly.

---

#### 🧹 Cleanup
```bash
docker rm -f macvlan-test hostmode-nginx

docker network rm macvlan_net
```

---
### Lab 18 - Identify Container's External IP and Port

> 📦 **Bonus Goal:** Learn how to assign a static IP address to a container using a custom bridge network. This is useful for predictable internal networking, service discovery, or integrating with tools like Consul and CoreDNS.

This lab explores how to determine the external-facing IP and exposed ports of a container.

---

#### 🧪 Lab Goals
- Inspect published ports.
- Identify host-mapped port and container IP.

---

#### 🛠️ Steps

1. **Run a container with a published port:**
```bash
docker run -d --name porttest -p 8081:80 nginx
```

2. **Check port mapping with `docker ps`:**
```bash
docker ps
```
_You should see `0.0.0.0:8081->80/tcp`._

3. **Use `docker inspect` to extract host port and IP:**
```bash
docker inspect porttest | grep -i "hostport"

docker inspect porttest | grep -i "IPAddress"
```

4. **Test with `curl`:**
```bash
curl http://localhost:8081
```

5. **(Bonus)** Run a container with a custom static IP in a user-defined bridge network:

```bash
# Create a custom bridge network with a defined subnet
docker network create --subnet=172.25.0.0/16 customnet

# Run a container with a static IP in this network
docker run -d --name customiptest \
  --network customnet \
  --ip 172.25.0.50 \
  nginx
```

> 🧠 This allows you to define predictable internal IPs between containers without relying on external DNS. It’s especially useful when combining Docker with service discovery or infrastructure tools that need fixed addressing.

```bash
docker network create --subnet=172.25.0.0/16 customnet
docker run -d --name customiptest \
  --network customnet \
  --ip 172.25.0.50 \
  nginx
```

---

#### ✅ Lab Complete When
- You can identify the host port and container IP of any service using inspect.
- You understand the difference between container IP and published ports.

---

#### 🧹 Cleanup
```bash
docker rm -f porttest customiptest

docker network rm customnet
```
## Phase 3
### Lab 19 - Docker Swarm Init & Service Deployment

This lab introduces Docker Swarm, Docker's native clustering and orchestration tool. You'll initialize a Swarm, join worker nodes, and deploy a replicated service across your three-node cluster.

---

#### 🧠 What is Docker Swarm?

Docker Swarm turns a group of Docker hosts into a single virtual host. It enables container orchestration: scheduling, replication, load balancing, and service discovery across multiple nodes.

---

#### ⚙️ Lab Requirements

- 3 Docker hosts: `docker01`, `docker02`, `docker03`
    
- All reachable over a private network (e.g., 172.16.3.0/24)
    
- SSH or shell access to each
    

---

#### 🛠 Step 1 - Initialize the Swarm on `docker01`

```bash
docker swarm init --advertise-addr 172.16.3.101
```

**Explanation:**

- `--advertise-addr` tells the other nodes how to reach the manager
    
- This generates a `docker swarm join` command for workers
    

---

#### ➕ Step 2 - Join `docker02` and `docker03` as workers

Run the tokenized command provided by `docker01`, for example:

```bash
docker swarm join \
  --token SWMTKN-1-xyz... \
  172.16.3.101:2377
```

To reprint the token if needed:

```bash
docker swarm join-token worker
```

---

#### 🔎 Step 3 - Verify the Cluster

Run this on `docker01`:

```bash
docker node ls
```

Expected:

```
ID     HOSTNAME   STATUS  AVAILABILITY  MANAGER STATUS
xxxx   docker01   Ready   Active        Leader
xxxx   docker02   Ready   Active        
xxxx   docker03   Ready   Active        
```

---

#### 📦 Step 4 - Deploy a Replicated Service

```bash
docker service create \
  --name hello \
  --replicas 3 \
  --publish 8080:80 \
  nginx:alpine
```

**Explanation:**

- Creates a service called `hello`
    
- Runs 3 replicas distributed across the nodes
    
- Exposes port 8080 on the cluster via ingress mesh
    

Check service:

```bash
docker service ls
docker service ps hello
```

---

#### 🔁 Step 5 - Test Load Balancing & Access

From any LAN-connected machine:

```bash
curl http://172.16.3.101:8080
```

Or open it in your browser to view the Nginx welcome page:

```
http://172.16.3.101:8080
```

Note: `curl localhost:8080` inside the VM may not work due to interface binding or NAT limitations. Always test from the LAN if using bridged networking.

To verify where each replica runs:

```bash
docker ps -a --filter label=com.docker.swarm.service.name=hello
```

---

#### 🧹 Clean Up

```bash
docker service rm hello
```

Optional: leave the swarm on workers:

```bash
docker swarm leave
```

And on the manager:

```bash
docker swarm leave --force
```

---

#### 📌 Key Concepts Recap

|Concept|Command / Setting|Notes|
|---|---|---|
|Swarm Init|`docker swarm init`|Initializes manager|
|Join Worker|`docker swarm join`|Adds nodes to the cluster|
|Node Mgmt|`docker node ls`|Shows Swarm members|
|Create Service|`docker service create`|Deploys distributed containers|
|Replication|`--replicas N`|Ensures multiple identical instances|
|Load Balancing|`--publish`|Swarm ingress routes traffic across nodes|

---

#### ✅ Lab Complete When:

- All 3 nodes appear in `docker node ls`
    
- You deploy a replicated service
    
- Accessing `http://172.16.3.101:8080` shows Nginx welcome page
    
- Replicas are balanced across nodes

---

#### 🐛 Known Limitation: `localhost` Does Not Work with Swarm Published Ports

When you use `--publish` with `docker service create`, the Swarm ingress routing mesh exposes the service on all nodes — but **not via localhost** inside the host. You must test using the node's external IP, e.g.:

```
curl http://172.16.3.101:8081  # ✅
curl http://localhost:8081     # ❌ (won’t work)
```

This is expected behavior due to how IPVS-based load balancing is implemented in Swarm.

### Lab 20 - Scaling & Rolling Updates in Swarm

This lab builds on your working Docker Swarm cluster by introducing service scaling and rolling updates. You’ll learn how to scale services up/down, perform controlled upgrades, and observe Swarm’s zero-downtime deployment behavior.

---

#### ⚙️ Prerequisites

- A running Swarm with 3 nodes (see Lab 12)
    
- Service `hello` either removed or replaced (we’ll deploy a new one)
    

---

#### 📦 Step 1 - Deploy a Versioned Web Service

We’ll use a simple versioned web app image:

```bash
docker service create \
  --name versioned \
  --replicas 3 \
  --publish 8081:80 \
  nginxdemos/hello
```

Verify:

```bash
docker service ls
docker service ps versioned
```

Access:

```bash
curl http://172.16.3.101:8081
```

You should see a demo page with container hostname and IP.

---

#### 📈 Step 2 - Scale the Service

Increase the number of replicas:

```bash
docker service scale versioned=6
```

Decrease again:

```bash
docker service scale versioned=2
```

Swarm will schedule or remove containers automatically.

Check running tasks:

```bash
docker service ps versioned
```

---

#### 🔁 Step 3 - Perform a Rolling Update

Update the image (simulate a new version):

```bash
docker service update \
  --image nginxdemos/hello:plain-text \
  versioned
```

Swarm will:

- Gracefully shut down one replica at a time
    
- Pull and start new containers with the updated image
    
- Maintain service availability throughout
    

Watch it live:

```bash
docker service ps versioned
```

---

#### ⚙️ Step 4 - Control Update Parameters (Optional)

Run an update with controlled rate:

```bash
docker service update \
  --image nginxdemos/hello:plain-text \
  --update-parallelism 1 \
  --update-delay 10s \
  versioned
```

**Explanation:**

- `--update-parallelism 1`: update one container at a time
    
- `--update-delay 10s`: wait 10 seconds between updates
    

---

#### 🧹 Clean Up

```bash
docker service rm versioned
```

---

#### 📌 Key Concepts Recap

|Concept|Command / Flag|Notes|
|---|---|---|
|Scaling|`docker service scale`|Adjusts replica count dynamically|
|Rolling Updates|`docker service update`|Zero-downtime service upgrades|
|Update control|`--update-parallelism`, `--delay`|Controls update strategy during upgrades|
|Versioned Image|`nginxdemos/hello`|Shows container hostname, useful for testing|

#### 🧱 Enhancement: `docker stats` and `docker system df`

Use these commands to inspect **runtime resource usage** and **Docker system footprint**.

- `**docker stats**` – Live metrics (CPU, memory, I/O, network) for running containers.
    

```
docker stats
```

Useful for debugging resource-heavy containers.

- `**docker system df**` – Summary of image, container, and volume disk usage.
    

```
docker system df
```

Great for cleanup planning and disk usage auditing.

🧠 **When to use:**

- Use `docker stats` to monitor runtime behavior.
    
- Use `docker system df` to investigate storage bloat or orphaned data.
    

---

---

#### ✅ Lab Complete When:

- You scale the service up and down successfully
    
- You perform a rolling update without downtime
    
- You observe the updated version on `http://172.16.3.101:8081`
    

### Lab 21 - Deploy Global Logging Agent in Docker Swarm

This lab introduces **global services** in Docker Swarm by deploying a simple logging agent (BusyBox) that runs on every Swarm node. You will compare this behavior with replicated services.

---

#### 🧠 Key Concepts

- **Global service**: Ensures **one task per node**.
    
- **Replicated service**: Spawns a specified number of **replicas**, regardless of node count.
    
- Useful for logging, monitoring agents (e.g., Fluent Bit, Promtail).
    

---

#### ⚙️ Step 1: Create a Global Service

```bash
docker service create \
  --name global-logger \
  --mode global \
  --mount type=bind,source=/var/log,destination=/host-logs,readonly \
  busybox:latest \
  sh -c "tail -F /host-logs/syslog"
```

- `--mode global`: One container per node
    
- Binds `/var/log` from host for log monitoring
    

Check deployment:

```bash
docker service ps global-logger
```

Should show **1 task per node** (docker01, docker02, docker03).

---

#### 🔄 Step 2: Compare With Replicated Mode

```bash
docker service create \
  --name replicated-logger \
  --replicas 3 \
  busybox:latest \
  top
```

Observe:

```bash
docker service ps replicated-logger
```

You may see uneven distribution if nodes are busy or constrained.

---

#### 🔍 Step 3: Inspect Running Containers

```bash
docker ps -a --filter name=global-logger
```

- Validate that all nodes have a `global-logger` container
    
- SSH into each node to confirm
    

---

#### 🧼 Step 4: Clean Up

```bash
docker service rm global-logger replicated-logger
```

---

#### ✅ Expected Outcome

- `global-logger` runs **exactly once** on each Docker host
    
- Demonstrates use case for monitoring agents
    
- `replicated-logger` shows flexible scaling
    

---

#### 📝 Notes

- Global mode is ideal for node-wide tools
    
- No need to define replica count for global services
    
- Bind mounts should be used carefully with global agents
    
- Ensure log directory (`/var/log`) exists and is readable on all nodes
    


### Lab 22 - Create Overlay Network & Interconnect Services

This lab demonstrates how to manually create a user-defined **overlay network** in Docker Swarm and interconnect multiple services across Swarm nodes.

⚠️ **Note**: Starting from this lab, the environment was migrated from VirtualBox to Hyper-V. Overlay networking was unreliable with VirtualBox bridged adapters due to VXLAN packet issues. Hyper-V resolved all cross-node communication problems.

---

#### 🧠 Key Concepts

- **Overlay network**: Enables cross-node container communication in Swarm.
    
- **Service discovery**: Built-in DNS for container-to-container resolution.
    
- **attachable**: Allows standalone containers to join overlay networks.
    

---

#### ⚙️ Step 1: Create a User-Defined Overlay Network (Run on Manager Node Only)

```bash
# Run this on the Swarm manager node only (e.g., docker01)
docker network create \
  --driver overlay \
  --attachable \
  my_custom_overlay
```

Check that the network is created:

```bash
docker network ls
```

Look for `my_custom_overlay` of type `overlay`.

---

#### 🐳 Step 2: Deploy Two Interconnected Services

**Service A** - web server

```bash
docker service create \
  --name webapp \
  --replicas 2 \
  --network my_custom_overlay \
  nginx:alpine
```

**Service B** - client tool (wget using BusyBox)

```bash
docker service create \
  --name client \
  --replicas 1 \
  --network my_custom_overlay \
  busybox \
  /bin/sh -c 'while true; do wget -qO- http://webapp; sleep 5; done'
```

---

#### 🔎 Step 3: Observe Inter-Service Communication

```bash
docker service logs client
```

Expected: Output from `wget` showing HTML from nginx.

---

#### 🧪 Step 4: Test Standalone Container Connectivity (Optional)

```bash
docker run -it --rm --network my_custom_overlay busybox sh
```

Inside container:

```bash
wget -O- webapp
```

This tests that a manually-run container can resolve and reach services on the overlay.

---

#### 🧼 Step 5: Clean Up

```bash
docker service rm webapp client
docker network rm my_custom_overlay
```

---

#### ✅ Expected Outcome

- `webapp` is reachable by `client` via DNS name
    
- Overlay network spans all three Swarm nodes
    
- Demonstrates how to manually connect and troubleshoot service communication
    

---

#### 📝 Notes

- Docker uses VXLAN to encapsulate traffic between hosts
    
- Overlay MTU should match physical interface MTU; 1450 often works well
    
- Use `docker network inspect my_custom_overlay` for detailed visibility
    
- Overlay is required for any multi-node Swarm communication outside ingress
    

---

Next: Lab 15.5 provides troubleshooting guidance for overlay network issues.

---


### Lab 22.5 - Troubleshooting Overlay Networks in Docker Swarm

This mini-lab helps you diagnose and resolve problems with **overlay networking** in a Swarm cluster, especially when services on different nodes fail to communicate.

---

#### 🧪 Symptoms of Overlay Issues

- `client` connects but hangs on `wget http://webapp`
    
- `client` task gets stuck in `Starting` or `Preparing`
    
- No logs appear in `docker service logs`
    
- `docker run` can resolve service name but never receives a response
    
- `ip link show | grep vxlan` returns nothing on any node
    

---

#### ✅ Step 1: Check for VXLAN Interfaces

```bash
ip link show | grep vxlan
```

Expected: At least one `vxlan...` interface should exist **if cross-node communication is active**.

---

#### ✅ Step 2: Test UDP 4789 Reachability

```bash
sudo ss -lun | grep 4789
```

Expected:

```
udp   UNCONN 0 0 0.0.0.0:4789 0.0.0.0:*
```

---

#### ✅ Step 3: Check Firewall Rules

```bash
sudo iptables -L -n | grep 4789
```

If you see DROP rules, allow explicitly:

```bash
sudo iptables -A INPUT -p udp --dport 4789 -j ACCEPT
```

---

#### ✅ Step 4: Confirm Node-to-Node Connectivity

```bash
ping -c 3 172.16.3.101
ping -c 3 172.16.3.102
ping -c 3 172.16.3.103
```

---

#### ✅ Step 5: Check and Set MTU

```bash
ip link | grep mtu
sudo ip link set dev eth1 mtu 1450
```

_(Replace `eth1` with your NIC name)_

---

#### ✅ Step 6: Recreate Overlay Network Cleanly

```bash
docker service rm client webapp
docker network rm my_custom_overlay

docker network create \
  --driver overlay \
  --attachable \
  my_custom_overlay

docker service create \
  --name webapp \
  --replicas 2 \
  --network my_custom_overlay \
  nginx:alpine

docker service create \
  --name client \
  --replicas 1 \
  --network my_custom_overlay \
  busybox \
  /bin/sh -c 'while true; do wget -qO- http://webapp; sleep 5; done'
```

---

#### ✅ Step 7: Force Client on Same Node

```bash
docker service create \
  --name client \
  --replicas 1 \
  --network my_custom_overlay \
  --constraint 'node.hostname == docker01' \
  busybox \
  /bin/sh -c 'while true; do wget -qO- http://webapp; sleep 5; done'
```

---

#### 📝 Notes

- Overlay networks only appear in `docker network ls` on a node **after tasks use them**
    
- Use `docker ps` to find the real container ID — not `docker service logs` with task ID
    
- ⚠️ **VirtualBox bridged adapters break overlay networks** — Hyper-V or cloud VMs are preferred
    

---

### Lab 23 - Use Placement Constraints to Schedule on Specific Nodes

This lab explores **placement constraints** in Docker Swarm, which allow you to control where services are deployed based on node attributes such as hostname or custom labels.

---

#### 🧠 Key Concepts

- **Placement constraints**: Control service scheduling to specific nodes
    
- **Node labels**: Custom metadata you can assign to Swarm nodes
    
- Useful for hardware-specific deployments, zone-aware scheduling, or affinity/anti-affinity
    

---

#### ⚙️ Step 1: View Swarm Node Hostnames

```bash
docker node ls
```

Take note of each node’s hostname (e.g., `docker01`, `docker02`, `docker03`).

---

#### 🐳 Step 2: Deploy a Service to a Specific Node

Force a service to run only on `docker01`:

```bash
docker service create \
  --name constrained-service \
  --constraint 'node.hostname == docker01' \
  busybox \
  sleep 3600
```

Check that the task is running on the correct node:

```bash
docker service ps constrained-service
```

---

#### 🏷️ Step 3 (Optional): Add a Custom Node Label

Apply a custom label to a node:

```bash
docker node update \
  --label-add zone=alpha \
  docker02
```

Then deploy a service using that label:

```bash
docker service create \
  --name zone-alpha-service \
  --constraint 'node.labels.zone == alpha' \
  busybox \
  sleep 3600
```

Inspect the label:

```bash
docker node inspect docker02 --format '{{ .Spec.Labels }}'
```

---

#### 🧼 Step 4: Clean Up

```bash
docker service rm constrained-service zone-alpha-service
```

(Optional: remove node label)

```bash
docker node update --label-rm zone docker02
```

---

#### ✅ Expected Outcome

- Services are only deployed to nodes that match the hostname or label constraint
    
- Placement constraints give you fine-grained control over Swarm scheduling
    

---


### Lab 24 - Deploy Stack via `docker stack deploy`

This lab introduces **Docker Stack files**, which define multi-service applications for Swarm deployment. You’ll learn how to structure a `docker-compose.yml` file for use with Swarm and deploy it using `docker stack deploy`.

---

#### 🧠 Key Concepts

- **Stacks**: Logical collections of services defined in a `docker-compose.yml` file
    
- **`docker stack deploy`**: Deploys and manages stacks on a Swarm cluster
    
- **Swarm-compatible Compose**: Only supports a subset of Compose features
    

---

#### 📁 Step 1: Create a Stack File

Create a file named `stack.yml` with the following contents:

```yaml
version: '3.8'
services:
  webapp:
    image: nginx:alpine
    deploy:
      replicas: 3
    ports:
      - "8080:80"

  client:
    image: busybox
    command: /bin/sh -c 'while true; do wget -qO- http://webapp; sleep 5; done'
    deploy:
      replicas: 1
```

---

#### 🚀 Step 2: Deploy the Stack

```bash
docker stack deploy -c stack.yml mystack
```

Verify the services:

```bash
docker stack services mystack
```

Check task placement:

```bash
docker stack ps mystack
```

---

#### 🧼 Step 3: Clean Up

```bash
docker stack rm mystack
```

---

#### ✅ Expected Outcome

- Stack is deployed with 2 services: `webapp` (3 replicas) and `client` (1 replica)
    
- `client` repeatedly fetches pages from `webapp`
    
- All services are deployed under the stack namespace (`mystack_` prefix)
    

---

#### 📝 Notes

- Stack files must use version `3.x` for compatibility with Swarm
    
- `deploy:` settings are **ignored** by `docker-compose up`, but **used** by `docker stack deploy`
    
- Logs can be viewed with:
    
    ```bash
    docker service logs mystack_client
    ```
    

---

### Lab 24.5 - Swarm + Traefik Ingress Routing

This advanced lab demonstrates how to integrate **Traefik** with **Docker Swarm** to dynamically route traffic to Swarm services using labels and a reverse proxy model.

> ❌ This lab was postponed due to persistent networking issues with Traefik in Docker Swarm. Traefik was running and the overlay network was configured, but traffic routing to services via hostname (`whoami.localhost`) failed. Dashboard was unreachable and no services were exposed via expected ports despite correct label setup.

> ✅ Traefik did run correctly as a Swarm global service with Docker socket access and discovered services, but actual ingress routing failed to complete in the given time.

> 💡 This lab will be revisited in Phase 4 or as a dedicated advanced topic. It is a cool example of using a reverse proxy with Docker Swarm.

---

#### 🧠 Key Concepts

- **Traefik**: Reverse proxy that auto-discovers Swarm services using labels
    
- **Ingress Routing**: Map hostnames or paths to services
    
- **Global Mode**: Traefik runs on every Swarm node to expose `:80`/`:443`
    
- **Docker Socket Access**: Required for dynamic configuration
    

---

#### ⚙️ Step 1: Create Overlay Network

```bash
docker network create \
  --driver=overlay \
  --attachable \
  traefik-public
```

---

#### 📦 Step 2: Deploy Traefik (Global Service)

Create a file `traefik.yml`:

```yaml
version: '3.8'
services:
  traefik:
    image: traefik:v2.11
    command:
      - "--providers.docker.swarmMode=true"
      - "--providers.docker.exposedByDefault=false"
      - "--entrypoints.web.address=:80"
      - "--api.dashboard=true"
    ports:
      - "80:80"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    networks:
      - traefik-public
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
networks:
  traefik-public:
    external: true
```

Deploy Traefik:

```bash
docker stack deploy -c traefik.yml traefik
```

---

#### 🧪 Step 3: Deploy Sample Web App With Labels

Create a file `web.yml`:

```yaml
version: '3.8'
services:
  whoami:
    image: traefik/whoami
    networks:
      - traefik-public
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
        - "traefik.http.services.whoami.loadbalancer.server.port=80"
networks:
  traefik-public:
    external: true
```

Deploy:

```bash
docker stack deploy -c web.yml webapp
```

---

#### 🌐 Step 4: Access The App

Add this to your host's `/etc/hosts` or Windows `hosts` file:

```
127.0.0.1 whoami.localhost
```

Visit: [http://whoami.localhost](http://whoami.localhost/)

You should see dynamic metadata about the container, routed via Traefik.

---

#### 🧼 Step 5: Clean Up

```bash
docker stack rm webapp
docker stack rm traefik
```

---

#### ✅ Expected Outcome

- Traefik listens on port 80 and routes requests based on labels
    
- `whoami` responds at `http://whoami.localhost`
    
- No need for manual container IPs or service discovery logic
    

---

#### 📝 Notes

- The `traefik-public` overlay is shared between Traefik and all routed services
    
- Traefik only watches **manager** node tasks in Swarm
    
- You can expose Traefik dashboard by adding another router rule

---

### Lab 25 – Swarm Lock and Token Rotation

#### Goal
Demonstrate swarm security operations including **swarm lock**, **unlock**, and **join token rotation**.

#### Steps
1. **Enable swarm autolock**:
   ```bash
   docker swarm update --autolock=true
   ```

2. **Simulate restart**:
   ```bash
   systemctl restart docker
   docker info  # should say "Swarm: (unlocked|locked)"
   ```

3. **Unlock the swarm**:
   ```bash
   docker swarm unlock-key  # shows the unlock key
   docker swarm unlock
   ```

4. **Rotate join tokens**:
   ```bash
   docker swarm join-token worker
   docker swarm join-token --rotate worker
   docker swarm join-token --rotate manager
   ```

5. Validate that **old tokens are invalidated**.

#### Lab Complete When
- You’ve locked and unlocked the swarm and successfully rotated both join tokens.

---

### Lab 26 – Node Labels and Task Placement Templates

> 🧪 **Goal:** Extend placement constraints by injecting dynamic task-specific metadata using node labels and task templates.

---

> 🧠 **Why this matters:**
> Node labels allow you to define custom metadata on Swarm nodes (e.g., `env=prod`, `zone=eu-west`). Placement constraints can then ensure that certain services run only on specific nodes that match this metadata.
> 
> Task templates, using Go-style placeholders like `{{.Node.Hostname}}`, enable services to inject runtime metadata (e.g., node name, ID) into container environments. This is useful for monitoring, logging, or automatically configuring containers based on their assigned node.

---

#### 🛠️ Steps

1. **Label nodes for roles:**
```bash
docker node update --label-add env=production docker02
```

2. **Deploy a service using a placement template:**
```bash
docker service create \
  --name app-prod \
  --constraint 'node.labels.env == production' \
  nginx:alpine
```

3. **Verify task placement:**
```bash
docker service ps app-prod
```

4. **Use task template env vars (advanced):**
```bash
docker service create \
  --name templated \
  --env NODE_ID={{.Node.ID}} \
  --env HOSTNAME={{.Node.Hostname}} \
  --constraint 'node.role == worker' \
  busybox \
  sh -c 'echo $NODE_ID $HOSTNAME && sleep 300'
```

> 🧠 Templates allow tasks to inherit contextual data at runtime.

🧠 **Important Note:**
Docker Swarm **does not dynamically respond** to label or environment changes on live services:

- Changing a node's label **after** a service is scheduled will **not** move the running task.
- Environment variables are fixed at deployment time — they cannot be updated without recreating the task.
- To re-evaluate constraints or task templates, you must run:
  ```bash
  docker service update --force <service-name>
	```
  This will restart the tasks and apply updated label constraints or templates.
 
✅ **Use this mechanism during maintenance, rebalancing, or dynamic scheduling updates.**.

---

#### ✅ Lab Complete When
- A service runs only on a labeled node.
- Template-based env vars resolve correctly in logs or output.



### Lab 27 – Host vs Ingress Publishing Modes

> 🧪 **Goal:** Understand how Docker Swarm exposes services using `ingress` vs `host` publishing modes and how ingress routing mesh compares to Kubernetes Ingress.

---

> 🧠 **Why this matters:**
> Swarm's ingress routing mesh is a fundamental networking model that resembles basic behavior found in Kubernetes with services of type `LoadBalancer` or `Ingress`. Understanding this now builds intuition for how traffic routing, service exposure, and node-level availability are handled in distributed systems.
> 
> - **Ingress mode** in Swarm uses an internal routing layer (`IPVS`-based) that ensures traffic sent to any node is routed to an available task.
> - **Host mode** bypasses this mesh and binds ports only on the actual node running the container. This is closer to `hostPort` in Kubernetes.


#### 🛠️ Steps

1. **Create two services with different publishing modes:**

**Ingress mode (default):**
```bash
docker service create \
  --name ingress-web \
  --publish published=8082,target=80 \
  nginx:alpine
```

**Host mode:**
```bash
docker service create \
  --name host-web \
  --publish mode=host,published=8083,target=80 \
  nginx:alpine
```

2. **Inspect differences:**
```bash
docker service ps ingress-web

docker service ps host-web
```

3. **Access each service:**
- `http://<any-node-ip>:8082` → Ingress
- `http://<task-node-ip>:8083` → Host mode (only on nodes running tasks)

4. **Optional: Use `docker ps` and `ss -tuln` on each node to confirm which ports are bound and whether IPVS forwarding is active.

```bash
ss -tuln | grep 8082
ss -tuln | grep 8083
```

> 💡 The ingress mode will show listening on all nodes, even those not running tasks. Host mode will only listen on the task node.

> 📌 **Key Differences:**
> - **Ingress**: Exposes service on *all* nodes using routing mesh
> - **Host**: Exposes service *only* on nodes running the container

---

#### ✅ Lab Complete When
- Both services are accessible at the correct ports.
- You understand how ingress routing forwards traffic from any node.
- You can explain the difference between Swarm’s ingress mesh and host mode, and relate it to Kubernetes concepts.
- You understand when to use `host` mode (e.g., load balancers, raw socket apps)

#### 🧹 Cleanup
```bash
docker service rm ingress-web host-web
```

---

### Lab 28 – Drain Node and Observe Task Redistribution

> 🧪 **Goal:** Simulate node maintenance by draining a node and watching tasks migrate automatically.

---

#### 🛠️ Steps

1. **Deploy a replicated service:**
```bash
docker service create \
  --name redis-test \
  --replicas 3 \
  redis:alpine
```

2. **Verify where tasks are running:**
```bash
docker service ps redis-test
```

3. **Drain a node:**
```bash
docker node update --availability drain docker03
```

4. **Observe task migration:**
```bash
docker service ps redis-test
```
Tasks on `docker03` should be rescheduled to other nodes.

5. **Restore availability (after maintenance):**
```bash
docker node update --availability active docker03
```

🧠 **Important Note:**
When you restore a node to `active` state, Swarm does **not** automatically move existing tasks back to it. Tasks remain on their new nodes until rescheduling is triggered.

✅ If you want tasks to redistribute across all eligible nodes, you must manually force an update:
```bash
docker service update --force redis-test
```

This causes all tasks to restart and re-evaluate placement, including the re-activated node..
- Re-activating the node allows future rescheduling.

---

#### ✅ Lab Complete When
- Draining a node forces tasks to move to healthy nodes.
- Re-activating the node allows future rescheduling.

---
#### 🧹 Cleanup
```bash
docker service rm redis-test
```

---
## Phase 4

### Lab 29 - Configure TLS Auth for Swarm Join Tokens

This lab hardens Docker Swarm by enforcing TLS mutual authentication when new nodes join the cluster.

---

#### 🎯 Objective

- Secure the swarm join process
- Understand how Docker uses mTLS (mutual TLS) for authentication

---

#### 🧠 Background

By default, Docker Swarm uses TLS certificates for encrypted and authenticated communication between nodes. However, **join tokens** are a key part of this trust model:

- `docker swarm join-token worker` returns a command that includes a temporary **join token**
- This token allows a node to join the cluster **if it can reach the manager and validate its certificate**

You’ll explore how this process works and how the TLS layer enforces identity.

---

#### 🧪 Step-by-Step

##### 1. Show the Join Token (on `docker01`)

```bash
docker swarm join-token worker
```

##### 2. Inspect TLS Certs

```bash
ls -l /var/lib/docker/swarm/certificates/
```

##### 3. Simulate a Secure Join (on `docker02`)

```bash
sudo docker swarm leave
sudo docker swarm join --token <token> 172.16.3.101:2377
```

---

#### 🔐 Validate mTLS Behavior

Example to simulate failure:

```bash
sudo mv /var/lib/docker/swarm/certificates/ca.crt{,.bak}
sudo systemctl restart docker
```

---

#### ✅ Expected Outcome

- You observe the mutual trust model enforced by Docker Swarm

---

#### 🧼 Cleanup

```bash
docker swarm leave --force
docker swarm init --advertise-addr 172.16.3.101
```

---

### Lab 30 - Scan Images Using Trivy (Manual and CI-Simulated)

---

#### 🎯 Objective

- Install and run Trivy to scan images
- Simulate CI scanning

---

#### 🧪 Step-by-Step

```bash
# Install Trivy
wget https://github.com/aquasecurity/trivy/releases/latest/download/trivy_0.63.0_Linux-64bit.deb -O trivy.deb
sudo dpkg -i trivy.deb

# Pull vulnerable image
docker pull python:3.9-slim

# Basic scan
trivy image python:3.9-slim

# Table and JSON output
trivy image --severity HIGH,CRITICAL python:3.9-slim
trivy image -f json -o trivy-report.json python:3.9-slim
jq . trivy-report.json | less
```

---

#### ✅ Expected Outcome

- Understand CVE output
- Simulate integration with CI

---

#### 🧼 Cleanup

```bash
rm trivy.deb trivy-report.json
```

---

### Lab 31 - Explore Storage Drivers and Image Layering

---

#### 🎯 Objective

- Identify the storage driver
- Explore image layers and cache behavior

---

#### 🧪 Step-by-Step

```bash
# Check storage driver
docker info | grep -i 'Storage Driver'

# Inspect image layers
docker pull nginx:alpine
docker image inspect nginx:alpine --format='{{json .RootFS.Layers}}' | jq

# Multi-stage build
cat > Dockerfile.layered <<EOF
FROM alpine AS base
RUN apk add curl

FROM base AS final
RUN curl https://example.com > /dev/null
EOF

docker build -f Dockerfile.layered -t layered-demo .
docker build -f Dockerfile.layered -t layered-demo .
```

---

#### ✅ Expected Outcome

- Confirm layer caching
- Understand image optimization

---

#### 🤝 Bonus Tips

```bash
du -sh /var/lib/docker/overlay2/
DOCKER_BUILDKIT=1 docker build -f Dockerfile.layered -t layered-demo .
```

---

#### 🧼 Cleanup

```bash
docker rmi layered-demo nginx:alpine
rm Dockerfile.layered
```

---

### Lab 32 - Create & Inspect Signed Docker Images with Cosign

---

#### 🎯 Objective

- Use `cosign` to sign and verify images

---

#### 🧪 Step-by-Step

```bash
# Install Cosign
wget https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64 -O cosign
chmod +x cosign
sudo mv cosign /usr/local/bin/

# Generate key pair
cosign generate-key-pair

# Push image
docker run -d -p 5000:5000 --name registry registry:2
docker tag alpine:latest localhost:5000/alpine:unsigned
docker push localhost:5000/alpine:unsigned

# Sign
cosign sign --key cosign.key localhost:5000/alpine:unsigned

# Verify
cosign verify --key cosign.pub localhost:5000/alpine:unsigned
```

---

#### ✅ Expected Outcome

- Signed image verified by public key

---

#### 🤝 Bonus Tips

```bash
# Get digest
docker inspect --format='{{index .RepoDigests 0}}' localhost:5000/alpine:unsigned

# Sign by digest
cosign sign --key cosign.key localhost:5000/alpine@sha256:<digest>
```

---

#### 🧼 Cleanup

```bash
rm cosign.key cosign.pub
```

---

### Lab 33 - Deploy Loki and Grafana for Log Aggregation

This lab introduces **Grafana Loki**, a log aggregation system, and **Promtail**, the agent that ships logs to Loki. You'll learn how to deploy both Loki and Grafana via Docker, and view logs in the Grafana dashboard.

---

#### 🧠 What is Loki?
Loki is a log aggregation tool by Grafana Labs. It stores logs and indexes only labels for efficiency, unlike Elasticsearch which indexes full log content.

- Lightweight and efficient
- Integrates directly with Grafana
- Designed to work like Prometheus, but for logs

#### 🧠 What is Promtail?
Promtail is an agent that ships logs to Loki. It tails log files, adds labels (e.g., host, job), and pushes them to a Loki endpoint.

Promtail is often deployed on each Docker host. It reads local log files (e.g., `/var/log/syslog`, container logs), applies labels like `job`, `host`, `path`, and sends the log data to the Loki service via HTTP.

In Grafana, once Loki is set up as a data source and receiving logs from Promtail, you can:
- Open the **Explore** tab
- Select the Loki data source
- Use LogQL (Loki Query Language) to filter logs by label or content
  ```
  {job="varlogs"} |= "error"
  ```
- View live tailing of logs and troubleshoot containers or services

---

#### 🚀 Step 1 – Start Loki (using default config)
```bash
docker run -d --name=loki \
  -p 3100:3100 \
  grafana/loki:2.9.4
```

**Check readiness:**
```bash
curl http://localhost:3100/ready
```
Expected output: `ready`  

---

#### 🚀 Step 2 – Start Grafana and Link to Loki
```bash
docker run -d --name=grafana \
  -p 3000:3000 \
  --link loki \
  grafana/grafana:10.2.3
```

**Access Grafana UI:** [http://localhost:3000](http://localhost:3000)  
**Default credentials:** `admin` / `admin`

---

#### 🔗 About `--link`
Great catch — `--link` is one of Docker's **legacy networking options**, and while it's still supported in the bridge network, it’s **deprecated** and rarely used in modern setups.

#### 🔗 `--link` Explained
The `--link` flag allows one container to access another container **by name**, with **predefined environment variables** and DNS resolution.

#### 👇 Example:
```bash
docker run -d --name=grafana \
  --link loki \
  grafana/grafana
```

This:

- Adds a DNS alias inside the `grafana` container pointing to the `loki` container.
- Sets environment variables like `LOKI_PORT_3100_TCP_ADDR`, `LOKI_PORT_3100_TCP_PORT`.

So inside `grafana`, you could access `loki:3100`.

---

#### ❌ Why It's Deprecated

- Doesn’t scale beyond single-host (not Swarm or Compose-friendly)
- Poor isolation and control
- Not customizable
- Replaced by **user-defined bridge networks** (preferred)

---

#### ✅ Modern Alternative

Use a custom bridge network:

```bash
docker network create loki-net

docker run -d --name=loki \
  --network=loki-net \
  -p 3100:3100 \
  grafana/loki:2.9.4

docker run -d --name=grafana \
  --network=loki-net \
  -p 3000:3000 \
  grafana/grafana:10.2.3
```

Now containers can still reach each other via hostname (`loki`) **but in a safer, more maintainable way**.

---

#### 🔁 TL;DR

| Feature        | `--link`            | Modern Approach                |
|----------------|---------------------|--------------------------------|
| Scope          | Legacy              | Preferred                      |
| Networking     | Implicit            | Custom bridge or overlay       |
| DNS Resolution | Yes (via aliases)   | Yes                            |
| Security       | Weak                | Stronger with scoped networks  |
| Future-proof   | ❌ Deprecated       | ✅ Yes                          |

---

#### ⚙️ Step 3 – Configure Grafana to Use Loki
1. Log in to Grafana using admin/admin
2. Go to **Connections > Data Sources**
3. Click **Add data source**
4. Select **Loki**
5. Set URL to: `http://loki:3100`
6. Click **Save & Test** – it should be successful

---

#### 🚀 Step 4 – Deploy Promtail (Optional for Real Logs)
To send real logs from the host to Loki:

```bash
# Download default config
wget https://raw.githubusercontent.com/grafana/loki/v2.9.4/clients/cmd/promtail/promtail-local-config.yaml

# Adjust the config as needed, then run:
docker run -d --name=promtail \
  -v $(pwd)/promtail-local-config.yaml:/etc/promtail/config.yaml \
  -v /var/log:/var/log \
  grafana/promtail:2.9.4 \
  -config.file=/etc/promtail/config.yaml
```

---

#### 🧪 Manual Test Log (Optional)
Once Promtail is running and configured to monitor `/var/log/*.log`, you can test log collection and Grafana integration manually.

**Emit a test error log:**
```bash
echo "ERROR: this is a test error from $(hostname)" | sudo tee -a /var/log/syslog
```

**Query it in Grafana:**
1. Go to the **Explore** tab
2. Select the **Loki** data source
3. Run the following query:
   ```
   {job="varlogs"} |= "ERROR"
   ```

**Tip:** If nothing shows up, try:
```logql
{} |= "ERROR"
```
Or use the **label browser** in Grafana to find the correct labels (e.g., `filename`, `job`, `host`).

---

#### ✅ Expected Outcome
- Loki is accessible at [http://localhost:3100](http://localhost:3100)
- Grafana is accessible at [http://localhost:3000](http://localhost:3000)
- Logs can be viewed via **Explore** tab in Grafana

---

#### 🧹 Cleanup
```bash
docker rm -f loki grafana promtail
```

---

#### 🧪 Lab Complete When...
- ✅ Loki responds with `ready`
- ✅ Grafana is running and connected to Loki
- ✅ You can view dummy or host logs via Grafana Explore
- ✅ A manually injected test error appears in Explore

This sets the foundation for centralized log visibility across multiple Docker nodes using Promtail + Loki + Grafana.

---
### Lab 34 - Enable Docker Content Trust and Pull Verified Images

---

#### 🎯 Objective

- Enable Docker Content Trust (DCT)
- Pull and validate signed images from trusted sources

---

#### 🧠 Background

Docker Content Trust uses **Notary** to ensure the images you're pulling are cryptographically signed and verified. This helps prevent man-in-the-middle attacks or unauthorized modifications.

---

#### 🧪 Step-by-Step

##### 1. Enable Content Trust Temporarily

```bash
export DOCKER_CONTENT_TRUST=1
```

##### 2. Pull a Signed Image

```bash
docker pull alpine:latest
```

✅ If the image is signed, the pull proceeds normally. If not, it fails.

##### 3. Try Pulling an Unsigned Image

```bash
docker pull busybox
```

❌ This may fail, depending on whether it's signed.

---

#### 📌 Notes

- Docker Hub only signs **official images**, not community uploads
- Content trust only works with **pull**, **push**, **build**, and **create**

---

#### 🧼 Cleanup

```bash
unset DOCKER_CONTENT_TRUST
```

---

### Lab 35 – Docker TLS Authentication and Secure Remote Access

In this lab, you'll explore how Docker Engine handles remote access, how to inspect its default security posture, and how to enable secure communication using TLS certificates.

---

#### 🧠 Default Docker Engine Behavior

By default, the Docker daemon (`dockerd`) listens on the **Unix socket**:
```
/var/run/docker.sock
```
This means:
- Only **local users** (usually root or in the `docker` group) can access Docker
- No remote API access is available out-of-the-box
- `docker info | grep -i tls` returns nothing unless TLS is explicitly configured

---

#### 🔒 Why Secure the Docker Daemon?

Exposing the Docker socket over TCP without TLS:
- Allows **root-equivalent** access to the host
- Is a massive security risk if unauthenticated
- Is disabled by default for this reason

When you **do** expose Docker for remote access (e.g., remote Docker clients or Swarm managers), you **must** use TLS.

---

#### 🔍 Check for Remote Access and TLS Support
```bash
ps aux | grep dockerd
```
Look for `-H tcp://0.0.0.0:2376` or similar. If it's missing, Docker is not exposed remotely.

Also try:
```bash
docker info | grep -i tls
```
If TLS is not configured, no output will appear.

---

#### ⚙️ Step 1 – Simulate a Remote-Exposed Docker Daemon with TLS

> ⚠️ **Pre-Step:** To prevent interference from the default Docker socket activation, stop both Docker and its socket before proceeding:
```bash
sudo systemctl stop docker.socket
sudo systemctl disable docker.socket
sudo systemctl stop docker
```

We'll simulate a secure Docker daemon that listens on TCP using TLS certificates.

**Generate TLS certificates** (no passwords, non-interactive):
```bash
mkdir -p ~/docker-certs && cd ~/docker-certs

# Generate CA key
openssl genrsa -out ca-key.pem 4096

# Generate CA cert
openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem \
  -subj "/C=GR/ST=Attica/L=Athens/O=Homelab/CN=docker-ca"

# Generate server key
openssl genrsa -out server-key.pem 4096

# Generate server CSR
openssl req -subj "/CN=docker-host" -new -key server-key.pem -out server.csr

# Sign server cert
openssl x509 -req -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem \
  -CAcreateserial -out server-cert.pem -extfile <(echo "subjectAltName = IP:127.0.0.1")
```

**Launch Docker daemon manually with TLS (detached mode):**

To keep the daemon running in the background, use `nohup`:
```bash
nohup sudo dockerd \
  --tlsverify \
  --tlscacert="$HOME/docker-certs/ca.pem" \
  --tlscert="$HOME/docker-certs/server-cert.pem" \
  --tlskey="$HOME/docker-certs/server-key.pem" \
  -H=0.0.0.0:2376 > ~/dockerd.log 2>&1 &
```
This will detach `dockerd`, redirect its output to `~/dockerd.log`, and keep it running in the background.

If you see an error like:
```
failed to start daemon, ensure docker is not running or delete /var/run/docker.pid
```
it means the regular Docker service is still active. Use `sudo systemctl stop docker` before proceeding.

---

#### ⚙️ Step 2 – Connect with a TLS-Enabled Client

> ⚠️ **Important:** After running a custom `dockerd` process with TLS-only configuration, you will no longer be able to use `docker ps` or `docker info` unless your client is configured for TLS with valid client certificates. This is expected behavior.

> ⚠️ **Reminder:** If `docker.socket` is still active, it may automatically restart the system-managed `dockerd` after you stop it. You should already have disabled it in Step 1.

> ⚠️ **Important:** If `docker.socket` is still active, it may automatically restart the system-managed `dockerd` after you stop it. To prevent conflicts, disable the socket temporarily:
```bash
sudo systemctl stop docker.socket
```
Or permanently if you're switching to TLS-only mode:
```bash
sudo systemctl disable docker.socket
```

> 🧪 **Troubleshooting Tip:** If you see the error:
> ```
> remote error: tls: certificate required
> ```
> it means the server expects **mutual TLS** (client certs), but you're not providing any.
> Ensure your client has the correct `DOCKER_CERT_PATH` with `ca.pem`, `cert.pem`, and `key.pem`, or launch `dockerd` without `--tlsverify` if mutual auth isn't needed.

**Generate client certificate and key:**
```bash
# Generate client key
openssl genrsa -out key.pem 4096

# Create a client certificate signing request
openssl req -subj "/CN=client" -new -key key.pem -out client.csr

# Sign client cert with CA
openssl x509 -req -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem \
  -CAcreateserial -out cert.pem
```

This creates:
- `cert.pem`: client certificate
- `key.pem`: client private key

Ensure these are placed alongside `ca.pem` in your `DOCKER_CERT_PATH` directory.

From another host (or same host if testing locally):
```bash
export DOCKER_TLS_VERIFY=1
export DOCKER_CERT_PATH=$HOME/docker-certs
export DOCKER_HOST=tcp://127.0.0.1:2376

docker info
```
You should see:
```
TLS Verify: true
```

---

#### ✅ Expected Outcome
- You understand that Docker is **not exposed** remotely by default
- You simulate TLS-based Docker access with certs
- `docker info` confirms TLS verify is enabled

---

#### 🧹 Cleanup
If you launched `dockerd` manually, stop it and revert to systemd-managed Docker:
```bash
sudo pkill dockerd
sudo systemctl start docker
```

Remove test certificates:
```bash
rm -rf ~/docker-certs
```

---

#### 🧹 Recovery: Revert Docker to Default (No TLS)

This block restores Docker to a clean, non-TLS state so you can run `docker ps` without TLS flags.

```bash
# 1. Remove custom TLS daemon configuration
sudo rm -f /etc/docker/daemon.json

# 2. Remove systemd override directory (if it exists)
sudo rm -rf /etc/systemd/system/docker.service.d

# 3. Kill any manually-run dockerd processes
sudo pkill -f dockerd

# 4. Remove stale PID file if present
sudo rm -f /var/run/docker.pid

# 5. Reload systemd and restart Docker normally
sudo systemctl daemon-reexec
sudo systemctl daemon-reload
sudo systemctl restart docker

# 6. Confirm Docker is running on default socket
docker ps
```

✅ Expected: Docker CLI should now work without requiring `--tls` or `DOCKER_HOST`.



---

#### 🔐 Optional: Swarm TLS Considerations

While this lab focuses on Docker Engine TLS authentication, note that **Docker Swarm** uses its own TLS-based certificate authority and mutual authentication between nodes:

- `docker swarm init` creates a root CA and node certs automatically
- You can view Swarm certificates in `/var/lib/docker/swarm/certificates`
- Nodes use **mutual TLS** for secure control-plane communication
- To rotate Swarm certs or CA:
  ```bash
  docker swarm ca --rotate
  docker swarm ca --rotate --ca-cert ... --external-ca ...
  ```

> 🧠 Note: Swarm TLS is not affected by `daemon.json` or the standalone `dockerd` TLS settings. It manages its own cert lifecycle internally.

> ⚠️ **Important Clarification:** Swarm’s internal TLS only secures inter-node control traffic. It does **not** expose the Docker API on TCP. If you want to securely access the Docker daemon on another Swarm node (e.g., `docker02`), you must:
> - Manually configure that node with `daemon.json` or TLS flags
> - Install matching CA/client certs
> - Enable `-H tcp://0.0.0.0:2376` in a secure way

If you expose Swarm’s TCP listener to external hosts (e.g., for remote manager join), always verify it is locked down and encrypted.

---

#### 🔐 Best Practices: Docker & Swarm Security

**For Docker Engine:**
- Always require TLS when exposing `dockerd` over TCP
- Rotate TLS certs periodically, and revoke if compromised
- Restrict listening IPs (e.g., use `127.0.0.1` or a management subnet only)
- Avoid using `--link` or legacy networking
- Protect the Unix socket (`/var/run/docker.sock`) with group-based permissions

**For Docker Swarm:**
- Use `docker swarm join-token` carefully and revoke tokens if leaked
- Regularly rotate Swarm CA certs with `docker swarm ca --rotate`
- Limit external access to the Swarm manager API
- Use firewall rules to restrict traffic between nodes to required ports only (e.g., 2377, 7946, 4789)
- Monitor `/var/lib/docker/swarm/` for certificate expiry and configuration drift


While this lab focuses on Docker Engine TLS authentication, note that **Docker Swarm** uses its own TLS-based certificate authority and mutual authentication between nodes:

- `docker swarm init` creates a root CA and node certs automatically
- You can view Swarm certificates in `/var/lib/docker/swarm/certificates`
- Nodes use **mutual TLS** for secure control-plane communication
- To rotate Swarm certs or CA:
  ```bash
  docker swarm ca --rotate
  docker swarm ca --rotate --ca-cert ... --external-ca ...
  ```

> 🧠 Note: Swarm TLS is not affected by `daemon.json` or the standalone `dockerd` TLS settings. It manages its own cert lifecycle internally.

If you expose Swarm’s TCP listener to external hosts (e.g., for remote manager join), always verify it is locked down and encrypted.

---

#### 🧪 Lab Complete When...
- ✅ `docker info` shows `TLS Verify: true`
- ✅ Client connects to Docker via TCP + TLS
- ✅ You understand the default and hardened posture of Docker networking

This lab demonstrates how to transition from a secure-by-default local-only daemon to a TLS-authenticated remote-access deployment.

---

### Lab 36 - Use External TLS Certs with Docker Daemon

This lab documents how to configure **Docker Engine with TLS authentication** permanently via systemd and `daemon.json`, using production paths.

---

#### 📁 1. Create and Secure Certificate Directory
```bash
sudo mkdir -p /etc/docker/certs
cd /etc/docker/certs
```

#### 🔑 2. Generate TLS Certificates (Self-Signed Example)
```bash
# Generate CA private key and certificate
sudo openssl genrsa -out ca-key.pem 4096
sudo openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -subj "/CN=docker-ca" -out ca.pem

# Generate server key and CSR
sudo openssl genrsa -out server-key.pem 4096
sudo openssl req -subj "/CN=docker01" -new -key server-key.pem -out server.csr

# Create server certificate with SAN for 127.0.0.1 and your LAN IP
echo "subjectAltName = IP:127.0.0.1,IP:172.16.3.201" | sudo tee extfile.cnf
sudo openssl x509 -req -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem \
  -CAcreateserial -out server-cert.pem -extfile extfile.cnf

# Cleanup
sudo rm server.csr extfile.cnf ca-key.pem ca.srl
```

Set secure permissions (allow Docker group read-only access):
```bash
sudo chown root:root *.pem
sudo chmod 640 *.pem
sudo chgrp docker *.pem
```

---

#### ⚙️ 3. Configure Docker Daemon for TLS
```bash
sudo nano /etc/docker/daemon.json
```
Paste:
```json
{
  "tls": true,
  "tlsverify": true,
  "tlscacert": "/etc/docker/certs/ca.pem",
  "tlscert": "/etc/docker/certs/server-cert.pem",
  "tlskey": "/etc/docker/certs/server-key.pem",
  "hosts": [
    "unix:///var/run/docker.sock",
    "tcp://0.0.0.0:2376"
  ]
}
```

---

#### 🛠 4. Reset Systemd ExecStart (Required)
```bash
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo nano /etc/systemd/system/docker.service.d/override.conf
```
Add:
```ini
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd
```

---

#### 🔄 5. Reload & Restart Docker
```bash
sudo systemctl daemon-reexec
sudo systemctl daemon-reload
sudo systemctl restart docker
```

---

#### 🔍 6. Test TLS Connection
On the same machine or remotely, use:
```bash
docker --tlsverify \
  --tlscacert=/etc/docker/certs/ca.pem \
  --tlscert=/etc/docker/certs/server-cert.pem \
  --tlskey=/etc/docker/certs/server-key.pem \
  -H=tcp://127.0.0.1:2376 info
```

---

✅ **Lab Complete When**:
- `docker info` succeeds over port 2376 with TLS
- Docker daemon starts cleanly on reboot
- Certificate files are secure in `/etc/docker/certs`, with root:docker ownership and 640 permissions

---

🧹 **Recovery Steps**: Revert Docker TLS Configuration
```bash
# Remove daemon TLS config
sudo rm -f /etc/docker/daemon.json

# Remove systemd override (if exists)
sudo rm -rf /etc/systemd/system/docker.service.d

# Kill any running custom dockerd process
sudo pkill -f dockerd

# Remove stale PID file (if any)
sudo rm -f /var/run/docker.pid

# Reload and restart Docker normally
sudo systemctl daemon-reexec
sudo systemctl daemon-reload
sudo systemctl restart docker

# Test default socket
docker ps
```

✅ Expected: Docker CLI works again without `--tls*` flags.

---

### Lab 37 - Simulate Access Control with Labels/Metadata

---

#### 🎯 Objective

- Apply metadata to simulate basic RBAC
- Use labels and node constraints for scheduling

---

#### 🧠 Background

While Docker Swarm doesn’t support fine-grained RBAC like UCP, you can simulate some access control patterns using:
- **Labels** on nodes/services
- **Constraints** in service definitions

---

#### 🧪 Step-by-Step

##### 1. Label Nodes

```bash
docker node update --label-add env=prod docker01
docker node update --label-add env=dev docker02
```

##### 2. Create a Service for Production Only

```bash
docker service create --name prod-nginx \
  --constraint 'node.labels.env == prod' \
  nginx
```

##### 3. Verify Placement

```bash
docker service ps prod-nginx
```

---

#### ✅ Expected Outcome

- You’ve restricted workloads to certain nodes

---

### Lab 38 - Configure Non-Default Logging Drivers

---

#### 🎯 Objective

- Replace `json-file` with `journald` or `syslog`
- Understand how to persist logs externally

---

#### 🧠 Background

Docker supports multiple logging drivers:
- `json-file` (default)
- `journald`, `syslog`, `gelf`, `fluentd`, etc.

---

#### 🧪 Step-by-Step

##### 1. View Current Logging Driver

```bash
docker info | grep 'Logging Driver'
```

##### 2. Run Container with Journald

```bash
docker run --log-driver=journald --name test-nginx -d nginx
```

##### 3. View Logs

```bash
journalctl -u docker.service | grep test-nginx
```

---

#### ✅ Expected Outcome

- Logs are sent to `journald` or system logging backend

---

### Lab 39 - CVE Severity Comparison and Patch Strategies

---

#### 🎯 Objective

- Understand CVE severities (LOW to CRITICAL)
- Learn how patching works for containers

---

#### 🧠 Overview

**CVE** stands for **Common Vulnerabilities and Exposures** — a standardized identifier for known security flaws in software. Each CVE includes:
- A unique ID (e.g., `CVE-2023-12345`)
- A **CVSS score** (0.0 to 10.0), which maps to severity:
  - LOW: 0.1–3.9
  - MEDIUM: 4.0–6.9
  - HIGH: 7.0–8.9
  - CRITICAL: 9.0–10.0
- Description of the vulnerability
- Potential impact

Security scanning tools (e.g., **Trivy**, **Clair**, **Anchore**) help detect known CVEs in container images, OS packages, and language-specific dependencies.

> 📦 **Also Recommended**: [docker-bench-security](https://github.com/docker/docker-bench-security) — a script that checks for dozens of best practices around Docker host and container security.

---

#### 🧪 Step-by-Step

##### 1. Install Trivy

```bash
sudo apt update
sudo apt install wget -y
wget https://github.com/aquasecurity/trivy/releases/download/v0.63.0/trivy_0.63.0_Linux-64bit.deb
sudo dpkg -i trivy_0.63.0_Linux-64bit.deb
```

Verify installation:
```bash
trivy --version
```

##### 2. Run a CVE Scan on a Known Vulnerable Image

```bash
trivy image python:3.9-slim
```

Observe the output. Pay attention to fields like:
- `Vulnerability ID`
- `Severity`
- `Installed Version`
- `Fixed Version`

> 🔍 Many base images (like Debian-based Python) contain dozens of vulnerabilities by default unless regularly updated.

##### 3. Understand Severity Levels

- **LOW**: Minor misconfigurations or theoretical exploits
- **MEDIUM**: Risk exists, but requires unlikely conditions
- **HIGH**: Exploitable in real-world conditions
- **CRITICAL**: Typically remote code execution, privilege escalation, or unauthenticated access

##### 4. Patch by Updating the Base Image

Create a new `Dockerfile`:

```bash
echo 'FROM python:3.9-slim-bullseye' > Dockerfile
```

Or manually create:

```dockerfile
FROM python:3.9-slim-bullseye
# OR
# FROM python:3.9-slim-bookworm
```

Then rebuild and scan again:

```bash
docker build -t patched-python:latest .
trivy image patched-python:latest
```

Observe if vulnerabilities are resolved or mitigated.

> ❗ **Note**: Sometimes newer base images contain **more CVEs** than older ones. This may be due to:
> - More comprehensive vulnerability reporting in newer distros
> - Timing of patch cycles (stable vs testing)
> - Inclusion of newer packages with known issues
>
> Patching isn't always linear — always evaluate **risk**, **exploitability**, and **update cadence**, not just raw CVE count.

---

#### ✅ Expected Outcome

- You understand how CVEs are classified
- You’ve seen real examples and learned how to detect and mitigate them
- You see the value of updating base images regularly

---

#### 🧹 Cleanup

```bash
# Optional: Remove Trivy binary and image layers
sudo rm trivy_0.63.0_Linux-64bit.deb
sudo apt remove --purge -y wget

# Optionally remove scanned images
docker rmi python:3.9-slim patched-python:latest || true

# Clean up Dockerfile
rm -f Dockerfile
```

---

```

Observe if vulnerabilities are resolved or mitigated.

---

#### ✅ Expected Outcome

- You understand how CVEs are classified
- You’ve seen real examples and learned how to detect and mitigate them
- You see the value of updating base images regularly

---

#### 🧹 Cleanup

```bash
# Optional: Remove Trivy binary and image layers
sudo rm trivy_0.63.0_Linux-64bit.deb
sudo apt remove --purge -y wget

# Optionally remove scanned images
docker rmi python:3.9-slim patched-python:latest || true

# Clean up Dockerfile
rm -f Dockerfile
```

---

---

#### ✅ Expected Outcome

- You understand how CVEs are classified
- You’ve seen real examples and learned how to detect and mitigate them
- You see the value of updating base images regularly

---

#### 🧹 Cleanup

```bash
# Optional: Remove Trivy binary and image layers
sudo rm trivy_0.50.1_Linux-64bit.deb
sudo apt remove --purge -y wget

# Optionally remove scanned images
docker rmi python:3.9-slim patched-python:latest || true
```

---

### Lab 40 - Deep Dive into Docker’s Layered File System

---

#### 🎯 Objective

- Understand how Docker uses layered filesystems
- Explore the structure of the `overlay2` driver
- Correlate Docker image layers to actual files

---

#### 🧠 Background

Docker images are built as **layered filesystems**. Each `RUN`, `COPY`, or `ADD` instruction in a Dockerfile creates a new image layer.

The `overlay2` storage driver (default on most Linux systems) manages these layers by:
- Stacking them in a **read-only lowerdir**
- Creating a writable **upperdir**
- Merging them into a single view via the **merged** directory

Containers launched from these images get their own writable layer on top.

---

#### 🧪 Step-by-Step

##### 1. Create a Simple Multi-Layer Image

```Dockerfile
# Dockerfile
FROM alpine:3.18
RUN echo "layer1" > /layer1.txt
RUN echo "layer2" > /layer2.txt
```

Build it:
```bash
docker build -t layered-demo .
```

##### 2. Identify the Container ID and Mount Path

Run a container from the image:
```bash
docker run -d --name demo layered-demo sleep infinity
```

Find the container ID and mountpoint:
```bash
CID=$(docker inspect --format '{{.Id}}' demo)
docker inspect demo | grep -i overlay
```

##### 3. Locate Files in `/var/lib/docker/overlay2`

```bash
sudo find /var/lib/docker/overlay2 -name "merged" | grep "$CID"
```

Or more directly:
```bash
sudo ls -l /var/lib/docker/overlay2/*/diff/layer*.txt
```

You can explore the `diff/`, `merged/`, and `work/` directories inside each layer:
```bash
sudo tree /var/lib/docker/overlay2/<layer_id>
```

##### 4. View the Full Merged Filesystem

Find the container’s full merged path:
```bash
sudo ls /var/lib/docker/overlay2/<container_layer>/merged
```

This shows the unified view as seen from within the container (`docker exec` equivalent).

---

#### 🧠 What to Look For

- Files in `diff/` are those added in a specific image layer
- The `merged/` directory is the final unified filesystem (lower + upper)
- Multiple layers may contribute to the final image
- `docker history` shows the logical build steps

```bash
docker history layered-demo
```

---

#### ✅ Expected Outcome

- You understand how layers correspond to image instructions
- You can map image history to physical files
- You understand how overlay2 merges read-only and writable content

---

#### 🧹 Cleanup

```bash
docker rm -f demo

docker rmi layered-demo
```

---

### Lab 41 - Simulate Secret Injection via Bind Mount & tmpfs

This lab demonstrates two common patterns for secret injection:
- Bind mounting a secret file from the host
- Copying that secret into a container-only tmpfs mount for ephemeral use

These techniques simulate what Kubernetes accomplishes with Secrets and `emptyDir` volumes.

---

#### 🎯 Objective

- Inject a secret into a container using a bind mount
- Copy that secret to an in-memory tmpfs mount
- Prevent secrets from being stored persistently in container layers or disks

---

#### 🧠 Why This Matters

Secrets should be:
- Delivered at runtime (not baked into images)
- Avoided in long-term mounts (host bind paths)
- Deleted or isolated after use

---

#### 🧪 Step-by-Step

##### 1. Create a Sample Secret

```bash
echo "super-secret-value" > ./secret.txt
```

##### 2. Run a Container With the Secret Mounted

```bash
docker run --rm \
  -v $(pwd)/secret.txt:/run/secret.txt:ro \
  busybox cat /run/secret.txt
```

✅ This proves bind-mounting secrets works. But they’re still on disk.

---

##### 3. Copy Secret Into `tmpfs` Inside Container

```bash
docker run --rm \
  --tmpfs /run/secret:rw,size=64k \
  -v $(pwd)/secret.txt:/host/secret.txt:ro \
  busybox sh -c 'cp /host/secret.txt /run/secret/secret && cat /run/secret/secret'
```

✅ Now the secret is copied to RAM (via `tmpfs`) and removed after the container exits.

---

##### 4. Verify Secret Isn’t Stored in Container Layers (Two Ways)

###### 🔎 Method A: With Bind Mount Present

```bash
docker run --name secret-test \
  --tmpfs /run/secret:rw,size=64k \
  -v $(pwd)/secret.txt:/host/secret.txt:ro \
  busybox sh -c 'cp /host/secret.txt /run/secret/secret && sleep 5'

docker export secret-test | tar -tvf - | grep secret
```

You will likely see:
- `host/secret.txt` (because it was a mounted file)
- `run/secret/` (as the directory structure exists)
- ❌ But **not** `/run/secret/secret`, confirming `tmpfs` data is not persisted.

###### ✅ Method B: tmpfs Only (Clean Test)

```bash
docker run --name secret-test \
  --tmpfs /run/secret:rw,size=64k \
  busybox sh -c 'echo "runtime-value" > /run/secret/secret && sleep 5'

docker export secret-test | tar -tvf - | grep secret
```

✅ You should only see `run/secret/` and **no files inside**, proving the file was truly ephemeral.

Then clean up:
```bash
docker rm secret-test
```

---

#### 🧼 Cleanup

```bash
rm secret.txt
```

---

#### ✅ Expected Outcome

- You injected a secret securely at runtime
- You prevented it from persisting on disk or in image layers

---

### Lab 42 - Serve Static HTML via Bind Mount

This lab demonstrates how to serve static HTML from your host into a containerized web server using a bind mount.

---

#### 🌟 Objective

- Bind mount a host directory with static files into an NGINX container
- Access the served files via web browser or curl

---

#### 🧠 Why This Matters

- Ideal for static site previews
- Demonstrates bind mounts for content delivery
- Helps reinforce directory mapping

---

#### 🧪 Step-by-Step

##### 1. Create a Static HTML Directory

```bash
mkdir html
```

Create `html/index.html`:

```bash
nano html/index.html
```

```html
<!DOCTYPE html>
<html>
  <head><title>Bind Mount Test</title></head>
  <body>
    <h1>Hello from your host filesystem!</h1>
  </body>
</html>
```

---

##### 2. Run NGINX With a Bind Mount

```bash
docker run -d --name static-web -p 8080:80 \
  -v $(pwd)/html:/usr/share/nginx/html:ro \
  nginx
```

️ NGINX now serves the local `html/` folder.

---

##### 3. Test the Web Page

```bash
curl http://localhost:8080
```

Or open in browser:
```
http://localhost:8080
```

Expected output:
```
<h1>Hello from your host filesystem!</h1>
```

---

#### 🧼 Cleanup

```bash
docker rm -f static-web
rm -rf html
```

---

#### ✅ Expected Outcome

- Your local HTML file is served from within the NGINX container via a bind mount

---

# 🧭 Docker → Kubernetes Transition Cheat Sheet

This cheat sheet helps bridge the concepts you already know in Docker to how they map (or change) in Kubernetes (K8s).

---

### 🐳 Docker CLI vs 🕸️ Kubernetes kubectl

| Docker                        | Kubernetes                         | Notes                                      |
|------------------------------|-------------------------------------|--------------------------------------------|
| `docker run`                 | `kubectl run` (dev only)            | Use Deployments or Pods in production      |
| `docker ps`, `logs`, `exec` | `kubectl get`, `logs`, `exec`       | Same functionality via `kubectl`           |
| `docker build`               | Use prebuilt images                 | K8s doesn't build images                   |
| `docker-compose.yml`        | `Deployment`, `Service`, `ConfigMap`| Translate to manifests or use Kompose      |

---

### 📦 Images and Containers

| Docker Concept   | Kubernetes Equivalent     | Notes                                              |
|------------------|---------------------------|----------------------------------------------------|
| Image            | Image                     | Same – pulled from registry                        |
| Container        | Container (in a Pod)      | Pods can have 1+ containers (sidecars, init)       |
| Volume (named)   | PersistentVolumeClaim     | More abstract and storage-class aware              |
| Bind Mount       | HostPath volume           | Not portable; avoid in cloud-native setups         |
| tmpfs            | `emptyDir` + memory       | Use `emptyDir` with `medium: Memory`               |

---

### 🔗 Networking

| Docker Concept     | Kubernetes Concept       | Notes                                              |
|--------------------|--------------------------|----------------------------------------------------|
| Bridge network     | Pod network (flat)       | All pods get unique IPs inside a flat cluster CIDR |
| Container DNS name | Service name             | Pods discover others via `svc-name.namespace` DNS  |
| Port mapping       | `NodePort` / `Ingress`   | Use Ingress for clean routing                      |

---

### 🧱 Multi-Container Apps

| Docker Compose             | Kubernetes                          | Notes                                        |
|----------------------------|--------------------------------------|----------------------------------------------|
| `docker-compose.yml`      | `Deployment`, `Service`, `ConfigMap`| Decompose into individual manifests          |
| `depends_on`              | `initContainers`, probes, ordering   | No direct analog, use startup ordering       |
| `volumes:` shared config  | `ConfigMap` / `Secret`               | Declarative config handling                  |

---

### 🚦 Lifecycle & Scaling

| Docker Swarm             | Kubernetes                         | Notes                                         |
|--------------------------|-------------------------------------|-----------------------------------------------|
| `docker service`         | `Deployment`                        | Manages pods, handles replicas, rollouts      |
| `docker stack deploy`    | `kubectl apply -f` or Helm          | Use Helm or raw YAML                          |
| `replicas: N`            | `replicas: N` in `Deployment`       | Same idea                                     |
| `placement constraints`  | `nodeSelector`, `affinity`, `taints`| More flexible in Kubernetes                   |

---

### 🔐 Secrets & Config

| Docker                 | Kubernetes         | Notes                                      |
|------------------------|--------------------|--------------------------------------------|
| `--env`                | `env:` in manifest | Can pull from `Secret` or `ConfigMap`      |
| `.env` file            | `ConfigMap`        | For non-sensitive configs                  |
| Secret via tmpfs mount | `Secret` volume    | Mounted into container or as env var       |

---

### 📈 Observability

| Docker                   | Kubernetes                       | Notes                                  |
|--------------------------|-----------------------------------|----------------------------------------|
| `docker logs`            | `kubectl logs pod-name`           | Add `-f` to stream                      |
| `docker stats`           | `kubectl top pods/nodes`          | Needs Metrics Server                    |
| `docker inspect`         | `kubectl describe pod`            | More human-readable than raw JSON      |

---

### ⚠️ Error Handling / Debug

| Docker Tool          | Kubernetes Equivalent        | Notes                                        |
|----------------------|-------------------------------|----------------------------------------------|
| `docker exec -it`    | `kubectl exec -it`            | Same for debugging containers                 |
| `docker inspect`     | `kubectl get -o yaml/json`    | Use `describe` for better readability         |
| `docker events`      | `kubectl get events`          | Cluster-level troubleshooting tool            |

---

### 🛠️ Building Blocks Summary

| Building Block    | Docker           | Kubernetes          |
|-------------------|------------------|----------------------|
| Single Unit       | Container        | Pod                  |
| Multi-Container   | Compose          | Pod with multiple containers |
| Service           | Swarm Service    | Service + Deployment |
| Orchestrator      | Swarm            | Kubernetes           |
| Config Mgmt       | `.env`, bind     | ConfigMap, Secret    |
| Networking        | User-defined net | Pod Network + Service|

---

### 🧠 Tips for Transitioning

- Start by deploying your Docker image as a single Pod using a `Deployment`.
- Use `kubectl explain` to explore manifest syntax.
- Familiarize yourself with `kubectl get`, `describe`, `logs`, `exec`.
- Avoid `hostPath` volumes unless you're running on bare metal.
- Use `kubectl port-forward` for local testing instead of port maps.

---