How to Build a Vision Capstone with YOLO and Transfer Learning

Which vision track should you choose?

Your dataset decides. If you have an images/ folder plus a labels/ folder of YOLO-format .txt files (class and box coordinates per line) and must output bounding boxes, pick Track 1: Autonomous Driving. If you have images organized into one subfolder per class and need to categorize structures, pick Track 2: Tourism. Both are vision tracks, but Track 1 is detection and Track 2 is classification — match your target prediction to the track before writing any code.

Every track has a fixed two-part structure. Track 1 Part 1 is YOLO detection; Part 2 is accident-data analysis. Track 2 Part 1 is transfer-learning classification; Part 2 is item-based recommendation. You complete both parts of whichever track you choose.

How do you build the model without training a CNN from scratch?

Don't build convolutional architectures from scratch — pretrained models win. For Track 2, load VGG16 with `include_top=False` to strip its ImageNet classification head. Then append `Flatten`, a `Dense` layer with ReLU, an optional `Dropout`, and a final `Dense` with Softmax whose neuron count exactly equals your number of classes. Compile with the Adam optimizer, categorical crossentropy, and accuracy as your metric.

Load your data with `image_dataset_from_directory()`, which turns each subfolder name into a class label and gives you an 80/20 train/validation split automatically. This keeps your data pipeline clean and exposes a `class_names` attribute you'll reuse when visualizing.

For Track 1, clone the YOLO repository, verify `data.yaml` points to your local images/ and labels/ directories, update its `names` field to match your vehicle classes, and run `train.py` with GPU enabled. Then run `detect.py` on test images and visualize bounding boxes with class labels.

How do you prove your model generalizes?

Train twice. For Track 2, run the same architecture once without augmentation and once with a `Sequential` block of `RandomFlip`, `RandomRotation`, and `RandomZoom` prepended. Compare validation accuracy between the two runs and explicitly state which generalized better and why. This with-and-without comparison is the single most convincing evidence of overfitting reduction you can put in a portfolio.

Visualize thoroughly: plot a 3x3 grid of sample images titled with their class names, and plot training-vs-validation accuracy curves for both augmentation runs. For detection, overlay bounding boxes on test images.

What mistakes should you avoid?

The biggest pitfall is forgetting `include_top=False` — without it, the original head stays attached and your custom layers can't connect. The second is mismatching your final Dense neuron count to your class count, which causes shape errors during training. The third is running vision training on CPU: always connect to a GPU runtime in Colab, or YOLO and CNN training will be prohibitively slow. Finally, never launch YOLO training without confirming `data.yaml` paths — misconfigured paths fail silently.

Audit your data first: run `.info()`, `.head()`, and check for nulls and duplicates before any modelling. For Track 1's accident CSV, fill numeric null columns with zero, since absence means no recorded event.

Next step

Pick your track based on your dataset, audit it with `.info()` and null checks, then implement Part 1 using the transfer-learning or YOLO pattern above. Reference the Deep Learning Lesson 9 (transfer learning) and Lesson 10 (YOLO) notebooks, and always start by connecting a GPU runtime.