Darkmatters

Day 15 & 16 V1 was chaos. Beautiful, productive chaos. I typed python train.py , watched accuracy hit 67.99% at Epoch 7, then watched it overfit into the ground for thirteen more epochs while I ate noodles and questioned my life choices. It worked, but it was messy. V2 was different. V2 was what happens when you finally read the manual. After V1 finished, I did something I probably should have done before writing any code: I studied the people who actually solved this problem. The 2015 EyePACS Kaggle competition had 661 teams competing for $100K in prizes. The winners didn't just build better models, they built better ways of looking at the data . And that changed everything. The Kaggle Winner Who Changed My Mind Ben Graham won first place. His secret wasn't a fancy neural network. It was preprocessing. His insight: raw retinal photographs are a mess. Different cameras. Different lighting. Different zoom levels. Some images look reddish, some yellowish. Some retinas fill t...

Day 14 7:55 PM to 11:55 PM They say you never forget your first. Your first car. Your first apartment. Your first time training a neural network on 35,000 medical images while sitting on your desk eating noodles and questioning every life decision that led to this moment. Tonight was that night. It was 7:55 PM on a Thursday. I had spent two weeks building the pipeline, downloading the EyePACS dataset, writing the model architecture, debugging preprocessing transforms, and triple-checking my training loop for bugs that would only reveal themselves three hours into training. Everything was ready. The data was clean. The code was tested. The conda environment was activated. This time I remembered to activate it. I typed the command: python inference-engine/train.py And then the terminal did something beautiful and slightly unhinged. It printed the same line nine times: Training on device: mps Training on device: mps Training on device: mps ... Nine times. One for each DataLoader worker th...

Day 12 & 13   The architecture is built. The preprocessing pipeline is done. The model knows how to accept an image and produce five output logits . What it does not know yet is what those logits should mean. That is what the training loop is for. This phase is where the model stops being a static structure and starts being something that learns. It is also the phase with the most moving parts, the most ways to silently fail, and the most satisfying moment when the loss curve finally starts going down. The Dataset Class: Teaching PyTorch to Read Your Data Before the training loop can run, PyTorch needs to know how to load your data. That is not automatic. You must write a Dataset class that tells it exactly how to find an image, open it, and pair it with the correct label. It is essentially a contract given an index, returning a sample. The EyePACS dataset gives you a CSV file mapping image filenames to DR grades, and a folder of JPEG retinal scans. The Dataset class reads the ...