Darkmatters

  Day 8 & 9 They say data is worthless until it is clean. That might be dramatic, but the reality is that 35,000 high-resolution retinal scans sitting in a folder do not mean anything to a neural network until they are resized, normalized, and shaped into the exact format the model expects. This phase was about building the preprocessing pipeline. The bridge between raw JPEG files and tensors ResNet50 can actually train on. Why Preprocessing Is Not Optional The temptation to skip this step is real. The data is downloaded, the model architecture is known, and the urge to just start training is strong. But feeding raw images directly into a pre-trained ResNet50 would fail immediately, not because the model is broken, but because the input format would be completely incompatible with what the pre-trained weights expect. ResNet50 was originally trained on ImageNet: 1.2 million images normalized to specific mean and standard deviation values. If you feed it images with a different...

Day 6 & 7 They say you cannot build an inference engine without data. That is technically true, but what they don't tell you is that getting the data, understanding the data, and actually seeing what your model will be trained on is a journey of its own. And by journey, I mean downloading 88GB, realizing the unzipped dataset is now 100GB, spending an embarrassing amount of time wondering why Jupyter could not find my libraries, and then finally understanding what conda activate sightx actually meant. Why Data Exploration Comes Before Everything Else? The urge to skip straight to model training is real. The environment is set up, PyTorch confirmed the M4 MPS backend is active, and ResNet50 is waiting. But training a model on data you have not actually looked at is like building a house without checking if the land is stable. You will get something, but it probably won't be what you wanted. This phase was about three things: acquiring the EyePACS dataset from Kaggle, underst...

Day 4 & 5 They say a craftsman never blames their tools, but they also never start building without setting them up properly. Before a single model weight gets initialized, the environment that will carry all of that work needs to be solid , reproducible , and hardware-aware. That was the entire focus of these two days. Why Bother With Setup First? My first instinct, honestly, was to skip straight to writing model code. I already understood the theory from Days 2 and 3: CNNs, ResNet50, transfer learning. So, the urge to just start building was real. But I have seen enough projects collapse under their own environmental chaos to know better. A broken dependency, a mismatched Python version, a library that silently falls back to CPU when it should be using the GPU, these are the bugs that waste hours and have nothing to do with the actual problem you are trying to solve. So I did it right, once, and locked it in. The Environment: Miniconda and Python 3.10 The foundation is a dedica...