Inspect and understand the data Setup and verify each component of full end-to-end training with a simple model to verify data, model, training flow is working as expected Use metric from previous end-to-end training as baseline Overfit with a large model Refine model, regularize, etc Squeeze last few bits of performance
Andrew Karpathy's Recipe for Training Neural Networks
Apr 29, 2019