Machine Learning in the Real World ๐ง ๐ค
ML for real-world applications is much more than designing fancy networks and fine-tuning parameters.
In fact, you will spend most of your time curating a good dataset.
Let's go through the process together ๐
#RepostFriday
Collect Data ๐ฝ
We need to represent the real world as accurately as possible. If some situations are underrepresented we are introducing Sampling Bias.
Sampling Bias is nasty because we'll have high test accuracy, but our model will perform badly when deployed.
๐
Traffic Lights ๐ฆ
Let's build a model to recognize traffic lights for a self-driving car. We need to collect data for different:
โช๏ธ Lighting conditions
โช๏ธ Weather conditions
โช๏ธ Distances and viewpoints
โช๏ธ Strange variants
And if we sample only ๐ฆ we won't detect ๐ฅ ๐คทโโ๏ธ
๐
Data Cleaning ๐งน
Now we need to clean all corrupted and irrelevant samples. We need to remove:
โช๏ธ Overexposed or underexposed images
โช๏ธ Images in irrelevant situations
โช๏ธ Faulty images
Leaving them in the dataset will hurt our model's performance!
๐
Preprocess Data โ๏ธ
Most ML models like their data nicely normalized and properly scaled. Bad normalization can also lead to worse performance (I have a nice story for another time...)
โช๏ธ Crop and resize all images
โช๏ธ Normalize all values (usually 0 mean and 1 std. dev.)
๐