"A Data-Based Perspective on Transfer Learning"

Different classes in a pretraining dataset can have different effects on downstream accuracy. And you can use this to your advantage. [1/9]

They assess these effects using a simple algorithm that trains different models on different subsets of the data and looks at both the class counts and the predictions for each model on each downstream sample. [2/9]
Using their scoring function, you can intelligently remove subsets of classes from the pretraining dataset in order to significantly raise downstream accuracy. [3/9]
Another use of their method is identifying more granular subpopulations than what a downstream task has annotated. E.g., you can find which CIFAR-10 images look most like ostriches even though CIFAR-10 only has the label “bird”. [4/9]
You can also use a similar idea to understand model failure modes or identify data leakage. [5/9]
And last but not least, you can use it to understand helpful/harmful samples in your pretraining dataset. [6/9]
Overall their algorithm seems like a great tool to have in the toolbox. [7/9]
Paper: https://t.co/CKg0nxmSxE

If you like this paper, consider RTing this (or another!) thread to publicize the authors' work, or following the authors: @saachi_jain_ @hadisalmanX @Alaa_Khaddaj… [8/9]
@saachi_jain_ @hadisalmanX @Alaa_Khaddaj …@RICEric22 @ssung_mminn @aleks_madry

For more paper summaries, you might like following @mosaicml, me, or my newsletter: https://t.co/5BMBC84xY8

As always, comments and corrections welcome! [9/9] https://t.co/8VRLAGmrfQ

More from All

You May Also Like

Moderna CEO Stephane Bancel was previously CEO of bioMerieux in France from 07-10.

Alain Merieux, who owns bioMerieux, was instrumental in the creation of the Wuhan Institute of Virology P4 Lab.

The same people who helped create the virus, also helped to create the vaccines...


Moderna partnered with French Pasteur Institute in 2015 to develop mRNA vaccine technology.

Pasteur Institute partnered with the Wuhan P4 Laboratory in 2017 along with the Merieux Foundation to study emerging viruses...
https://t.co/yFsHwrNYaK
https://t.co/9M5lydBKhM


Nobel prize winning scientist Luc Montagnier asserts that Sars-Cov-2 is man-made and originated from the Wuhan Institute of Virology.

Montagnier did extensive work with the Pasteur Institute in France which was partnered with the Wuhan P4.

Merieux Foundation & the Chinese government have worked together since 1965, and partnered to study emerging pathogens in Africa in 2015.

Their research included "PATHOGENS CARRIED BY BATS" that provoke respiratory diseases.

🚨🚨🚨
https://t.co/gVwpT0ssqI