"A Data-Based Perspective on Transfer Learning"

Different classes in a pretraining dataset can have different effects on downstream accuracy. And you can use this to your advantage. [1/9]

They assess these effects using a simple algorithm that trains different models on different subsets of the data and looks at both the class counts and the predictions for each model on each downstream sample. [2/9]
Using their scoring function, you can intelligently remove subsets of classes from the pretraining dataset in order to significantly raise downstream accuracy. [3/9]
Another use of their method is identifying more granular subpopulations than what a downstream task has annotated. E.g., you can find which CIFAR-10 images look most like ostriches even though CIFAR-10 only has the label “bird”. [4/9]
You can also use a similar idea to understand model failure modes or identify data leakage. [5/9]
And last but not least, you can use it to understand helpful/harmful samples in your pretraining dataset. [6/9]
Overall their algorithm seems like a great tool to have in the toolbox. [7/9]
Paper: https://t.co/CKg0nxmSxE

If you like this paper, consider RTing this (or another!) thread to publicize the authors' work, or following the authors: @saachi_jain_ @hadisalmanX @Alaa_Khaddaj… [8/9]
@saachi_jain_ @hadisalmanX @Alaa_Khaddaj …@RICEric22 @ssung_mminn @aleks_madry

For more paper summaries, you might like following @mosaicml, me, or my newsletter: https://t.co/5BMBC84xY8

As always, comments and corrections welcome! [9/9] https://t.co/8VRLAGmrfQ

More from All

You May Also Like

And here they are...

THE WINNERS OF THE 24 HOUR STARTUP CHALLENGE

Remember, this money is just fun. If you launched a product (or even attempted a launch) - you did something worth MUCH more than $1,000.

#24hrstartup

The winners 👇

#10

Lattes For Change - Skip a latte and save a life.

https://t.co/M75RAirZzs

@frantzfries built a platform where you can see how skipping your morning latte could do for the world.

A great product for a great cause.

Congrats Chris on winning $250!


#9

Instaland - Create amazing landing pages for your followers.

https://t.co/5KkveJTAsy

A team project! @bpmct and @BaileyPumfleet built a tool for social media influencers to create simple "swipe up" landing pages for followers.

Really impressive for 24 hours. Congrats!


#8

SayHenlo - Chat without distractions

https://t.co/og0B7gmkW6

Built by @DaltonEdwards, it's a platform for combatting conversation overload. This product was also coded exclusively from an iPad 😲

Dalton is a beast. I'm so excited he placed in the top 10.


#7

CoderStory - Learn to code from developers across the globe!

https://t.co/86Ay6nF4AY

Built by @jesswallaceuk, the project is focused on highlighting the experience of developers and people learning to code.

I wish this existed when I learned to code! Congrats on $250!!