"A Data-Based Perspective on Transfer Learning"

Different classes in a pretraining dataset can have different effects on downstream accuracy. And you can use this to your advantage. [1/9]

They assess these effects using a simple algorithm that trains different models on different subsets of the data and looks at both the class counts and the predictions for each model on each downstream sample. [2/9]
Using their scoring function, you can intelligently remove subsets of classes from the pretraining dataset in order to significantly raise downstream accuracy. [3/9]
Another use of their method is identifying more granular subpopulations than what a downstream task has annotated. E.g., you can find which CIFAR-10 images look most like ostriches even though CIFAR-10 only has the label “bird”. [4/9]
You can also use a similar idea to understand model failure modes or identify data leakage. [5/9]
And last but not least, you can use it to understand helpful/harmful samples in your pretraining dataset. [6/9]
Overall their algorithm seems like a great tool to have in the toolbox. [7/9]
Paper: https://t.co/CKg0nxmSxE

If you like this paper, consider RTing this (or another!) thread to publicize the authors' work, or following the authors: @saachi_jain_ @hadisalmanX @Alaa_Khaddaj… [8/9]
@saachi_jain_ @hadisalmanX @Alaa_Khaddaj …@RICEric22 @ssung_mminn @aleks_madry

For more paper summaries, you might like following @mosaicml, me, or my newsletter: https://t.co/5BMBC84xY8

As always, comments and corrections welcome! [9/9] https://t.co/8VRLAGmrfQ

More from All

ChatGPT is a phenomenal AI Tool.

But don't limit yourself to just ChatGPT.

Here're 8 AI-powered tools you should try in 2023:

1. KaiberAI

@KaiberAI helps you generate beautiful videos in minutes.

Transform your ideas into the visual stories of your dreams with this Amazing Tool.

New features:
1. Upload your custom music
2. Prompt Templates
3. Camera Movements:

Check here

https://t.co/ivnDRf628L


2. @tldview TLDV

Best ChatGPT Alternative for meetings.

Make your meetings 10X more productive with this amazing tool.

Try it now:

https://t.co/vOy3sS4QfJ


3. ComposeAI

Use ComposeAI for generating any text using AI.

It’s will help you write better content in seconds.

Try it here:

https://t.co/ksj5aop5ZI


4. Browser AI

Use this AI tool to extract and monitor data from any website.

Train a robot in 2 minutes to do your work.

No coding required.

https://t.co/nNiawtUMyO

You May Also Like