BC ALL
Saved by @kumar_praveen96

Barlow Twins: a new super-simple self-supervised method to train joint-embedding architectures (aka Siamese nets) non

 Basic idea: maximize the normalized correlation between a variable in the left branch and the same var in the right branch, while making the normalized cross-correlation between one var in the left branch and all other vars in the right branch as close to zero as possible.
2/N
In short: the loss tries to make the normalized cross-correlation between the embedding vectors coming out of the left branch and the right branch as close to the identity matrix as possible.
3/N
The 2 branches are always fed with differently-distorted version of the same image, and there is no need for dissimilar training pairs.

The objective makes the embedding vectors of the two branches as similar as possible, while maximizing their information content.
4/N
No contrastive samples, no huge batch size (optimal is 128), nor predictor, no moving-average weights, no vector quantization, nor cut gradients in one of the branches.
5/N
Competitive results on ImageNet with a linear classifier head.
Great results on semi-supervised ImageNet in the low labeled-data regime and on transfer tasks.
6/N
Results on ImageNet with linear classifier head
7/N
Results with 1% and 10% of ImagNet labeled images
8/N
Results on transfer tasks.
9/N
Arch is standard ResNet50 with 2048-D feature vec.
But contrary to others, the embedding size (projector output) is larger. The perf keeps going up as the embedding dim grows (we stopped at 16384).
Probably cause the feature vars are made independent, not just decorrelated.
10/N
Why Barlow? Horace Barlow was a pioneer of visual neuroscience who proposed the idea that the brain tries to minimize redundancy in representations.

By Jure Zbontar, Li Jing, Ishan Misra, yours truly, and Stéphane Deny.
All from FAIR.
To appear at ICML 2021
11/N
Don't you just hate slicing what would be a decent-size post into threaded thin tweets?
12/N
No, really. Don't you hate reading those long thread slices?
If you do, you could just read my Facebook post:
https://t.co/dQii7BEPQ5
13/N
N=13
Typo: optimal batch size is 1024, not 128.
14/13 (haha).

More from All

Anu Satheesh
Anu Satheesh
@AnuSatheesh5
Guruvayur Sri Krishna & Coconut with Horns.
#Krishnaleela
#Bhakthi
🕉
We’ve all grown up listening to stories of Shri Krishna. This one is a famous story of Shri Krishna in Guruvayur Temple, also called Dakshina Dwaraka.


There lived a villager who planted coconut saplings and had promised that he would offer the 'first coconut' from each of his coconut trees to "Guruvayurappan". When the trees started to yield coconuts, he collected the first coconut from all the trees in a sack 👇


and set forth to Guruvayur.

On the way he was stopped by a robber who asked all the items in the sack. The villager told the robber that the coconuts in the sack belonged to Guruvayurappan and so he was unable to hand it over.
@SriramKannan77 @Jayalko1


The robber with anger asked the villager "Is Guruvaurappan's coconut any different? Does it have horns ?".
Saying so the robber pulled the sack forcefully out of the villager's hand, and coconuts came out..👇


To their astonishment each and every coconut in the sack had horns! Even today, the coconut with horns are displayed in the temple for devotees to see.

Below Picture is not original, as we cannot take photos inside temple.
Hare Krishna
Guruvayurappa
🕉🕉🕉🕉🕉
Shubham
ALL
The Chartians
The Chartians
@chartians
Took me 5 years to get the best Chartink scanners for Stock Market, but you’ll get it in 5 mminutes here ⏰

Do Share the above tweet 👆

These are going to be very simple yet effective pure price action based scanners, no fancy indicators nothing - hope you liked it.

https://t.co/JU0MJIbpRV

52 Week High
One of the classic scanners very you will get strong stocks to Bet on.

https://t.co/V69th0jwBr

Hourly Breakout
This scanner will give you short term bet breakouts like hourly or 2Hr breakout

Volume shocker
Volume spurt in a stock with massive X times
SCREENERS , ALL , CHARTLINK SCREENERS
Advertisement
Ranvijay Singh\U0001f1ee\U0001f1f3
Ranvijay Singh🇮🇳...
@ranvijayT90
#Thread Two divine forms of Khatu Shyam ji...


On the day of Amavasya, devotees get the darshan of Shyam Baba in Shyam varan (black color). In ordinary days, the form of Baba is saffron. The reason for this is a custom. The real head of Shyam Baba which had appeared from Shyam Kund is of Shyam varan.


Every day Shyam Baba's grand Shringar is done by Pujaris. The main part of Shyam Baba's shringar is Tilak. This Shringar is done by the members of Nij Mandir sevaks. The sevaks of the Nij Mandir do sandalwood tilak to Baba everyday in the auspicious time.


This Divine Tilak is applied from the forehead to the nose. When Baba is given a bath by sevaks, this tilak comes off. Then in the auspicious time, the sevaks again apply tilak to Baba.


There is no auspicious time on the Amavasya. Because of this, tilak is not done on that day. That's why devotees have darshan of Shyam Baba on the Amavasya only in the actual real form (Shyam Swaroop). Baba's tilak is done on Amavasya only on the day of Deepawali.
ALL
Buzz Chronicles
Buzz Chronicles
@buzz_chronicles
@SaveToBookmarks Ciao! Of course! Original tweet by @AnJamison: "@bellingcat @buzz\_chronicles @buzz\\\_chronicles @buzz_chronicles save as categ...". Cheers! 👌

Nǐ hǎo! You can find it here. Original tweet by @SaveToBookmarks: "@AnJamison @bellingcat @buzz @buzz_chronicles @pdfmakerapp @Readwiseio @threader...". Enjoy 👌
ALL
Angela Jamison
Angela Jamison
@AnJamison
@JeffJacksonNC @buzz\_chronicles @buzz\\\_chronicles @buzz_chronicles save as category @pdfmakerapp grab @pdfmakerapp grab this @readwiseio save @readwiseio save thread @threader compile summarize @Threadreaderapp unroll please @ttttreads unroll @unrollthread.com @SaveToBookmarks
ALL

You May Also Like

Brianne Kimmel \U0001f4ac
Brianne Kimmel 💬...
@briannekimmel
Shocking fact: Millennial men are less likely to work than any other age and gender demographic in America.

Today, there are 500,000 young men missing from the U.S. workforce.

Research suggests video games & improved leisure tech plays a role in the problem. 👇 Thread:

Following the 2007 to 2009 recession, 25 to 34 year old men exited high school with fewer middle-skill job opportunities than years prior.

During this time, we saw an increased number of men living with parents & choosing unemployment over lower paying jobs.

It's estimated that 24M millennials live w/ their parents.

1 in 4 living in their parents’ home neither go to school nor work.

What's more surprising? 9 in 10 who lived with their parents a year ago are still living there w/ no plans to leave.

Economists are calling millennial men a lost generation.

According to economist David Dorn:

“If you get to the point where you’re turning 30, you’ve never held a real job and you don’t have a college education, then it is very hard to recover at that point.”

Economists suggest this choosiness is a generational trait.

Forbes interview w/ a high school educated man:

"I’m very quick to get frustrated when people refuse to pay me what I’m worth."
“People feel that they have choice nowadays, and they
SOCIETY
Lu_lu\U0001f498
Lu_lu💘
@luciame16
Some thoughts on episode 54 of #Hercai

If you ever cross the distance and come to me, come with love that blooms in your heart.
~ Şems-i Tebrizi

#Reyyan #Miran #ReyMir #EbruŞahin #AkınAkınözü
#HercaiThoughts


1
I have to make a note before writing about this episode. I was the person (still am) who thinks that bringing back D. was a bad idea. I repeat it that for me the writing of S1 and S2 is like tossed into the bin with that plot evolution, but they had to invent something+
#Hercai

2
+ to be able to continue to prolong the general story after Aslan’s story did not work out. All the suffering of M., all the crying seems for “nothing” for two seasons and we viewers are also kind of fed up in seeing him cry all the time when with the upcoming episodes+
#Hercai

3
+he is justified the most to cry. With her resurrection now the attention shifts back to Miran’s storyline and I don’t know if I like this to be honest. This said, Dilsha is here and we have to deal with her.
#Hercai

4
So, my comments are going to be how I find they wrote her into the story, the details we get to know, the flashbacks we get to see, the way it was filmed and directed, with leaving out if I agree on it or not but describing along the way what I saw.
#Hercai
CULTURE
Advertisement
Jeannette Ng \u5433\u5fd7\u9e97
Jeannette Ng 吳志麗...
@jeannette_ng
So, you’re a writer and you’re worried about writing outside your immediate experiences and identities.

Here is the thread of practical advice for you.

EXAMINE YOUR MOTIVES. Why do you want to write this?

Are you trying to solve racism/sexism/colonialism with your work?

As writers we love stories about heroic writers whose work has changed the world. And as such we like to look to our own writing to solve societal problems.

And I understand this completely, not the least because I’ve felt the pull.

But if you’re looking to play saviour with your words, it is unlikely that you will do the marginalised people you are trying to save justice.

And I understand this very often comes from place of good intentions, but there is a reason that most of the moralising plays written by white abolitionists are deeply uncomfortable to read.

It is incredibly easy for works looking to play saviour to become patronising or traffic in simplistic stereotypes that ultimately hurt the people they are looking to rescue.
WRITING
Billy Bostickson \U0001f3f4\U0001f441&\U0001f441 \U0001f193
Billy Bostickson 🏴👁&👁 ...
@BillyBostickson
Tip from the Monkey
Pangolins, September 2019 and PLA are the key to this mystery
Stay Tuned!


1. Yang


2. A jacobin capuchin dangling a flagellin pangolin on a javelin while playing a mandolin and strangling a mannequin on a paladin's palanquin, said Saladin
More to come tomorrow!


3. Yigang Tong
https://t.co/CYtqYorhzH
Archived: https://t.co/ncz5ruwE2W


4. YT Interview
Some bats & pangolins carry viruses related with SARS-CoV-2, found in SE Asia and in Yunnan, & the pangolins carrying SARS-CoV-2 related viruses were smuggled from SE Asia, so there is a possibility that SARS-CoV-2 were coming from
ALL
Abigail See
Abigail See
@abigail_e_see
Can we control the output of neural generation systems?
Can we build a better chatbot using this control?
And what makes a good conversation anyway?

Check out the blog post for our NAACL paper:

https://t.co/UAAUxSeA0L

#AI #NLProc #DeepLearning #ConversationalAI @facebookai

This post got a little delayed since the paper's publication, but the intervening time has given me an opportunity to reflect.

So this post has some extra thoughts regarding more recent NLG developments - e.g., pretrained models like GPT(2), and the suitability of beam search.
ALL