#Indiamart - Did see what i just saw? Something about to happen here @caniravkaria https://t.co/gk8LUD0Qlu

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

UNTOLD STORY OF QUEEN BHAVASHANKARI OF BENGAL - A 🧵u must read

Know the role of Naga Sadhus

PART-1

One of the most underrated empires to ever grace India is the kingdom of Bhurshut from central Bengal.

1/

A medieval powerhouse this kingdom stood firm against the Pathan invasion from Orissa twice, and with each time devastating the enemy forces.

Along with this, we also have to note that this was the time when the empire was under the reign of a widowed queen named Bhavashankari.

She wasn’t even born in a royal family and yet she achieved what most of the kings couldn’t do

This begs the question, why does no one remember this great kingdom?

3/

Bhurshut empire thrived from 15th century till late 18th century as an independent kingdom while acting as tributary state with full autonomy.

4/

What’s even more puzzling is that no one knows about the great queen Bhavanshakari who thrashed the Pathans single handedly without the aid of their allies.

5/

1/ there is this sense when you are young that your accomplishments need to be a list of things that seem impressive to others. A list of several items you did.

This isn't actually right, so here is another suggestion.

2/ I remember being 26 and writing about reading 52 books a year. I wrote blog posts about it. They got copied. It became "a thing." Now it's in Twitter bios. It looks impressive but it's insanely useless and I shouldn't have done it.

3/ what I should have known at that time is that only young idiots like myself, with no accomplishments, find list of tiny achievements impressive. Anyone who has actually done anything of substance doesn't gaf

4/ what is actually difficult, and worthwhile, instead is to do ONE single thing for a very, very long time. It's much harder and much rarer and results in outlier outcomes much more often.

Of course you can find this out too late if you are chasing the dragon of Ted talks etc

5/ if I had only worked on a startup for a year, I would've gotten nowhere, the same way that if you lift for 3 months, it achieves nothing. Everything good in life comes from perseverance, but at the beginning, you're just like "I need to be somebody!!!"