3iInfotech

#returngift

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

Hinduism is a minority religion in Punjab.

During the Khalistan movement, the minority Hindus and Jains were subjected to several attacks. Hindu and Jain temples were frequently vandalized and desecrated. This thread documents a few instances.

Thread

Durgiana Mandir is one of the oldest temples of Amritsar. According to the regional Hindu tradition, the temple was built on the spot where Lava and Kusha bound Hanuman. The temple annually attracts millions of Hindus from all over the country.

Amristsar has a very ancient Hindu history.

According to the regional Hindu tradition recorded in the "Survey of Amritsar", Amritsar got its name from Amrit immersed by Lava and Kusha in a pond.

Durgiana Mandir was rebuilt in 1921.

It was used to house the Murtis of Vishnu and Durga which were thrown out from the Parikrama of Harmandir Sahib in 1905.

Because of its presence in Amritsar, it became a frequent target of radicals during the Khalistan movement.

In 1982, in what was a completely unprovoked attack, the Dal Khalsa "activists" threw a severed cow head inside Durgiana temple.

(An extract from Field Marshal Sam Manekshaw, Soldiering with Dignity)

The man eventually destined to be free India's first Field Marshal was born on 3rd April 1914 at Amritsar. How did a Parsi couple settle for the holy city of the Sikhs? I once asked him and was told that in+

1899, his father recently qualified as a doctor and just married, could make no professional headway in Bombay, and was advised to try his luck at Lahore in the Punjab. With his young wife, he set off by train for Lahore. The long dusty and hot journey took five days and by the+

end of it, his young wife, who had never left the comforts and civilization of Bombay, was in hysterics and cried to go back. Poor Dr Manekshaw did all he could to comfort her, but as the train steamed into Amritsar, with her first sight of the Sikhs the young bride screamed her+

lungs out and refused to go any further. So they left the train at Amritsar, and there they stayed for forty-five years.

The Manekshaws had six children, four boys and two girls, and Sam was the fifth child. Sam had his schooling at Nainital's Sherwood College.+

After completing his schooling, he should have gone to England to pursue higher studies; this was the promise made to him by his father but, fortunately for the Indian Army, Dr Manekshaw felt that this particular son was far too young to be on his own in a foreign country, even+