#HindCopper 170

Breakout zone marked on the

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

1 of 16: WE ARE ALL JEWS HERE: THE STORY OF RODDIE EDMONDS

One of the most moving and relevant stories of the Battle of the Bulge, or any American Soldier in any war, is that of Master Sergeant Roddie Edmonds, a Knoxville, Tennessee native, who served with the 106th Infantry.

2 of 16:

Roddie was captured early on in the Battle of the Bulge, on December 19th, when Panzer forces plowed through his unit.

He, along with almost his entire regiment, was forced to surrender.

3 of 16:

The men were transported to the Stalag IX-A POW camp in Ziegenhain, Germany.

Roddie was the senior enlisted American Soldier at the site. As such, he was the conduit between all American Soldiers and their German captors.

4 of 16:

In late January, the camp’s commandant, Major Siegmann [pictured here], ordered Roddie, a Christian, to identify all Jewish Soldiers and order them to stand in formation by themselves the next day.

[Jewish Soldiers were a minority within American units]

5 of 16:

Throughout WWII, captured Jewish Soldiers were often tortured and then killed by their German captors.

In fact, Jewish Soldiers had been told to bury their dog tags before capture. The dog tag identified Jews with the letter “H” for Hebrew.

Ashta Siddhi

Ashta Siddhi is mentioned in Patanjali Yog sutra and can be obtained by very few through countless years of practice and devotion to all eight limbs of Ashtanga Yoga.

Siddhi can loosely be translated as accomplishment or a special power or unusual skill.

The eight Siddhis are:

1. Anima: Reducing one’s physical self to the size of an atom.

2. Mahima: Expanding one’s physical self to infinite level

3. Garima: Making one’s physical self so heavy as immovable by others

4. Laghima: Becoming almost weightless

5. Prapti: Being able travel anywhere in the Universe

6. Prakamya: Can know & understand what is going on in one’s mind. Can understand other person’s min.

7. Ishitva: Possessing title of Bhagwan

8. Vashitva: A seeker can subdue any person/thing

(Hanuman Jayanti is coming)

Hanumanji not only possessed all eight siddhis, but was also blessed by Sita mata as “Ashta Siddhi Nau Nidhi Ke Daata”.

When Hanumanji first reaches to Lanka on his mission to find Sita, he makes use of anima so as to be discrete in enemy territory.

He uses it again when he approaches Sita mata for the first time – reducing his size to that of a schoolboy, to avoid scaring her. He uses mahima to outwit & overpowers
the demons.

You want to know something about how bullshit insane our brains are?
OK, so there's a physical problem with our eyes: We move them in short fast bursts called "saccades", right? very quick, synchronized movements.
The only problem is: they go all blurry and useless during this

having your vision turn into a blurry mess every time you move your eyes is obviously not a good idea, so our brains hide it from us. Now, imagine you're an engineer and you have this problem.

You've got some obvious solutions you could do.
1. make the vision go black during movement. (Some VR games do this!)
2. just keep showing the last thing we saw prior to movement

both are good options with different downsides, but OH NO. this is assuming everything makes sense and is chronological and (regular) logical.
Your brain does neither of these options, really.

first, it basically puts your visual system on "pause". You're not seeing blackness or even nothing, you're just not seeing period.
then when you finish your saccade, it shows you what you now see at the new position. and then it pretends it can time travel.