The Ultimate Guide to Unit Testing in JavaScript.

A Thread...

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

On December 25 Mr Pool retweeted August 16 message about the Phoenix and that the worst was still on the horizon to come. 1/*

Maybe now we can assume that November 3 message meant "FUD" nor "FUN". Putting the word in capital letters seems to indicate the aspect the word fud has in our space, FUD. 2/*

Brad started it on December 22 (1/3 posts (Don't say? Sure) = 13 = Plan B). 3/*

And immediately the SEC followed dropping the bomb (1.3 billions (Don't say? Sure) = 13 = Plan B). 4/*

FUD materializes, Bitstamp is one of the first that haltes XRP trading and so on. 5/*

1. Now that Stan Lee is being eulogized far and wide, it's important to remember how marginal Stan Lee and his collaborators (Jack Kirby and Steve Ditko) were in the 1950s and 1960s.

2. Marvel Comics (before that Atlas) was just a cog in the machine of a bottom pulp publisher run by Martin Goodman, the husband of one of Lee's cousins. It was the lowest of the low in the publishing world.

3. Now Mario Puzo (not yet the author of the Godfather) shared offices with Stan Lee in the 1950s and 1960s. Puzo wrote for garrish men's adventure magazines and, like Lee, dreamed of writing a novel & breaking out. But Puzo looked down on Lee.

4. Flo Steinberg, 1960s secretary at Marvel: "They were always making jokes about us. They'd come in and giggle. mario Puzo would look in and would see us all working on his way to the office and say, 'Work faster, little elves. Christmas is coming.'"

5. When JFK was killed, the whole office of Magazine Management was stunned and quiet. Except Lee. He continued working. "He was still working on the comic books," Puzo said. "Like that was the most important thing in the world."