BC MACHINE LEARNING

Saved by @zmbnski

Roger Grosse
@RogerGrosse 7 years, 6 months ago 3710 views

Save to PDF Share See On Twitter

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

The time-to-target-validation is explained purely by optimization considerations. There's a regime where variance dominates, and you get linear speedups w/ batch size. Then there's a regime where curvature dominates and larger batches don't help. As theory would predict.

Incidentally, this paper must have been absurdly expensive, even by Google's standards. Doing careful empirical work on optimizers requires many, many runs of the algorithm. (I think surprising phenomena on ImageNet are often due to the difficulty of running proper experiments.)

More from Machine learning

IM POCUS Focus
@IMPOCUSFocus

A tweetorial on optimal machine settings for B line image acquisition, with a focus on: focal position at pleura, harmonics off, and increased far gain.

Inspired/based on the recent article - https://t.co/vv2xBSZm6N
@cameron_baston @IsaacMatthiasM1

#POCUS #IMPOCUS

1/18

Let's start with some polls
We will stick to low frequency probes here (i.e. curvilinear and phased array - linear has its own purpose for detailed pleural assessment)

Which preset do you prefer for B line assessment?

2/

What depth do you typically set?

3/

After selecting preset and selecting depth, do you ever adjust any other machine settings to optimize B line assessment?

4/

Importantly, there is no right answer to the above questions!

We won't discuss much about preset or probe type. But we will discuss machine settings to optimize B line quality, based on
1) Theory based on physical basis of B lines
2) In vitro study
3) Study in patients

5/

Mads Brodt
@madsbrodt

I've applied for a total of 10 development jobs in my life:

- 7 lead to interviews
- Of those 7, 4 was a good fit for me
- Of those 4, 3 offered me the job
- In all 3 cases, I negotiated a better deal

🧵 on what worked for me and what YOU can do to get similar results 👇

1. Apply for the RIGHT jobs. Look for midsize to large companies, not necessarily tech related. These tend to get way fewer applications than traditional tech companies, so it's easier to get your foot in the door - and they also tend to be great environments for learning.

2. Spend time on your application. Make sure your resumé and written application is tailored to the exact position. Recruiters or hiring managers can smell a copy/pasted resumé from miles away. Use as many words from the job spec in your application as possible.

3. Have a portfolio. Show off your skills and some personality on your website. This is a nobrainer for front-end devs, but it's true for other areas too. It's a great way to stand out over other candidates by actually *showing* the value you can create and your experience.

4. Establish personal contact. Send a quick message or make a phone call before applying. Just introduce yourself to the person in charge of hiring and ask a simple question. It's the best way I find to make them remember your application when you submit it.

Advertisement

Pratham Prasoon
@PrasoonPratham

Here's everything you need to know about the math for machine learning.

(+ free resources )
🧵👇

Before diving into the math, I suggest first having solid programming skills.

For example👇

(2 / 18)

In Python, these are the concepts which you must know:

- Object oriented programming in Python : Classes, Objects, Methods
- List slicing
- String formatting
- Dictionaries & Tuples
- Basic terminal commands
- Exception handling

(3 / 18)

If you want to learn those concepts for python, these courses are freecodecamp could be of help to you.

🔗Basics:youtube∙com/watch?v=rfscVS0vtbw
🔗Intermediate :youtube∙com/watch?v=HGOBQPFzWKo

(4 / 18)

👉You need to have really strong fundamentals in programming, because machine learning involves a lot of it.

It is 100% compulsory.

(5 / 18)

Pratham Prasoon
@PrasoonPratham

How to learn Python for machine learning. 🐍

The beginner's guide.
(based on my real life experience of mentoring 300+ students)

🧵👇

In this thread we'll look at all the concepts in Python you need to know for machine learning along with all the free resources that you'll need.

All of this is based on my experience of successfully teaching 300+ students how to code using Python.

(2 / 19)

You can use many languages for machine learning, why Python?

Because of 2 reasons:
- Comparatively easier to learn than other languages
- Has the biggest and most mature community

This makes Python a no-brainer to learn for machine learning as a beginner.

(3 / 19)

These are the absolute basics which you must know about:

- Basic terminal commands
- Basic arithmetic (+,-,/,*)
- Accepting user input
- For & While loops
- Exception handling
- If-Else statements
- Functions, modules & Imports

(4 / 19)

Then comes the more tougher concepts which you must know about:

- Object oriented programming in Python:Classes, Objects, Methods
- PIP (Pypi)
- List slicing
- String formatting
- Dictionaries & Tuples
- Managing environments
- Dunder methods like __init__

(5 / 19)

Jan Giacomelli
@jangiacomelli

🤔 Python generators

What are they? How to use them?

#Python

🧵Let's find out 👇

1️⃣ Python generators are lazy iterators delivering the next value when their .next() is called.

They are created by using the yield keyword

next() can be called explicitly or implicitly inside for loop

They can be finite or infinite

2️⃣ yield - where a value is sent back to the caller, but the function doesn’t exit afterward as with the return statement

The state of function is remembered.

For example, the number is incremented and sent back from yield at the consecutive call next()

3️⃣ Generator stores only the current state of the function - it generates next element on next() call and forgets the previous one -> it saves memory

For example, we don't need to store 1mio elements in memory to do something with each element

4️⃣ When you call a generator function generator object is returned - it's not executed yet

It executes only when next() is called

For example, that's why Exception is raised only on the next() call

You May Also Like

STEALTH JEFF
@drawandstrike

Andy McCabe has some explaining to do.

Andrew McCabe was reported to the FBI’s Office of Public Affairs for making an unauthorized leak of classified info to the media about @GenFlynn in early February 2017.

If you’ve wondered exactly who it was who leaked the classified info from that intelligence report on @GenFlynn’s phone calls with then-Russian ambassador Kislyak to the media, you are right now seeing a huge honking clue who it was.

To those responding to this info by sneering "Nothing's gonna happen":

Up.

Your.

Game.

You just might be too stupid to be following me.

The then-Deputy Director of the FBI get's caught targeting @GenFlynn with an illegal leak so he can 'investigated' by Peter Strzok?

Now that we know it was McCabe who set up Flynn for a fake 'perjury' investigation by his pet attack dog, Peter Strzok, the next name that's going to surface is...Joseph Pientka.

Simon DeDeo
@SimonDeDeo

"I lied about my basic beliefs in order to keep a prestigious job. Now that it will be zero-cost to me, I have a few things to say."

As a dean of a major academic institution, I could not have said this. But I will now. Requiring such statements in applications for appointments and promotions is an affront to academic freedom, and diminishes the true value of diversity, equity of inclusion by trivializing it. https://t.co/NfcI5VLODi
— Jeffrey Flier (@jflier) November 10, 2018

We know that elite institutions like the one Flier was in (partial) charge of rely on irrelevant status markers like private school education, whiteness, legacy, and ability to charm an old white guy at an interview.

Harvard's discriminatory policies are becoming increasingly well known, across the political spectrum (see, e.g., the recent lawsuit on discrimination against East Asian applications.)

It's refreshing to hear a senior administrator admits to personally opposing policies that attempt to remedy these basic flaws. These are flaws that harm his institution's ability to do cutting-edge research and to serve the public.

Harvard is being eclipsed by institutions that have different ideas about how to run a 21st Century institution. Stanford, for one; the UC system; the "public Ivys".

Advertisement

Jaya_Upadhyaya
@Jayalko1

NARHARI AND BHAGWAN PANDURANG/VITTHAL

Once a goldsmith by the name of Narhari lived in Pandharpur. He excelled in his craft. He had vowed not to look at Pandurang or visit his mandir ever, as he was a Shiva Bhakt.
One day, a wealthy merchant visited him.

He wanted a gold girdle studded with precious gems, to be made for Prabhu Vitthal. Narhari refused to make it as it was for Vitthal.
After many requests, he agreed to make the girdle but asked the merchant to bring the measurement from the mandir as he himself would not go there

The measurement was given n he made the girdle. When the girdle was put on Vitthal, it was loose. Narhari had to shorten it. Now the girdle became tight. So Narhari was forced to take measurement on his own. He had to go to the mandir but went blindfolded to avoid seeing Vitthal

In the Mandir, when blindfolded Narhari touched Vitthal, he felt as if he was touching Shiva with matted hair, the moon, snake in His neck and Trishul in His hand. Ecstatic with joy, he removed the blindfold to see Shiva but he found the murti of Vitthal with Rukmini there.

He realised the truth that there was no difference between Vitthal and Shiva. Narhari at once fell at the feet of Panduranga. He then took the measurement of the waist of Vitthal and this time the ornament fit the Murti perfectly.

Narayan Hari🙏🏻

$\uc2e0\ub150\uc758\uc0ac\ub098\uc774$

신념의사나이...
@Belposto_korea

The RV and the Redemption Funds.

ALL Exchanged Currencies as well as all Zim Redemption funds will be placed in the accounts they set up in the Quantum Financial System./1

Absolutely none of these funds will be placed in a Bank at the redemption appointment. Bank computers cannot be programmed to transfer Digital Gold Certificates./2

The QFS can and will transplant funds into a bank account as has been explained earlier,/3

so a person can transfer funds to their own bank account but the desirability will become less and less as the banks go into receivership and their customer’s deposits are confiscated to avoid the Bank’s demise./4

Which bank can compete with the QFS when the QFS is free, secure, private, and stable? One of the major problems in the transition will be the use of Debit Cards for merchant purchases. Will VISA or Mastercard be available?/5

foone
@Foone

An interesting set of games that I've been vaguely interested in since I was a kid, after seeing one at a friend's house and never being able to find it again, was the BATTLES IN A DISTANT DESERT/BATTLES ON DISTANT PLANETS series.

These are turn-based wargames for windows 3.x, sharing the same engine.
Battles in a Distant Desert is from 1992, and is based on the first Iraq war (Desert Storm)

and Battles on Distant Planets is from 1991, and takes place in SPACE!
I'm pretty sure this is the one I played as a kid.

They've got 3 options to play with:
* Player vs. Player
* Player vs. Computer
* Computer vs. Computer

So it's a 0-2 player game!

They also did a DOS strategy game called "STARDATE 2140.2: GALACTIC CONQUEST" in 1990, but it seems to be lost.
All the games share an experimental AI system based on neural networks.