Docker run

The `docker run` command is one of the most used commands when we use the docker CLI (Command Line Interface).

What does happen when we use it and which are the most used options? Check the docs!

But if you don't have time, let’ see it in 2 minutes:

1/12

Under the Hood, `docker run` are 2 commands:

docker create+docker start

- a container layer is created on the top of an image
- the container just created is started

It's important to understand this to avoid confusion between docker run, docker start, and docker create

2/12

Docker RUN Common Options:

--name: assign a name
--rm: remove it when it exits
-p: publish ports
-e: set environment variable
-d: run it in the background
--network: connect it to a network
-i: keep stdin open
--mount: set a volume or bind mount
--user: set a user

3/12

Assign a name (--name)

By default, a random name is assigned, but in general is a good idea to assign a custom name to our containers, as we do to our pets!

4/12

Remove the container when exits (--rm)

Useful for disposable containers and examples. When a container is stopped, it’s automatically removed. Handy

5/12

All ML projects which turned into a disaster in my career have a single common point:

🚨 I didn't understand the business context first, got over-excited about the tech, and jumped into coding too early.

When someone asks you for a model, always ask:

👉 why do you need it?
👉 what is your current solution (e.g. what is the baseline)?
👉 who is going to use the predictions and how?
👉 what is the impact of the model’s downtime or mistakes?
👉 which metrics do we care about?

Once you have your answers, back them up with a solid exploratory data analysis, and, when done, loop in the biz team again.

This is a critical moment as your results will translate into 3 potential outcomes:

💡 “Really? This is weird. Well, in this case, the ML model doesn’t make much sense anymore”. You are off the hook 🔴
💡 “Interesting. I guess we’ll have to change requirements/scope then.” Course-correct before moving forward 🟠
💡 “This is what I expected. Let’s go ahead”.🟢

Might seem silly, but skip the above and you are all set for failure.
Trust me, I learned it the hard way 😱

Also, always remember that the best model is no model.

Machine Learning in the Real World 🧠 🤖

ML for real-world applications is much more than designing fancy networks and fine-tuning parameters.

In fact, you will spend most of your time curating a good dataset.

Let's go through the process together 👇

#RepostFriday

Collect Data 💽

We need to represent the real world as accurately as possible. If some situations are underrepresented we are introducing Sampling Bias.

Sampling Bias is nasty because we'll have high test accuracy, but our model will perform badly when deployed.

👇

Traffic Lights 🚦

Let's build a model to recognize traffic lights for a self-driving car. We need to collect data for different:

▪️ Lighting conditions
▪️ Weather conditions
▪️ Distances and viewpoints
▪️ Strange variants

And if we sample only 🚦 we won't detect 🚥 🤷‍♂️

👇

Data Cleaning 🧹

Now we need to clean all corrupted and irrelevant samples. We need to remove:

▪️ Overexposed or underexposed images
▪️ Images in irrelevant situations
▪️ Faulty images

Leaving them in the dataset will hurt our model's performance!

👇

Preprocess Data ⚙️

Most ML models like their data nicely normalized and properly scaled. Bad normalization can also lead to worse performance (I have a nice story for another time...)

▪️ Crop and resize all images
▪️ Normalize all values (usually 0 mean and 1 std. dev.)

👇

@svpino I understand ReLu in this way:

1. a neuron is a number in the model

2. not every neuronal is important for each label you want to predict

3. if you apply a “linear” activation, redundant neurons will influence the prediction a bit

4. but the ReLu activation won't consider a neuron relevant until it overpass a threshold

5. therefore removing noise for the predictions

would you agree ?

One of the most popular activation functions used in deep learning models is ReLU.

I asked: "Is ReLU continuous and differentiable?"

Surprisingly, a lot of people were confused about this.

Let's break this down step by step: ↓

Let's start by defining ReLU:

f(x) = max(0, x)

In English: if x <= 0, the function will return 0. Otherwise, the function will return x.

If you draw this function, you'll get the attached chart.

Notice there are no discontinuities in the function.

This should be enough to answer half of the original question: the ReLU function is continuous.

Let's now think about the differentiable part.

A necessary condition for a function to be differentiable: it must be continuous.

ReLU is continuous. That's good, but not enough.

Its derivative should also exist for every individual point.

Here is where things get interesting.

We can compute the derivative of a function using the attached formula.

(I'm not going to explain where this is coming from; you can trust me on this one.)

We can use this formula to see whether ReLU is differentiable.

The worst taught skill in machine learning is model validation.

If you can’t validate your models well, you have no idea if they will actually work.

Here are 3 steps I’d take if I was relearning model validation from scratch 🧵

1. Learn the essential evaluation metrics

Think accuracy should be your primary metric? You’re sorely mistaken.

Most of the best metrics instead focus on how far your were from the correct answer. Think RMSE and MAE.

Others point to how well calibrated your model is, like F1.

2. Learn the common forms of cross validation

Before diving in too deep, make sure you understand the basics.

You can’t become an expert in validation in the classroom, but knowing what is out there (simple k-fold, stratified, grouped, roll forward, etc.) is crucial.

3. Read old Kaggle competition solutions

Every day, or multiple times a week, pick an old Kaggle competition.

Read every solution that is posted and skip to their validation schemes.

There are nuances to every dataset, and this is the best way to see how pros navigate them.

4. Build simple models and try different CV schemes

Get a dataset and create a random test set.

Then, build some simple models and switch validation strategies in and out and see how well your models generalize for each scheme.

This will cement the importance of validation.

I’m excited to share our new paper on HyperTransformers, a novel architecture for few-shot learning able to generate the weights of a CNN directly from a given support set. 🧵👇

📜: https://t.co/vcm67G6P6t with Andrey Zhmoginov and Mark Sandler.

2) We train a transformer model to `convert` a few-shot task description into a small CNN network specialized in solving it on new images.

3) This effectively decouples a high-capacity transformer generator from a much smaller inference model. It is different from most of the existing methods, e.g. MAML where the generator and the executing model share the same architecture.

4) CNN weights are generated layer-by-layer from a combination of layer embedding (features from the last generated layer), and image w/ class embeddings (features directly from the data). The final weights are extracted from output of self-attention (similar to [CLS] tokens).

5) What is cool is that we can also add unlabeled samples from the support set into the mix, effectively allowing for semi-supervised few-shot learning!

2022 has started 🚀

If you want to
🐍 Learn Python
🧑‍💻 Master Data Scientist & ML
💰 Use Python for Financial Analysis

Then follow me ✔️

I will share a lot of great content this year 🔥

See my top 10 pages in 2021 👇🧵

No 1

Learn Python - a 8 hours video course

- Includes 17 lessons
- 34 prepared Notebooks
- a FREE eBook

No 2

No 2

Calculate the Relative Strength Index (RSI) with pandas
- Learn what RSI is
- Read stock prices with PDR
- How to calculate RSI using pandas

No 3

No 3

Calculate the MACD with pandas
- Get stock prices
- How to calculate MACD
- Make a plot with MACD lines

No 4

No 4

FREE eBook
- Backtesting a investment strategy
- Use Python and pandas
- 82 pages with source code

No 5

Lessons learned debugging ML models:

1/ It pays to be paranoid. Bugs can take so long to find that it’s best to be really careful as you go. Add breakpoints to sanity check numpy tensors while you're coding; add visualizations just before your forward pass (it must be right before! otherwise errors will slip in).

2/ It's not enough to be paranoid about code. The majority of issues are actually with the dataset. If you're lucky, the issue is so flagrant that you know something must be wrong after model training or evaluation. But most of the time you won't even notice.

3/ The antidote is obsessive data paranoia. Without this, data issues will silently take away a few percentage points of model accuracy.

4/ You can unit test ML models, but it's different from unit testing code. To prevent bugs from re-occurring, you have to curate scenarios of interest, then turn them into many small test sets ("unit tests") instead of one large one.

We introduce a new paradigm for dataset creation based on human 🧑‍💻 and machine 🤖 collaboration, which brings together the generative strength of LMs and the evaluative strength of humans. And we collect 🎉 WaNLI, a dataset of 108K NLI examples! 🧵

Paper: https://t.co/IUXcm9wIh2

Our pipeline starts with an existing dataset (MNLI), and uses data maps 📜 to automatically identify pockets of examples that demonstrate challenging 🧐 reasoning patterns relative to a trained model. Then we use GPT-3 to generate new examples likely to have the same pattern. 2/

Next we propose a new metric, also inspired by data maps, to automatically filter generations for those most likely to aid model learning. Finally, we validate ✅ the generated examples through crowdworkers, who assign a gold label 🟡 and (optionally) revise for quality ✍️. 3/

Remarkably, replacing MNLI with WaNLI (which is 4x smaller) for training improves performance📈 on seven OOD test sets🧪, including by 11% on HANS and 9% on ANLI. Under a data augmentation setting, combining MNLI with WaNLI is more effective than using other augmentation sets. 4/

Our method addresses limitations of crowdsourcing, where workers may resort to repetitive writing strategies 🤷, and leverages the great progress in text generation 📃. We get the best of both worlds: 🤖’s ability to produce diverse examples, and 🧑‍💻’s ability to evaluate them. 5/

@praveen0582 I use a combination of Jupyter notebooks and (just a bit biased :)) Scale Nucleus.

With average salaries of $145,000 for remote positions and an open end to what you can earn, Solidity developers are in high demand.

Time to become one and enter an interesting field in the industry.

This is your roadmap to becoming a Solidity Developer in 2022. ↓

Before we get into it, one clarification:

This is the roadmap for a very specific tech stack, used to develop smart contracts on Ethereum-like blockchains.

This includes:

- Ethereum
- Polygon
- Binance Smart Chain
- etc.

It contains many of the fundamentals you need to branch out to other blockchains if you want to, but please don't expect to become a competent Solana smart contract developer by following this roadmap to the end.

Now that we got this out of the way, let's get into it! ↓

1. Learn The Basics Of Computer Science

Depending on where you currently stand in terms of your skills, it might be that you first need an introduction to CS overall.

Harvard offers its CS50 for free, and it'll take you a while, but it's worth

You intuitively know that poor quality images lead to poor object detection and vice-versa.

Image-Adaptive (IA)-YOLO improves object detection in adverse weather conditions using a hybrid task: image improvement combined with object detection.

• The paper, Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions, was published 3 days ago:

• The authors present an end-to-end hybrid data training task

• Each input image is adaptively enhanced to obtain better detection performance

• The hybrid architecture has a CNN-Parameter Predictor (PP) network that learns some appropriate Differentiable Image Processing (DIP) parameters to adaptivily enhance images for object detection, in a weakly supervised manner

• The DIP module consists of six differentiable filters with adjustable hyperparameters: Defog, White Balance(WB), Gamma, Contrast, Tone, and Sharpen.

• The CNN-PP network is trained using low-resolution images (256x256px) to speed the training

• CNN-PP parameters are used by the DIP module to improve the quality of the high-resolution images

• DIP processed (enhanced) images feed a YOLOv3 model for the object detection task

"Attention is all you need" implementation from scratch in PyTorch. A Twitter thread:
1/

There are two parts: encoder and decoder. Encoder takes source embeddings and source mask as inputs and decoder takes target embeddings and target mask. Decoder inputs are shifted right. What does shifted right mean? Keep reading the thread. 2/

The encoder is composed of N encoder layers. Let's implement this as a black box too. The output of one encoder goes as input to the next encoder and so on. The source mask remains the same till the end 3/

Similarly, we have the decoder composed of decoder layers. The decoder takes input from the last encoder layer and the target embeddings and target mask. enc_mask is the same as src_mask as explained previously 4/

Let's take a look at the encoder layer. It consists of multi-headed attention, a feed forward network and two layer normalization layers. See forward(...) function to understand how skip-connection works. Its just adding original inputs to the outputs. 5/