I’ll retweet this every time i see it
21 thoughts from 2021 I'd like to take into 2022: pic.twitter.com/hdHCgGiYVj
— Tim Urban (@waitbutwhy) December 31, 2021
More from All
Introducing Voltron: Language-Driven Representation Learning for Robotics!
Paper: https://t.co/gIsRPtSjKz
Models: https://t.co/NOB3cpATYG
Evaluation: https://t.co/aOzQu95J8z
🧵👇(1 / 12)
Videos of humans performing everyday tasks (Something-Something-v2, Ego4D) offer a rich and diverse resource for learning representations for robotic manipulation.
Yet, an underused part of these datasets are the rich, natural language annotations accompanying each video. (2/12)
The Voltron framework offers a simple way to use language supervision to shape representation learning, building off of prior work in representations for robotics like MVP (https://t.co/Pb0mk9hb4i) and R3M (https://t.co/o2Fkc3fP0e).
The secret is *balance* (3/12)
Starting with a masked autoencoder over frames from these video clips, make a choice:
1) Condition on language and improve our ability to reconstruct the scene.
2) Generate language given the visual representation and improve our ability to describe what's happening. (4/12)
By trading off *conditioning* and *generation* we show that we can learn 1) better representations than prior methods, and 2) explicitly shape the balance of low and high-level features captured.
Why is the ability to shape this balance important? (5/12)
You May Also Like
Like company moats, your personal moat should be a competitive advantage that is not only durable—it should also compound over time.
Characteristics of a personal moat below:
I'm increasingly interested in the idea of "personal moats" in the context of careers.
— Erik Torenberg (@eriktorenberg) November 22, 2018
Moats should be:
- Hard to learn and hard to do (but perhaps easier for you)
- Skills that are rare and valuable
- Legible
- Compounding over time
- Unique to your own talents & interests https://t.co/bB3k1YcH5b
2/ Like a company moat, you want to build career capital while you sleep.
As Andrew Chen noted:
People talk about \u201cpassive income\u201d a lot but not about \u201cpassive social capital\u201d or \u201cpassive networking\u201d or \u201cpassive knowledge gaining\u201d but that\u2019s what you can architect if you have a thing and it grows over time without intensive constant effort to sustain it
— Andrew Chen (@andrewchen) November 22, 2018
3/ You don’t want to build a competitive advantage that is fleeting or that will get commoditized
Things that might get commoditized over time (some longer than
Things that look like moats but likely aren\u2019t or may fade:
— Erik Torenberg (@eriktorenberg) November 22, 2018
- Proprietary networks
- Being something other than one of the best at any tournament style-game
- Many "awards"
- Twitter followers or general reach without "respect"
- Anything that depends on information asymmetry https://t.co/abjxesVIh9
4/ Before the arrival of recorded music, what used to be scarce was the actual music itself — required an in-person artist.
After recorded music, the music itself became abundant and what became scarce was curation, distribution, and self space.
5/ Similarly, in careers, what used to be (more) scarce were things like ideas, money, and exclusive relationships.
In the internet economy, what has become scarce are things like specific knowledge, rare & valuable skills, and great reputations.
