Authors Gail weiss
7 days
30 days
All time
Recent
Popular
EXTREMELY excited to announce RASP, a programming language whose goal is to provide a computational model for transformers in much the same way that automata have served for RNNs. Work with @yoavgo and @yahave , accepted into ICML 2021. https://t.co/3Vp57mHhbJ
RASP abstracts away low-level operations into simple primitives, allowing a programmer to "think like a transformer" without getting bogged down in the details of how their solution is realised in practice. We use it to imagine how a transformer might solve several formal tasks
Using RASP, we found solutions for problems such as Dyck-k (for any k and depth!), and logical reasoning (in a simplified variant of the task considered in "Transformers as Soft Reasoners over Language"), previously unsolved for the Transformer model
Anyway, this one goes out to my hater, @mmgm
RASP abstracts away low-level operations into simple primitives, allowing a programmer to "think like a transformer" without getting bogged down in the details of how their solution is realised in practice. We use it to imagine how a transformer might solve several formal tasks
Using RASP, we found solutions for problems such as Dyck-k (for any k and depth!), and logical reasoning (in a simplified variant of the task considered in "Transformers as Soft Reasoners over Language"), previously unsolved for the Transformer model
Anyway, this one goes out to my hater, @mmgm
getting rave reviews on my latest venture pic.twitter.com/zXruI4NZvb
— gail weiss (@gail_w) December 7, 2020