Ok, ok. Lots of exciting events happened this past week that we want to share with you. While peeps are in-flight to Vancouver for NeurIPS 2019 (starting today), Twitter has been a’buzz with data galore. And last not but least, we gave a talk about hosting a low-resource BERT in the cloud for Question Answering. Check out my “pointing at screens” skills here… 👇
This Week:
Top Papers From NeurIPS Ranked by GitHub Popularity
Twas the Night Before NeurIPS
Decoding Transformers
Hyper-Excited for Graphs
TensorFlow Kills Python 2
Moving Away From Intents in Conversation
SOTA Multi-Hop QA for Open Domain
Top Papers From NeurIPS Ranked by GitHub Popularity
Twas the Night Before NeurIPS
When all through the house not a creature was stirring, not even a mouse. On a cold December night, IBM dropped this…
2 days later, Facebook Research dropped this…
Ian GoodFellow:
Decoding Transformers
The lads from Brooklyn just added decoders to their encoders. Yea, this may be a surprise to some of you but most transformers (excluding Bart and T5) only have an encoder stack:
In a really enjoyable read from Hugging Face, they share how they added decoders to its library in an attempt to get better at language understanding and generation in the same model. They also provide an intuitive history of the transformer architecture.
Medium:
In addition, 🤗 has a new demo with even more decoding controls (BOOM):
Hyper-Excited for Graphs
Yes graphs are cool. Yes they are popular. But have you heard of hypergraphs? In this piece from Grakn.AI, they define hypergraphs and show how they differ from traditional SQL databases and regular property graphs (RDF triplet stores).
Remember: edge is a bridge, vertices are endpoints of bridges…
- A hypergraph consists of a non-empty set of vertices and a set of hyperedges;
- A hyperedge is a finite set of vertices (distinguishable by specific roles they play in that hyperedge);
- A hyperedge is also a vertex itself and can be connected by other hyperedges.
Medium:
If you are interested in knowledge graphs and you are in NYC this week, Grakn will be giving a talk (we will be there)… They are calling for all hands on deck:
TensorFlow Kills Python 2
For those of you who refuse to use parentheses in your print commands, Python 2.7 will be deprecated after TensorFlow 2.1.
R.I.P.
Moving Away From Intents in Conversational AI
In a thought piece by Alan Nichol of Rasa, he discusses how intents limit the possibilities of conversation and if we want to move towards the next wave of innovation in chatbot technology, intents need to be replaced by a more flexible means of NLU.
Blog:
SOTA Multi-Hop QA for Open Domain
Salesforce/Allen Institute/ Uni of Wash. dropped heat this week with their open domain question answering research revealing they had advanced SOTA results on HotpotQA by 10 points. 🤫 It’s a retriever / reader model:
Paper:
Sad news: In a pithy tweet exchange with Francois Chollet, he told us he won’t be at NeurIPS (don’t cry for me Argentina):
This column is a weekly roundup of NLP news and code drops from researchers worldwide.
Follow our Twitter for complete coverage: @Quantum_Stat
