DALLE

NATURAL LANGUAGE PROCESSING (NLP) WEEKLY NEWSLETTER

The NLP Cypher | 05.01.22

>>> curl -L http://git\.io/unix

Hey Welcome back! Want to start off by giving a few shout outs!!!

1. Hectiq.AI hosted the big-yaml — Neural Magic’s config to serve 19 BERT models simultaneously, all under 16GBs of RAM! 😍 Demo
2. If you want to know more about the demo above, you can read about it here: Thank you KDnuggets and Towards AI! 🚀
3. Scrape tweets with Twint and classify it with a Neural Magic Sparse Transformer: Code 🧙‍♂️

ICLR Happened:

Stanford

Stanford AI Lab Papers and Talks at ICLR 2022

Google

Google at ICLR 2022

Meta

ICLR 2022

Amazon

ICLR 2022

Apple

ICLR 2022

DeepMind

DeepMind's latest research at ICLR 2022

FormNet: A New Model for Document Understanding

Gets SOTA performance on the CORD, FUNSD, and Payment benchmarks.

CORD benchmark

FormNet: Beyond Sequential Modeling for Form-Based Document Understanding

WaNLI: Generate Your Own NLI dataset

Run Python in the Browser via HTML

The makers of Anaconda came out with this. 🍾❤️

Has only been tested on Chrome thus far.

Anaconda | New from Anaconda: Python in the Browser

Pyscript.net

Code:

GitHub - pyscript/pyscript

DALL-E-2 | Performance and Limitations

Limitations of DALL-E-2 | a thread 🧵

DALL-E-2 PyTorch Implementation

Lucidrains for the win!

GitHub - lucidrains/DALLE2-pytorch: Implementation of DALL-E 2, OpenAI's updated text-to-image…

SEAL 🦭 Search Engines w/ Autoregressive LMs

Thread 🧵

Thread by @MicheleBevila20 on Thread Reader App

Code

GitHub - facebookresearch/SEAL: Search Engines with Autoregressive Language models

GPT-NeoX Annotated

GitHub - labmlai/neox: Simple Annotated implementation of GPT-NeoX in PyTorch

Play

DiffCSE — Meta’s New Sentence Embeddings Library

GitHub - voidism/DiffCSE: Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive…

Paper

OpenAIs New Clip Model

(a silent drop)

ViT-L/14@336px (#234) · openai/CLIP@b4ae449

Get Stoic

Stoic Quotes | The best quotes from the great Roman Stoics

Papers to Read 📚

https://arxiv.org/pdf/2204.13074.pdf

https://arxiv.org/pdf/2204.07705.pdf

NLP Index👨‍💻

A collection of recently released repos that caught our 👁

LitMind Dictionary

An open-source online generative dictionary that takes a word and context containing the word as input and automatically generates a definition as output.

GitHub - blcuicall/litmind-dictionary

Connected Papers 📈

SkillSpan Repository

A novel skill extraction dataset consisting of 14.5K sentences and over 12.5K annotated spans from job postings.

GitHub - kris927b/SkillSpan: SKILLSPAN: Competences as Spans for Skill Extraction from Job Postings

Models

jjzha (Mike Zhang)

Connected Papers 📈

LM-Debugger

An interactive debugger tool for transformer-based LMs, which provides a fine-grained interpretation of the model’s internal prediction process.

GitHub - mega002/lm-debugger: The official code of LM-Debugger, an interactive tool for inspection…

Connected Papers 📈

SalesBot: Transitioning from Chit-Chat to Task-Oriented Dialogues

The first large-scale dataset of dialogues transitioning from chit-chat to task-oriented scenarios.

GitHub - MiuLab/SalesBot: Transitioning from Open-Domain Chit-Chat to Task-Oriented Dialogues

Connected Papers 📈

PLOD: An Abbreviation Detection Dataset

An abbreviation detection dataset for scientific documents.

GitHub - surrey-nlp/PLOD-AbbreviationDetection: This repository contains the PLOD Dataset for…

Connected Papers 📈

Ricky ҈̿҈̿҈̿҈̿҈̿҈̿Costa̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈