Add resource "The Annotated Transformer" Accepted
Changes: 4
-
Add The Annotated Transformer
- Title
-
- Unchanged
- The Annotated Transformer
- Type
-
- Unchanged
- Interactive
- Created
-
- Unchanged
- 2018-04-03
- Description
-
- Unchanged
- The Transformer from “Attention is All You Need” has been on a lot of people’s minds over the last year. Besides producing major improvements in translation quality, it provides a new architecture for many other NLP tasks. The paper itself is very clearly written, but the conventional wisdom has been that it is quite difficult to implement correctly. In this post I present an “annotated” version of the paper in the form of a line-by-line implementation. I have reordered and deleted some sections from the original paper and added comments throughout. This document itself is a working notebook, and should be a completely usable implementation. In total there are 400 lines of library code which can process 27,000 tokens per second on 4 GPUs.
- Link
-
- Unchanged
- https://nlp.seas.harvard.edu/2018/04/03/attention.html
- Identifier
-
- Unchanged
- no value
Resource | v1 | current (v1) -
Add Transformer (machine learning model)
- Title
-
- Unchanged
- Transformer (machine learning model)
- Description
-
- Unchanged
- A transformer is a deep learning model that adopts the mechanism of attention, differentially weighing the significance of each part of the input data. It is used primarily in the field of natural language processing (NLP) and in computer vision (CV). Like recurrent neural networks (RNNs), transformers are designed to handle sequential input data, such as natural language, for tasks such as translation and text summarization. However, unlike RNNs, transformers do not necessarily process the data in order. Rather, the attention mechanism provides context for any position in the input sequence.
- Link
-
- Unchanged
- https://en.wikipedia.org/?curid=61603971
Topic | v1 | current (v1) -
Add Transformer (machine learning model) discussed in The Annotated Transformer
- Current
- discussed in
Topic to resource relation | v1 -
Add Deep learning has tool Transformer (machine learning model)
- Current
- has tool
Topic to topic relation | v1