New Neural Trick Helps Models Think in Longer Patterns

:::info
Authors:

(1) Hung Le, Applied AI Institute, Deakin University, Geelong, Australia;

(2) Dung Nguyen, Applied AI Institute, Deakin University, Geelong, Australia;

(3) Kien Do, Applied AI Institute, Deakin University, Geelong, Australia;

(4) Svetha Venkatesh, Applied AI Institute, Deakin University, Geelong, Australia;

(5) Truyen Tran, Applied AI Institute, Deakin University, Geelong, Australia.

:::

Table of Links

Abstract & Introduction

Methods

Methods Part 2

Experimental Results

Experimental Results Part 2

Related Works, Discussion, & References

Appendix A, B, & C

Appendix D

2.3 Pointer-Augmented Neural Memory (PANM)


2.3.1 Pointer Unit



2.3.2 Pointer-based Addressing Modes



2.3.3 The Controller



Table 1: Algorithmic reasoning: mean sequence-level accuracy (%) over testing lengths Other Max is selected as the best numbers at each length mode from other baselines.


Table 2: SCAN (Left): Exact match accuracy (%, median of 5 runs) on splits of various lengths. Mathematics (Right): mean accuracy over 5 runs. The baselines’ numbers are from Csord´as et al. [2021] and we run PANM using the authors’ codebase.

:::info
This paper is available on arxiv under CC BY 4.0 DEED license.

:::

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.