목차
Abstract
“
Alternatives to recurrent neural networks , in particular, architectures based on self-attention, are gaining momentum for processing input sequences. In spite of their relevance, the computational properties of such networks have not yet been fully explored. We study the computational power of the Transformer , one of the most paradigmatic architectures exemplifying self-attention. We show that the Transformer with hard-attention is Turing complete exclusively based on their capacity to compute and access internal dense representations of the data. Our study also reveals some minimal sets of elements needed to obtain this completeness result.
목차 관련 글 LLM에 대한 촘스키의 견해 이 글을 인용: “나도 연결주의가 만능이며 신경망만 가지고 모든 게 해결될거라고 보지는 않는다.” Attention is all you need “We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing…” 트랜스포머 아키텍쳐의 오리진 스토리 관련 절: Chapter 9. The Goliath Paradox Connectionist approaches to language “In research on theoretical and Computational linguistics and Natural language processing, the dominant formal a…” Defining and identifying attention capture deceptive designs in digital interfaces 관련 절: 3. Systematic Literature Review of Attention Capture Deceptive Designs Vector Symbolic Architectures as a Computing Framework for Emerging Hardware “This article reviews recent progress in the development of the computing framework vector symbolic architecture…” The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity “Through extensive experimentation across diverse puzzles, we show that frontier LRMs face a complete accuracy c…” Measuring AI Ability to Complete Long Tasks “This is the time humans typically take to complete tasks that AI models can complete with 50% success rate.” Nested Learning: The Illusion of Deep Learning Architectures “Over the last decades, developing more powerful neural architectures and simultaneously designing optimization …” So how does the mind work? “First, my claim that the mind is a computational system is different from the claim Fodor attacks (that the min…” 계산주의-연결주의 논쟁 관련 절: 각종 메모 BioGPT “Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their gre…” 완전 단백질 “(It is commonly, although mistakenly, thought that the amino acid intake may be inadequate in vegetarian diets.” Evolving artificial neural networks “There has been a great interest in combining learning and evolution with artificial neural networks (ANNs) in r…” A simple way to prevent neural networks from overfitting “However, overfitting is a serious problem in such networks.” 모드 (인터페이스) “A more complete definition of modes must incorporate how the user views the interface.” MatFormer: Nested Transformer for Elastic Inference “We present MatFormer, a novel Transformer architecture designed to provide elastic inference across diverse dep…” A complexity measure “Several properties of the graph theoretic complexity are then proved which show, for example, that complexity i…” Text-to-LoRA: Instant Transformer Adaption “T2L is a hypernetwork trained to construct LoRAs in a single inexpensive forward pass.” 정보시각화: 디자인을 위한 지각 관련 절: 메모