Sparks of artificial general intelligence: Early experiments with GPT-4

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Abstract

• LLMs show near-human performance across diverse fields and tasks.
• An evaluation of an early version of GPT-4 under development at OpenAI suggests it might reasonably be considered an early version of artificial general intelligence.
• This research focuses on GPT-4’s limitations and challenges to be addressed.
• The conclusion discusses the technology’s social implications and future research directions.

1. Introduction

• Intelligence is very difficult to define. Debate continues across psychology, philosophy, computer science, and other fields. (ak: Alan Turing’s Computing machinery and intelligence paper was a full 73 years ago.)
• To count as intelligent, a system should perform diverse tasks across broad domains rather than merely a particular task in a particular field. Building such intelligence (AGI) has long been an ambition of AI research, with little success until now.
• This study uses AGI to mean “a system demonstrating near-human or better-than-human performance across broad intellectual capabilities such as reasoning, planning, and learning from experience.” Other definitions are discussed in the conclusion. (ak: It introduces various previous definitions and notes shortcomings in each.)
• The most notable recent achievements have been in NLP (Natural language processing) based on large language models (LLMs), thanks to the transformer architecture, large datasets, and self-supervised learning.
• This study primarily compares GPT-4 with ChatGPT. Although GPT-4 still has various problems, including hallucinations and bias, it appears to be approaching AGI.
• We view GPT-4’s intelligence as signaling a true paradigm shift in computer science. (ak: This reminds me of Schillace laws.)

1.1. Our approach to studying GPT-4’s intelligence

• The traditional method evaluates performance on standard benchmark datasets independent of the training data.
• This is unsuitable for GPT-4 for two reasons.
  • GPT-4’s training data is not public. (ak: The concern is that benchmark datasets may already have been used in training. The human analogy would be not trusting college entrance exam scores because the exam papers may have leaked.)
  • GPT-4 also performs well in situations without a fixed answer, with multiple possible answers, or involving interaction. Traditional methods struggle to assess these aspects.
• We therefore borrow methods resembling traditional psychology rather than machine learning. However, the methodology is not rigorous, making quantitative evaluation difficult. Example tasks:
  • “Write a poem about prime numbers”: requires a creative combination of elementary mathematics, poetic expression, and sentence generation.
  • “Draw a unicorn in TiKZ”: requires creatively combining visual imagination and programming. (ak: TiKZ is an extension language for diagrams in LaTeX.)
• We ask follow-up questions that slightly modify these tasks, aiming to show that the answers are not “mere memorization.” (ak: Here, “mere memorization” means answering by having memorized the training data verbatim.)

1.2. Organization of our demonstration

(ak: Describes the rest of the paper’s organization. Omitted.)

2. Multimodal and interdisciplinary composition

3.1. From instructions to code

• On HumanEval, a benchmark dataset of docstring–code pairs, it substantially outperforms other LLMs.
• Because it may already have trained on HumanEval, it was also evaluated on LeetCode (a problem-bank site for people learning programming), where it again scored highly.
• It passed LeetCode’s mock interview with a perfect score.
• Many point out that coding test problems differ from real programming. Real programming is complex, involving domain knowledge, creativity, many existing components and libraries, and modifications to existing code. Considering these conditions, the researchers tried more realistic tasks such as data visualization, LaTeX coding, frontend development, and deep learning development (ak: Oh!? Recursive self-improvement?). It did quite well.

3.2. Understanding existing code written by humans

• Reverse engineering assembly into a high-level language: using programmers’ tools such as debuggers, it “explained” how assembly code worked and then translated it into Python.
• Inferring execution results: shown C source code and asked to infer the result of running it, it did so without trouble.
• Explaining execution: given a Python algorithm and asked to explain it, it did well.
• Executing pseudocode: because pseudocode is less rigorous, pretend code, it requires somewhat more “human-like” thought. It performed this without trouble. In the future, coding in natural language may be possible.

4. Mathematical abilities

…

6. Interaction with humans

6.1. Understanding Humans: Theory of Mind

A modified Sally-Ann test, replacing physical boxes with a shared Dropbox folder and using the names Alice and Bob. It passed.

9. Societal influences

9.1. Challenges of erroneous generations

…

9.2. Misinformation and manipulation

…

10. Directions and Conclusions

• GPT-4 is judged comparable to human performance across multiple domains and tasks. This agrees with OpenAI’s own conclusion.
• The main goal of the study is to assess GPT-4’s intelligence. We hope this work provides a useful first step.
• Future benchmarks should move from evaluating narrow AI models toward methods resembling human assessment.
• The central claim is that GPT-4 appears to display some characteristics of general intelligence. Many challenges remain before full AGI, however.

10.1. Definitions of intelligence, AI, and AGI

• The introduction defined AGI as “a system demonstrating near-human or better-than-human performance across broad intellectual capabilities such as reasoning, planning, and learning from experience,” but this is too superficial.
• Several other definitions have been proposed, each with shortcomings. For example:
  • Intelligence measures an agent’s ability to achieve goals in a wide range of environments(Leg08): excludes passive or reactive intelligence that does not actively pursue goals.
  • A system that can do anything a human can do(LH07): overlooks diversity among humans and defines intelligence anthropocentrically.
• Although this study adopts none of these existing definitions, each highlights important aspects of intelligence.

10.2. On the path to more general artificial intelligence

Problems common to GPT-family LLMs.

• Confidence calibration: Current models cannot reliably distinguish confidence from guessing. They fabricate facts absent from the training data and give inconsistent answers, yet tend to speak confidently even in these cases. (See Why Language Models Hallucinate, which proposes ways to reduce this problem.)
• Long-term memory: Very limited context and stateless operation make it difficult to teach models new facts. (ak: Recently, various efforts have connected long-term memory to GPT-family models, usually using a vector database.)
• Continual learning: Current models cannot yet update themselves and remain fixed at their training point.
• Personalization: Current models are difficult to personalize. (ak: Techniques such as LoRA are attracting attention. They enable fine-tuning at very low cost without performance degradation.)
• Planning and conceptual leaps: They struggle with tasks requiring “eureka”-like conceptual leaps.
• Transparency, interpretability, and consistency: It is difficult to verify whether generated content is consistent with training data. They sometimes give good post-hoc explanations, but these can be inconsistent with the actual content, among other problems.
• Cognitive fallacies and irrationality: They sometimes exhibit human-like cognitive errors and irrationality, such as confirmation bias and anchoring.
• Challenges with sensitivity to inputs: Small differences in prompts greatly affect response content. In other words, robustness is low.

How might the current “next-word prediction” approach be extended?

• Call external modules such as calculators and databases.
• Treat existing “next-word prediction” as “fast thinking” and add a “slow thinking” mechanism. (ak: Terms from Thinking fast and slow. Reasoning models released after GPT-4 have improved this somewhat. However, some studies suggest that the reasoning they output may have little connection to their actual results.)
• Integrate vector-based long-term memory that stores context.
• Introduce a paragraph–sentence–word hierarchy instead of token sequences.

10.3. What is actually happening?

• This study is limited to analyzing observable phenomena.
• Explanations of what happens internally and why these phenomena occur are lacking.
• Analyzing LLMs has suddenly become an extremely consequential problem.

Appendix A. GPT-4 has common sense grounding

Appendix B. Appendix for multimodal and interdisciplinary composition

Appendix C. Appendix for the Coding section

Appendix D. Additional examples for mathematical reasoning

Appendix E. Additional Interpretability Examples

Appendix F. Additional examples for interaction with the world

Appendix G. Supplementary Materials: Discriminative Capabilities