AI isn’t very good at history, new paper finds

Quick Summary:

The results, which were presented last month at the high-profile AI conference NeurIPS, were disappointing, according to researchers affiliated with the Complexity Science Hub (CSH), a research institute based in Austria.

The best-performing LLM was GPT-4 Turbo, but it only achieved about 46% accuracy — not much higher than random guessing.“The main takeaway from this study is that LLMs, while impressive, still lack the depth of understanding required for advanced history.

The researchers shared sample historical questions with TechCrunch that LLMs got wrong.

---

More Pictures

---

Read Quick SummarySave Time

or

Article Details

Author / Journalist: Charles Rollet

Category: Technology

Markets: United StatesAustriaChinaEgypt

Topics: Research TC AI LLMs Hallucinations Benchmarks

Source Website Secure: Yes (HTTPS)

News Sentiment: Neutral

Fact Checked: Legitimate

Article Type: News Report

Published On: 2025-01-19 @ 15:01:00 (14 hours ago)

News Timezone: GMT -5:00

News Source URL: techcrunch.com

Language: English

Article Length: 570 words

Reading Time: 4 minutes read

Sentences: 25 lines

Sentence Length: 23 words per sentence (average)

Platforms: Desktop Web, Mobile Web, iOS App, Android App

Copyright Owner: © TechCrunch

News ID: 25255642

View Article Analysis

About TechCrunch

Main Topics: Technology

Official Website: techcrunch.com

Update Frequency: 8 posts per day

Year Established: 2005

Headquarters: United States

News Last Updated: 8 hours ago

Coverage Areas: United States

Ownership: Independent Company

Publication Timezone: GMT -5:00

Content Availability: Worldwide

News Language: English

RSS Feed: Available (XML)

API Access: Available (JSON, REST)

Website Security: Secure (HTTPS)

Publisher ID: #20

Publisher Details

Frequently Asked Questions