AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt

Quick Summary:

Attackers explain how an anti-spam defense became an AI weapon.

Shortly after he noticed Facebook's crawler exceeding 30 million hits on his site, Aaron began plotting a new kind of attack on crawlers "clobbering" websites that he told Ars he hoped would give "teeth" to robots.txt.

Watching the controversy unfold was a software developer whom Ars has granted anonymity to discuss his development of malware (we'll call him Aaron).

---

More Pictures

---

Read Quick SummarySave Time

or

Article Details

Author / Journalist: Ashley Belanger

Category: Technology

Markets: United StatesMali

Topics: Bots Anthropic Ai crawlers Features Iocaine Facebook LLMs Policy Quixotic Meta Google Cybersecurity Ai poisoning Openai Artificial Intelligence Tarpits Ai scraping Nepenthes

Source Website Secure: Yes (HTTPS)

News Sentiment: Negative

Fact Checked: Legitimate

Article Type: News Report

Published On: 2025-01-28 @ 21:16:31 (2 days ago)

News Timezone: GMT -5:00

News Source URL: arstechnica.com

Language: English

Article Length: 148 words

Reading Time: 1 minutes read

Sentences: 119 lines

Sentence Length: 2 words per sentence (average)

Platforms: Desktop Web, Mobile Web, iOS App, Android App

Copyright Owner: © Ars Technica

News ID: 25590244

View Article Analysis

About Ars Technica

Main Topics: Technology

Official Website: arstechnica.com

Update Frequency: 4 posts per day

Year Established: 1998

Headquarters: United States

News Last Updated: 8 hours ago

Coverage Areas: United States

Ownership: Independent Company

Publication Timezone: GMT -5:00

Content Availability: Worldwide

News Language: English

RSS Feed: Available (XML)

API Access: Available (JSON, REST)

Website Security: Secure (HTTPS)

Publisher ID: #32

Publisher Details

Frequently Asked Questions