Meta drops terms that let it scrape news sites for AI training

Meta has pulled a sentence from its commercial terms that experts said read as a right to scrape any site a business advertises on and train AI on what it found. The Verge reports the removal on 17 October, after Nieman Lab and a Cornell law professor flagged the earlier draft and Meta told reporters its web crawlers had not changed.

The sentence that is gone, as it appeared in the terms Meta posted for an October 30 start, said: "Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products." It sat under a paragraph that let Meta use web scrapers, bots and crawlers to pull content from any domain a business was associated with on Facebook, Instagram, WhatsApp or Messenger.

How it was read, and by whom

Nieman Lab reported the language on 16 October. James Grimmelmann, a professor of digital and information law at Cornell, told the outlet the paragraph "clearly represents permission to Meta to train AI on any content from their sites and possibly to deploy some of that content, even in competition with the publishers themselves". The worry Grimmelmann set out was that a commercial agreement could argue it overrides a site's robots.txt, which is how most publishers already tell AI crawlers to keep out.

What Meta says now

Andy Stone, Meta's vice president of communications, pushed back the same day. Spokesperson Daniel Roberts told The Verge: "There is no change to how Meta uses web crawlers; any suggestion to the contrary is false." The follow-up statement, published on 17 October, says the consolidated terms "only applied to our ads and business products" and that content owners "can use robots.txt to signal their preferences. This is not about scraping news sites, and these updates don't give us new rights that supersede those preferences."

The retraction leaves the live commercial terms with no sentence that touches training at all, and the robots.txt directive stays the single machine-readable control a site has over Meta's crawlers.

What a site owner is left to do is the same thing as yesterday: open robots.txt, name each Meta crawler the site wants to exclude, and point to the paths. The business terms were only ever one place a right to crawl could have been written down, and today's statement says they will not be that place.

Source

The retraction and Meta's updated statement are reported by Terrence O'Brien at The Verge. The earlier terms and the Nieman Lab reading are in The Verge's piece from the previous day.