Image: GitHub
Why it mattersAgents that scrape sites now fill forms, solve CAPTCHAs and reuse a saved login without a second tool stitched in.
Most teams building AI data pipelines still glue a request library, a browser driver and a CAPTCHA solver together. Scrapling v0.5 was released on 17 October. The Python framework now carries a much bigger Model Context Protocol server. An AI agent can run a full browser session on its own. The repository has 91,033 stars on GitHub. The project started in October 2024. It ranked ninth on Trendshift on 1 November.
What the new MCP server does for an agent
The release notes say the server lets an agent do a complete browser workflow in one ordered run. The agent can fill a form, click, hover, scroll, use the keyboard, handle a dialog, and wait for the result. It can read the current page as HTML, Markdown or plain text. It can read the whole page or just one element. It can search a snapshot that shows each element's role, name and position. It can run its own JavaScript, including async work. It can take a focused screenshot as PNG or JPEG. It can inspect network traffic, save and restore browser login state, and reach into open or closed Shadow DOM. The notes also say the server can "solve almost any CAPTCHA."
New features for the human developer
Scrapling now records every completed request made by a page. It records across loads, actions and open tabs. The developer turns it on with record_requests=True on a session. The saved responses can be read later. The developer filters them by URL, method, type or status. The default limit is 1,000 per session. Headers, request bodies, API responses, HTTP errors and redirects are kept.
A login can be saved once with save_state(path) and reused with load_state(path). The file is JSON in Playwright's own format. Cookies, local storage and IndexedDB travel between tools. Open Shadow DOM is now reachable with pierce_shadow=True. It works in browser fetchers, sessions and the CLI. CSS or XPath selectors still apply. Crawlers can pick crawl_strategy="bfs" for oldest first or "dfs" for newest first.
What else changed, and the breaking parts
The release page lists breaking changes big enough that the author points readers to a separate migration guide. The bug fix list is long. It covers Cloudflare challenge handling with delayed widgets. It covers browser startup from an elevated Windows shell. It covers charset detection and header insertion order. It covers dictionary form encoding for lists and tuples. It fixes robots.txt failure handling. A failed robots.txt lookup is no longer cached as permission to crawl.
The documentation site now carries ten languages. The list is English, Arabic, Spanish, Brazilian Portuguese, French, German, Simplified Chinese, Japanese, Russian and Korean. Arabic has right-to-left support. The agent skill, its examples and its downloadable archive were updated for the new browser and MCP tools.
Scrapling is a one-year-old project. The 91,033 stars are a total, not a gain. The trending ranking on 1 November is what makes the current release the news. Teams that already use Scrapling for AI scraping get a new way to let an agent drive the browser. The extra network and login tools sit around it.
Source
GitHub, by D4Vinci.
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.