SocketBot is the fetcher operated by Stedfast Softworks LLC for the Socket application. When a Socket user asks their assistant to compare products for a project, SocketBot retrieves the manufacturer pages the user's own search results point at, and reads the specifications off them.
1. What SocketBot Does
- Fetches individual pages, on demand, in response to a specific request from a signed-in user.
- Reads specification tables and
schema.orgproduct markup from those pages. - Requests one page at a time per host.
2. What SocketBot Does Not Do
- It does not train models. Nothing SocketBot fetches is used to train an AI model, ours or anyone else's.
- It does not crawl. SocketBot does not follow links to discover new pages, does not build an index, and does not sweep a site.
- It does not execute JavaScript. SocketBot requests the HTML document and nothing else. There is no headless browser.
- It does not disguise itself. SocketBot always identifies as the user agent above. It never presents itself as a browser or as another crawler, and it does not rotate identities or addresses.
3. How SocketBot Behaves
SocketBot reads robots.txt before every fetch, evaluating the specific
path it is about to request — not just the host — against the group that names
SocketBot, or the * group if none does. It re-runs that
check after every redirect, so a redirect cannot carry it onto a path you disallowed.
If robots.txt cannot be retrieved, SocketBot treats the host as
disallowed and does not fetch.
SocketBot honors Crawl-delay for its group, and honors the
Content-Signal directive: a host declaring
ai-input=no is not fetched.
4. How to Control SocketBot
To block SocketBot entirely, add this to your robots.txt:
User-agent: SocketBot
Disallow: /To allow it but slow it down:
User-agent: SocketBot
Crawl-delay: 10
Allow: /
Changes take effect within an hour, which is how long SocketBot caches a
robots.txt file.
5. Contact
If SocketBot behaved in a way this page does not describe, or you want it to stop and would rather tell us than edit a file, write to us:
Stedfast Softworks LLC Email: [email protected] Website: https://socket.link/bot