StockistoMarketDataBot

Stockisto is operated by Goty Invest AB (org. no. 559006-5651).

How shops are ranked

It sends this User-Agent on every request:

StockistoMarketDataBot/0.1 (+https://stockisto.com/bot; contact=crawler@stockisto.com)

If you see anything claiming to be us that does not match this string, it is not us: tell us at crawler@stockisto.com. Anyone can send a User-Agent, so do not allow-list traffic on this string alone. Our proxy pool uses the IP addresses listed below. Some sources connect directly from our hosts. To check a request, email crawler@stockisto.com with the time and path.

It does not sign in, pass a paywall or a bot check, or go around a block.

We also read shops' public product indexes hosted by these vendors:

• Algolia: .algolia.net
• Loop54: .54proxy.com
• Norce: storeapi.jetshop.io

• 38.225.18.100
• 38.225.18.130
• 38.225.18.131
• 38.225.18.62
• 38.225.18.67
• 38.225.19.104
• 38.225.19.111
• 38.225.19.38
• 38.225.19.75
• 38.225.19.82
• 38.225.6.133
• 38.225.6.148
• 38.225.6.224
• 38.225.6.242
• 38.225.7.122
• 38.225.7.197
• 38.225.7.66
• 92.112.231.109
• 92.112.231.227
• 92.112.231.29

• Before its first request to your site in a run, it reads robots.txt and obeys its Allow and Disallow rules. A group that names StockistoMarketDataBot wins over the * group. The longest matching path decides, and Allow wins a tie. If a page it wants is disallowed, it skips that page and can still read the rest of your site.
• If robots.txt is missing, or answers with anything other than 200 and plain text, your site has no rules for us. If it answers with a server error, or is larger than 512 KiB, we read nothing from your site in that run.
• It does not read Crawl-delay. Each site has its own pace instead. A site we have not tuned starts at 2 requests a second, with at most 4 at once. After 20 clean answers in a row the pace rises by a tenth, never above 3 a second. A server error, or an answer that takes 5 seconds or more, halves it.
• A 403 or a challenge stops us on every IP, with a weekly retry or later if Retry-After says so. A 429 makes us wait and slow down. We never change IP because of a 429. A shop-wide refusal also stops vendor indexes for that shop.

Block it in robots.txt. Add this. Each run reads robots.txt again, so the rule applies from the next run.

User-agent: StockistoMarketDataBot
Disallow: /

Email us. Write to crawler@stockisto.com with your host name. For a store's own website we record a stop that every source checks before each request. For any other site we pause the sources that read it. A written stop also withdraws the shop's offers without a further request.

Take a listing down. Blocking the crawler stops future reads. To remove the business listing, use our removal form. A person reviews every request, and an approved removal also blocks the listing from coming back on a later import.

Own the business? Claiming your listing is free, and it lets you correct it instead of removing it.

If the data is about you rather than about a website you run, the notice you want is the public-sources section of our Privacy Policy. It covers what we hold, our lawful basis, how long we keep it, and your right to object. Send privacy requests to privacy@stockisto.com, not to the crawler address: that way they reach the right people.