Anthopic-AI Bots and Amazon Web Services Support

A few weeks ago, a new bot (to us) started visiting our site. The IPs are from Amazon Cloud (AWS), located in the Ashburn, Virginia (US), and the only identifying information is the user agent “anthropic-ai.”

ID: 1691778765-054077-7179480296 »
Date/Time: Fri, 11 Aug 2023 21:32:45 +0300 »
IP address: 54.161.193.200 »
Query: ver=5.12.75 »
User agent: anthropic-ai
Reconstructed URI: https://www.valueweb.gr/wp-content/plugins/yuzo-related-post/assets/js/jquery.equalizer.js?ver=5.12.75 »
Infractions: 2 »
Country code lookup: US »
Request method: GET »
Hostname: ec2-54-161-193-200.compute-1.amazonaws.com »

The strange thing is that this bot is only searching for JS files, cache files and other unusual things like queries like “Query: ver=5.12.75”, rather than pages and posts. It’s also searching for very old files from WordPress plugins that were removed years ago, some of which had security issues (like the above example). Yuzo Related plugin was the cause of thousand WordPress sites hacked, including ours.

We find it very strange that “anthropic-ai” bot is searching data from 4-5 years back. That means that all that old data was collected somehow, stored and now verified. Why?

So What (TF) this bot is looking for? And what company is behind that bot? Since there are no details available about all that, so we consider this bot to be an attacker, probing for issues.

ID: 1691774167-499804-9625199232 »
Date/Time: Fri, 11 Aug 2023 20:16:07 +0300 »
IP address: 44.200.85.194 »
Query: ver=1.5.5 »
User agent: anthropic-ai
Reconstructed URI: https://www.valueweb.gr/wp-content/plugins/wpforo/wpf-assets/js/ajax.js?ver=1.5.5 »
Infractions: 1 »
Country code lookup: US »
Request method: GET »
Hostname: ec2-44-200-85-194.compute-1.amazonaws.com »

We contacted AWS support and reported all of this, providing our full logs as evidence. Amazon replied that they had notified the “customer” and that the issue would be resolved soon.

After a week, the issue was still not resolved, so we contacted Amazon again. They replied that they had contacted their customer again, who assured them that the issue would be solved.

Another week passed, and the issue was still not resolved. We contacted Amazon again, but this time their reply was different. They asked us which rule of Robots.txt the scanner bot was breaking.

We wasted our time with all of this. Since the bot doesn’t use fixed IPs, we can’t block it. We will block the “anthropic-ai” user agent and see how it goes. If that doesn’t work well, we will have to reconsider whether AWS should be 100% whitelisted.

It seems that many services, scanners, seo bots, scrapers, use AWS for ONE and ONLY reason: AWS is big and most websites are very cautious when it comes to block or ban an Amazon (AWS) IP or CIDR.