Global Relay for Website Archive captures and stores web content, such as web pages, images, and other digital assets, to preserve it for compliance purposes. As part of this service, the Global Relay Website Archive Bot crawls websites your company identifies to archive a snapshot of the relevant pages and files for that point in time.
Note: The Global Relay Website Archive Bot crawls only the web pages your company owns and has identified to be archived according to an agreed-upon schedule.
To ensure the Global Relay Website Archive Bot can crawl your websites, you need to safelist the following Global Relay IP addresses. Please contact Global Relay support for the list of IPs.
support@globalrelay.net
Identifying the Website Archive Bot
You can identify the Global Relay Website Archive Bot by its unique user agent, where the User-Agents Match Pattern is “GlobalRelayWebArchiveBot/1.0”
“GlobalRelayWebArchiveBot” along with additional agent information displays in the user-agent string. For example:
Mozilla/5.0 (compatible; GlobalRelayWebArchiveBot/1.0; +http://www.globalrelay.com/webarchivebot/)
When traffic to your website is coming from the Global Relay Website Archive Bot, you can typically identify it by conducting a reverse DNS lookup in the “*.globalrelaywebarchivebot.globalrelay.com” domain.
Note: Because Global Relay can employ parallel crawls to capture website data, you may notice multiple Global Relay machines crawling your website data at the same time with the “GlobalRelayWebArchiveBot/1.0” user agent.
Note: The information provided on this page is for transparency and verification purposes only. While Global Relay endeavours to ensure accurate identification and operation of Global Relay’s Web Archive Bot, we make no warranties, express or implied, regarding the bot’s behaviour on third-party systems or the effects of automated access to publicly available content.
For more information on Global Relay for Website Archive, contact Global Relay Support at support@globalrelay.net