Crawl fixture hub

gtm-crawl-fixture:hub:v1

Crawl test fixtures (OD-115). Each link below is one fixture; docs/sites/crawl-fixtures.json in the repository says what each answers.

• noindex page
• X-Robots-Tag page
• canonical elsewhere
• canonical target
• 2-hop redirect
• robots-disallowed page
• CSR-only page
• CSR gaps page
• 500 route
• page that changes on cue