Crawler Test Scenarios
This page provides deliberate PASS and FAIL scenarios for XeoPix crawler checks #8, #9, #14, #47, #163, and #220. Each section is labelled with the check number and expected result so the script output can be validated against the expected state.
Check #9 — Broken Internal Links
CRITICALThe script follows every <a href> pointing to an internal URL and records the HTTP status code. Anything non-200 (excluding intentional 301s) is flagged as broken.
✗ FAIL — Links that return 404
/this-page-does-not-exist-404/old-product-page-no-redirect/pricing/enterprise/legacyCheck #163 — Anchor Text Quality
IMPORTANTThe script reads every internal <a> element's visible text and flags any that match a generic phrase blocklist: "click here", "read more", "here", "this page", "learn more".
✓ PASS — Descriptive keyword-rich anchor text
Contact the Xeopix support teamBrowse our SEO and AEO guidesLearn about the Xeopix team and missionRead our SEO basics guide for beginnersCheck #8 & #220 — Redirect Verification
CRITICAL / IMPORTANTThe script requests each URL and inspects the HTTP response status. URLs that were moved must return 301 with a valid destination — never a bare 404. Redirect rules are defined in public/_redirects.
✓ PASS — 301 redirect rules in place
/old-pricing → 301 → /contact/old-features → 301 → //old-blog → 301 → /blog✗ FAIL — Moved page with no redirect (returns 404)
/moved-page-missing-redirect → 404 (no rule)This URL was moved but has no entry in _redirects, so it returns a raw 404 — flagged by check #220.
Check #47 — Internal Linking Strategy
IMPORTANTThe script counts internal links per page and validates that link anchor text is keyword-relevant. Pages with fewer than 3 contextual internal links or no links to sibling content are flagged.
✓ PASS — Keyword-rich internal links covering core content
Check #14 — Crawl Depth
IMPORTANTThe script measures the shortest path from the homepage to each page. Pages requiring more than 3 clicks are flagged. This section links into a deliberate depth chain — the depth-4 page exceeds the 3-click limit.
| URL | Depth | Status |
|---|---|---|
| / | 0 | ✓ Root |
| /crawl-test | 1 | ✓ PASS |
| /crawl-test/depth-2 | 2 | ✓ PASS |
| /crawl-test/depth-2/depth-3 | 3 | ✓ PASS (boundary — linked from depth-2) |
| /crawl-test/depth-2/depth-3/depth-4 | 4 | ✗ FAIL — exceeds 3-click limit |
Broken Image Test
The script requests every <img src> and records the HTTP status. A non-200 response is flagged as a broken image.
✗ FAIL — Image src returns 404
✓ PASS — Image src returns 200
Script Check Summary
| Check # | Signal | FAIL scenario on this page | PASS scenario on this page |
|---|---|---|---|
| #9 | Broken internal links | /this-page-does-not-exist-404 | /about, /blog, /contact |
| #163 | Anchor text quality | "Click here", "Read more", "here" | Descriptive keyword phrases |
| #8 | Redirects correct | /moved-page-missing-redirect → 404 | /old-pricing → 301 → /contact |
| #220 | Moved pages → 301 | /moved-page-missing-redirect → 404 | /old-features → 301 → / |
| #47 | Internal linking | — | 5 keyword-rich internal links |
| #14 | Crawl depth ≤ 3 | /crawl-test/…/depth-4 (depth 4) | /crawl-test/depth-2/depth-3 (depth 3) |
| — | Broken images | /images/broken-hero-image.jpg → 404 | /placeholder.jpg → 200 |