ΕΛΛΗΝΙΚΟ ΜΕΤΡΟ Α.Ε. – ELLINIKO METRO S.A.

ELLINIKO METRO
visual_id_ESPA
Menu
7

Screaming Frog Seo Spider 184 Neverb Link __hot__ File

Screaming Frog Seo Spider 184 Neverb Link __hot__ File

It seems you're asking about the Screaming Frog SEO Spider (version 184) and a specific issue or report related to "neverb link" — likely a typo for "no follow", "never linked", or a custom filter.

Based on common SEO Spider usage, here’s how to generate a complete report for detecting "never linked" internal URLs or no-followed internal links in version 18.4 (build 184). screaming frog seo spider 184 neverb link

Blocked links (never accessible)

  • Response codes – 403, 401, 404, 5xx
  • Robots.txt blocked – flagged in the “Blocked by Robots.txt” column
  • Canonicalized links – shown as redirected

So if you want to find links that search engines will never crawl/trust, Screaming Frog provides that clearly. It seems you're asking about the Screaming Frog

Common Types of Never-Blocked Links:

  1. Direct href links on pages not blocked by robots.txt.
  2. Links within JavaScript-rendered content (Screaming Frog can render these).
  3. Links from HTTPS to HTTP (still followed, despite browser warnings).
  4. Redirect chain links where the final destination is disallowed, but the chain is not.
  5. Links embedded in XML sitemaps (not blocked by user-agent rules).

1. Screaming Frog SEO Spider 18.4 – Overview

Screaming Frog SEO Spider is a desktop-based website crawler for technical SEO audits. Version 18.4 (released late 2024/early 2025) focuses on: Response codes – 403, 401, 404, 5xx Robots

  • JavaScript rendering improvements (better handling of modern frameworks)
  • Core Web Vitals integration (directly from CrUX or Lighthouse)
  • AI-powered insights (optional OpenAI API integration for content analysis)
  • Link extraction from images, iframes, and JS-generated DOMs

Technique A: Using Screaming Frog’s robots.txt Tester

Never-blocked links often arise because of typos in robots.txt. To check:

  1. Download the robots.txt of your site.
  2. In Screaming Frog, go to Tools > Robots.txt Tester.
  3. Enter a sample URL from your 184 list.
  4. If the tester says "Allowed" but you intended "Disallowed", that URL is a never-blocked link.