Building an Automated Tool for Internal Links

Internal linking is one of the highest-leverage, most-neglected SEO levers — it costs nothing to implement, directly helps search engines discover and understand your site structure, and keeps visitors engaged across more pages. Automating the audit process makes it practical to maintain at scale.

Why Internal Linking Matters

• Crawl discovery — search engines find new pages partly by following links from already-indexed pages; an orphaned page with zero internal links is genuinely harder to discover and index.
• Topical authority signals — linking related content together signals to search engines which pages belong to the same topic cluster, reinforcing relevance for target queries.
• Pages-per-session — relevant internal links keep visitors reading additional content instead of leaving after one page, directly benefiting engagement metrics and (for ad-supported sites) ad impressions per visit.

A Simple Python Tool for Auditing Internal Links

import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin, urlparse

def find_orphan_and_thin_link_pages(sitemap_urls, domain):
    all_urls = set(sitemap_urls)
    linked_urls = set()

    for url in sitemap_urls:
        try:
            resp = requests.get(url, timeout=10)
            soup = BeautifulSoup(resp.content, 'html.parser')
            for link in soup.find_all('a', href=True):
                full_url = urljoin(url, link['href'])
                if urlparse(full_url).netloc == domain:
                    linked_urls.add(full_url.rstrip('/'))
        except Exception:
            continue

    orphans = all_urls - linked_urls
    return orphans

This pulls every internal link found across your site’s pages and compares it against your full URL list (from a sitemap) — any URL never linked to from elsewhere on the site is a genuine orphan page worth addressing, either by adding relevant internal links to it or reconsidering whether it should exist at all.

Practical Internal Linking Strategy

• Link from high-authority pages to newer or underperforming ones — your best-performing content passing link value to pages that need it is more effective than linking uniformly regardless of a page’s own authority.
• Use descriptive anchor text — a link reading “click here” conveys nothing about the destination; anchor text matching the target page’s actual topic helps both users and search engines.
• Build genuine topic clusters — a pillar page linking out to several detailed sub-topic posts, which link back to the pillar, reinforces topical relevance far more than scattered, unstructured linking.

Frequently Asked Questions

How many internal links should a typical blog post have?
There’s no universal fixed number — the right count is however many genuinely relevant related pages exist, added naturally within the content, not forced to hit an arbitrary target.

Conclusion

Internal linking is a free, high-leverage SEO lever that’s easy to neglect at scale — a simple crawl script that identifies orphaned pages turns a manual audit into a repeatable process, making it practical to maintain good internal linking structure as a site grows.