LINK TO IDEAS.
NOT JUST FEEDS.

Primary description of web-crawl datasets, formats, and coverage; inclusion is not comprehensive.

Explore open WARC, WAT, WET datasets and publicly available crawl snapshots.

Peer-reviewed study showing how dataset filters can disproportionately remove minority-related text.

Official guide separating useful publishing from unproven GEO tricks; Google does not use llms.txt for search.

What is and is not required for appearance in AI Overviews and AI Mode.

Check whether a URL appears in specific open crawl indices; not proof of model training.

Official description of GPTBot, OAI-SearchBot and ChatGPT-User; separates possible training crawls from search discovery and user requests.

Primary documentation for looking up a URL in specific open crawls; an index match cannot establish training in a closed model.

Start an independent personal website and build connections through human links.

Publish on your own site first, syndicate links elsewhere.

A real organization explaining a site-first publishing strategy.

Semantic metadata vocabulary for descriptive article pages; not an AI-inclusion switch.

Standard format for listing site URLs to crawlers.

Accessibility standard helping more humans read and participate.

Community guidance for curating useful links to independent writers and allowing readers to traverse an intentionally recommended web.

Original standards-track specification for portable subscriptions that deliver publishing updates to different feed readers.

Learn about operating and participating in sustained, moderated discussions.

Standard for link notifications between independent websites.

Federated social protocol for cross-server participation.

Keep a durable personal home for your public writing and references.

Voluntary framework for assessing reliability, transparency, fairness and risk.

International normative framework discussing inclusion, human rights and diverse expression.

Documented eligibility and limitations, in contrast with folklore about hidden algorithm parameters.

Actual crawler-access protocol; not a mechanism for requiring data inclusion.

Official published repository describing retrieval, ranking and visibility enforcement; code versions matter, and old engagement weights should not be treated as permanent.

Observed U.S. search behavior in 2025: 8% clicked conventional results with AI summary versus 15% without; not a causal experiment.

2024 measurement of web-page and link disappearance, with methodology.

Practical model for backups, integrity checks and archive governance.

How to request page captures; capture is not universal or guaranteed.

Public source-code preservation and persistent references.

How scholarly registration maintains lasting work identifiers; membership required.

Foundational research on findable, accessible, interoperable, reusable digital research objects.

Documented disparate effects from blocklist filtering in one prominent dataset.

Human-rights, culture and language considerations for AI governance.

Find scholarly studies and trace original research rather than reposted summaries.

Improve participation for people with different access needs.

Original URL on your site plus cited copies and links in relevant channels.

Notify sites about genuine references to them; implement with abuse protections.

Open federation for distribution beyond a single platform.

Practical example of a hub-and-spoke content distribution strategy.

Inspect broken links, fragments and redirects so readers can keep following citations.

Read the tool documentation before running comprehensive local or site-wide link audits.

Official introduction to accessible site reviews, combining practical tools and human checks.

Find maintained specifications for linking independent sites through feeds, identity and mentions.