Skip to content
Original study

Found Is Not Cited: ChatGPT Source Telemetry Compared

The median prompt-level answer recorded 20 found hosts but five explicitly linked hosts; only 23.8% of found hosts were linked in the primary summary.

8 min read

A source recorded as found for a ChatGPT answer is not automatically an explicit citation. In 522 eligible responses with both telemetry objects present, the median across prompts of each prompt's median share of found hosts that were explicitly linked was 23.8%. The typical prompt-level median contained 20 found hosts per response and five explicitly linked hosts.

The result is conditional on answers that recorded at least one host in both detected_urls and the explicit cite_map. A missing citation map was treated as missing attribution telemetry, not as zero citations. This distinction is essential: dashboards that label every detected URL “cited” can overstate observable attribution.

The main comparison

Prompt-level summaryMedianInterquartile range
Found hosts per eligible response2016–22
Explicitly linked hosts per eligible response54–5
Share of found hosts explicitly linked23.8%20.0%–26.2%

For each prompt, the study took the median response value, then summarized the 64 prompt medians. This prevents prompts with more eligible answers from defining the result. Across the 522 response-level pairs, the median Jaccard similarity between found and explicitly linked sets was also 23.8%, with an interquartile range of 18.8% to 28.6%.

After canonicalization, none of the eligible responses contained a host in the citation map that was absent from its detected set. That consistency check supports the interpretation of explicit citations as a subset of the broader recorded-source object for this integration and frozen period.

Two source fields, two questions

detected_urls supplied the broad set of recovered or recorded source hosts associated with the answer. response_json.cite_map[].u supplied hosts explicitly connected to a response fragment in the stored structured output. Both sets were converted to canonical hosts and deduplicated within each response.

The broad field helps answer “which source hosts did the integration record around this answer?” The explicit field helps answer “which recorded hosts were linked to a response fragment in the available attribution structure?” Neither field by itself proves what every user saw in a particular ChatGPT interface.

Using one label for both creates a measurement error. A team may celebrate an increase in “citations” when only the broader discovery field grew. Or it may classify a missing citation map as a zero and report a decline caused by telemetry coverage. A robust schema stores the two objects separately and tracks field availability as its own quality metric.

How the cohort was selected

The snapshot was frozen at 18:00 UTC on 21 July 2026. After excluding demo accounts, incomplete executions, empty responses and errors, 1,043 valid ChatGPT responses contained at least one detected source host. Of those, 554 also contained at least one explicit citation-map host.

The primary cohort then required at least five observations with both non-empty sets for a prompt. That produced 522 responses across 64 prompts, ten projects and seven accounts. Eligible observations ran from 13 July at 07:45:45 UTC through 21 July at 15:50:03 UTC. Every public coverage and privacy gate was met.

This filtering means the 23.8% value is not an unconditional rate across all 1,043 source-bearing responses. It describes the distribution of prompt medians where both fields were measurable often enough. Extending it to missing-map responses would require an assumption the data do not support.

How the linked share was calculated

Within one eligible response, let F be the set of normalized found hosts and C the set of normalized explicitly linked hosts. The linked share is |F ∩ C| / |F|. Because all citation-map hosts were inside the found set after normalization, the numerator equalled the size of C in this cohort.

Imagine an answer with 20 distinct detected hosts and five explicitly linked hosts. The linked share is 5 divided by 20, or 25%. It does not say that the other 15 hosts were irrelevant, that the five were displayed identically, or that any host caused the wording of the answer. It only classifies their recorded relationship.

The analysis calculated this value for every eligible response, took the median inside each prompt, and then reported the median and quartiles across prompts. The hierarchy keeps the prompt inventory as the unit of interpretation and reduces distortion from uneven execution counts.

Why absent attribution telemetry is not zero

A numerical zero means the source field was observed and contained no matching items. Missing means the necessary field was not available. Conflating them changes denominators and can create false comparisons between engines, periods or integration versions.

In this cohort, 1,043 responses had detected sources but only 554 had a non-empty explicit citation map before the five-per-prompt rule. If all other responses were assigned zero citations, the resulting rate would combine source behaviour with citation-map coverage. A release that changed structured telemetry could move the metric even if user-facing attribution stayed constant.

Every reporting layer should therefore expose at least three counts: valid answers, answers with the broad source object and answers with explicit attribution telemetry. The AI visibility metrics framework applies the same principle to mentions, failures and traffic.

Practical consequences for source monitoring

First, choose the metric that matches the decision. If the team wants to audit domains visibly attributed to answer claims, use an explicit citation field and describe its interface limitations. If it wants to study the wider retrieval or detected-source pool, use the broad field and avoid calling every host a citation.

Second, preserve page URLs as well as normalized hosts when privacy and storage rules permit. Host-level analysis is stable enough for aggregate overlap, but it collapses different pages on one domain. A page can be detected while another page from the same host is linked.

Third, repeat the observation. The five-host prompt median is not a fixed allocation. Source sets can vary between answers, and a single execution cannot determine recurrence. Pair source monitoring with a stable prompt inventory and the workflow for tracking brand mentions.

Fourth, audit examples under controlled access. Aggregate rates can locate prompts with unusually low or high linked shares. A reviewer can then inspect authorized records to determine whether the difference reflects answer format, provider telemetry, duplicate handling or genuinely different attribution.

A clean data model

  • answer_status: valid, empty, error or unavailable.
  • detected_source_urls: the raw broader URLs with collection provenance.
  • explicit_citation_urls: URLs connected to answer fragments by the provider structure.
  • normalized_host: a derived value with documented prefix and wrapper rules.
  • telemetry_available: a boolean or state for every source field.
  • engine and surface: the product context in which each observation was collected.
  • prompt and execution keys: the information needed for repeated and paired analysis.

Keeping raw and derived fields makes future definitions reproducible. If host normalization changes, the team can recompute without losing original URLs. If a provider changes citation mapping, a data-quality break can be annotated rather than silently rewriting history.

What the study cannot establish

The recorded citation map does not guarantee that every linked host was displayed identically in every interface. Product surfaces, rendering and user context may differ from the stored structure. The study did not inspect screenshots or individual user sessions.

Host overlap does not measure causal use. A source can be retrieved or linked without being decisive for a statement, and an answer can reflect information learned elsewhere. The analysis cannot rank source quality, accuracy, authority, traffic potential or influence on model generation.

The cohort is conditional and not market representative. It uses eligible Bee LLM production prompts, projects and accounts through a fixed cutoff. Other languages, topics, ChatGPT versions, access modes and collection integrations can yield different ratios.

Finally, medians summarize a distribution. Prompt-level linked shares ranged from 4.6% to 55.6%. The 23.8% headline is useful for the cohort but should not replace the per-prompt value when a team decides what to investigate.

Reproducibility and evidence integrity

The extraction used a repeatable-read, read-only transaction. The aggregate contains denominators, quartiles and controls without exposing prompts, accounts, projects or hosts. Its SHA-256 is 232565ebb6d5246fd0a9d5b1d42fbceeee2264f024af859c83b6371c329641db, and the accompanying methodology, CSV and analysis wrapper are separately checksummed.

The defensible conclusion is not that ChatGPT “uses only five sources.” It is that, within 522 eligible answers where both fields existed, the typical prompt-level median recorded a much broader found-host set than the explicitly linked subset. Found and cited telemetry answer different questions, and an honest visibility system must keep them apart.

Frequently asked questions

Is every URL detected for a ChatGPT answer a citation?

No. In this study, detected_urls is the broader recorded-source set and cite_map contains hosts explicitly linked to response fragments. The two fields must remain separate in collection and reporting.

Does an empty citation map mean ChatGPT cited nothing?

Not in this dataset. An empty or missing cite_map is absent attribution telemetry for this analysis, so those responses were not assigned zero citations and were excluded from the conditional overlap calculation.

Does the 23.8% result apply to all ChatGPT answers?

No. It is the median across 64 eligible prompt medians in a frozen Bee LLM cohort of responses with both source fields present. Other prompts, integrations, periods and interfaces can differ.