Crawl Foundry
Competitive Research

Create and read one comparable domain claim

A credible comparison begins before the charts. Domains, market, depth, reuse window, quote, request identity, snapshot completion, and module coverage must describe the same question.

Comparable inputs

Normalize domains and hold country, language, and depth constant across every reference.

Priced reuse

The quote names which snapshot modules are reused and which new work reserves balance.

Completion proof

Aggregate status and module coverage distinguish admission from usable evidence.

Set up one comparable claim

1
Use the active tracked own domain or enter the intended own domain explicitly.
2
Add 1 to 20 normalized competitor domains that differ from the own domain.
3
Choose the country and language that define the search market.
4
Select Summary for headline evidence or Deep for stored keyword, gap, page, and link modules.
5
Inspect every quote line, its reuse state, and the remaining spendable balance.
6
Start once, keep the request identity stable during retries, and wait for terminal snapshot evidence.

Domains and market travel together

Crawl Foundry normalizes and deduplicates domains, rejects the own domain from the competitor set, and limits each comparison to the current domain ceiling. The country and language apply to all referenced snapshots so the domains are read in one search context.

A competitor that matters in German search results in Germany may not be the same competitor in English results in the United States. Changing the market changes the claim and can prevent snapshot reuse even when the domain names stay the same.

Summary and Deep answer different levels of question

A Deep competitor snapshot stores the own domain as its directed gap target. The own Deep snapshot has no such target. A Deep snapshot for the same competitor against another own domain is therefore not an equivalent reusable gap claim.

DepthEvidence requested
SummaryBase snapshots for authority, organic and paid estimates, keyword totals, backlinks, referring domains, ranking buckets, movement, links, and monthly history.
DeepThe base evidence plus detail snapshots for ranked keywords, competitors, top pages, directed keyword gaps, anchors, referring domains, and indexed pages.

Reuse depends on context, freshness, and depth

The reuse search checks workspace, normalized domain, country, language, depth, deletion state, and, for a competitor Deep snapshot, the same gap target. It inspects a bounded set of recent candidates rather than every historical snapshot.

Current defaults allow 7-day reuse for matching competitor snapshots, 30-day reuse for the own Summary snapshot, and 30-minute reuse for compatible running work. Reuse avoids another provider charge, but it also means the compared modules can carry different fetch times. The quote and coverage ledger expose that distinction.

The server quote charges only missing work

Each domain and module appears as a reused or new line. Reused evidence contributes zero to the estimate. Every newly required snapshot or module receives its own reservation, and the estimated total is the sum of those new lines.

Spendable balance is the organization balance after active reservations. The start action should remain unavailable until the current inputs have a current server estimate and the balance can cover it. A displayed quote is still an estimate, not proof that a worker completed or that every module returned data.

Admission is atomic and retry-safe when identity stays stable

The server reprices the request, checks permissions, feature access, domain limits, comparison capacity, active-run capacity, and spendable balance, then creates the comparison, snapshot references, and reservations together.

A stable UUID request key and fingerprint make an identical client retry idempotent. Reusing that key with different domains, market, depth, or plan is rejected. This protects against duplicate paid claims without allowing a key to stand for changing work.

Accepted is not the same as acquired

In production, every newly minted snapshot is dispatched through the durable outbox. A successful comparison response proves that the request and reservations were admitted. It does not prove that a worker claimed the outbox item or that the provider returned usable rows.

The snapshot records and module coverage provide the later evidence. A dispatch failure can fail work that was not delivered and release it through the Domain Overview failure path. Running, partial, failed, and completed states must therefore be read with the individual snapshot and module statuses.

Depth determines which statuses count

Summary comparisons derive their state from base snapshots. Deep comparisons derive it from both base and detail snapshots. Any running reference keeps the comparison running. All failed references produce failed. Any mix containing failure or partial evidence produces partial; only the remaining fully completed set produces completed.

Missing snapshot records count as failed. A completed comparison can still contain a module marked truncated or unsupported, because aggregate snapshot completion and module-level coverage answer different questions.

Read the head-to-head ledger as provider estimates

These are directional measurements under a shared context, not audited analytics, revenue, or causal explanations. Missing values remain unknown. A domain can lead on links and trail on traffic, or lead on keyword count while ranking for a less relevant mix.

MetricCurrent meaning
AuthorityThe provider backlink-rank value stored on the base snapshot.
Organic trafficEstimated organic traffic, stored as organic ETV.
Organic keywordsThe provider estimate of ranking keyword count.
Backlinks and referring domainsStored link totals from the base snapshot.
Paid trafficEstimated paid traffic, stored as paid ETV.
Traffic valueEstimated paid cost of the organic traffic.

A ratio badge shows scale, not significance

The current badge compares a competitor value with the own-domain value as a ratio. It hides when the value is missing, non-finite, equal, or when the own value is zero. An upward marker means the competitor ratio is above one; a downward marker means the own domain leads.

It is not a percentage change over time, a confidence interval, or proof that the larger value caused a ranking difference. Use the raw values, dates, and coverage alongside the badge.

The coverage matrix is part of the result

For each domain and module, the matrix can report available, pending, truncated, failed, unavailable, or not requested. It can also show loaded rows, known total, provider limit, freshness, and support code.

Summary normally covers rank overview, history, and backlink summary. Deep adds ranked keywords, competitors, top pages, keyword gap, anchors, referring domains, and indexed pages. Do not infer a missing Deep module from a healthy base metric.

Retry and Deep preparation create new claims

Retrying missing domains opens setup with the previous competitor set and depth. Preparing Deep from a Summary comparison opens setup at the deeper level. Neither control patches the old comparison in place.

The new request receives a current quote and can reuse still-matching snapshots. This preserves the older comparison as dated evidence and makes any new charge, context, status, and settlement independently reviewable.

Check the comparison before using it

  • The own domain and every competitor were normalized as intended.
  • Country and language match the audience behind the decision.
  • Summary or Deep is proportionate to the question.
  • Reused and new quote lines are understood, including their fetch times.
  • The comparison reached a terminal state and partial modules are named.
  • Head-to-head conclusions use raw values and coverage, not only ratio badges.