Crawl Foundry
Competitive Research

Review the competitor catalog, gaps, pages, and keyword handoffs

Competitor evidence becomes useful when a person can recover why a domain appeared, which stored rows support the claim, how much of the universe was scanned, what was paid for, and exactly what an import will change.

Bounded gaps

Aggregate and per-domain reads expose row ceilings, truncation, and snapshot coverage.

Explicit states

Active, confirmed, automatic, hidden, and saved-comparison membership remain separate.

Controlled handoff

Paid full analysis and keyword import have independent permissions, limits, costs, and effects.

Start with specific queries and pages

A Deep comparison can expose competitor gap keywords and top pages from stored detail snapshots. Gap rows show the query, competitor rank, available search volume, difficulty, intent, and contributing domain context. Top-page rows show the stored path and estimated organic traffic.

A gap is a lead for review. It does not prove that the query is relevant, winnable, profitable, missing from the site, or suitable for a new page. A top page is provider evidence about estimated visibility, not a content brief or a copy instruction.

The aggregate gap is deliberately bounded

The aggregate reads only already stored competitor Deep gap rows. It scans at most 1,000 rows overall, at most 100 per competitor, and divides the total allowance across the selected competitor references. It returns at most 50 normalized keywords.

For a repeated keyword it counts distinct competitor domains, keeps the best observed rank, the highest observed search volume, the lowest observed difficulty, and the first available intent. Sorting favors competitor count, then volume, then best rank. A truncated flag appears when the output or a per-domain read exceeds its allowance. These rules matter when a row is used as a priority claim.

Open a competitor for its own stored dossier

Each competitor can lazily load paginated gap keywords and top pages from its referenced Deep snapshot. Opening the panel does not buy more data. Search and selection operate on the rows currently returned to the browser.

Bars are scaled against the largest value in the loaded rows, not against the full competitor universe. Continue pagination and read the module coverage before treating a visually dominant row as the global maximum.

The operational universe collects several kinds of evidence

An evidence source explains why the domain exists in the catalog. It does not certify business relevance or strategic priority. One row can carry several sources and observation times.

SourceWhat it can establish
ManualA person named the domain as relevant enough to retain.
Position TrackingThe domain is part of the active tracked comparison set.
Keyword Database SERPThe domain recurred in stored SERP positions for workspace keywords.
Domain Overview or discoveryA stored analysis or discovery flow observed the domain.
ComparisonThe domain was included in a saved paid comparison.

The SERP score is a local prioritization rule

Recomputation reads up to the latest 1,000 stored SERP enrichments and summarizes distinct keywords per domain. The current score weights top-10 appearances three times, adds top-20 appearances, then adds a logarithmic covered-volume term. It excludes the own domain and hidden or denied domains.

Gap count means the competitor is in the top 10 while the tracked own domain is not in the top 10 for the same stored keyword. Covered volume sums the current enrichment volume across covered keywords. The score ranks review candidates; it is not provider authority, market share, or a probability of winning.

Coverage views are samples of the active keyword evidence

Catalog counts and overlap use stored keyword and SERP evidence within explicit read ceilings. The own ranking distribution scans at most 2,000 rank snapshots, keeps the newest row per keyword, and stops at 500 unique keywords. Unknown rows can be filled from the bounded workspace keyword total up to that ceiling.

A displayed top-10, gap, or not-ranking count is therefore scoped to the stored and sampled set. The response exposes sample size, limit, and truncation so a reviewer can tell a complete small workspace from a bounded large one.

Pagination and sorting define what is on screen

The catalog is genuinely paginated. The current default page contains 40 rows and the server allows up to 100. Server indexes provide the initial score or confirmation order. Any additional client sorting applies to the loaded rows, not to every candidate stored in the workspace.

The current workbench excludes hidden rows and has no visible reveal control. Confirmed-only filters to confirmed or manually tracked rows. State the loaded page and filter when a list is used as evidence.

Catalog actions change different records

  • Confirm marks the catalog row and activates or manually pins the domain in the tracked competitor set.
  • Hide marks the catalog row as irrelevant for the current catalog view. It does not automatically remove an already active tracked-domain record.
  • Recompute rebuilds local summaries from stored evidence and can fill free active competitor slots when automatic fill is enabled.
  • Prepare comparison copies the selected current domains into a new setup. It does not change an existing saved comparison.
  • Open detail reads stored keyword and profile evidence; Domain Overview opens the wider domain dossier.

Active, confirmed, hidden, and automatic are not synonyms

Confirming a domain activates it as a manually selected and pinned competitor. Automatic fill uses free tracked slots and marks its selections as automatic rather than pinned. The current default allows automatic fill unless the workspace setting disables it, and the active competitor ceiling is currently 20.

Hiding affects catalog visibility. Removing a tracked competitor is a separate action. A domain can therefore be active while hidden in the catalog, and a saved comparison can still reference a domain that is no longer active. Check each state separately before describing the competitor set.

Competitor detail separates four reads

The overview profile can stitch authority, traffic, and keyword values from different recent stored snapshot modules. That profile may use another market or fetch time than the saved comparison. Read its source and date instead of assuming one unified live scan.

TabCurrent source
OverviewStored competitor summary and already-paid Domain Overview profile modules when available.
AllA merge of paginated SERP-derived rows and stored full-analysis rankings.
SERPPaginated current workspace SERP evidence for the selected domain.
Full analysisThe active generation of a separately paid ranked-keyword analysis.

Keyword rows can join ranking and enrichment clocks

Competitor keyword reads prefer stored rank snapshots where available and otherwise use stored SERP enrichments. A row can join current Keyword Database volume, CPC, difficulty, intent, and own position onto a competitor rank captured at another time.

Pagination is bounded and source-specific. A page can be sparse after domain filtering, and a bounded all-rows read does not promise every historical matching keyword. Carry the ranking source, capture time, current metric time, page state, and truncation into any export or handoff.

Full analysis is a separate paid keyword acquisition

The current depth choices are 100, 500, 1,000, 5,000, and 10,000 ranked keywords, subject to the workspace limit. The server prices task and requested-row cost, reports spendable balance, and requires keyword edit access to start. A fresh completed result at an equal or greater depth can be reused for the current 30-day cache window unless a person explicitly refreshes it.

A new job reserves the quote for 10 minutes, writes a running generation, and must complete before that generation becomes active. The worker stores results in bounded batches. Success settles the measured provider cost under the reservation cap; failure or the current 3-minute stale timeout releases the reservation. Requested depth is a ceiling, not a promised yield, and the result reports found and available totals when known.

Import is explicit, even when opening a new keyword

Selected competitor keywords can be imported into the database alone or into an existing or newly created manual list. Each mutation accepts at most the current 200-row chunk, and the client repeats chunks when a larger loaded selection is approved. Import requires keyword add access; list targets additionally require list-management access.

Opening a keyword that is not yet in the database first imports that one row, then opens it. Active duplicates are skipped as new rows but can still be added to the selected list. Soft-deleted rows are restored. The handoff carries available volume, CPC, difficulty, competition, and intent, but not the competitor rank as a durable Keyword Database metric.

Selection does not trigger the next workflow

Importing or creating a list does not refresh metrics, purchase a SERP, begin Position Tracking, create a comparison, assign a page, generate a brief, draft an article, or publish content. Sending selected gap terms to Keyword Research only prefills a separate research setup within its seed limit.

Use the originating domain, market, source, capture time, and reason for selection as handoff context. Without that record, a useful competitor observation becomes an unexplained keyword in the shared inventory.

Finish with a smaller, justified action set

  • Each retained domain has a stated relationship to the audience or tracked keyword set.
  • Catalog score, active state, confirmation, hiding, and saved comparison membership are not conflated.
  • Gap and page claims include read caps, truncation, market, and snapshot date.
  • Full-analysis depth, cache state, found rows, and settlement are recorded.
  • Imported keywords have a human owner and an intended database or list destination.
  • No tracking, content, or publication action is implied by competitor evidence alone.