Plan recurring measurements without mistaking due for done

Treat a schedule as a recurring measurement contract

A Position Tracking schedule stores who and what should be measured again: workspace, fixed keyword selection or live list, data kind, cadence, SERP context or keyword-data packages, optional budget cap, status, and next due time. It is the configuration for future claims. It is not a promise that a provider call happened at the due time or that every requested row returned data.

The Schedules view can show list-backed SERP and keyword-data jobs for the selected list, fixed keyword-selection schedules at workspace scope, and a bounded workspace-wide jobs summary. The current default resource and display limit is 30 active or paused schedules per workspace, although the effective plan or safety policy can lower it. Archived schedules are removed from the ordinary lists.

Know whether membership is fixed or resolved at run time

A direct-selection schedule stores a normalized, deduplicated keyword array. It is SERP-only and currently supports up to 200 keywords by default. Its membership changes only through the dedicated selection editor, which cannot save an empty set. A list-backed schedule stores the list ID instead. Every run resolves the list's live membership again, so additions and removals change the next acquisition without rewriting past run counts.

The current per-run ceiling is 500 keywords. Creation and the estimate reject an oversized list, and the dispatcher checks the current list again before every run. A list that grows beyond the effective limit can therefore make a previously valid schedule fail before dispatch. The run records that membership error rather than silently taking the first 500 rows.

• A direct schedule preserves its fixed cohort until a person edits it.
• A full-list schedule follows current membership, so its cost and comparability can change over time.
• A subset chosen in the creation wizard becomes a fixed direct selection rather than a dynamic list slice.
• Run requested counts are immutable evidence of the membership resolved for that claim.

Match cadence to the data kind and review decision

SERP schedules support daily, weekly, every two weeks, or monthly acquisition. Keyword-data schedules support monthly or quarterly refreshes and require at least one package. The form locks the data kind during editing; changing a SERP schedule into a keyword-data schedule, or the reverse, requires a separate schedule. Choose a cadence because someone will review and use the resulting evidence, not because a shorter interval looks more complete.

The monthly projection uses an average frequency: 365 divided by 12 for daily, 52 divided by 12 for weekly, 26 divided by 12 for every two weeks, one for monthly, and one third for quarterly. Actual calendar months contain different numbers of days and weekly occurrences. Monthly and quarterly due dates use UTC calendar months and clamp impossible dates, such as moving a January 31 schedule to the last day of February.

Keep depth, device, location, and language attached

A SERP schedule requires Top 10 or Top 100 depth, a positive location code, a valid language code, and at least one of desktop or mobile. Selecting both devices creates one logical run group with two child runs, two reservations, and two context-specific result sets. The schedule row can aggregate their requested and fetched counts, while the run log keeps the child rows visible. A doubled requested total can therefore mean two devices, not duplicate keywords inside one request.

Top 10 cannot prove whether an absent domain ranks at position 11 or 100. Top 100 buys a deeper observation and changes the price. Location and language describe the acquisition request; they do not translate keywords or make different markets interchangeable. Device, depth, location, language, and membership must stay stable if the timeline is meant to support a before-and-after comparison.

Understand what can be edited and what counts as a duplicate

For a list-backed schedule, the duplicate guard compares list, data kind, cadence, and, for SERP tracking, the device set, location, and language. Depth, package choice, and budget cap are not part of that key. Two live schedules on the same list cannot therefore differ only by Top 10 versus Top 100, or only by keyword-data packages, while keeping the same duplicate key. Archived rows do not collide with a new schedule.

Editing can change cadence, SERP depth, context, devices, keyword-data packages, and budget cap. The data kind and list binding stay fixed. A direct-selection schedule can change its keywords through the separate editor, within current per-keyword, total-character, selection, and run limits. Changing cadence recalculates the next due time from the edit; changing other fields does not rewrite completed runs or their requested counts.

• Edit changes future claims, not historical snapshots or settled run rows.
• Duplicate for mobile prefills a new mobile SERP schedule; it does not mutate the source row.
• A duplicate warning is an admission rule, not evidence that two reports would have identical histories.
• If a context must change without breaking comparability, finish the old series and create a clearly named new contract.

Read the preview as an admission estimate

The setup card prices the membership known at that moment. It shows estimated cost per run, average runs per month, resolved keyword count, a displayed per-keyword average, estimated monthly cost, current spendable organization balance, and the balance after one estimated run. Spendable balance already subtracts active reservations. The per-keyword line divides and rounds the run estimate for display; it is not a provider rate card.

SERP pricing responds to keyword count, Top 10 or Top 100 depth, queue tier, and device count. Keyword-data pricing responds to count and selected packages. The monthly estimate multiplies the standard per-run quote by average runs per month and rounds up to cents. It does not include ad hoc priority runs, future list growth, later pricing changes, retries that lead to a new paid claim, or other product spending from the same organization balance.

Separate the schedule cap from organization balance

The optional monthly cap belongs to one schedule and currently has a minimum of EUR 5. Unlimited means that this per-schedule cap is absent. It never means unlimited organization balance. Before a claim, Crawl Foundry totals this schedule's settled run costs and still-reserved estimates since the start of the current UTC month. Manual, recurring, standard, priority, and per-device child runs all use the same schedule ledger and can consume the cap.

If the next complete run group would exceed the cap, nothing is dispatched. Balance admission then reserves the full estimate for every device before worker execution. If a later device reservation cannot be made, earlier reservations and provisional run rows from that claim are rolled back. This prevents a two-device schedule from starting only one device because the balance ran out during admission.

Budget-cap and insufficient-balance outcomes both appear under the `skipped_balance` run state, with different error codes. Each increments the schedule's consecutive blocked count. After two consecutive cases, the schedule moves to `paused_balance` and the owner can receive a pause email. A successful or partial completed group resets the failure counter. Raising a cap or adding balance does not itself resume a paused schedule.

A manual run is a paid claim, not a harmless refresh

Run now asks for the same standard queue used by recurring SERP work. A SERP schedule can also offer Measure fast, which uses a separately priced priority queue for that run only. Keyword-data schedules have no priority option. Either choice resolves current membership, checks the monthly cap, reserves balance, writes dispatching run rows, and then sends worker events. Opening the schedule or report does none of those things.

An active manual claim resets the next cadence anchor from the manual start time. It is not an extra observation inserted without affecting the calendar. Run now can also start nothing when the schedule is paused or archived, another run is still live, balance is insufficient, or the cap is reached. The schedule-list action reports these skip reasons and opens the run log; an HTTP 200 alone is not proof that a worker started.

Creating a schedule stores the row and then calls the same run-now route for the first baseline. Schedule creation can succeed while that immediate request is skipped or fails. The creation flow currently checks the HTTP response but does not use the returned fired count as a separate completion gate. Confirm the first ledger row and terminal coverage before treating the tracker as initialized.

Pause, resume, and end have different timeline effects

Pause changes the schedule to `paused_user`. It also closes its current in-flight child runs as failed with a cancellation code and releases their still-active reservations. Resume returns the schedule to active, clears consecutive failures, and makes it due immediately. The hourly dispatcher can then claim it, or a person can use Run now. Resume does not reconstruct the cancelled run or fill the gap it left.

End archives the schedule, stamps its deletion time, closes current in-flight runs, releases their reservations, and removes the row from ordinary active and paused lists. It is not a temporary pause and the interface offers no resume action for it. Historical run and billing evidence are not rewritten into success by archiving, and a new schedule can later reuse the previously archived configuration without a duplicate collision.

Treat the displayed next time as a due boundary

The recurring dispatcher currently sweeps hourly at minute 20 UTC and claims a bounded batch of due active schedules. A row whose due time has passed can therefore say it will run soon while waiting for the next sweep. Once claimed, the recurring next time is calculated from the prior scheduled boundary, which limits drift from provider latency. A manual run or resume deliberately establishes a new anchor.

If dispatch never reaches the worker, the run becomes `skipped_dispatch`, its reservation is released, and the schedule uses a one-hour retry backoff for the first current retries before returning to its normal cadence. The schedule row labels this as retrying instead of presenting the backoff timestamp as a healthy next run. A feature gate, invalid current membership, tenancy failure, bounded dispatcher backlog, or worker outage can all prevent a due configuration from producing measurements at the expected moment.

Read requested, fetched, no-data, pending, and missing separately

Every child run begins as dispatching and becomes running when progress arrives. While it is live, requested minus fetched minus no-data is pending. Only after a terminal state does the remainder become missing. The interface keeps those states separate so a fresh run with zero returned rows does not look failed before the worker has finished.

Succeeded means the run completed under its terminal contract. Partial means some evidence or explicit no-data returned but full success did not. Failed means no usable completion, unless the read model repairs a failed row with positive fetched or no-data counters to partial for display. `skipped_balance` means budget or balance admission blocked work. `skipped_dispatch` means the claim existed but dispatch failed. No-data is an explicit provider outcome, while missing is the unaccounted terminal remainder.

Use the run ledger as proof of execution and cost

The per-schedule run log is paginated newest first and currently uses a policy page size of up to 50 rows by default. Its health chart summarizes a bounded recent set, currently 12 rows. Each ledger row carries state, start time, requested, fetched, no-data, missing where terminal, support code where available, and settled cost when present. A multi-device claim produces one row per device under one group, while the live progress band and schedule summary can aggregate the group.

The worker settles each reservation against actual billable work and releases unused reservation. A zero settled cost can accompany an all-no-data run; it does not make the schedule or future acquisitions free. A partial run can still have positive cost. Stale in-flight rows and stranded reservations have reconciliation and recovery paths, but the visible terminal status, counters, error code, cost, and Activity record remain the evidence to review before retrying.

Review a schedule before trusting its timeline