Skip to content
Back to Blog

The scans that reported success and stored nothing: how to tell an agent is really working

Our competitor scan recorded 23 successful runs and 440 successful skips. On real brands it had stored no competitor ads at all. What the records showed, why every one of them read as success, how we fixed it, and six questions that tell you whether any agent is really working.

8 min readStrategy

One of the jobs our agents run is a daily competitor scan. Olivia, our creative strategist, reads the ads a competitor is running from Meta's public Ad Library and takes a capture of the competitor's homepage, so the Competitor ads page can show what is running and what changed. On 10 October we read every record that job had ever written. Every one said it had worked.

What the records said

RecordCountMarked as
Scan runs, 10 August to 9 October23Succeeded
Of those, runs whose record says nothing about what they did14Succeeded
Runs skipped because the previous run was still going, 30 September to 1 October440Succeeded
Competitors watched for real brands2Both marked as ads synced, neither with an error
Competitor ads stored for real brands0
Homepage captures stored for real brands0
Read from our production records on 10 October 2026, without changing them. Demo brands are counted separately below.

So the job had a perfect record and, for the brands that were not demos, nothing to show for it. Nobody was alerted, because nothing had failed. The Competitor ads page showed the sync as current, because the record said it was.

Why every record read as success

There were four separate ways a run could do nothing and still be written down as a success, and the records show all four.

  1. A run's success meant the code finished, not that anything was read. Fourteen runs from August and September have an empty message. The record cannot say how many competitors they looked at, what they found or what they stored. It can only say they ended.
  2. A skip was written as a success. On 30 September and 1 October the job was started 440 times while an earlier run was still going. Each start stood down, correctly, and each was recorded as succeeding. A job that did no work 440 times looked healthier than one that worked once.
  3. "Synced" was stamped whether or not anything came back. Until a rebuild on 1 October, the scan kept a summary of each competitor rather than the ads themselves, and it marked the ads as synced whenever the Ad Library returned a result, whatever that result held. One real competitor's last summary says zero ads, with no error. From the record alone you cannot tell whether that competitor ran no ads or whether the answer was empty. That is the finding: the record could not tell the difference.
  4. A count is not a copy. The other real competitor's last summary, from 5 October, counts 21 active ads. None of them is stored. That record is in the older summary shape, so the 21 was a number about the ads rather than the ads, and once the scan was rebuilt to keep each ad, there was nothing behind the number to show.

Meanwhile the demo brands we use for previews held 128 stored competitor ads, 18 homepage captures and 9 recorded changes. The page looked full every time we showed it. It was full of demo data, and none of the counts above would have looked wrong to anybody who only ever saw the demo.

The fix, and where the first fix was wrong

The rebuild on 1 October changed what was stored: each ad is kept one by one with when it was first and last seen, each homepage capture is kept, and a change is only reported when it can be built from two stored readings. That made it possible to see that nothing was stored. It did not stop a run that stored nothing from calling itself a success.

The first fix, on 9 October, named the cause in one line: a run was only marked failed when the Ad Library did not answer and the homepage could not be captured, so a run that got neither stored nothing, wrote no error and stamped the ads as synced. That fix made "stored nothing" a failure.

It was wrong the same day. A competitor that genuinely runs no new ads this week stores nothing new, and calling that a failure would have raised an alert every morning for a correct reading. An alert that fires on correct readings is one people stop reading. The rule that replaced it splits the cases the old record had merged:

What happenedHow the run is recorded now
Read everything, something newSucceeded, with what was stored
Read everything, nothing newSucceeded, and says it read nothing new
Could not read: the Ad Library did not answer, the answer had no list of ads, the homepage was empty or not captured, or the code failedFailed, and raises one alert per competitor until a later scan reads it
Not run because a rule said so, for example the brand is pausedSkipped, and not counted as a success

Each of these is pinned by a test that fails on the old behaviour, and the page shows the sync as current only when something has actually been stored. The counts in this post are from 10 October, the day after the fix landed in our code, so they describe the records the old rules wrote.

Six questions that tell you whether any agent is really working

None of this is specific to competitor scans. Any agent that runs on a schedule, ours or anybody's, can fail the same four ways. These are the questions we now ask of every job.

  1. Count what was stored, not how often it ran. Ask for the number of records the agent has written for your brand. A run count is a statement about a clock.
  2. Read the message, not the status. A run should say what it examined and what it found: so many competitors due, so many read, so many with nothing new, so many failed. A blank success is not information.
  3. Make sure a skip cannot pass as a success. "Did not run" and "ran and found nothing" need different words on the record, or a stalled job looks like a quiet one.
  4. Separate "nothing new" from "could not read". The first is a result. The second is a failure. A tool that shows the same empty state for both cannot tell you when it is blind.
  5. Check that "last synced" only moves when something was read. A timestamp that advances on an empty answer turns every failure into a fresh-looking page.
  6. Look at your own data, not the demo. If every example of the feature working is a demo account, the feature has not yet been shown working on yours.

The same questions apply to changes an agent makes as well as to things it reads. That is why every change in Cresva writes a record before it is attempted and, where the platform allows, reads the account back afterwards, as set out in what agents should be allowed to do in an ad account. A scan that reads nothing is an annoyance. A change that reports success and did nothing, or did something else, is the version that costs money.

See what the agents read and wrote for your brand Every change an agent makes is recorded before it runs, and every scan now says whether it read something, read nothing new, or could not read.

Frequently asked questions

How can I tell whether a scheduled AI agent is actually working?
Count what it has stored for your account rather than how many times it ran, and read what each run says it examined. A run marked successful with no description of what it read, or a skipped run marked as a success, tells you only that the code finished.
What is the difference between "nothing new" and "could not read"?
Nothing new means the source answered and had nothing that had changed, which is a correct result. Could not read means the source did not answer, or answered with nothing usable. They need different records, because only the second is a failure.
Where does Cresva read competitor ads from?
From Meta's public Ad Library, plus one capture of the competitor's homepage. Meta publishes ads in its library only for some regions, so a competitor that runs ads elsewhere can show none there, and the page says so rather than showing zero.

Written by Shubham Raghav, Founder, Cresva

Have a question? Email us