Say your agent checks your online store's numbers every morning. One day the report reads: "Yesterday, visits with an unknown source jumped and the engagement rate fell below half its usual level. The tracking tag may be broken. I need more access to check the deploy history and the analytics change log." It is a diligent report. Two days later you open the same date again and the odd numbers have quietly returned to normal. Nothing was ever broken.
What should an agent ask first when a number looks wrong?
Recent numbers are not evidence of a failure; they are drafts that have not settled yet. Before an agent goes looking for a break, it should ask when the number settles. Most measurement tools do not finalize data the moment it arrives. They attach late events, reattribute sources and remove duplicates, and only then does the value stop moving. A value seen before that is not wrong. It is unfinished.
Why does an unsettled number look like a failure?
Because an unsettled number goes wrong in exactly the way a broken setup does.
- Sources look empty. Visits that have not been attributed yet show up as "(not set)" or "unknown", just as they would with a broken tag.
- Ratios collapse. If the denominator (visits) arrives first and the numerator (engagement, conversions) is filled in later, the ratio briefly hits the floor.
- Yesterday always looks worse than the day before. The most recent date is the least settled one, so a daily report shows a "drop yesterday" almost every day.
Google Analytics states in its data freshness documentation that processing takes time and that report values can change in the meantime. It is not only analytics: ad conversion attribution, payment refunds and cancellations, and search performance reports all settle late. How late depends on the tool, so check each tool's own documentation.
Where does an agent go if it does not know this?
Somewhere wrong, and diligently. Once it assumes a failure, the next question is "what changed?" So it goes to the deploy history, the settings change log and the tags on the live page, and when it cannot reach them it records that it was blocked by missing access.
This happened to us. Our operations agent read same-day numbers as a tag or deploy failure twice in a row, went after the change log and deploy history, got blocked, and logged "no tool". Once that access was opened and the trail followed to the end, there had been no deploy and no settings change in between, and for all three odd dates (across two services) the "(not set)" share had moved back to its real sources when checked a day or two later. What was missing was not a tool. It was one piece of knowledge: when does this number settle? More access would not have held the answer.
The cost comes in two layers. One is wasted effort. The other is larger: a misdiagnosed block turns into a request for more permissions. Try to fix a knowledge gap with access and you widen access while the same problem shows up again the next day.
What should go into the agent's instructions?
Add this to the instructions of any agent that reads metrics. It is ready to paste.
When a recent number looks wrong, before looking for a failure:
1. Check when that data settles (the tool's documentation, or the table in this document).
2. Report anomalies on unsettled dates as "provisional" and recheck on the date they settle.
3. Investigate deploys, settings and tags only if the anomaly remains after the data has settled.
4. If you were blocked by unsettled numbers, record it as "wait", not as "missing access".
Keep a settling-time table next to those instructions. Writing it down once is cheaper than having the agent look it up every time.
- Web analytics: treat dates from a few days ago onward as final
- Ad conversions: after the attribution window closes
- Revenue: after the refund and cancellation window passes
- Search performance: up to the date the tool marks as final
How do you tell a real failure apart?
Waiting does not mean doing nothing. If any one of these holds, look right away instead of waiting.
- The anomaly is also on settled dates. If three days ago and a week ago look wrong too, it is not lag.
- It is zero. Unsettled numbers look smaller; they do not vanish. If visits drop to zero, collection has stopped.
- Something changed at the same moment. If a deploy or settings change lines up with when the anomaly began, check that before blaming lag.
If none of the three holds, the anomaly can wait until it is still there two days later.
PreviousTaking content down means acting on its list of copies, not the original