Once the decision to buy rather than build was made, the implementation looked like data entry: an inventory of seventy-five URL profiles across roughly twenty sites — sixty-one of them in scope once the optional properties are excluded — each needing a synthetic check, plus alert rules and dashboards. Days of clicking through a web console.
It became about a hundred and sixty lines of Terraform instead, and along the way two vendor constraints turned an obvious design into a better one.
TL;DR: Grafana Cloud caps browser checks at ten per tenant — a number that isn’t on the pricing page and arrives as a 403. Since billing is per probe location and per started minute, not per page, one check can walk several sites for the same money, so the configuration groups sites into checks generated from an inventory YAML. The cap forced the design that was cheaper anyway: about €61/month grouped, against €170 for one check per site. The sting in the tail is that the labels you carefully assign to a check never appear on the browser metrics — which makes the check naming scheme the only axis you can filter web vitals by.
Why Generate the Checks Instead of Clicking Them?
Because clicking doesn’t survive contact with a changing estate, and three reasons make that concrete:
- It’s the exit protection. The build-versus-buy analysis listed vendor lock-in under harder. Test definitions in the repository mean a vendor change costs the history, not the definitions.
- The estate grows. New sites land regularly. An inventory entry plus
terraform applyis a minute; the console equivalent is an afternoon. - The alert rules are not trivial. Majority-of-three-locations, two consecutive runs, plus an all-sites-simultaneously rule. Clicking that three times identically goes wrong. Writing it once and rolling it out doesn’t.
The honest cost line: the generator is itself a day of work — about a quarter of what the self-hosted option would have spent on initial setup alone. The difference is that it’s reusable and clicking isn’t.
The provider covers exactly the four objects needed:
| Object | Terraform resource |
|---|---|
| Synthetic check per URL profile | grafana_synthetic_monitoring_check |
| Probe locations | grafana_synthetic_monitoring_probes (data source for the IDs) |
| Alert rules | grafana_rule_group |
| Dashboards | grafana_dashboard |
Why Does One Check Cover Four Sites?
Because of a cap that isn’t in the pricing table. The first terraform apply with one check per site ended like this:
403 {"msg":"browser checks quota exceeded (current: 12, max: 10)"}
Ten, per tenant. The number comes from the Synthetic Monitoring API as MaxBrowserChecks, and searching for the vendor’s synthetics limits turns up execution quotas and per-execution prices — not the hard ceiling on how many browser checks may exist at all. With twenty sites, one check per site was over before it started.
The instinct is to call that a limitation and work around it. But look at what billing actually counts — per probe location, per started minute — and grouping stops being a workaround:
# Grafana Cloud caps browser checks at 10 per tenant (MaxBrowserChecks), which
# is fewer than we have sites. Several sites therefore share one check. That is
# not a workaround but the cheaper design anyway: billing is per probe location
# and per started minute, so a script visiting nine pages costs the same as one
# visiting three. The metrics carry a url label, so per-page analysis survives.
A check running nine pages in fifty seconds and a check running three pages in ten seconds cost exactly the same. Priced across the three shapes — twenty sites, three probe locations, hourly:
grouped (4 sites per check) 19.440 executions ~€61/month 9 checks
one check per site (20) 43.200 executions ~€170/month over the cap
one check per URL (61) 131.760 executions ~€578/month over the cap
The two shapes that violate the cap are also the ones that cost three and nine times as much. The cap pushed toward the design that was already cheaper — which only became visible because the billing unit got read carefully enough to notice it wasn’t per page.
That leaves group size as the real tuning knob, bounded from both directions:
# 3 sites -> 10 checks, at most 10 pages -- exactly at the cap, no headroom
# 4 sites -> 9 checks, at most 13 pages -- one slot spare
# 5 sites -> 7 checks, at most 16 pages -- three spare, longest runs
variable "sites_per_check" {
description = "How many sites one browser check covers"
type = number
default = 4
}
Four, because the estate keeps growing and a cap with no headroom is the worse problem to have. Upward, group size is limited by the billed minute: a run over sixty seconds costs two minutes instead of one.
The grouping key is worth a sentence, because the obvious choice is wrong. Sites are bucketed by content category rather than by hosting app. The app says where a site runs; the category says what it is. A regression on a recipe template is one story, a regression across the regional news sites is a different one — and the alert you want fires along the second axis. The app stays on the check as a label, so nothing is lost.
buckets = {
for e in local.entries :
"${lookup(var.app_short_names, e.app, e.app)}-${e.category}" => e.site...
}
sites_of_bucket = { for b, sites in local.buckets : b => sort(distinct(sites)) }
# Chunk each bucket into groups of at most var.sites_per_check sites. Keeps a
# single run short enough to stay inside one billed minute. Always numbered,
# so a bucket that grows past the limit does not rename its first group.
groups = merge([
for b, sites in local.sites_of_bucket : {
for i in range(ceil(length(sites) / var.sites_per_check)) :
"${b}-${i + 1}" => slice(
sites,
i * var.sites_per_check,
min((i + 1) * var.sites_per_check, length(sites))
)
}
]...)
The -1 suffix on every group, including groups that currently have only one, is the kind of detail that pays for itself once. Without it, the first group of a bucket is named mp-food until a fifth site arrives, at which point it becomes mp-food-1 — and a renamed check in Terraform is a destroy-and-create that discards the metric series behind it.
Why Is the Check Name the Only Filter That Works?
Because the labels never arrive where you want to use them — and that is the single most expensive thing to find out late.
Start with the obvious part. The vendor allows at most five labels per check, each at most 32 bytes. With several sites per check, a site label is not merely a squeeze — it’s false, because the check covers four of them.
# At most 5 labels, each at most 32 bytes -- the vendor's limit.
# No site label: with several sites per check it would be wrong, and the
# per-page view comes from the url label on the metrics anyway.
labels = {
app = local.meta_of_group[each.key].app
category = local.meta_of_group[each.key].category
device = "desktop"
ownership = anytrue([...]) ? "third-party" : "own"
}
So far, so ordinary. Then you go to build a dashboard filtered by category and discover that your labels are not on the browser metrics at all. They surface as label_category, label_app and so on — but only on sm_check_info and in the logs. The metrics that carry the actual measurements look like this:
probe_browser_web_vital_lcp{job="mp-food-1", url="https://…/", probe="Frankfurt", instance="…"}
job, url, probe, instance. Nothing else. Every label the Terraform carefully assigns is invisible from the vitals.
Which means filtering by category has to go through the job name:
probe_browser_web_vital_lcp{job=~".*-food-.*"}
count by (url) (probe_browser_data_received{job=~".*-news-.*"})
And that turns the naming scheme from a cosmetic decision into the load-bearing one. <property>-<category>-<n> is the only dimension along which web vitals can be split. Name the checks check-1 through check-9 and that capability is gone — not broken, gone, with nothing to migrate back from and no error to tell you. The dashboard’s own template variables are built on exactly this, job=~"($app)-.*" and job=~".*-($category)-[0-9]+(-mobile)?", which only works because the names were structured before anyone needed them to be.
Per-page analysis does survive, via the url label the metrics carry natively. The general shape: check labels describe the check, metric labels describe the measurement, and the two label sets do not meet. Before designing around any label, query the metric you actually intend to alert on and look at what came back.
One more translation layer sits in the variables, and it exists for a reason worth copying. The inventory stores the hosting app as the slug the platform CLI returns, because that’s measured data comparable against the platform. Nobody says those slugs out loud. The mapping to the names people actually use is a naming convention and lives in Terraform:
variable "app_short_names" {
description = "VIP app slug -> internal short name used in check names"
default = {
funke = "rwp"
funkevrt = "vrt"
funkespc = "spc"
partisans = "mp"
}
}
Measured data keeps its measured form; conventions live where conventions live. Rewriting the inventory to match how people talk would have broken the drift check that compares it against the platform.
Which Constraints Belong in the Config as Code?
The ones that cost money or silently break things when violated. Every vendor limit here is a validation block or a comment, not tribal knowledge:
# NOTE: Grafana Cloud only accepts 60 to 3600 seconds. The four runs per day
# from ADR-0001 cannot be configured; 3600 is the sparsest possible cadence.
# Every halving doubles the bill.
variable "frequency_seconds" {
default = 3600
validation {
condition = var.frequency_seconds >= 60 && var.frequency_seconds <= 3600
error_message = "Grafana Cloud only accepts 60 to 3600 seconds."
}
}
# 60 s is the natural ceiling: billing rounds up to whole minutes, so a run of
# 61 s costs two. Better to fail and be seen than to silently double the bill.
variable "timeout_seconds" {
default = 60
validation {
condition = var.timeout_seconds >= 1 && var.timeout_seconds <= 180
error_message = "Grafana Cloud only accepts 1 to 180 seconds."
}
}
The timeout default is the interesting one. The vendor permits up to 180 seconds. Setting it to 60 means a check that would have taken 90 seconds fails instead of quietly billing double. A visible failure beats an invisible cost — and that’s a policy choice, so it needs the comment explaining it, or the next person raises it to 180 to "fix the failing check."
The same reasoning drives a default that looks like a style preference and isn’t:
# Armed on 2026-08-28 after the checks had been reviewed in the UI. The default
# lives here rather than in a -var flag: passing it on the command line means
# the next apply without the flag silently disables every check.
variable "enabled" {
default = true
}
Any state that lives only in how you invoked the command is state that reverts the next time somebody invokes it differently. Monitoring that silently disarms itself is worse than monitoring that was never set up, because the dashboards keep looking fine.
And the outputs exist to make the two things that cost money checkable before apply, not on the invoice:
output "browser_checks" {
description = "Number of browser checks. Grafana Cloud allows at most 10 (MaxBrowserChecks)."
value = length(local.groups) + length(grafana_synthetic_monitoring_check.mobile)
}
output "executions_per_month" {
description = "Billed executions: checks x locations x runs per month. The free tier is 10,000."
value = (length(local.groups) + length(grafana_synthetic_monitoring_check.mobile)) *
length(var.probes) * floor(2592000 / var.frequency_seconds)
}
What Bit During Setup That the Docs Don’t Say?
Two credentials, not one — and a region that fails in a misleading way.
# Two credentials, because Synthetic Monitoring has its own token:
# auth -> the Grafana instance (folders, alert rules)
# sm_access_token -> the Synthetic Monitoring backend (checks)
provider "grafana" {
url = var.stack_url
auth = var.grafana_auth
sm_access_token = var.sm_access_token
# Without this the provider talks to the US region and an EU token is
# rejected with "invalid API token". The address for your stack is shown
# under Synthetics -> Config, "Your backend address is:".
sm_url = var.sm_url
}
invalid API token is exactly the wrong error message for "you are talking to the wrong region," and it will send you to rotate a perfectly good token first. Worth a comment at the point of use rather than a note in a README nobody opens while debugging.
The alert rules are generated the same way, with the ADR’s requirements as expressions rather than prose — a majority of locations, confirmed across two consecutive runs:
# ADR-0003 requires two consecutive runs. At an hourly cadence that is two hours.
confirmation_window = "${var.frequency_seconds * 2}s"
# Majority of locations: with 3 probes that is 2.
majority = floor(length(var.probes) / 2) + 1
Deriving both from the variables rather than hardcoding 7200s and 2 means adding a fourth probe location doesn’t quietly leave a majority rule that now means "half."
That gets you two rules: a budget that needs no history and applies from the first run, and a deviation from the seven-day median confirmed by a majority of locations. The third rule is the one worth stealing, because it points the alerting at itself:
(
count(count by (job) (
probe_browser_web_vital_lcp
> 1.2 * quantile_over_time(0.5, probe_browser_web_vital_lcp[7d])
))
/
count(count by (job) (probe_browser_web_vital_lcp))
) > 0.5
All three probe locations run at the same cloud provider. A regional event there moves them together — and then the majority rule dutifully reports a regression for every site at once. That more than half of a set of independently operated websites got slower in the same run is the less likely explanation, so the rule says so in its annotation: More than half of all sites degraded in the same run. Check the measurement path first, not the sites.
It prevents no outage. It prevents three people debugging the wrong thing at three in the morning, which on a monitoring system is the more common failure.
One caveat carried in the file itself, because it’s the honest state of things:
# NOTE: the PromQL below is written against the documented metric names
# (probe_browser_web_vital_lcp with labels job, url, probe) but has never run
# against real data. Before arming, run it once in Explore and check the unit
# of LCP -- k6 reports web vitals in milliseconds, which the budget assumes.
Alert rules written against documented metric names and never executed are a plausible-looking guess. Saying so in the file is the difference between a known gap and a silent one.
What’s the Generalizable Lesson Here?
- Read the billing unit, not just the price. "Per started minute per location" rather than "per page" is what turned a hard cap of ten checks from an obstacle into the cheaper design. The unit determines the shape of the configuration far more than the rate does.
- Group by the axis your alerts care about, not the axis your infrastructure is organized on. Hosting location is a label; content category is what makes a regression one story rather than twenty.
- Number your groups from the first one. A name that changes when a collection grows past a threshold is a destroy-and-create in Terraform, and it takes the metric history with it.
- When a label would be false, drop it rather than approximate it, and find the granularity at the layer that actually has it. Check labels describe the check; metric labels describe the measurement.
- Encode vendor limits as
validationblocks with the reason in the error message. The person who hits "only accepts 60 to 3600 seconds" a year from now will not have the pricing page open. And expect the limit that actually shapes the architecture to be absent from the pricing page entirely — this one arrived as a 403 from the API. - Check where your labels land before you design around them. Labels attached to a check may not exist on the metrics that check emits, and when they don’t, the naming scheme becomes the only filter dimension you have. That is a decision made before the first
applyand effectively unmakeable afterward. - Give the alerting one rule that suspects the measurement rather than the subject. If more than half your independently operated targets degrade in the same run, the measurement path is the likelier culprit, and saying so in an annotation is cheaper than the incident where nobody thought of it.
- Defaults that only exist as command-line flags are defaults that revert. Anything whose absence silently disarms production monitoring belongs in the file.
- Write down the parts you haven’t verified. A PromQL expression that has never run against real data is fine to ship behind a comment saying so; the same expression presented as finished is a monitoring gap disguised as coverage.
The check script itself had a subtler problem — it reported Largest Contentful Paint for one page out of every nine. That’s the next post.