A number you cannot find is a question to investigate, not proof of an empty market. It may be costly to gather, restricted, weakly demanded, available under a different name or useful only in a very specific workflow.
Before building a data product, test two things separately: whether someone needs the answer and whether you can obtain it accurately, lawfully and affordably enough to serve that need. Passing one does not establish the other.
Four historical internal project screens motivated this lesson for Plain Reckoner. Their outcomes are not a representative study of data businesses. This article turns the practical concern into a feasibility workflow and removes the old universal rules about coverage and update frequency.
Start with the decision the number supports
Write the user's task before describing the dataset. Someone may need to compare service availability, choose a supplier, understand a revenue change or prepare a purchasing decision. Each task implies different fields, timing and consequences of error.
A large table is not automatically valuable. If its records do not connect to the decision, more rows may add cost without improving usefulness. A small verified set can be enough for one local question while being wholly insufficient for a market-wide estimate.
The GOV.UK user-needs guidance emphasizes learning what people are trying to do and treating unsupported opinions as assumptions. Apply that principle to the proposed product: ask about actual decisions and current workarounds before declaring demand.
Record who needs the answer, when they need it and what they currently do without it. A person saying a dataset sounds interesting is different evidence from a documented recurring task that the data can help complete.
Search for substitutes before claiming absence
A missing exact phrase in search results does not mean the underlying information is unavailable. It may exist in a trade report, a paid service, a provider dashboard, an association dataset or a process that relies on direct contact.
Look for functional substitutes, not only products with the proposed name. A buyer might use a spreadsheet, an existing supplier relationship or a manual check instead of buying a dedicated data feed. Those alternatives matter even when they are less polished.
Keep a dated search record with the terms, sources and scope checked. A defensible statement is that no suitable source was found in that review. “Nobody has this data” requires much stronger evidence and is usually unnecessary for explaining what the product does.
Do not treat a competitor's silence as proof of a hidden opportunity or misconduct. A service may have changed audience, distribution or business model. Inspect the available evidence and leave motives unknown unless they are documented.
Test reachability, meaning and permission separately
A successful request tells you that a resource was reachable at that time. It does not establish that the content contains the required field, that the field has been interpreted accurately or that reuse is permitted for the proposed purpose.
A public rate card may be straightforward to read but insufficient for a date-specific quote. A calendar may show some availability without revealing every booking condition. A source's terms may constrain aggregation even when a technical endpoint responds.
Keep separate columns for collection outcome, interpretation status and permission status. A project that needs all three should not call a record usable merely because the parser found a number. Missing one component may require a provider relationship or a narrower product scope.
The price-versus-availability guide explains this distinction with service and store examples. It also shows why an unknown response is not automatically evidence that a business deliberately hides information.
Use a pilot to estimate work, not proclaim a market law
A small pilot can reveal common source formats, missing fields and manual review time. Choose cases that exercise the likely variation instead of only collecting the easiest sites. Preserve failures and ambiguous cases so the cost estimate reflects the actual work.
The old note proposed checking forty sources and using publication shares of 25% and 50% as hard viability boundaries. Those were internal screening heuristics, not validated universal thresholds. This revision does not present them as rules that determine whether a business can succeed.
A low public coverage rate may still support a valuable permissioned survey or specialized service. A high rate may be commercially unattractive if the information is easy for users to obtain themselves. Coverage is one input, not a verdict.
Write a project-specific stopping rule before the pilot. Tie it to the required coverage, acceptable uncertainty, available budget and actual user task. Explain the reasoning instead of borrowing a neat percentage from a different market.
Price the full collection process
Include finding sources, requesting data, parsing, matching entities, checking results, resolving exceptions and publishing a usable output. Automated fetch time may be a small part of the total. Human review often matters precisely where the data is valuable or ambiguous.
Use chosen example inputs: 200 records require two minutes each for initial review and one minute each for validation. That is 600 minutes, or ten hours, before setup, permission work and unresolved cases. At a chosen labor allowance of $25 per hour, that part of the exercise is $250.
The example is a cost scenario, not a measured industry benchmark or a claim about appropriate pay. Replace the inputs with actual observed work from the pilot. Keep one-time setup separate from recurring collection and review.
Do not assume that doubling the records doubles every cost exactly. Some work scales efficiently, while exceptions, source relationships and quality checks may not. A useful estimate identifies those components rather than hiding them in one optimistic cost-per-row figure.
Account for failed and ambiguous records
Suppose a chosen pilot attempts 100 records and produces sixty usable ones after five hours of total work. The relevant output rate is twelve usable records per hour, not twenty attempted records per hour. Both counts are worth preserving, but they answer different questions.
If the product requires near-complete coverage of a named group, the forty unresolved records may dominate the remaining effort. Easy early cases do not establish the cost of the hard final cases. A full-census promise should be assessed against that requirement.
Do not convert unresolved values into guessed numbers to make the dataset look complete. Keep the status explicit and explain which outputs can be calculated without them. A missing denominator or applicability condition can invalidate a seemingly simple aggregate.
The Government Data Quality Framework treats quality as fitness for purpose across several dimensions. The relevant question is not just whether a file contains values, but whether those values support the intended use.
Match freshness to the user's decision cycle
The old article stated that a number changing annually could not support a product people return to. That conclusion was too broad. An annual regulatory deadline, renewal decision or planning cycle can create recurring need even if the underlying information changes infrequently.
Conversely, rapidly changing information may not justify frequent updates for every user. The decision may happen only once, or the user may tolerate some delay. Freshness has value when the difference affects the task, not merely because a daily label sounds impressive.
Ask when a stale value becomes materially misleading. Record the source's update pattern and the user's acceptable delay. Distinguish checking a source from finding a changed value; a fresh review may confirm the same observation without creating new market data.
Choose an update schedule that can be maintained. Do not promise daily analysis when the source is quarterly or the team cannot inspect changes responsibly. A clear known schedule is more useful than a frequent badge detached from actual collection.
Estimate the recurring maintenance burden
A source can redesign its website, rename fields, change access rules or stop publishing. A working initial scraper does not establish that the dataset will remain inexpensive. Reserve capacity for detection, repair and review.
Track source versions and pipeline versions. When a parser changes, test known fixtures and inspect affected records. A new extraction method should not silently reinterpret old values while the article still describes a stable series.
The W3C Data on the Web Best Practices addresses metadata, provenance, licensing and versioning. These are useful operational concerns for a small project too: readers need to know what a value represents and which version they are using.
Build a change log and a recovery path. If a source is temporarily unavailable, say which coverage is affected. Do not leave a current-looking result active indefinitely with no visible indication that its inputs are no longer being checked.
Validate usefulness before adding more scope
Offer a bounded prototype to suitable users and ask them to use it for the relevant task. Watch where the data helps, where it is insufficient and whether the output changes their next action. Avoid pitching the product throughout the exercise.
A user may need a smaller decision aid rather than the full dataset. Another may need permissioned details the public collection cannot provide. Those findings can change the product without requiring the original idea to be declared either brilliant or worthless.
Separate usefulness from willingness to pay. A free prototype can be useful without supporting a commercial price. A stated willingness to pay can be encouraging without being an actual purchase. Record each evidence type honestly.
Do not count your own need as proof of a large customer base. It is a concrete starting problem, which is valuable. The next work is to find whether other people share it and whether the proposed solution fits their circumstances.
Publish a bounded coverage statement
A defensible claim names the group, date, collection method and unresolved cases. “Forty-four named businesses reviewed on this date” can be useful if the list and rules are appropriate. It does not establish that every business in the wider market was included.
A census of a specified list still depends on the list's coverage and the accuracy of the field definitions. Checking every entry does not remove duplicates, stale entities or missing members automatically. Completeness within a frame is different from completeness of a population.
Avoid using a large count as a substitute for a method. A table with thousands of rows can repeat a systematic error. Show the provenance, definitions and validation steps that make the records usable for the stated purpose.
Where publication is appropriate, make a sample or documentation available without calling it the entire underlying archive. The description should match what a reader can actually download or request.
Use competing evidence to improve the idea
If another provider already serves the task, inspect what remains difficult for the user. The answer may be usability, cost, scope or a different decision rather than absence of data. Do not assume that a competitor's existence kills every useful variation.
If a previous service stopped, investigate documented reasons where available. An inactive page can reflect priorities, acquisition, staffing or distribution changes as well as weak economics. Its existence is a signal to ask questions, not proof that the market cannot work.
If a provider makes strong claims with little visible support, use that as a prompt to improve your own evidence trail. It does not establish what all users, search engines or advertising networks can distinguish. Avoid unsupported accusations in the product's marketing.
A useful competitive comparison states observable capabilities and limits. It does not need to declare that nobody else measures anything correctly in order to explain what the new service offers.
Make the next decision explicit
At the end of the pilot, decide whether to continue, narrow the scope, seek a source agreement or stop. State the evidence that supports that choice and the conditions that would change it. A stopped project can still leave a useful method lesson.
The Test Planner can hold a hypothesis, pilot protocol, observations and next action. It cannot infer demand or permission from a website address. Supply the actual evidence and preserve unknowns.
The core lesson is not that unpublished data is always too expensive, nor that daily updates are always valuable. It is that novelty, collection feasibility, quality, permission and user value are separate questions. Test the uncertain links before making a large promise.
A professional research product does not need to know everything. It needs to explain what it knows, what that evidence supports and which missing fact matters next. That makes a small pilot useful without turning it into a universal law about markets.
Check the economics without inventing a forecast
A simple scenario can expose an assumption. Suppose chosen monthly collection and review costs are $400, and a proposed subscription contributes $10 per paying customer after its directly associated costs. Forty paying customers would cover that $400 component. The calculation does not establish that forty buyers exist or that all business expenses are covered.
Add the omitted costs before calling the project profitable: support, development, payment charges, administration and whatever other expenses actually apply. If the product remains free, identify the real funding source rather than assuming traffic will automatically pay the bills.
Use the scenario to ask a concrete research question: what evidence would support the required customer count or funding level? A cost spreadsheet is a planning tool. Demand still needs to be observed, and no attractive arithmetic removes that requirement.