A site score can rank your worst store highly because it grades the ground, not the operation, and because it learned from locations your company already chose. Test any vendor by holding out stores you already know, sending addresses only, locking the scores in writing, then checking whether the rank order matches your actuals.
If you have sat through enough site selection demos, you know the awkward beat. The sales rep drops a pin on a store you already own, the score comes back an 87, and you know for a fact that location has missed its budget three years running. Nobody says anything out loud. The demo moves right along to drive-time polygons.
That silent moment is worth more than the rest of the meeting. It is a free validation test the vendor never meant to run, and how their team responds tells you most of what you need to know. When you are deciding whether to renew an existing contract or evaluate a second tool to sit beside what you already have, you do not have to wait for an accidental reveal during a screen share. You can run the test on purpose.
Why a model can rank your worst store first
A location model can miss badly on your home turf without anyone acting in bad faith. Real estate involves messy operations, incomplete data, and statistical limits that explain why a score misfires.
It learned from the sites you picked, not the ones you passed on
Most location models train on a retailer's existing stores. Those locations represent the survivors of years of internal screening. Your team already filtered out the awkward access points, the dying strip centers, and the spaces where the landlord demanded impossible rents, so the model never saw them.
Statisticians call this range restriction. A model trained only on sites that cleared your bar has very little information about what a failing site looks like. It can tell you which of your good sites is best, but asking it to flag a bad deal asks it to extrapolate past the edge of everything it has ever observed.
Fleet size compounds the issue. Paul Sill, who heads JLL's Visionary Insights Group, put the arithmetic plainly to Modern Retail in December 2025: "If I'm a 500-unit coffee shop, 500 observations is not enough data for AI to do anything intelligently." Most brands running an active expansion program operate far fewer than 500 units.
The score grades the ground, your P&L grades the business
A location model scores the physical coordinates and the trade area. Your store P&L records the real estate plus everything that happened after you signed the lease: the store manager, staffing turnover, negotiated rent, build-out costs, format choices, and whether you opened during an unseasonal freeze.
A high score on a bad store sometimes means the real estate was fine and the problem sits inside the four walls. That distinction is genuinely useful to know before committee, but only if the vendor talks through it directly instead of rushing past the score.
Everything comes back in the same narrow band
Look at the distribution before you study individual numbers. If every site the vendor evaluates scores between 70 and 85, the model is not separating anything. A score that never drops low cannot tell you what to avoid, which is the primary reason to buy a site selection tool.
Compressed output also breaks the validation math. When the range of an outcome is squeezed into a narrow band, any correlation calculated against real performance is suppressed. That allows a model to look defensible in a vendor's own testing and still fail to separate winning sites from duds on your portfolio.
Some inputs are estimates wearing the clothes of facts
Several layers feeding a composite score are modeled estimates rather than primary records. Esri's own technical paper on its 2026 Business Locations data, published in February 2026, is direct about this: "Data Axle sales volume attributes are modeled, as verifiable sales volume figures are extremely difficult to obtain from private businesses. The primary source of information for the model is the U.S. Department of Commerce's data for sales per employee for each NAICS code."
A competitor's sales figure in your trade area may be an employee count multiplied by a national industry average. That is a reasonable estimate, but it is not a measurement. When you are underwriting millions in capital expenditure, relying on modeled competitor revenue can quietly skew your rating, which is why choosing which site selection data to trust requires inspecting the pedigree of every layer.
Foot traffic panels are relative signals, not counts
Mobile location panels observe a sample of devices and extrapolate to the broader population. A peer-reviewed audit of SafeGraph data published in PLOS ONE in January 2024 found that the panel's sampling rate averaged 7.5% over five years, noting that device counts tracked census population well at the county level while the association "at the tract and block group levels becomes weaker." The same study found that Hispanic populations, low-income households, and residents with less formal education were underrepresented, with the bias varying by place and over time.
Census tracts and block groups are the exact geographic resolution where site selection decisions happen. Use foot traffic to compare one site against another, and do not treat the absolute number as a count of people. Our guide to evaluating location data providers covers how to audit these underlying datasets before they reach a composite score.
Back-test the score against a portfolio you already know
You do not need an in-house data science team to evaluate site selection software. You just need locations whose actual performance you already know.
- Hold out real stores. Pull 20 or more locations from your fleet, deliberately mixing your strongest performers, chronic underperformers, and unremarkable units. A test composed only of good stores gives the model nothing to separate.
- Send addresses only. Provide street addresses and nothing else: no sales figures, no internal rankings, and no clues about which ones you like. Anything you volunteer is an input the system no longer has to evaluate on its own.
- Lock the scores in writing first. Require the vendor to return their ratings in writing with timestamps before you reveal your actuals. A score delivered after the reveal is not validation.
- Compare the rank orders. Line the vendor's score order against your actual sales ranking. Use rank correlation rather than debating absolute points. The question is not whether store 14 should have received an 82, but whether the model arranged the portfolio in roughly the right sequence.
- Check the two ends. Middle-of-the-pack noise is normal in real estate. The critical failure is seeing your top revenue generators clustered near the bottom or your known underperformers graded near the top, because those extremes represent the green lights and red flags in your pipeline.
- Check the spread. Note the highest and lowest score in the set. If the entire portfolio fits inside a fifteen-point window, you have a model that agrees with everything.
A vendor worth keeping will take the test as written and walk through the specific inputs that moved each score up or down. A vendor who explains away misses by calling the method a trade secret has told you everything you need to know.
What a good vendor answer sounds like
During your next demo or renewal review, push on four points:
- Which inputs moved this score, and in which direction. You want the specific variables and their directional weight on that exact parcel, not a generic brochure list of data partnerships.
- How the model was validated, and on whose data. Ask whether the testing used true holdout stores or whether the platform was evaluated on the same locations used to calibrate it.
- Whether you can change the weights. Inline mall retail, drive-through QSR, and neighborhood services depend on fundamentally different traffic and demographic drivers.
- How it behaves on a format it hasn't seen. A system calibrated on suburban strip centers will generate confident ratings on urban street retail whether or not it has earned that confidence.
Published validation methodology is scarce across the retail site selection platforms teams typically shortlist. When we looked at platforms such as Placer.ai, Buxton (now part of Audiense), SiteZeus and its Atlas platform, Kalibrate, Esri Business Analyst, CoStar, MRI Software, and SiteWise, we found very little public documentation detailing how scoring and forecasting models are tested. That is not an accusation of flawed math, but it means the burden of verification rests entirely on you. If you are assembling a shortlist, our comparison of retail site selection platforms details what each system is designed to handle.
Assumptions worth re-testing on your own portfolio
A blind back-test often uncovers something more interesting than a vendor verdict: the variables that predict performance across your fleet frequently differ from retail rules of thumb. A demographic factor your team screened on for years might carry almost no weight, while an unheralded co-tenant or traffic pattern explains most of the variance. If your model says household income drives your business and your own stores say otherwise, the model is describing a category, not your brand. That divergence is why forecast accuracy work belongs directly alongside site scoring rather than downstream of it.
While your performance data is organized, run the same test against your historical approvals. Rank those stores the way your committee ranked them before opening, then see how well that order matches today's actuals. Finding the line between when to trust your gut and when to trust the data produces an answer specific to your portfolio. The same caution applies when studying competitor footprints: if your expansion relies on evaluating the operators who already won in a market you're entering, run your own units through that filter first to confirm whether it can order a portfolio accurately.
How GrowthFactor approaches the same problem
Real estate teams run blind validation tests because they are weary of black boxes: an opaque number arrives, nobody can explain the math, and the committee is expected to sign off on a ten-year lease.
GrowthFactor's Custom Evaluators are built the other way around. Your team writes the evaluation criteria in plain language, the way you would brief an analyst, and selects the data sources the evaluator checks: demographics, foot traffic, hourly vehicle traffic, nearby businesses, cannibalization against your existing network, the analog stores a candidate most closely resembles, zoning, and live web research. It delivers a rating on the scale you set, backed by written reasoning that names every input behind the result. If a required data layer cannot answer, the run fails rather than scoring around the gap. A drive-through restaurant and a bookstore aren't graded on the same assumptions, and every rating comes with evidence you can defend directly in committee.
That design makes back-testing a standard part of your workflow. Custom Evaluators score your active stores as easily as new prospects. You can run an evaluator across twenty existing units, compare its ratings against your actual sales, and adjust your criteria where the logic misses your operational experience. Every site maintains a complete run history so you can track how scores shifted as criteria evolved, while a comprehensive site report with the five stores in your portfolio it most resembles loads in about 10 seconds.
Forecasting sales follows an equally transparent path. Labs begins with Discovery, a 30-day analysis of your brand's historical sales that identifies the specific factors driving your performance, free for brands with 40 or more stores. A named data scientist builds the model alongside your team, back-testing it on held-out stores it has never seen before it touches a live deal. Once built, it produces revenue projections as a range in under 10 seconds. Cavender's went from 9 new stores in 2024 to 27 in 2025, and every new location is performing at or better than expected.
Technology should never overrule what your team knows about landlords, local markets, or operational talent. It should give your judgment clear evidence to lean on. Our ratings deserve the same holdout test as anyone else's. Run it. If you already track cannibalization and market saturation across your markets, your data is already prepped. Book a 30-minute demo and we will run it live on one of your own markets.
Frequently Asked Questions about site selection software accuracy
Common questions from real estate teams evaluating scoring models.
Why does site selection software give a high score to a store I know is underperforming?
Usually because the model grades the trade area rather than the operation. Rent, staffing, build-out, format fit, and local execution move store results but rarely appear in a vendor's training data. A second common cause is that the model learned from sites your company already chose, so it never saw the locations you rejected and has little basis for recognizing a bad one.
How do you test the accuracy of site selection software?
Back-test it against stores you already know. Pick a mix of your strongest, weakest, and middling locations, send the vendor addresses only, and get every score in writing before you share your actual performance. Then compare the two rank orders. The question is whether the model puts your best stores near the top and your worst near the bottom, not whether any single score looks correct.
What is score compression in a site selection model?
Score compression is when nearly every site the model evaluates comes back in a narrow band, for example 70 to 85. A model whose output barely varies cannot separate good sites from bad ones, and the narrow range also suppresses any correlation you try to compute against real performance. If a vendor's scores never go low, the score is not doing the work you are buying it for.
What should I ask a site selection vendor about their model?
Ask which inputs moved this specific score and in which direction, how the model was validated and on whose data, whether you can change the weights, and how the model behaves on a category or format it has not seen before. Ask for the score distribution across your own portfolio. A vendor who can answer those in plain language is a different proposition from one who treats the method as a trade secret.
How does GrowthFactor compare to Placer.ai for validating a site score?
They solve different parts of the problem, and plenty of teams run both. Placer.ai sells foot traffic measurement from its own panel, the category benchmark, which is one input into a site decision. GrowthFactor's Custom Evaluators rate each site against criteria your team writes, on the scale you choose, and show the data and reasoning behind every rating. Evaluators also run on the stores you already operate, so the back-test above is something you can rerun on your own portfolio inside the platform whenever the criteria change, rather than a one-off vendor exercise.
