Executive Summary

"Spatial AI" effectively became a new category between 2024 and 2025. In September 2024, Stanford's Fei-Fei Li founded World Labs with a $230M seed round. By November 2025, the company released Marble and crossed $1B in cumulative funding. World Labs framed the thesis bluntly: "Spatial Intelligence is the next frontier of AI, rivaling and complementing language intelligence." The conceptual foundation traces back to Andrew Davison of Imperial College, who defined Spatial AI in his 2018–2019 FutureMapping essays as the evolution of SLAM — a system that perceives, models, reasons, and acts on space in an integrated loop. That academic framing has now been absorbed as an industry term.

1.1Five Concurrent Signals

Throughout 2025, tools and papers following the same pattern — natural language to 3D spatial structure — appeared in rapid succession. Within the same category, however, the tools differ in flavor. World Labs Marble focuses on navigable environments. STF Labs UrbanGPT 2.0 is specialized for urban and architectural domains, handling GFA and FAR (Floor Area Ratio). Sat2City reconstructs cities from satellite imagery alone. Cesium and Esri layer AI on top of established GIS infrastructure.

1.2Market Numbers — Three Sources Pointing the Same Way

Three market research firms cover different slices but point in the same direction. AI urban planning is projected to grow from $2.26B in 2025 to $13.60B by 2035 at CAGR 19.65% (Metatech Insights). Geospatial AI is forecast to expand from $38B in 2024 to $64.6B by 2030 at CAGR 9.25% (Arizton), while the broader Geospatial Intelligence market reaches $62.88B by 2030 (MarketsandMarkets). What began as a narrow category — natural-language-to-3D generation — is expanding across GIS, digital twins, and urban planning as a whole.

1.3⚠️ UrbanGPT Namesake Disclaimer

1.4STF Labs UrbanGPT 2.0 — Public Information

STF Labs (Studio Tim Fu) is a London-based RIBA and ARB-registered architecture studio founded by Tim Fu. UrbanGPT 2.0 Beta combines GPT-4o (language layer) with real-time diffusion (spatial generation) and Rhino/Grasshopper (precision geometry) to generate cities from natural language. A prompt such as "70% residential, 30% commercial, FAR 450%" returns a 3D city layout along with automatic GFA optimization. Disclosed partners include NVIDIA and Lenovo, and the STF Labs roadmap states a clear direction: "integrate real-world city data → move from form-generation to spatial decision-making."

However, the evaluation methodology is not disclosed. The Beta announcement consists of qualitative phrases like "production workflow integration, reaching stability and precision," with no public quantitative benchmark, evaluation dataset, or baseline comparison. The phrase "automatic GFA calculation" appears, but the underlying regulatory database and algorithm are unspecified. This information asymmetry is the starting point of the present report.

A Spatial AI output has three dimensions: (1) the 3D model (geometry of buildings, roads, and vegetation), (2) metadata (coordinate system, GFA, FAR, zoning), and (3) scenarios (population, traffic, and environmental simulation inputs). All three must be coherent for the output to qualify as a "usable urban design proposal." Yet existing LLM benchmarks (MMLU, HELM, MMBench) measure only the accuracy and consistency of text outputs. 3D geometry, coordinate systems, regulatory conflicts, and the validity of simulation inputs sit outside their measurement scope.

2.1The Academic Diagnosis — "Reliance on Human Evaluation"

In 2024, researchers from Tsinghua and SJTU published T3Bench (arXiv:2310.02977), formally diagnosing that Text-to-3D evaluation has "largely relied on subjective user experiments." T3Bench's Quality metric reaches a Spearman correlation of 0.784 with human ratings (via ImageReward), and Alignment reaches 0.780 (via GPT-4) — strong, but still not a single automatic metric that closes the loop. The concurrent 3D Scene Generation Survey (arXiv:2505.05474, S-Lab NTU) summarized the same consensus: "no single metric performs consistently across all datasets." Even SOTA models like CityDreamer (CVPR 2024) rely on user studies of N=20 with 1–5 scales. Industry and academia stand on the same blank space.

2.2"Visually Plausible ≠ Usable"

The UrbanWorld authors (arXiv:2407.11965) identified the central weakness of their own model: "homogeneous styles, limited diversity." A city that looks plausible but produces near-identical variants regardless of seed is unusable for decision-making in urban planning. Even a halving of distribution distance from FID 213.56 (SceneDreamer) to 97.38 (CityDreamer) in just one year does not guarantee usability. The gap between visual plausibility and operational applicability remains large.

2.3The Standards Lag

ISO/IEC 23894:2023 (AI risk management) and the ISO/IEC 5259 series (data quality — 5259-4 published 2024-07, adopted as the EU standard EN ISO/IEC 5259-4:2025 in February 2025) are starting to apply to LLMs. They have not yet been applied to 3D spatial outputs. Esri's Trusted AI guide centers on GIS analysis workflows, leaving natural-language-to-3D generation blank. ISO 23247 (digital twins) is manufacturing-centric and requires complementary standards (such as ITU T-REC-Y.3090) to apply to urban use cases. OGC CityGML 3.0 reached its conceptual model in 2021, and implementation accelerated in 2025 with 3DCityDB v5 — but a formalized role as "Spatial AI output validation" has not yet emerged. The gap between standards and technology runs roughly 2–3 years.

These five criteria are an evaluation framework that Pebblous proposes from the PebbloSim perspective. They are not standards we operate today; they are a hypothetical proposal meant to fill the Spatial AI evaluation gap. The five are derived from the intersection of metrics that academia has partially formalized and validation dimensions that industry has left empty. For each criterion we provide the definition, measurement approach, academic priors, and the distinctive contribution Pebblous proposes.

3.1Geographic Coherence

This criterion asks whether the coordinates, scale, and road topology of the generated city align with real geography (GIS, satellite imagery). When the prompt specifies, for example, "Lower Manhattan, 700,000 m²," does the output match the actual GIS polygon, area, and road network topology to a reasonable degree?

3.2Scale Consistency

This criterion measures whether the relative proportions of building heights, road widths, vegetation density, and block sizes match urban reality. A 30-story single-family home appearing in a low-rise residential district is visually plausible but contextually wrong. We quantify the statistical distance between the generated distribution and the city-specific reference distribution.

3.3GFA Validation (Gross Floor Area)

This criterion auto-verifies whether the generated city's FAR, lot coverage, and zoning distribution comply with regulations. When UrbanGPT 2.0 receives "FAR 450%" as a prompt, two questions follow: does the actual sum of building footprint × number of floors really reach FAR 450%, and is FAR 450% permitted under the zoning code of that lot? In B2B and regulated environments, this criterion delivers the most immediate value.

3.4Scenario Coverage

This criterion measures the breadth and coverage of city variants produced from a single prompt or seed, and the risk of mode collapse. When only the seed changes, how distinct are the resulting cities? Low diversity erodes the system's value as a decision-support tool. This is precisely the dimension the UrbanWorld authors flagged as their own weakness — "homogeneous styles, limited diversity."

3.5Sim-to-Real Gap

This criterion measures the distributional and geometric distance between generated city data and real city data (LiDAR, aerial imagery, GIS). Can a model trained on simulated data work in the real world? The question sits at the core of autonomous driving, robotics, and digital twins. UCF's HiFi DT framework has effectively assembled the standard package.

3.6The Five Criteria, Side by Side

The five criteria, summarized across academic priors, industry priors, and Pebblous's proposed contribution. ✅ marks areas with rich priors that can be adopted immediately; ⚪ marks areas where both academia and industry are blank — defining the criterion is itself a contribution.

4.1Application Table (External-Observation Estimate)

We attempt an evaluation for each criterion using only public information. Where the evaluation methodology is undisclosed, we mark the entry as ⚪ unmeasured.

4.2Estimated Strengths (Two)

The evaluation methodology is undisclosed, but the public technical stack and demo videos consistently surface two strengths.

4.3Estimated Weaknesses (Three)

On the other hand, the undisclosed methodology produces visible weaknesses across three dimensions. All three are gaps the proposed five criteria can fill precisely.

If UrbanGPT 2.0 generates the city layout from natural language, PebbloSim can simulate the population, traffic, and environmental scenarios on top of that layout — turning it into an "executable urban design proposal." In that framing, UrbanGPT output → PebbloSim simulation → five-criteria evaluation → feedback becomes a natural pipeline. This section sketches that pipeline at the blueprint level.

5.1Workflow — A Five-Step Pipeline

From input to feedback loop, each step maps directly to one or more Pebblous products.

5.2Pebblous Products ↔ Five Criteria Mapping

DataClinic (training-data diagnostics), PebbloScope (distribution and visual analytics), PebbloSim (synthetic data generation), and Data Greenhouse (ground-truth curation) together compose the measurement infrastructure for the five criteria.

5.3Positioning Against NVIDIA Omniverse Smart City Blueprint

NVIDIA's Smart City AI Blueprint (Palermo case study, 50B pixels processed per second) sits at the infrastructure layer — Cosmos, NeMo, and Metropolis analyze more than 1,000 video streams in real time. Bentley iTwin is a digital twin platform for infrastructure assets; Cesium is 3D geospatial infrastructure. All three are powerful, and all three leave the evaluation methodology for natural-language-to-3D outputs blank. Pebblous's place is the evaluation meta-layer that runs on top of these stacks. While NVIDIA Omniverse runs the simulation, Pebblous evaluates the output against the five criteria and feeds the results back. The relationship is complementary, not competitive — Pebblous operates on top of these infrastructures rather than entering the same category.

This blueprint represents one form of a possibility we are exploring with STF Labs. For teams in AI urban planning, digital twins, and simulation who are wrestling with evaluation methodology, the five criteria offer a starting point.

Between 2023 and 2025, ISO/IEC, IEC, OGC, and NIST have been steadily updating standards across AI trustworthiness, data quality, and digital twin security. Yet no formal urban-domain standard directly applicable to Spatial AI (natural-language-to-3D generation) exists. That gap is a time window for Pebblous to enter the standards conversation.

6.1Four Core Standards

Four standards connect directly or indirectly to Spatial AI evaluation. We summarize their date, scope, and applicability to Spatial AI.

6.2The Gap in Urban-Domain Standards

The ISO 23247 series is manufacturing-centric and requires complementary standards (ITU T-REC-Y.3090) to apply to urban and smart-city contexts. The Digital Twin Consortium and OGC UDTIP (the Urban Digital Twin Implementation Pilot, which concluded in February 2025) remain in the consensus-building phase. No formal urban-domain standard exists — which is to say, the time window is open for whichever team proposes an evaluation framework first to set the agenda.

Among the ten major players in this report — STF Labs, World Labs, Cesium, Esri, Mapbox, NVIDIA, Bentley, Autodesk Forma, TestFit, Hypar — only Esri publishes a methodology for verifying the quality of its own AI outputs. The other nine excel at automated generation but stay silent on "how outputs are verified." The implication is simple: the team that owns an evaluation framework owns a market entry barrier.

7.1An Evaluation Framework Is Reusable IP

The five criteria are not merely measurement tools. They are an IP asset reusable across four layers: (a) definitions academia can cite in follow-on research, (b) checklists potential partners can apply to their own tools, (c) line items policymakers can adopt as procurement criteria, and (d) self-diagnostic tools that Pebblous's prospective customers can apply to their own data quality. Once defined, an evaluation framework compounds — citation, reuse, and extension create network effects.

7.2Pebblous's Place — "Spatial AI Data Quality" Thought Leader

Pebblous is an AI-Ready Data infrastructure provider. In the Spatial AI era, we plant a flag in the place that defines what "AI-Ready" means — which data, which evaluation criteria, which standards. DataClinic for training-data diagnostics, PebbloScope for distribution and visual analytics, PebbloSim for synthetic data generation, Data Greenhouse for ground-truth curation. These four products together constitute the measurement infrastructure for the five criteria. The evaluation framework is both the integrating shell of the Pebblous product line and the interface to the outside world.

7.3Next Steps

The EN adoption of ISO 5259-4 in February 2025 and the acceleration of OGC CityGML 3.0 implementation (with 3DCityDB v5 in 2025) both signal that standards are moving. The five criteria are the agenda Pebblous brings to that conversation. This report is the first step. Signature Series #2 (Palantir) will extend the same "evaluation lens" to data operations platforms.

References

Academic papers, industry announcements, and standards / market sources cited in this report, organized by track.

Academic (Track A)

1. Wen, B., Xie, H., Chen, Z., Hong, F., Liu, Z. (2025). "3D Scene Generation: A Survey." arXiv:2505.05474. Link
2. He, Y., Bai, Y., Lin, M., Zhao, W. (2024). "T3Bench: Benchmarking Current Progress in Text-to-3D Generation." arXiv:2310.02977. Link
3. 3DGen-Bench Team (2025). "3DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models." arXiv:2503.21745. Link
4. Xie, H., Chen, Z., Hong, F., Liu, Z. (2024). "CityDreamer: Compositional Generative Model of Unbounded 3D Cities." CVPR 2024 / arXiv:2309.00610. Link
5. Shang, Y., Lin, Y., Zheng, Y. (2024). "UrbanWorld: An Urban World Model for 3D City Generation." arXiv:2407.11965. Link
6. Shahbaz, M., Agarwal, S. (2025). "High-Fidelity Digital Twins for Bridging the Sim2Real Gap in LiDAR-Based ITS Perception." arXiv:2509.02904. Link
7. Hua, T. et al. (2025). "Sat2City: 3D City Generation from A Single Satellite Image." ICCV 2025 / arXiv:2507.04403. Link

Industry (Track B)

8. STF Labs (Studio Tim Fu). "Introducing UrbanGPT 2.0 Beta." LinkedIn announcement. Link
9. World Labs. "Marble: Multimodal World Model." World Labs Blog. Link
10. Esri. "Trusted AI — Implementation Best Practices" (ISO 23894 aligned). Link
11. NVIDIA. "Smart City AI Blueprint" (Palermo case study). NVIDIA Blog. Link
12. Fei-Fei Li. "From Words to Worlds: Spatial Intelligence." World Labs Substack. Link

Standards & Market (Track C)

13. ISO/IEC 5259-4:2024 — AI Data Quality Process Framework (adopted as EN ISO/IEC 5259-4:2025 in the EU). Link
14. ISO/IEC 23894:2023 — AI Risk Management. Link
15. Metatech Insights. "AI in Urban Planning Market 2025-2035." Link
16. Arizton. "Geospatial AI Market Report." Link
17. OGC. "CityGML 3.0 Standard." Link