Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation
1School of Internet, Anhui University
2National Engineering Research Center for Agro-Ecological Big Data Analysis & Application, Anhui University
3State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University
*Corresponding author
IEEE TGRS 2026Referring remote sensing image segmentation (RRSIS) segments the single object that an aerial image's natural-language expression refers to. Existing training-free methods resolve the expression through implicit vision–language activations or region–text similarity, which gives weak control over the spatial, superlative, and ordinal relations that dominate aerial referring and cannot represent constructions such as the rightmost ship or the second court from the left. We propose GeoSelect, a training-free pipeline that reframes referring as the execution of a typed spatial program. A frozen, text-only language model synthesises a program in a small domain-specific language from the expression alone; a well-formedness checker accepts it; and a deterministic executor evaluates it over a single scored candidate set type that unifies continuous geometric fields with discrete set and order operators (extremum, ordinal, top-k, binary relation). A reliability ladder degrades any failing program to the field-only special case rather than forfeiting an answer. GeoSelect attains 58.86 mIoU / 59.48 oIoU on RRSIS-D test (more than twice the best prior training-free method and about 93% of the supervised specialist RMSIN with no training), and the identical configuration transfers to RISBench at 55.27 mIoU. Holding candidates and the segmenter fixed, explicit program execution improves the selector over implicit matching, attributing the gain to the executed representation rather than the backbone. An image-level exposure audit shows comparable accuracy on the audited unseen subset.
Same candidate boxes, same segmenter; only the selector differs. A real RRSIS-D example: “the airplane on the right”, with several same-class candidates.
GeoSelect executes the program and picks the airplane matching the ground truth ✓
The executed program scores every candidate; brighter is a higher score, and the chosen box (green) wins.
A typed spatial program, synthesised then executed over one scored candidate set; every component is frozen and off-the-shelf.
A frozen text-only LLM synthesises a typed program in a small DSL from the expression alone; a well-formedness checker accepts it; a deterministic executor runs it bottom-up. Every intermediate program, field, and ranking is inspectable.
One scored-candidate-set type unifies continuous geometric fields (position, proximity) with discrete set and order operators (extremum, ordinal, top-k, relation) that express the superlative, ordinal, and compositional constructions a closed lexicon cannot.
A failing program falls back to the field-only special case on the same candidates, which the executor recovers exactly. In aggregate the system stays at or above that field-only floor.
The expression is compiled into a typed program, and the program is executed over one scored candidate set. Step through it, or edit an operator and watch the selection change. Every candidate box below is a real detection from the frozen pipeline.
Scores are computed in your browser by a port of the executor under the frozen configuration. Detection and segmentation are not rerun: the mask overlay is the pipeline's real output, so editing an operator changes the selection, not the mask.
Supervised specialists and trained VLMs are an upper reference, not training-free competitors.
| Method | Venue | mIoU | oIoU |
|---|---|---|---|
| Fully-supervised specialists | |||
| LAVT | CVPR'22 | 61.12 | 76.48 |
| LGCE | TGRS'24 | 60.98 | 76.33 |
| RMSIN | CVPR'24 | 63.38 | 76.55 |
| FIANet | TGRS'24 | 64.01 | 76.81 |
| CroBIM | arXiv'25 | 64.24 | 76.37 |
| ProVG | arXiv'26 | 65.44 | 77.62 |
| BTDNet | arXiv'25 | 66.04 | 79.23 |
| RSRefSeg2 | TGRS'26 | 69.17 | 79.45 |
| Trained vision–language models | |||
| GeoGround | arXiv'24 | 60.50 | – |
| Text4Seg++ | TPAMI'26 | 62.80 | 74.20 |
| SegEarth-R1 | arXiv'25 | 66.40 | 78.01 |
| Sosa et al. (LoRA) | arXiv'26 | 67.6 | – |
| Training-free | |||
| SAM3 | arXiv'26 | 15.50 | 16.87 |
| EKP-HRM | GRSL'26 | 19.21 | 20.68 |
| DGL-RSIS | JAG'26 | 21.50 | – |
| Sosa et al. (zero-shot) | arXiv'26 | 24.9 | – |
| RSVG-ZeroOV | AAAI'26 | 28.35 | 22.83 |
| GeoSelect (ours) | – | 58.86 | 59.48 |
×2.08 over the best prior training-free method (RSVG-ZeroOV) and ×3 over EKP-HRM, about 93% of the supervised specialist RMSIN, with no training.
| Method | Venue | mIoU | oIoU |
|---|---|---|---|
| Fully-supervised | |||
| RRN | CVPR'18 | 43.18 | 49.67 |
| BRINet | CVPR'20 | 42.91 | 48.73 |
| CMPC+ | TPAMI'21 | 46.73 | 53.98 |
| CRIS | CVPR'22 | 55.18 | 69.11 |
| LAVT | CVPR'22 | 61.93 | 74.15 |
| RMSIN | CVPR'24 | 63.07 | 74.09 |
| CARIS | MM'23 | 65.79 | 75.10 |
| CroBIM | arXiv'25 | 67.32 | 73.61 |
| RSRefSeg2 | TGRS'26 | 72.57 | 74.77 |
| Training-free | |||
| SAM3 | arXiv'26 | 21.09 | 26.20 |
| RSVG-ZeroOV | AAAI'26 | 31.84 | 26.35 |
| GeoSelect (ours) | – | 55.27 | 55.88 |
A single frozen configuration, selected before test evaluation, transfers directly, at ×1.74 the best prior training-free method, surpassing the early LSTM-based supervised methods RRN, BRINet, and CMPC+ with no test-set recalibration.
A typed program, synthesised by a frozen text-only LLM and executed over one scored-candidate-set type that unifies continuous fields with discrete operators, reaches 58.86 mIoU on RRSIS-D, ×2.08 the prior training-free baseline.
With detection and segmentation held fixed, explicit program execution beats the stronger of two implicit selectors by +12.06 mIoU (+18.8 on image-relative, +0.9 on pure-attribute).
The identical frozen configuration transfers across domains to RISBench. Separately, an image-id-level seen/unseen audit on RRSIS-D puts the unseen subset at 58.44 mIoU (54.46 under a conservative seen-set), both well above prior training-free full-test results. We read this descriptively: the gap is small on this audit, though it does not rule out other forms of pretraining exposure, and we make no exposure-robustness claim for RISBench.
What each operator looks like on a real expression. Pick any card and it opens in the inspector above, where you can step through its execution.
@article{jiang2026geoselect,
title = {GeoSelect: Spatial-Program Execution for
Training-Free Referring Remote Sensing Image Segmentation},
author = {Jiang, Yuhang and Deng, Guohui and Xu, Miaozhong and Ruan, Chao and Zhao, Jinling and Huang, Linsheng},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
year = {2026},
volume = {64},
pages = {5642022-5642022},
doi = {10.1109/TGRS.2026.3734378}
}