Publications
Selected Works
11 of 11-
01
GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation
Recast referring segmentation as executing a typed spatial program rather than matching embeddings. Training-free, single GPU, and more than twice the best prior training-free mIoU on RRSIS-D.
Yuhang Jiang, Guohui Deng*, Miaozhong Xu, Chao Ruan, Jinling Zhao, Linsheng Huang
IEEE Xplore Project page arXiv Code
Abstract & figures
AbstractReferring remote sensing image segmentation segments the object named by a natural-language expression in an aerial image. Existing training-free methods resolve the expression through implicit vision–language activations or region–text similarity, which gives weak control over the spatial, superlative, and ordinal relations that dominate aerial referring, such as the rightmost ship or the second court from the left. We propose GeoSelect, a training-free pipeline that reframes referring as the execution of a typed spatial program. A frozen, text-only language model synthesises the expression into a small domain-specific language, a well-formedness checker accepts the program, and a deterministic executor runs it. The central abstraction is a single scored candidate set type under which every operator composes: continuous geometric fields realise position and proximity, while discrete set and order operators add the extremum, ordinal, top-k, and relational constructions that fields alone cannot express. Execution is explicit, so every intermediate is inspectable, and a reliability ladder degrades any failing program to the field-only special case. GeoSelect achieves 58.86 mIoU on RRSIS-D test and 55.27 mIoU on RISBench test, more than twice the best prior training-free method on RRSIS-D, with no referring supervision and on a single GPU. Under a fixed detector and segmenter, explicit execution improves over implicit selectors under the same backbone; the best-box-IoU and outcome-partition diagnostics motivate complementary tests of proposal recall and program-path behaviour, with the program path the clearer priority on RISBench, and an exposure audit shows comparable accuracy on the audited unseen subset. Code and configurations are available at github.com/Avalon-S/GeoSelect.
Fig. 01Why explicit execution. For a spatial expression over several same-class objects, implicit region–text matching (left, GeoRSCLIP) selects the wrong instance, whereas GeoSelect synthesises and executes a typed spatial program that scores the candidate set (middle, brighter = higher score; the chosen box in green) and returns the instance matching the ground truth (right). Same candidate boxes and segmenter; only the selector differs. Yellow outline: ground-truth target.
Fig. 02GeoSelect executes a typed spatial program over a single scored candidate set, with a real per-operator trace above each stage (RISBench, “the westernmost storage tank in the line of tanks”). A frozen text-only LLM synthesises the program π; the executor evaluates it bottom-up: SELECT returns the detected candidates, the continuous FILTER (teal) re-weights them by a geometric field, and the discrete ARGMIN (teal) re-scores the set to its extremum (green), which SAM segments. IEEE TGRS2026
Early Access -
02
The Scissors Effect: When Resize-Based Input Diversity Helps or Hurts Transfer Attacks
Input Diversity is assumed to help transfer attacks. On adversarially trained surrogates it reverses sign, costing a robust source 10.3 points of attack success — so leaving it on by default overstates the robustness being measured.
Yuhang Jiang, Xiaojing Chen
Project page arXiv OpenReview Code
Abstract & figure
AbstractInput Diversity (DI), a random resize and pad applied at each attack iteration, is a near-default ingredient of transfer-based attacks, widely assumed to improve transferability. We show this assumption is regime-dependent and, for adversarially trained surrogates, often reversed. Holding the attack fixed and varying only the surrogate, raising the DI probability improves transfer from standard surrogates but degrades it from robust ones: the two response curves separate like a pair of scissors, a pattern we call the Scissors Effect. On ImageNet, blind DI costs a robust source 10.3 percentage points of attack success across four architecturally diverse targets; the effect is several times smaller at 32×32. Direct measurement supports a bias–variance account: DI displaces the gradient by a comparable amount on both groups but reduces its variance only where the gradient is noisy, and robust surrogates have little noise left to average away. A gradient-consistency probe, frozen and hashed before the runs, predicts the sign of the effect on seven unseen surrogates, and we report where it fails alongside where it works. The practical consequence holds independently of the mechanism: leaving DI enabled by default understates the attack a robust surrogate can mount, and so overstates the robustness of the model being evaluated.
Fig. 01Change in transfer ASR relative to p=0. Raising the Input-Diversity probability improves transfer from standard surrogates but degrades it from adversarially trained ones. Left: ImageNet, four standard targets. Right: CIFAR-10, two standard targets. Bands are 95% image-paired bootstrap intervals. TMLR2026
Accepted as is -
03
PInVerify: An Offline Embodied Benchmark for Active Instance Verification
Reaching the goal vicinity is not finding the right instance. 3,000 offline episodes for active, fine-grained verification; within the tested next-best-view strategies, active viewpoint selection gave no reliable gain.
Yuhang Jiang (sole author)
Project page arXiv OpenReview Code
Abstract & figures
AbstractEmbodied agents have made strong progress in navigating to target objects, but reaching the goal vicinity does not guarantee that the agent has found the correct instance: subtle attribute differences (e.g., “white floral” vs. “white striped”) often require close-range, multi-view inspection. We address this gap with Active Instance Verification (AIV), a task in which an agent actively selects viewpoints around a candidate object to decide whether it matches a fine-grained natural-language description. We formalize AIV as a finite-horizon decision process and introduce PInVerify, an offline embodied benchmark for AIV: 3,000 evaluation episodes across 18 object categories, delivered as multi-view captures with a 6-sector navigation topology that exposes trap views (navigable but uninformative) and unreachable sectors. As reference baselines we build a training-free pipeline and a LoRA-fine-tuned end-to-end agent around open-source multimodal large language models (MLLMs) at on-device scale (≤8B parameters), with attribute decomposition, a visibility-weighted multi-view tracker, and three next-best-view (NBV) strategies. In our evaluation across Qwen3-VL (4B/8B), SenseNova-SI-1.2-InternVL3-8B, CLIP, and SigLIP2, the best MLLM-based baseline exceeds the best embedding baseline by 4.9 pp; GT-box ablations show a +3.1 pp detection gap; and we do not observe reliable gains from active viewpoint selection within the tested NBV strategies. A LoRA-fine-tuned agent (SFT+GSPO) reaches 85.6%. PInVerify aims to support further work on active, fine-grained semantic verification in embodied AI.
Fig. 01Geometric success ≠ semantic success. Left: the agent arrives at the goal vicinity but cannot decide whether this mug is the one the description names. Right: Active Instance Verification moves over a six-sector viewpoint topology, confirming colour, logo colour, logo pattern and mug type one clause at a time until every attribute is resolved.
Fig. 02Dataset capture: each offline episode is built from top-down camera poses around the goal and per-sector RGB observations with paired segmentation masks.
Fig. 03Two reference agents: a training-free modular pipeline (top) with attribute decomposition, per-view verification, multi-view tracking, and next-best-view selection; and an end-to-end LoRA-fine-tuned agent (bottom). CVPR 2026FMEA Workshop
Poster -
04
When Dynamic Scaling Fails: A Scale–State Feedback Governs Recurrent-State Quantization in Linear-Attention and SSM LLMs
Quantizing the recurrent state puts the quantizer inside the recurrence: the scale is derived from the state it then corrupts. INT4 that looks lossless at step 0 destroys recall a thousand steps later, and yoking the scale to a full-precision reference removes the norm explosion but recovers only 1.8 of the 5.4 nats lost — contain the scale rather than locate outliers.
Yuhang Jiang (sole author)
Project page arXiv OpenReview Code
Abstract & figures
AbstractLinear-attention and state-space LLMs carry long context in a fixed-size recurrent state, and quantizing that state is the natural way to cut its cost during long decode. Unlike a weight tensor, the state is rewritten at every step, so a quantizer applied at each step sits inside the recurrence and its errors compound. INT4 that leaves synthetic recall perfectly intact at the start destroys it a thousand steps later. The mechanism is a feedback between the quantizer scale and the state it is derived from. Yoking that scale to a full-precision reference removes the resulting norm explosion entirely, yet recovers only 1.8 of the 5.4 nats lost, so numerical instability and functional degradation are separable failures. The remainder is a resolution problem that locating outliers does not solve, since every outlier-aware scheme here loses to plain persistent scaling. Allocating bits across layers does help, but under article-disjoint splits a learned allocation collapses onto the trivial rule of protecting the first few layers. Run out to 32K, group-wise INT4 is worth no more than discarding the state. Contain the scale rather than locate outliers, and report the decode horizon at which a bit-width was certified.
Fig. 01The phenomenon. (a) NIAH recall over 12 seeds against quantized decode horizon H. Every bit-width is intact at H=0 and decays as the state is overwritten; INT5 traces the frontier. (b) Mean recurrent-state norm on independent articles, log scale: per-token dynamic INT4 inflates it by two to three orders of magnitude, while the static curves and the unquantized reference coincide along the bottom.
Fig. 02The feedback, and cutting it. (a) Single-layer decode trajectory, boxcar smoothed. Norm and tail concentration get separate axes because they carry different units; both expand under dynamic INT4 and stay bounded under a persistent scale. (b) Taking the quantizer step size from the FP state instead of the quantized one removes the explosion at both scales. The three columns are independent arms, not a time course. NeurIPS 2026LCFM Workshop
Non-archival -
05
Retrieved but Not Delivered: Multimodal Memory Delivery for Long-Term Agents
Between retrieving a memory and answering from it sits a stage evaluations never isolate: delivery — what of the retrieved memory reaches the model, and in what form. With the retrieved set held fixed, delivering the original pixels is worth +13.87 points while making retrieval perfect adds 2.31. DeliverMem trains nothing and leads both benchmarks from one fixed configuration.
Yuhang Jiang*, Qingwei Liao, Kaize Yin, Xingling Liu, Luca Cuomo, Silvio Bacci
Abstract & figures
AbstractWork on memory for multimodal agents optimizes what is written, updated and retrieved. Between retrieval and the answer, however, is a stage that multimodal memory evaluations do not isolate: what of the retrieved memory reaches the model, and in what form. We call it delivery, and a controlled decomposition on MemLens locates the remaining room there. With the retrieved evidence set exactly fixed, delivering the original pixels instead of withholding them raises accuracy by 13.87 points on an 8B backbone, whereas making retrieval perfect on those same messages improves it by 2.31. Delivery is the larger term on all three MemLens backbones and grows with backbone strength; retrieval grows too, without closing the gap. We propose DeliverMem, an instantiation of delivery as three decisions: keep the original modality, give each item a readable identity, and state when it was seen, with a retrieval-side adapter for the one property delivery cannot supply. Each is measured against a delivery-matched control that alters only its own variable. DeliverMem leads the strongest published memory agent on MemLens at all four context lengths, and beats DMV-Bench’s own strongest method at every setting on both backbones. On MemLens it does this on a tenth to a seventieth of the input. Each decision helps only where the question lacks what it supplies, and is null elsewhere. A single fixed configuration nonetheless leads both benchmarks, without training any component or modifying the stored records.
Fig. 01Left: one MemLens question from our runs, answered with the pixels withheld and with them delivered. Right: the two quantities this paper separates, with the arrow marking the difference between them.
Fig. 02The memory pipeline, with the stage this paper isolates. MLLM is the answering model M. PreprintarXiv -
06
Median Temporal Ensembling: Training-Free Robust Aggregation for Action-Chunked Visuomotor Policies
Temporal ensembling blends the overlapping predictions of an action-chunked policy with a weighted mean, whose breakdown point is 0 — one corrupted prediction moves it without bound. Taking the coordinate-wise median of the same predictions is a one-line change that needs no retraining: never worse across 25 settings and significantly better in 15.
Yuhang Jiang (sole author)
Abstract & figures
AbstractAction-chunked visuomotor policies predict overlapping trajectories, so every executed action is covered by several predictions. Temporal ensembling smooths execution by combining these predictions with an exponentially weighted mean. One corrupted prediction can move the aggregate without bound: its breakdown point is 0. We use adversarial corruption to stress this deployed aggregator and to compare two kinds of guarantee. A metric guarantee bounds the response to a perturbation of a given size. A combinatorial guarantee instead bounds the damage when at most q of the M candidates covering a timestep are corrupted, whatever their size. Encoder adversarial fine-tuning recovers 44% of the loss under the published patch attack, but only 7.3% after the attacker’s step size is increased. By contrast, the coordinate-wise median of the same candidate set keeps its recovered fraction flat as attack optimisation increases. Median temporal ensembling costs one line and requires no retraining. Across 25 (configuration, corruption-level) combinations it is never worse than the mean and is significantly better in 15. It also transfers to a second policy class, and it recovers performance under a failure with no attacker in the loop at all: camera frames that arrive blank. Its effect on clean data is configuration-dependent, from −0.04 to +0.07. We also give the boundary: corruption that shifts every covering prediction by the same amount is invisible to this whole family of statistics, and no equivariant aggregator can remove it.
Fig. 01Metric-type robustness degrades as attack optimisation increases; combinatorial robustness persists. The vertical axis is the fraction of attack-induced loss the defence recovers, with the perturbation bound fixed throughout. The horizontal axis is optimisation effort relative to each attack’s own baseline, measured in PGD iterations for the two burst curves and in optimiser step size for the patch curves. Stars mark released-patch results; error bars are ±1 standard error, paired for the burst curves and binomial for the patch curves.
Fig. 02One episode of can at h=2, q=3, run twice from the same seed and under the same attack, changing only the aggregator: the weighted mean (top) fails and the median (bottom) succeeds. A circle marks the can in each frame; columns are equally spaced in time and a star marks a corrupted query window. Frames are a third-person render for visualisation only; the policy sees two 84×84 views. PreprintarXiv -
07
Refit the Probe: Single-Direction Ablation Is Not a Necessity Test
Ablating the direction a probe found does not remove what the probe read — a refitted probe recovers its accuracy — so neither a large task drop nor a near-zero one settles necessity.
Yuhang Jiang (sole author)
Summary & figure
SummaryProbes are routinely paired with an intervention: ablate the direction the probe found, rerun the model, and read the change in task accuracy — a large drop taken as evidence that the computation depends on what the probe read, a near-zero drop as evidence that it does not. Either inference requires that the ablation actually removed the target from the layer. It does not. A probe refitted on the ablated activations recovers its original accuracy in every cell tested, and keeps recovering when the probe's entire row space is deleted rather than a single axis, because the quantity survives in the orthogonal complement. One probe fit is enough to detect this. Replacing the ablation with iterative nullspace projection, scored against random subspaces of matched dimension, reverses the conclusion: representations that looked causally inert carry most of the task. The correction also separates where a variable is most readable from where deleting it does most damage, and those are not the same layer in any pretrained model studied. Context for the figure below: the same architecture changes its layerwise probe profile when the task changes — holding inputs and probed positions fixed and varying only the supervised target moves when a variable becomes linearly readable. An earlier version proposed a task taxonomy for this grouping; a pre-registered test on three held-out task designs did not support it, and that claim is withdrawn. The erasure here is defined by a linear probe family, so removing a nonlinearly encoded quantity remains open.
Fig. 01Task-dependent profiles: Mamba concentrates readable state late on Parity but builds it gradually on bounded-depth Dyck, and Transformer does the reverse. PreprintarXiv -
08
A Circuit, Not The Circuit: Non-Unique Causal Localisation of the Mamba-2 State Sink
A probe’s top units are routinely called the circuit. In Mamba-2 a near-disjoint head set matches their ablation effect — probing localises where state is linearly available, not where the computation is bottlenecked.
Yuhang Jiang, Bowen Zhang
Abstract & figures
AbstractMechanistic interpretability routinely reads a probe and labels its top-activating units as the circuit executing the computation. We test the move in Mamba, on the state sink: the selective state-space analogue of the Transformer attention sink, where the Delta-gate fires disproportionately on boundary tokens such as BOS and newline. At Mamba-1 channel granularity the probe’s units carry the causal effect. At Mamba-2 head granularity the label-to-locus link breaks in three ways. The causal set is not unique: a near-disjoint set of top-activation heads, sharing almost none of the specialists, matches or exceeds their ablation effect. Representation does not track function: dual heads, a quarter to over a third of all heads, respond alike to BOS and newline yet do not reproduce the specialists’ causal profile. And the effect dissociates by intervention surface: perturbing the input-dependent Delta the probe reads moves the loss by at most 0.16 nats, while suppressing the same heads’ output moves it by 1.5 to 7.3 nats. The heads still matter behaviourally, dropping needle-retrieval success on a RULER-style probe from 30/30 to 0/30 at 1024 context on both flagship checkpoints. A representational signature in a selective state-space model thus marks a behaviourally important head set without pinning down the circuit: a single probe recovers a circuit, not the circuit.
Fig. 01Size-matched BOS-specialist causal effect across granularities and architectures under gate-zero ablation: specialist−complement ΔNLL for (a) Mamba-1 channels, (b) Mamba-2 heads, and (c) Pythia attention heads. Dark bars wikitext-2, paler bars Pile-10k; labels in nats.
Fig. 02Functional decomposition across three Mamba-2 scales: specialist minus size-matched random-complement differentials for (a) the BOS-context target and (b) the newline target. Dual heads share the representational signature but not the causal profile. PreprintarXiv -
09
CL-RAG: A Closed-Loop Multimodal Retrieval-Augmented Generation Architecture for Robust Human-Robot Control Interaction
Bowen Zhang, Yuhang Jiang, Lingxiang Hu, Dun Li, Qianqian Hu*
WRC SARA 2025Oral -
10
Improved lightweight identification of agricultural diseases based on MobileNetV3
Yuhang Jiang*, Wenping Tong
CAIBDA 2022Oral -
11
Intelligent contract software for agricultural insurance compensation
Software copyright — registration 2022SR1568111
CopyrightNov 2022
Note — * indicates the corresponding author.