Adversarial Robustness · Transfer Attacks · Evaluation

The Scissors Effect

When Resize-Based Input Diversity Helps or Hurts Transfer Attacks

Yuhang Jiang1, Xiaojing Chen2

1University of Trento  ·  2Anhui University

Accepted at TMLR 2026
Paper OpenReview Code BibTeX

Same operation. Opposite sign.

Input Diversity applies a random resize and pad at every attack iteration, and it is enabled by default in most transfer attacks. Hold the attack and the targets fixed, change only the surrogate, and raising the DI probability helps standard surrogates while it hurts robust ones.

1.0 ImageNetCIFAR-10

Turn robustness up. Watch the sign flip.

Maybe two arbitrary models were compared? We fix the architecture (ResNet-50) and the recipe (PGD-AT) and vary only the adversarial-training budget. The sign flips between the first two settings, and the harm is graded rather than binary.

0
MI-FGSM (no DI)
41.2
transfer ASR
DI-FGSM
54.2
transfer ASR
DI effect
+13.0
pp · helps

Tab. 7 · target Swin-B, N=500, 3 seeds, all-images basis. Effects in pp.

It is the resize, not the diversity.

DI is a resize composed with a translation. A 2×2 factorial separates them. On robust surrogates the resize main effect is about four times the translation effect, and the interaction is the one term whose interval contains zero, so the two factors do add and the resize carries the harm.

Robust surrogatesStandard surrogates

Tab. 12 · 6 sources × 3 targets, N=1000, 5 seeds, p=1, all-images basis, r > 1. Bars are point estimates; thin lines are 95% image-paired bootstrap intervals.

Noisy gradients gain from averaging. Smooth ones only pay for it.

DI averages the gradient over transformed views, trading a bias against a variance reduction. We measured both directly. The displacement is comparable across the two groups; the variance reduction is not.

What the gradients look like

A standard surrogate's input gradient is scattered and high-frequency. A robust one is spatially consistent and tracks the object. That difference is what the averaging acts on.

ImageNet input gradients: original image, standard ResNet-50, robust Engstrom

Fig. 2 (ImageNet) · Columns: the original image, the standard ResNet-50 gradient (LGC 0.64), and the robust Engstrom gradient (LGC 0.98).

Variance of the DI-averaged direction

Relative to a single clean gradient. Below 1 means DI reduced the variance.

Tab. 14 · m=20 EOT draws, image-bootstrap intervals. All six standard ratios lie entirely below 1 and eight of the nine robust ones entirely above.

Local Gradient Consistency

How stable a surrogate's gradient is under a tiny input perturbation. Measurable before any attack is run.

ModelHF ratioLGC

Tab. 9 · N=200 images, K=5 probes at 1/255. HF is a ratio of log-magnitudes, not of energies.

Should you turn DI on?

Probe the surrogate's LGC with five local gradient queries, compare against a threshold fixed in advance, and switch. The threshold, the probe scale and the sample count were frozen at their published values, and the per-surrogate predictions were hashed before any held-out attack was run.

0.64
τ = 0.92

Sec. 5 · The binary choice costs 0.95 pp [0.50, 1.42] against the best grid value on the one standard surrogate where the two differ, and zero on both robust ones. The curves are monotone in p, so there is no interior optimum to miss.

Held out: seven surrogates it had never seen

Five standard architectures and two adversarial-training recipes not in the development set. The frozen rule matched the sign on all seven; six of the seven effects are individually resolved.

Held-out surrogateLGCrulemeasured D (pp)95% CI

Tab. 19 · N=1000, 5 seeds, r=0.9, source-and-target clean-correct basis.

Ten attacks, 2018–2024.

DI degrades all ten on the robust surrogate and helps nine of ten on the standard one. These attacks share images, models and algorithmic ancestry, so this is a dependent panel and not ten independent observations. The tally is a description; the pooled statistics below are what carry the weight.

Attackvenuerobust sourcestandard source

Tab. 5 · Engstrom / ResNet50 → Swin-B, N=1000, 5 seeds, all-images basis. D in pp.

With the dependence taken into account
Robust, Swin-B
−3.92
[−4.81, −3.04] pp
Standard, Swin-B
+0.83
[+0.27, +1.40] pp
Robust, ConvNeXt-B
−6.25
[−7.30, −5.23] pp
Standard, ConvNeXt-B
+1.38
[+0.68, +2.06] pp

Tab. 6 · The mean pairwise correlation of the per-image effect vectors is 0.12–0.15, giving an effective 4.2–4.9 independent attacks rather than ten. An image-block sign-flip permutation test gives p < 1e−4 on three of the four cells and p = 0.003 on the fourth.

What we did not establish.

The effect is the claim we stand behind. The mechanism is a bounded candidate and the rule is preliminary. These are the places where each runs out.

  • Scale. The effect is strong on ImageNet and several times smaller at 32×32. On CIFAR-100, registered in advance as a scope test, it is within 1.5 pp of zero and the direction does not hold.
  • The theory does not fit the measurement. Modelling the DI gradient as a fixed linear operator applied to the clean gradient leaves a residual of 1.04–1.28 relative to the gradient norm. The crossover theorem shows a reversal is possible under a simplified model; it does not describe the measured gradients.
  • The per-surrogate theoretical threshold failed. Estimated from measured constants and scored on seven held-out surrogates, it is correct on 2 of 7. We report the constants and offer no per-surrogate predictor.
  • LGC is a regime classifier, not a within-regime predictor. It saturates across the robust region while the harm still varies several-fold. Its threshold does not carry to CIFAR-10, where it sits inside the standard cluster and misroutes two of four.
  • A corner case. Salman eps=8/255 ResNet-50 has LGC 0.80 and is routed as standard, on a surrogate DI harms. Its gradient SNR is 0.4, in the range of the standard models rather than of the other robust ones.
  • Provenance is a strong baseline. Over 27 model–dataset cases the probe and a training-metadata rule are each correct on 21. We do not claim the probe beats knowing how the surrogate was trained.

BibTeX

@misc{jiang2026scissors,
  title         = {The Scissors Effect: When Resize-Based Input Diversity Helps or Hurts Transfer Attacks},
  author        = {Yuhang Jiang and Xiaojing Chen},
  year          = {2026},
  eprint        = {2606.22516},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2606.22516}
}

Every number on this page is transcribed from the accepted manuscript and labelled with the table it comes from. A difference between two attack success rates is reported in percentage points (pp); % is used for rates, image fractions and relative changes.