Align to Misalign: Automatic LLM Jailbreak
with Meta-Optimized LLM Judges

Hamin Koo1, Minseon Kim2, Jaehyung Kim1
1Yonsei University, 2Microsoft Research
Paper arXiv Code

Abstract

Disclaimer: This paper contains potentially harmful or offensive content.
Identifying the vulnerabilities of large language models (LLMs) is crucial for improving their safety by addressing inherent weaknesses. Jailbreaks, in which adversaries bypass safeguards with crafted input prompts, play a central role in red-teaming by probing LLMs to elicit unintended or unsafe behaviors. Recent optimization-based jailbreak approaches iteratively refine attack prompts by leveraging LLMs. However, they often rely heavily on either binary attack success rate (ASR) signals, which are sparse, or manually crafted scoring templates, which introduce human bias and uncertainty in the scoring outcomes. To address these limitations, we introduce AMIS (Align to MISalign), a meta-optimization framework that jointly evolves jailbreak prompts and scoring templates through a bilevel structure. In the inner loop, prompts are refined using fine-grained and dense feedback using a fixed scoring template. In the outer loop, the template is optimized using an ASR alignment score, gradually evolving to better reflect true attack outcomes across queries. This co-optimization process yields progressively stronger jailbreak prompts and more calibrated scoring signals. Evaluations on AdvBench and JBB-Behaviors demonstrate that AMIS achieves state-of-the-art performance, including 88.0% ASR on Claude-3.5-Haiku and 100.0% ASR on Claude-4-Sonnet, outperforming existing baselines by substantial margins.

Motivation

1 LLM-Based Optimization for Jailbreak

  • Prompt Generation is an Optimization Problem: Jailbreak prompts are improved through iterative search.
  • Judge Feedback Guides the Search: An attacker LLM refines jailbreak prompts based on feedback from a judge model.
LLM-based jailbreak framework illustration

2 Limitation of ASR as an Optimization Signal

  • Binary Signal (0/1) is insufficient: ASR only indicates success or failure.
  • Sparse Feedback Provides Little Guidance: It does not reveal how close a prompt is to succeeding, making optimization inefficient.
Evaluation metric in LLM jailbreak

3 Effect of Scoring Template During Optimization

  • Highly Sensitive to Scoring Template: Optimization results vary significantly depending on the judge’s scoring template.
  • Dense Signals Help, but Design Still Matters: Richer templates improve ASR, yet performance remains highly sensitive to template design.
Effect of optimization signals

😈 AMIS: Align to MISalign

We introduce AMIS (Align to MISalign), a meta-optimization framework that jointly evolves jailbreak prompts and scoring templates to produce stronger attacks through better evaluation signals.

(a) Inner loop: Query-level prompt optimization The attacker iteratively generates jailbreak prompts guided by a fixed scoring template.

(b) Outer loop: Dataset-level scoring template optimization. The scoring template is updated using ASR alignment score from inner-loop logs.

Experiments

Table 1: Main Result on AdvBench. ASR and StR (StrongREJECT) scores with jailbreaking methods across five target LLMs. The best and second best scores are highlighted in bold and underline, respectively.

Target → Llama-3.1-8B GPT-4o-mini GPT-4o Haiku-3.5 Sonnet-4
Attacks ↓ ASR StR ASR StR ASR StR ASR StR ASR StR
Vanilla30.00.154.00.030.00.00.00.00.00.0
PAIR90.00.3082.00.2184.00.1346.00.1428.00.04
TAP98.00.3590.00.3374.00.1346.00.1322.00.07
PAP76.00.4248.00.2244.00.266.00.046.00.02
SeqAR90.00.8238.00.100.00.014.00.08.00.01
AutoDAN-Turbo84.00.6154.00.3138.00.1642.00.0538.00.04
AMIS (Ours)100.00.8498.00.87100.00.8788.00.42100.00.70

Table 2: Main Result on JBB Behaviors. ASR and StR (StrongReject) scores with jailbreaking methods across five target LLMs. The best and second best scores are highlighted in bold and underline, respectively.

Target → Llama-3.1-8B GPT-4o-mini GPT-4o Haiku-3.5 Sonnet-4
Attacks ↓ ASR StR ASR StR ASR StR ASR StR ASR StR
Vanilla41.00.193.00.092.00.071.00.043.00.05
PAIR91.00.3283.00.2477.00.2061.00.1329.00.08
TAP91.00.3980.00.2472.00.1753.00.2137.00.07
PAP97.00.2284.00.2369.00.2367.00.1620.00.09
SeqAR89.00.740.00.00.00.09.00.1216.00.15
AutoDAN-Turbo85.00.6160.00.3845.00.2833.00.1231.00.15
AMIS (Ours)100.00.95100.00.8597.00.8578.00.4888.00.67

AMIS achieves state-of-the-art jailbreak performance on both AdvBench and JBB Behaviors, reaching up to 100% ASR on multiple target LLMs while consistently outperforming six strong baselines. Across five open- and closed-source models, it significantly improves both ASR and StR, demonstrating strong robustness and transferability.

Analysis

[ Ablation studies ]


Ablation Study

Ablation results show that every component of AMIS (e.g., prompt inheritance) contributes significantly to performance, and removing any of them degrades ASR and StR. We also find that highly safety-aligned attacker models perform worse, while the AuT scoring template achieves the best overall results.

[ Transferability ]


We evaluate whether optimized jailbreak prompts transfer across different LLMs. Prompts from strongly safety-aligned models (e.g., Claude) generalize better across models, while those from weaker ones transfer poorly, suggesting that stronger safety alignment improves cross-model robustness.

Transferability

BibTeX

          @article{
            koo2025align,
            title={Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges},
            author={Koo, Hamin and Kim, Minseon and Kim, Jaehyung},
            journal={International Conference on Learning Representations (ICLR)},
            year={2026}
          }