1. Better supervision for critics
FIRM-Edit scores editing with execution and consistency, while FIRM-Gen focuses on instruction following with explicit scoring plans. This yields cleaner reward data and more reliable critic behavior.
FIRMReward Series Project Page
Reliable Reward Modeling for Visual Generation
For Image Generation & Edit
Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
FIRM uses a difference-first editing pipeline and a plan-then-score generation pipeline to reduce critic hallucination and improve reward quality.
Abstract
Accurate critics are the bottleneck in RL for image editing and generation.
Reinforcement learning has become a promising tool for improving image editing and text-to-image generation, but current reward models often hallucinate, miss details, and assign noisy scores that misguide optimization. FIRM addresses this problem with tailored data curation pipelines, specialized reward models, a human-annotated evaluation benchmark, and reward formulations that better balance competing goals. The framework produces FIRM-Edit-8B and FIRM-Gen-8B, then uses them to guide RL for faithful editing and instruction-aligned generation.
Method
One framework, three connected pieces.
2. Dedicated evaluation benchmark
FIRM-Bench measures critic alignment with human judgment for both editing and generation, using balanced score distributions and controlled prompt difficulty.
3. Reward shaping for RL
CME for editing and QMA for generation use a Base-and-Bonus design, preventing easy reward hacking shortcuts and improving credit assignment during optimization.
Editing pipeline
Instead of asking a model to directly judge edited images end-to-end, FIRM first describes the visual differences between source and edited images, then scores execution and consistency from that structured evidence.
Generation pipeline
For text-to-image prompts, an LLM first expands the prompt into a checklist, and a multimodal evaluator scores the generated image against this plan dimension by dimension.
Reward design
CME makes execution a prerequisite for high editing reward, while QMA strengthens alignment with an explicit quality term to suppress low-quality but superficially compliant generations.
Reward formulas
CME
R = Execution * (0.6 + 0.4 * Consistency)
QMA
R = InsFollowing * (0.4 + 0.6 * Quality)
Both formulations are designed to stop the policy from maximizing the easiest sub-score while ignoring the real task objective.
Results
FIRM improves both critic quality and downstream RL performance.
Editing guided by FIRM-Edit-8B
The paper highlights stronger edit execution while preserving task-irrelevant content, addressing the common failure mode of returning an almost unchanged image.
Generation guided by FIRM-Gen-8B
Structured scoring plans improve instruction following, especially on prompts with multiple entities, styles, and spatial constraints.
FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling
Abstract
Reliable supervision is the bottleneck in text-to-video reward modeling.
Existing holistic video judges often miss prompt-specific details, provide unfaithful rationales, or penalize the same issue across multiple dimensions. FIRM-Video replaces open-ended judging with dimension-specific checklists grounded in temporal visual evidence. It evaluates Instruction Following, World Coherence, and Perceptual Quality, then transforms verified criteria and scores into high-quality supervision for efficient end-to-end reward models.
Method
One principle, three dimension-specific checklists.
1. Instruction Following
Decomposes prompts into weighted atomic requirements and verifies each requirement against temporal visual evidence before scoring.
2. World Coherence
Grounds visible entities and actions, then applies prompt-calibrated checks for physical and temporal violations without penalizing intended fantasy.
3. Perceptual Quality
Uses a stable taxonomy of visual defects, including blur, flicker, color instability, compression artifacts, and local distortions.
Construct the checklist
Each evaluation dimension receives a checklist tailored to its semantics: prompt requirements for IF, grounded entities and actions for WC, and a generic defect taxonomy for PQ.
Verify temporal evidence
Every criterion is inspected against the video rather than inferred from an unconstrained holistic rationale.
Aggregate verified decisions
Only verified criterion-level decisions contribute to the final score and the natural-language supervision used to train FIRM-Video-8B.
FIRM-Video resources
88,044
dimension-specific instances from 29,348 videos
3,012
unique text-to-video prompts
750
expert annotations across 250 benchmark videos
Results
FIRM-Video improves human alignment and Best-of-N selection.
Reliable absolute scores and candidate rankings
The Qwen3-VL-based FIRM-Video-8B reduces overall MAE on FIRM-Video-Bench from 1.33 to 0.78, achieving the best overall MAE among the evaluated proprietary and open-source models.
In Best-of-8 sampling, it achieves the highest VBench Total, Quality, and Semantic Scores across LaVie-Base, CogVideoX-2B, and Wan2.1-T2V-1.3B.
BibTeX
@misc{zhao2026trustcriticrobustreward,
title={Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation},
author={Xiangyu Zhao and Peiyuan Zhang and Junming Lin and Tianhao Liang and Yuchen Duan and Shengyuan Ding and Changyao Tian and Yuhang Zang and Junchi Yan and Xue Yang},
year={2026},
eprint={2603.12247},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2603.12247}
}
@misc{zhang2026firmvideocheckscorereliable,
title={FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling},
author={Peiyuan Zhang and Xiangyu Zhao and Hongbo Liu and Xiaoxing Hu and Mingxin Liu and Shuran Ma and Yunhang Shen and Jian Hu and Haihan Gao and Haoyu Cao and Xue Yang},
year={2026},
eprint={2608.21839},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2608.21839}
}