Recently, Tencent has introduced a new method aimed at enhancing the realism and aesthetic score of AI-generated images. According to reports, this fine-tuning technique achieves significant convergence within just 10 minutes of training using 32 H20 GPUs, with human evaluation scores increasing by more than 300%.

Although current diffusion models can optimize image quality through reward mechanisms, they face several challenges. First, the number of model optimization steps is limited, leading to a phenomenon known as "reward cheating," where the model generates low-quality images to achieve high scores. Second, the offline adjustment of the reward model is not flexible enough, limiting the ability for real-time optimization.

image.png

To address these issues, the Tencent team proposed two innovative methods. The first is called "Direct-Align," which enables the model to recover the original image from any point in time by pre-injecting noise. This method reduces gradient explosion during early backpropagation, allowing the model to be optimized throughout the entire diffusion process, not just in the final few steps.

The second innovation is "Semantic Relative Preference Optimization" (SRPO). This method transforms the reward signal into a text-controlled signal. By adding positive and negative prompt words, the model can flexibly adjust the style of generated images without requiring additional data. This means users can simply add a control phrase before the prompt word to achieve functions such as brightness adjustment or style transfer.

Experimental results show that the FLUX.1-dev model trained with SRPO has significantly improved performance in terms of realism and aesthetic quality. In a test involving 3,200 prompts, the excellent rate of the SRPO-trained model in the realism dimension increased from 8.2% to 38.9%, while the excellent rate for aesthetic quality rose from 9.8% to 40.5%. Compared to other methods, SRPO not only maintains high aesthetic quality but also produces more natural image textures.

This successful application of the technology demonstrates Tencent's further exploration in the field of AI painting and points the way for future AI image generation technologies.

Paper link: https://arxiv.org/pdf/2509.06942