ClothMotion: Pose-Guided Temporal Consistent Garment Mask Generation

Abstract

With the advancement of diffusion models and Transformer-based architectures, pose-guided human image animation has achieved remarkable progress. However, existing methods often treat garments as static textures attached to the body, ignoring the non-rigid nature of clothing deformation during motion. This modeling limitation leads to unrealistic artifacts such as skirt misalignment and sleeve discontinuity, especially in the presence of loose or dynamic garments. To address this issue, ClothMotion is introduced as an appearance flow-based approach that explicitly models the evolution of garment contours driven by pose changes. The method employs a multi-scale flow estimation framework to predict dense non-rigid displacement fields, which deform a reference garment mask according to the target pose sequence. The resulting warped mask serves as a strong geometric prior for a pose-aware decoder, enabling temporally consistent and spatially coherent segmentation across frames. To support training, a dynamic garment segmentation dataset is constructed. Pseudo-labels are generated on TikTok and UBC-Fashion videos using a DeepFashion2-pretrained detector. A global majority voting strategy and manual correction are applied to improve annotation consistency and accuracy. Experimental results show that ClothMotion significantly improves mask accuracy and garment stability, achieving 0.85 mIoU on real-world sequences. When integrated into existing animation frameworks such as DisCo, the method further enhances visual quality and structural consistency. These results highlight the importance of explicit contour modeling in pose-guided image animation and demonstrate improved controllability over dynamic garment behavior, offering a generalizable path toward realistic clothing synthesis.