廖振, 林国军, 胡鑫, 游松, 兰江海, 周旭, 罗春兰. 改进CycleGAN的素描头像到现实头像转换[J]. 内江师范学院学报, 2024, 39(4): 65-71. DOI: 10.13603/j.cnki.51-1621/z.2024.04.011
    引用本文: 廖振, 林国军, 胡鑫, 游松, 兰江海, 周旭, 罗春兰. 改进CycleGAN的素描头像到现实头像转换[J]. 内江师范学院学报, 2024, 39(4): 65-71. DOI: 10.13603/j.cnki.51-1621/z.2024.04.011
    LIAO Zhen, LIN Guojun, HU Xin, YOU Song, LAN Jianghai, ZHOU Xu, LUO Chunlan. Improved sketch-to-realistic avatar conversion with CycleGAN[J]. Journal of Neijiang Normal University, 2024, 39(4): 65-71. DOI: 10.13603/j.cnki.51-1621/z.2024.04.011
    Citation: LIAO Zhen, LIN Guojun, HU Xin, YOU Song, LAN Jianghai, ZHOU Xu, LUO Chunlan. Improved sketch-to-realistic avatar conversion with CycleGAN[J]. Journal of Neijiang Normal University, 2024, 39(4): 65-71. DOI: 10.13603/j.cnki.51-1621/z.2024.04.011

    改进CycleGAN的素描头像到现实头像转换

    Improved sketch-to-realistic avatar conversion with CycleGAN

    • 摘要: 当前,由素描头像转换的现实头像还存在不够逼真和人脸识别率不高的问题.为此,提出了一种改进CycleGAN的现实头像转换方法.首先,在U-Net自编码器的基础上增加了一个人脸特征提取器.其次,将人脸特征提取器提取的特征与U-Net解码器中的特征采用通道连接的方式进行特征融合,再对融合后的特征做进一步解码处理.最后,将基础模型CycleGAN转化为监督学习模型,从而对转换头像与真实头像添加图像空间损失和风格损失.实验结果表明:改进模型较基础模型转换的现实头像,在CUHK测试集上FID值降低了27.31、Rank-1提高了19 %,在XM2VTS测试集上FID值降低了8.65、Rank-1提高了4.1 %.

       

      Abstract: Currently, the realistic avatars converted from sketch avatars has the problems of insufficient lifelikeness and low face recognition rate. In response to this, a realistic avatar conversion method with improved CycleGAN is proposed. Firstly, a face feature extractor is added to the U-Net self-encoder. Secondly, the features extracted by the face feature extractor and the features in the U-Net decoder are fused using the channel connection method, and the fused features are further decoded. Finally, the base model CycleGAN is transformed into a supervised learning model so as to add image spatial loss and style loss to the converted avatar and the real avatar. The experimental results show that the improved model reduces the FID value by 27.31 and improves Rank-1 by 19 % on the CUHK test set, and reduces the FID value by 8.65 and improves Rank-1 by 4.1 % on the XM2VTS test set, compared with the results collected from the converted real avatar of the base model.

       

    /

    返回文章
    返回