科研之友
微信
新浪微博
Facebook
分享链接
摘要
With the rapid development of multimedia and digital technology, people now are able to obtain a large number of high-quality images or photos easily with the newest devices like digital camera or smartphone. Dressed people are always the focus in images or photos, and people are desired to acquire data with higher reality than two-dimensional data such as image. As a consequence, numerous methods attempt to reconstruct human body and outer garments based on a single image or photo. However, plenty of recent image-based methods reconstruct human body and garments separately without considering the interaction between them, which leads to mutual interpenetration. This paper purposes a method to reconstruct human body as well as outer garments from a single image based on 2D collision detection. It separately utilizes SMPL and TailorNet as body and garment parametric models, and then establishes an energy to jointly optimize the shape and pose parameters of the human model as well as the style parameters of the garment models. Our method starts with an preprocessing stage towards the input image which includes semantic segmentation of all garments and 2D joints estimation of the human body. After preprocessing stage, we employ human mesh recovery with the input image to estimate the shape and pose parameters of the human body as the initial values of the optimization. Although human mesh recovery is able to estimate the shape and pose parameters of SMPL based on the input image, the reconstructed 3D human body suffers the inaccurate shape and pose because of the ambiguity between the human body and the garments in the image. Hence, it is necessary to conduct a further optimization to estimate the shape and pose parameters of human body as well as the unknown style parameters of the outer garments with higher accuracy. Our optimization energy consists of two parts: The first is the shape and pose constraint, which penalizes the difference of the 2D joint positions and the region of dressed person between the image and the projection of the 3D parametric models; The other is a collision constraint between human body and garments, which introduces an error measurement of 2D projection areas between human body and outer garments to prevent interpenetration; In addition, considering that projection-based constraint is sensitive to viewpoints, we take a further step to sample more viewpoints to project the 3D models onto 2D spaces so as to reinforce the 2D collision constraint. All constraints mentioned above are unified into one optimization framework in order to take all factors into account. Considering the difficulty of computing the gradients of TailorNet due to the complicated network structure, we employ a global optimization algorithm named the hill-climbing algorithm to alternatively optimize shape, pose and style parameters in the energy. In the end, we conduct a variety of experiments to compare our results with those of state-of-the-art methods in both qualitative and quantitative analysis, which shows that the proposed approach can effectively alleviates penetration between body and garments, and achieve higher accuracy. ? 2023 Science Press.
- 单位
