Adjustable Method Based on Body Parts for Improving the Accuracy of 3D Reconstruction in Visually Important Body Parts from Silhouettes

Abstract

This research proposes a novel adjustable algorithm for reconstructing 3D body shapes from front and side silhouettes. Most recent silhouette-based approaches use a deep neural network trained by silhouettes and key points to estimate the shape parameters but cannot accurately fit the model to the body contours and consequently are struggling to cover detailed body geometry, especially in the torso. In addition, in most of these cases, body parts have the same accuracy priority, making the optimization harder and avoiding reaching the optimum possible result in essential body parts, like the torso, which is visually important in most applications, such as virtual garment fitting. In the proposed method, we can adjust the expected accuracy for each body part based on our purpose by assigning coefficients for the distance of each body part between the projected 3D body and 2D silhouettes. To measure this distance, we first recognize the correspondent body parts using body segmentation in both views. Then, we align individual body parts by 2D rigid registration and match them using pairwise matching. The objective function tries to minimize the distance cost for the individual body parts in both views based on distances and coefficients by optimizing the statistical model parameters. We also handle the slight variation in the degree of arms and limbs by matching the pose. We evaluate the proposed method with synthetic body meshes from the normalized S-SCAPE. The result shows that the algorithm can more accurately reconstruct visually important body parts with high coefficients.