Install the dependencies using the following command. Install the downloaded Anaconda-3 package using the following command.Įnable the conda directory to run further commands, %cd content/neuralbody/Īnd activate the environment and NeuralBody by providing the following commands inside the inner base mode command cell as shown below. Verify the downloaded contents by exploring the directory.Ĭhange the current directory to refer content/neuralbody/ by providing the line-magic command.ĭownload the Anaconda-3 package using the following command, if the local machine does not have a conda environment. The following command downloads source code to the local machine. The following commands install PyTorch 1.4.0 compatible with CUDA 10.0. Neural Body requires Python 3.6+, CUDA 10.0, PyTorch 1.4.0 and a GPU runtime. Neural Body on Novel view synthesis and 3D reconstruction ( Source) Python Implementation This model enables quick inference on 3D reconstruction and novel view synthesis. By anchoring the latent representations to this SMPL model, a dynamic mesh of the human body is developed. For this, the famous SMPL (Skinned Multi-Person Linear Model) is employed that is governed by the shape parameters and the pose parameters. Once latent code is obtained for any inference pose, they are fed into feed-forward networks for colour and density regression.ĭeformable mesh is designed by connecting its vertices with structured latent codes. Latent code for any inference 3D point can be obtained by performing trilinear interpolation of the neighbour points in the latent code volume. Thus 3D space representation is enabled from the input data. While training the Neural Body framework, the structured latent codes are fed as input into a sparse convolutional neural network ( SparseConvNet) that outputs a latent code volume. Neural Body generates different implicit 3D representations of a human body based on the input poses from a common structured latent code anchored to a deformable mesh ( Source). Moreover, this framework needs no pre-trained networks to learn the representations. Neural Body synthesizes photorealistic novel views of a human performer in complex motions and varying illustrations from sparse multi-view video frames. The deformable mesh can be deformed to any possible human position based on the input pose. Thus, the sparse capturing can be integrated to form a continuous 3D view representation. This approach assumes that the learnt implicit neural representation among different sparse camera capturing frames share the same structured latent space representation code set anchored to a deformable mesh. Neural Body performs 3D reconstruction and Novel view synthesis from a sparse multi-view video captured with limited RGB cameras ( Source). To this end, Sida Peng, Yuanqing Zhang, Qing Shuai, Hujun Bao and Xiaowei Zhou of Zhejiang University, Yinghao Xu of The Chinese University of Hong Kong, and Qianqian Wang of Cornell University introduced a powerful approach named Neural Body that employs sparse cameras to capture the poses of dynamic human body and renders high-quality 3D view as well as 3D scene of the original human body. An approach to novel view synthesis with limited number of cameras or sensors has become a need nowadays. This causes ill-posed representation learning of views and thus results in poor view rendering. But, the reduced number of cameras causes sparsity in the view continuity. ![]() These dense camera requirements may be overcome by employing a relatively lesser number of cameras or sensors. High hardware requirements make the system highly expensive or impossible to establish due to spatial constraints and strict-configurations requirements. View synthesis requires either a dense array of cameras to capture the object from different views and orientations or a few high-definition depth sensors. However, the major hindrance in these view synthesis approaches is the hardware complexity. Present view synthesis methods employ either image-based rendering or implicit neural representation to develop the 3D view. Human body view synthesis is one of the challenging problems, especially the human body, which is in motion. Novel view synthesis is the process of generating a 3D view, or sometimes 3D scene, with the available 2D images captured from different poses, orientations and illuminations. Novel view synthesis finds interesting applications in movie production, sports broadcasting and telepresence.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |