The classical portrait can be changed into 3D in seconds, and the perspective can be cut at will. It is jointly produced by Huawei and Shanghai Jiaotong

如果Gan生成的真实肖像变成3D版本呢？

What if the realistic portrait generated by Gan becomes a 3D version?

此外，这些效果是由静态单视图图片生成的！

Moreover, these effects are generated by static single view pictures!

它甚至可以使卡通人物立体化。

It can even make cartoon figures three-dimensional.

它是一种基于GaN的三维感知发生器，仅使用原始单视图图像，无需任何上采样×256清晰图像，即可生成256分辨率的图像。

It is a GaN based 3D perception generator, which can generate 256 resolution images only using the original single view image without any up sampling × 256 clear images.

镜像对称

Mirror symmetry

在高清晰度人脸数据集ffhq上，cips-3d的性能可以说是非常好的。即使是古典绘画也可以变成三维版本。

On the high-definition face data set ffhq, the performance of cips-3d can be said to be very nice. Even classical paintings can be turned into three-dimensional versions.

它只需要输入少量的静态图片，就可以从多个角度实现逼真的3D效果。

It only needs to input a small number of static pictures to achieve realistic 3D effects from multiple perspectives.

该cips-3d也是基于nerf开发的。

This cips-3d is also developed based on nerf.

它主要使用两个网络：nerf和INR（隐式神经表示）。

It mainly uses two networks: nerf and INR (implicit neural representations).

为了获得更精确的3D图像，通常需要在每个灯光上采样多个点，这导致nerf需要大量内存。

In order to obtain more accurate 3D images, it is often necessary to sample multiple points on each light, which leads to the large memory required by nerf.

因此，如果在神经网络中仅使用nerf，则网络的深度将受到限制，从而导致图像模糊和缺乏细节。

Therefore, if only nerf is used in the neural network, the depth of the network will be limited, resulting in blurred images and lack of details.

在研究过程中，工作人员发现cips-3d仍然存在镜像对称问题。

During the research, the staff found that cips-3d still had the problem of mirror symmetry.

这种现象存在于许多3D Gan中，例如长颈鹿和stylenerf。

This phenomenon exists in many 3D Gan, such as giraffe and stylenerf.

例如，在以下情况下，初始单角度图像的条纹被保留，但在生成的具有不同角度的图像中，条纹将随着透视的变化而变化，就像镜像一样。

For example, in the following case, the fringe of the initial single angle image is left, but in the generated images with different angles, the fringe will change with the change of perspective, just like a mirror image.

例如，图中a和C的坐标完全镜像对称。

For example, the coordinates of a and C in the figure are completely mirror symmetrical.

这对于生成完全对称的对象没有问题。

This is no problem for generating perfectly symmetrical objects.

但如果把它放在只有一个侧面角度的肖像上，那可能是一场灾难。

But it may be a disaster to put it on a portrait with only a side angle.

为此，研究人员在神经网络中添加了一个鉴别器，以帮助识别这个问题。

To this end, researchers added a discriminator to the neural network to assist in identifying this problem.

最后，结果表明，与其他生成三维人像的方法相比，FID和kid值显著降低。这两个值越低表示图像质量越好。

Finally, the results show that compared with other methods that can generate 3D portrait, the FID and kid values are significantly reduced. The lower these two values mean the better image quality.

谢灵曦，学士、博士，毕业于清华大学计算机视觉与自动机器学习专业。他目前是华为的高级研究员。

Xie Lingxi, bachelor and doctor, graduated from Tsinghua University, majoring in computer vision and automatic machine learning. He is currently a senior researcher of Huawei.

Link：https://new.qq.com/omn/20211030/20211030A0876400.html

update time：2021-10-31 18:57:54