变脸技术

1. 变脸技术 Deepfake Speaker Li Yubin

2. What's the Deepfake? • Generating fake videos(changing face) Change face and become Yang Mi Zhu Yin’s Huang Rong

3. Application Area • Filming • Changing the weather and time in the video

4. What's the Deepfake? • Implementation process

5. Face Conversion • Training a neural network with supervised learning to restore Zhang's distorted face to the original face, and expecting this network to have the ability to restore any face to the face of Zhang San.(Self-coding model). • formula:

6. Deepfake's process Deepfake's entire process consists of three steps: • extract data • training • conversion The first and third steps need to use data preprocessing, and the third step also uses image fusion technology. So I am mainly technically divided into three aspects: • image preprocessing • network model • image fusion

7. Deepfake's process image preprocessing: • Identify from the big picture (or video) and pull out the face image. • The original version uses the face recognition library in dlib (this recognition module can be replaced). • This library can not only locate the face, but also give people The coordinates of the 36 key points of the face, according to these coordinates can calculate the angle of the face, and finally the face that is pulled out is the face after the square.

8. Deepfake's process image preprocessing:

9. Deepfake's process network model The whole network is not complicated, just convolution plus full connection, coding -> decoding

10. Deepfake's process image fusion After the face is replaced, the following problems occur: • Skin color differences, even for the same kind of people, will have subtle differences. • Light differences, the lighting environment of each photo is different • False face boundary is obvious The first two causes the objective difference, and the second is related to the size of the data set.As for the last point, it is caused by the former two.

11. Deepfake's process image fusion • As for false face boundary is obvious,it can be alleviated by reducing the resolution. This is also the reason why many online video fake face boundaries are not obvious, because there is rarely a face that accounts for 80% of the screen. But if you use it directly on the 256x256 avatar, the boundary effect is obvious.

12. Deepfake's process image fusion As for the first two,there are two methods to reduce: • Smooth_mask: The boundary is processed directly with Gaussian blur, making the transition look natural. • Adjust_avg_color: Assuming that the A picture is to be replaced by the B picture, then the difference between the A picture plus the mean of each pixel of the B picture is considered as a color reconciliation.

13. Deepfake's process image fusion(Poisson fusion) In addition, deepfake gives a processing based on the Poisson fusion algorithm. • The general idea of Poisson fusion provides a mask matrix with a background area of 0. The region of the ROI region (region of insteresing, here is the same as ROI) is 255. The algorithm knows which part is fused through the matrix. Part, then by calculating the gradient, using the gradient field as an indication, modify the pixel value of the ROI, making the boundary more relevant to the original image.

14. Deepfake's process image fusion(Poisson fusion)

15. Deepfake's process image fusion(Poisson fusion) Poisson Fusion in Deepfake can choose between two modes. • One is bounded by a rectangular frame of a human face. • And the other is bounded by a feature point of a person (a face boundary and an eye boundary). Original After Possion Fusion

16. Combating Deepfake Videos Deepfake can easily lead to bad phenomena： • Fake news • Fake videos • And even some people use it in A film production

17. Combating Deepfake Videos • Blink detection

18. DOWNLOADSathttp://vcc.szu.edu.cn Thank You!