1 / 73

Distributed Video System realized on mobile device with efficient Feedback channel

Distributed Video System realized on mobile device with efficient Feedback channel. 分散 式 影像編碼在手機上的實現與有效率的回饋通道. Chen, chun-yuan 陳群元 Advisor: Prof. Wu, Ja -Ling 吳家麟 教授 2012/6/28. Outline. Motivation and introduction DVC architecture overview DVC architecture with mobile device

dirk
Télécharger la présentation

Distributed Video System realized on mobile device with efficient Feedback channel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率的回饋通道 Chen, chun-yuan陳群元 Advisor: Prof. Wu, Ja-Ling 吳家麟 教授 2012/6/28

  2. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Experiment Result • Conclusion • Future work

  3. Motivation • Video communication is essential in Mobile device • Decrease the coding time and battery consuming on mobile.

  4. Introduction • For Mobile device video codec • Conventional video coding (ex:MPEG-4 H.264) • Heavy weight encoder, light weight decoder. • Distributed Video Coding • Light weight encoder, heavy weight decoder.

  5. Conventional video codec Decoding Encoding

  6. DVC to H.264 Transcoder Clouding server: DVC to H.264 transcoder Feedback channel Parity bits Encoded sequence H.264 decoder DVC encoder

  7. DVC codec baseline: • DISPAC+DVC transcoder 1.Encoder no feedback channel ! 2.Decoder Tse-Chung Su. Yun-Chung Shen. and Ja-Ling Wu. 2011. Real-time Decoding for LDPC Based Distributed Video Coding. National Taiwan University

  8. Time complexity analysis without feedback channel • DVC Decoding without Feedback channel • Side-Information generation • LDPCA Total decoding time: 17.10 sec 31%

  9. Time complexity analysis with feedback channel • DVC Decoding • Side-Information generation • Send-Request at Feedback channel • LDPCA Total decoding time: 446.83 sec 1% 96% Bottle neck!

  10. Outline • Motivation and introduction • DVC architecture Overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Experiment Result • Conclusion • Future work

  11. Distributed Video Codec Decoded WZ Frames Key Frame Key Frame WZ Frames CRC-8 Mode selection CRC check CRC IDCT& Deblocking filter DCT Feedback channel LDPCA Encoder LDPCA Decoder WZ bitstream level Quantizer SI Reconstruction Buffer Bitplanes WZ Frame WZ Frame WZ Frame Correlation Noise Modeling DCT SI generation& refinement GOP size 4 Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  12. LDPC decodingSum-Product Algorithm(Message Passing) Side Information (real number) +  0 -  1 decode output hard decision a25 b25 c25 d25 e25 f25 g25 a1 b1 c1 d1 e1 f1 g1 a b c d e f g 4 6 7 1 2 3 5 Horizontal processing Vertical processing 甲 乙 丙 0 1 1 From DVC encoder (syndrome bits) Kschischang, F.R., Frey, B.J., and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm.IEEE Trans. Inform. Theory

  13. LDPC Accumulate (LDPCA) codes Rate adaptivity D. Varodayan et al., "Rate-adaptive codes for distributed source coding," EURASIP Signal Processing Journal, Special Section on Distributed Source Coding, 2006

  14. 65 LDPC codes

  15. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Experiment Result • Conclusion • Future work

  16. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  17. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  18. Key frame encoding on Mobile • Previous ver. use JM9.5 • Transcoder ref x264 on PC Keyframe encoding Cross compiler x264 static library x264 lib

  19. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  20. WZ coding with NDK Jni Makefile Native code DVC encoder C/C++ code with NDK rule

  21. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  22. Transcode to mobile codec • Because DVC only encode luminance part in YUV, so we got only Y for output. • Our mobile device can only read mp4 and 3gp. Y G R B Frame 1 Frame 2 Frame 3 Frame4 Frame 5 Frame 6 Frame 7 U V .mp4 Frame n

  23. Distributed Video Codec with mobile Decoded WZ Frames CRC-8 Decoded Video sequence With specific form Transcode level Quantizer Bitplanes CRC check CRC Video deliver IDCT& Deblocking filter Feedback channel DCT LDPCA Encoder LDPCA Decoder WZ bitstream SI Reconstruction Buffer Mode selection Correlation Noise Modeling DCT WZ Frames SI generation& refinement Conventional Intraframe Encoder Conventional Intraframe Encoder Key Frames Decoded Key Frames

  24. Network connection • Create network connection to connect remote server. • TCP socketin C Clouding server: DVC to H.264 transcoder

  25. ACK at Feedback channel TCP socket syndromes syndromes syndromes syndromes … Send syndrome Request

  26. Decoding complexity onFeedback channel • Packet header occupy large part of network bandwidth. • 48 bits syndromes vs. 24 bytes header • Network latency in communication at Feedback channel • About 20 thousands times requests for 10 sec sequence.

  27. Outline • Motivation and introduction • DVCarchitecture with mobiledevice • DVC architecture overview • Propose an efficient feedback channel • Experiment Result • Conclusion • Future work

  28. Correlation with LDPCA Residual SI quality

  29. 統計關係圖

  30. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

  31. Average number of requests per bitplane AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  32. Estimate the syndromes size per WZ frame • Estimate syndromes size by correspond WZ frame in previous GOP • Ex. GOP 4 WZ frame WZ frame WZ frame WZ frame Key frame Key frame WZ frame WZ frame Key frame Second GOP First GOP

  33. Efficient feedback channel • we propose two methods to decrease time consuming at feedback channel. • Estimate the syndromes size per WZ frame • Estimate the syndromes size per bitplane

  34. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  35. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  36. Number of requests per bitplane per band AC11 AC13 AC9 AC10 AC12 AC14 AC6 AC7 AC8 AC1 AC2 AC4 AC5 DC AC3 Bitplane Number

  37. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  38. Test condition • 12 CPU, 24 processor • Intel(R) Xeon(R) CPU X5650 @ 2.67GHz • GPU: Tesla M2050 • Mobile device: HTC sensation

  39. Test materials Motion: High Low Soccer Foreman Coastguard Hall Monitor • Test sequences : • QCIF, 15Hz, all frames • GOP Size 2, 4 and 8 • Only luminance component is used

  40. Decoding time

  41. Request amount

  42. Speed up ratio

  43. Bitrate

  44. Bitrate

  45. RD(Foreman)

  46. RD(hall)

  47. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Transcoding • Experiment Result • Conclusion • Future work

  48. Conclusion • We combine DISPAC and transcoder, then port it’s encoder on mobile. • Implement and propose Feedback channel which the previous work doesn’t discuss or improve. • Record video with mobile device’s camera. • Transcode the decoded sequence to mobile codec.

  49. Outline • Motivation and introduction • DVC architecture overview • DVCarchitecture with mobiledevice • Propose an efficient feedback channel • Experiment Result • Conclusion • Future work

  50. Future Work • Take a more precise estimation of syndrome size. • Use motion information to estimate the syndrome size. • Build a reliable DVC codec which is loss tolerant.

More Related