1 / 12

Final Project: Video Transcoding on Cloud Environments

CMPT 880. Final Project: Video Transcoding on Cloud Environments. Queenie Wong. Introduction. Cloud computing technology has become mature and accessible to the public Many complex and high computational operations can be distributed and processed on cloud environments

ipo
Télécharger la présentation

Final Project: Video Transcoding on Cloud Environments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMPT 880 Final Project:Video Transcoding on Cloud Environments Queenie Wong

  2. Introduction • Cloud computing technology has become mature and accessible to the public • Many complex and high computational operations can be distributed and processed on cloud environments • Complex video transcoding operations can be distributed to available nodes for paralleling processing • The encoding time of a MJEPG file was reduced from 7.5 hours to 2 minutes by scaling up to 6 nodes Queenie Wong

  3. Problem Statements • Excess Key-frame problem: Inappropriate splitting position on the original video file can create excess key-frames in the final output • How to perform video transcoding on Hadoop • Do not have native video transcoding library • Find out the right level of parallelism and split size for maps tasks • Too small split size: synchronization overheads • Too large split size: lack of dynamic load balancing Queenie Wong

  4. Proposed Solutions • Excess Key-frame problem: • Solution: MKVmerge program • Video transcoding on Hadoop • Solution: FFmpegtranscoding tool • The right level of parallelism and split size for maps tasks • Solutions: • Performance indicator to measure the efficiency of parallelism level • Video Transcoding Performance Test to find the optimal split size Queenie Wong

  5. Video Transcoding Performance Test • Studies suggested: • The right level of parallelism for maps seems to be around 10-100 maps/node • The size of each map task is roughly 16MB to 64MB • Performance tests against different parallelism levels and split size to find out the idealset upfor video transcoding • Implementation: • Apache Hadoop, FFmpeg for video transcoding and MKVmerge for video splitting at key frames boundary Queenie Wong

  6. Parallel Nodes Test Results Queenie Wong

  7. Threshold for Adding Extra Nodes • Threshold: 5% ofinitialprocessing time (1 node) Reduction: (initial time – current time)/initial time Efficiency of parallelism level: (previous time – current time)/initial time (23.3 - 20.67) / 69.35 = 3.7 % Queenie Wong

  8. Split Size Test Results Queenie Wong

  9. Challenges • Tuning programs for files with different size • Get familiar with FFmpeg for video transcoding operations • Repositioning to key frame boundary • Measurement variance caused by I/O operations • Unstable software problems • Learn, setup,useand debug Hadoop and MapReduce program within a short timeframe Queenie Wong

  10. Conclusion • Video transcoding performance has been diminished when the system overhead exceed the benefit of parallel processing • Efficiency indicator is proposed to measure the efficiency of the level of parallelism • The optimal split sizes for transcoding is 64MB for files with common sizes • Enforce load balancing on every node in order to maximize the benefit of paralleling processing Queenie Wong

  11. References [1]  R. Pereira, M. Azambuja, K. Breitman, and M. Endler, “An architecture for distributed high performance video processing in the cloud,” 2010 IEEE 3rd International Conference on Cloud Computing, pp. 482–489, 2010. [2]  “mkvtoolnix - matroska tools for linux/unix and windows.” [3]  “Ffmpeg.” [Online]. Available: http://www.ffmpeg.org [4]  R. Schmidt and M. Rella, An approach for processing large and non-uniform media objects on mapreduce-based clusters. Springer Berlin Heidelberg, 2011. [5]  “Yahoo! hadoop tutorial,” 2013. [Online]. Available: http://developer.yahoo.com/hadoop/tutorial/index.html [6]  J. Lin and C. Dyer, Data-Intensive Text Processing with MapReduce. Morgan & Claypool, 2010. [7]  J. Dean and S. Ghemawat, “Mapreduce: simplified data processing on large clusters,” Comunications of the ACM, vol. 51, no. 1, pp. 107–113, 2008. Queenie Wong

  12. References [8]  Z. Y, C. Wang, C.Thomborson, J. Wang, S. Lian, and A. Vasilakos, “Multimedia applications and security in mapreduce: opportunities and challenges,” Concurrency and Computation: Practice and Experience, vol. 17, no. 24, pp. 2083–2101, 2012. [9] A.Garcia, H.Kalva,andB.Furht,“Astudy of transcoding on cloud environments for video contentdelivery,” inProceedings of the 2010 ACM multimedia workshop on Mobile cloud media computing. ACM, 2010, pp. 13–18. [10]  C. Yan, X. Yang, Z. Yu, M. Li, and X. Li, “Incmr: Incremental data processing based on mapreduce,” in 2012 IEEE 5th International Conference on Cloud Computing (CLOUD). IEEE, 2012, pp. 534–541. [11]  N. S. Chahal and B. S. Khehra, “A comparative study for optimization of video file compression in cloud environment,” International Journal of Computer Applications, vol. 60, no. 13, pp. 27–30, 2012. [12]  Yzyero. Hadoop + ffmpeg on mapreduce. [Online]. Available: http://yzyzero.iteye.com/blog/1900489 [13]  “Apache hadoop.” [Online]. Available: http://hadoop.apache.org Queenie Wong

More Related