1 / 12

Image Magick in the Cloud Scalable Image Processing Service

This paper discusses a cloud-based scalable image processing service developed by Sanketh Beerabbi at the University of Central Florida. The service addresses the challenges of compute-intensive image processing in online applications, enabling users to upload, manipulate, and create images using filters and face detection. Implemented on AWS with tools like ImageMagick and PHP SDK, it utilizes services such as EC2, S3, and SimpleDB for efficient processing and load balancing. Key learnings include cloud architecture, message queuing, and batch processing.

bikita
Télécharger la présentation

Image Magick in the Cloud Scalable Image Processing Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Magick in the CloudScalable Image Processing Service Sanketh Beerabbi University of Central Florida

  2. Introduction • Image processing is a very compute intensive task which requires multiple compute units to provide near instantaneous results. • It’s more challenging in online applications involving many simultaneous, “impatient” users creating varying workloads. • Cloud computing provides the scalability needed for running such applications online.

  3. Service • Allows users to upload photos and store them in the cloud. • Allows photos to be manipulated by applying Instagram style image filters. • Allows users to create profile pics by detecting faces and cropping them out of the original photo.

  4. Implementation • Amazon 64 bit Linux AMI • AWS PHP SDK with cURL and PEAR packages • GD and Image Magick for image manipulation • AWS services: • EC2 & S3 • SimpleDB (SDB) • Simple Queue Service (SQS) • Elastic Load Balancer (ELB) • Cloudfront CDN

  5. What took time • Plan and Design (~2 weeks) • LAMP, AWS PHP SDK, GD and ImageMagick (~2 weeks) • Browser based uploading to S3 and server side operations using S3 API (~2 weeks) • Database operations using Simple DB(1 week) • Message passing using SQS API (~2 weeks)

  6. Architecture Simple DB AWS CLOUD

  7. Load Balancing & Redundancy

  8. Modules • Upload – • Stores file names and task status in SDB • Creates bucket and uploads image to S3 • Sends message to SQS • Process – • Reads details from SDB and updates task status. • Receives and deletes message from SQS • Downloads from S3 and processes image . • Uploads processed image to S3.

  9. Message Queing • A single Amazon SQS queue can be shared by multiple instances simultaneously. • An instance that is processing a message can prevent other instances from processing the same message at the same time by temporarily “locking” a message. When the instance is done processing the message, it deletes the message. • If the instance fails while processing the message, another instance can get the message after the lockout period. • The number of processing servers can be changed dynamically to handle varying load.

  10. What I Learnt • PHP, hundreds of new *NIX commands, tools and Linux from the “terminal” point of view. • Virtual networking, load balancing and DNS. • Batch processing on the cloud using multiple instances • Inter-instance communication using message queuing.

  11. DEMO • Main Website: • http://imagecloud.vf1.us/ • AWS Public address: • http://ec2-54-204-45-246.compute-1.amazonaws.com/ • Cloudfront CDN: • http://d3emazpmic2hkb.cloudfront.net/

  12. Questions?

More Related