1 / 50

IPSI Belgrade Ltd.

IPSI Belgrade develops a dynamic virtual reality art gallery with advanced search capabilities, optimized for user satisfaction. Learn about the technology and design behind this cutting-edge platform.

maryjmorgan
Télécharger la présentation

IPSI Belgrade Ltd.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. IPSI Belgrade Ltd. ARTSHOP GALLERY vm@ipsi.co.yuoffice@ipsi.co.yuhttp://www.ipsi.co.yu This material was developed with financial help of the WUSA fund of Austria.

  2. Authors Marinkovic Ivan Stojanovski Aleksandar Nikezic Gavro Radakovic Miroslav Skundric Nikola Milutinovic Darko Toskov Ivan Vujovic Ivana Milutinovic Veljko Anucojic Goran

  3. Introduction – IPSI Bgd IPSI Belgrade is a company jointly founded by German and Serbian capital Partners: - IPSI Fraunhofer, Darmstadt, Germany- Telecom Italia Learning Services, Italy- NYU, School of Continuous Professional Studies, USA- Instituto Tecnologico de Durango, Mexico

  4. Introduction – IPSI Bgd - Workspaces of the Future- Environments for Cooperative Working and Learning- Virtual Information and Knowledge Environments- Mobile Interactive Media- Open Adaptive Information Management Systems- Publication Engineering and Technology- Hardware Design and Operating Systems- Networks and WWW

  5. Introduction – IPSI Bgd • Products: • Advanced Virtual Gallery • Semantic Web tutorial and book development • The injection cache, the STS cache, VLSI Detection for Internet/Telephony Interfaces, Genetic Search with Spatial/Temporal Mutations, Browser Acceleration, Technology transfer, Testing the Infrastructure for EBI, Socratenon Distant Web Educating Machine, e-Tourism…

  6. Problem Statement - Creating Web based art gallery with “look and feel” of the real world exhibitions - Visitor moves through the gallery by “walking with options”

  7. Existing Solutions • Musee national des Arts asiatiques http://www.museeguimet.fr/tour-guimet/index.html • Web Server of the Galleria degli Uffizi in Florence http://www.uffizi.firenze.it • The Distributed Interactive Virtual Environment (DIVE) http://www.sics.se/dive/ • The Web3D Repository http://www.web3d.org/vrml/artgal.htm

  8. Proposed Solution - Virtual reality gallery - Advanced search capabilities - Visitor’s criteria based room generation

  9. Why is it better? - Dynamically generated gallery - Content based search engine - User satisfaction

  10. Conditions and Assumptions - PC- Internet connection- Internet Explorer 5.0 or higher - Netscape 7.0 - Cortona VRML plug-in for IE

  11. Analysis and Implementation • - Application is written in ASP.NET using C# as code-behind, and ADO.NET for database access. • - Database server is SQL Server 2000. • Communication with the database is entirely made through XML (using SQLXML3.0 framework). • Queries are made in XPath, while adding, changing and deleting of the records is done through UpdateGrams. • - Application is optimized for Internet Explorer 5.0 or higher, at the 1024x768 screen resolution. Netscape 7.0 or higher is also supported. • - 3D gallery is completely generated on the server side (dynamically) using VRML.

  12. Track 1 Track Requirements: • To stand as the integrative part for the other two tracks • To provide: • User interaction • Database connectivity (database independent) • Search functions (simple and advanced using Track3 output) • Information brokering between artists and buyers • Administration tools • Artworks management tools, etc. • Thin client (3D scene generation on server side)

  13. Track 1 Development Tools: • Application server platform: • Windows XP Professional • IIS 5.1 • MS SQL Server 2000 • Development platform: • ASP.NET • C# as code-behind. • Communication with the underlying database: • XML & XSD using XPath queries (DB independent) • Currently using SQLXML3.0 add-on for ADO.NET

  14. Track 1 Database Design:

  15. Track 1 Administrator Tools: • Separate entry point:http://<server_address>/artshop/admin

  16. Track 1 • Entry point:http://<server_address>/artshop/index.htm Users & Exhibitors:

  17. Track 1 Interesting Details: • Native XML DBMS under development at IPSI Fraunhofer • Practical testing of the XML/XPath database access • Dynamic addition (to the system) of new multimedia types • 3D view of search results

  18. Track 1 Interesting Details: • Application that can connect on the fly to any DBMS which supports XML/XPath is an interesting and possibly useful idea (user just has to set one XML file containing local field mapping, and one XSD to map the database fields to the pre-defined scheme) • Cons: • XPath queries are lot less powerful then standard SQL queries • Inherently, loss of speed (one complex SQL query had to be simulated with couple of XPath queries and additional processing in the code). • For now, SQLXML3.0 does not support complete XPath standard.

  19. Track 1 Errors Made: • Initially, content analysis, picture processing, and adding data to database were completely separated (as specified in the contract), with the idea of later (partial) integration. • Turned out to be a bad idea(required a lot of interventionfrom the ArtShop system administrator when adding artworks).

  20. Track 1 Lessons Learned: • Problem solved by complete integration of forementioned tasks into the one system process which monitors input directory, automatically schedules picture processing and content analysis, and takes care of updating of all necessary fields in all required databases. • With that, we achieved maximum automation, reduced time needed for artwork addition, and reduced amount of data transferred through the Internet (between the administrator’s machine and the application host).

  21. Track 2 Image-Content-Oriented Search Track requirements: • Images used for extracting objects are artistic paintings • Image analyses • Extraction of the features • Create XML file for each image • Fetch the database with the features

  22. Load parameters, input and output directory Open the picture Determine the filter value Put the picture into the reduced matrix Determine histogram Create objects Merge objects into bigger objects Create sorted array of objects Create database objects, prepare them, and put them in XML file and tables in database Track 2 Algorithm:

  23. Create general histogram Sort histogram Remove zero values Remove redundancies Sort histogram Refresh matrix Track 2 Determine histogram:

  24. P8 P8 P3 P8 P3 P5 P6 P5 P4 P5 P4 P8 P3 P8 the first colon the last row the rest of the matrix the last colon the last element in the last row Track 2 Regions for processing matrix:

  25. false Any pixel left in the current region? false true p8 = the current pixel p8.oi == 0 (it doesn’t belong to any object) false true create new object Any neighbour pixel left? false true px = the neighbour pixel p8.oi != px.oi (don’t belong to the same object) true call matrix.unite_pixels method take the next neighbour pixel take the next pixel Creating objects:

  26. Is Picture_Objects list empty? true false q = true; Edge[0] = Picture_Objects[0] false q==true (are there any objects inside Border which colors after translation are the same as the color of Core object) true i < number of objects inside Edge list false true Take ith object Find all neighbour objects, which colors after translation are the same as the color of Core object, and put them into Border i = i+1 Move all pixels from objects inside Edge to Core object Remove objects that are inside Edge from Picture_Objects list Is Border empty true false Move all objects inside Border to Edge q=false; add Core object to big list; Remove pixels belonging to Core object from Picture_Objects Merge objects into bigger objects:

  27. Track 2 Tools used in development: • C# programming language – the chosen tool • Advantage: Includes the best properties from other programming languages (C++, Java, Visual Basic) • Disadvantage: slower processing speed than C++, which is not necessary in this application • C++ - the best alternative tool • Advantage: faster processing speed (unnecessary) • Disadvantage: more complicated code, 50% of all bugs due to use of pointers

  28. Track 2 Original picture:

  29. Track 2 Picture after applyinghistogram values:

  30. Track 2 Picture representedthrough extracted objects:

  31. Index of histogram array Hue Saturation Luminance Description 0 any any <=30 black 1 any any >=lummax white 2 any <20 >30 <30+lum the darkest grey ... … … … … ilmax-1 any <20 >=30+(ilmax-3)*lum <30+(ilmax-2)*lum the lightest grey ilmax <=hue/2 >=240-hue/2 >20 <20+1*sat >30 <30+lum the darkest red with the smallest saturation ilmax+1 <=hue/2 >=240-hue/2 >=20+1*sat < 20+2*sat >30 <30+lum the darkest red with smaller saturation … … … … … ilmax+isat-2 <=hue/2 >=240-hue/2 >=20+(isat-3)*sat < 20 +(isat-2)*sat >30 <30+lum the darkest red with the biggest saturation ilmax+1*(isat-1) <=hue/2 >=240-hue/2 >20 <20+1*sat >=30+lum < 30+2*lum darker red with the smallest saturation ilmax+1*(isat-1)+1 <=hue/2 >=240-hue/2 >=20+1*sat < 20+2*sat >=30+lum < 30+2*lum darker red with smaller saturation … … … … … ilmax+1*(isat-1)+isat-2 <=hue/2 >=240-hue/2 >=20+(isat-3)*sat < 20 +(isat-2)*sat >=30+lum < 30+2*lum darker red with the biggest saturation ilmax+2*(isat-1) <=hue/2 >=240-hue/2 >20 <20+1*sat >=30+2*lum < 30+3*lum dark red with the smallest saturation Track 2 3D HSL space => 1D histogram

  32. ilmax+2*(isat-1)+1 <=hue/2 >=240-hue/2 >=20+1*sat < 20+2*sat >=30+2*lum < 30+3*lum dark red with smaller saturation … … … … … ilmax+(ilmax-3)*(isat-1)+1 <=hue/2 >=240-hue/2 >=20+1*sat < 20+2*sat >=30+(ilmax-3)*lum < 30+(ilmax-2)*lum the brightest red with smaller saturation … … … … … ilmax+(ilmax-3)*(isat-1)+(isat-2) <=hue/2 >=240-hue/2 >=20+(isat-3)*sat < 20 +(isat-2)*sat >=30+(ilmax-3)*lum < 30+(ilmax-2)*lum the brightest red with the biggest saturation ilmax+1*(ilmax-2)*(isat-1) > hue/2 < 3*hue/2 >20 <20+1*sat >30 <30+lum the darkest orange-red with the smallest saturation ilmax+1*(ilmax-2)*(isat-1)+1 > hue/2 < 3*hue/2 >=20+1*sat < 20+2*sat >30 <30+lum the darkest orange-red with smaller saturation … … … … … ilmax+1*(ilmax-2)*(isat-1)+(ilmax-3)*(isat-1)+(isat-2) > hue/2 < 3*hue/2 >=20+(isat-3)*sat < 20 +(isat-2)*sat >=30+(ilmax-3)*lum < 30+(ilmax-2)*lum the brightest orange-red with the biggest saturation ilmax+2*(ilmax-2)*(isat-1) >=3*hue/2 < 5*hue/2 >20 <20+1*sat >30 <30+lum the darkest red-orange with the smallest saturation ilmax+2*(ilmax-2)*(isat-1)+1 >=3*hue/2 < 5*hue/2 >=20+1*sat < 20+2*sat >30 <30+lum the darkest red-orange with smaller saturation … … … … … ilmax+(ihmax-1)*(ilmax-2)*(isat-1)+1 >=(2*ihmax-1)*hue/2 <(2*ihmax+1)*hue/2 >=20+1*sat < 20+2*sat >30 <30+lum the darkest magenta-red with smaller saturation … … … … … ilmax+(ihmax-1)*(ilmax-2)*(isat-1)+(ilmax-3)*(isat-1)+(isat-2) >=(2*ihmax-1)*hue/2 <(2*ihmax+1)*hue/2 >=20+(isat-3)*sat < 20 +(isat-2)*sat >=30+(ilmax-3)*lum < 30+(ilmax-2)*lum the brightest magenta-red with the biggest saturation Track 2 3D HSL space => 1D histogram

  33. Track 2 Lessons learned: • It is impossible to extract objects using only colors as a criterion • It is impossible to extract objects, even using textures, edges, different transformations as criteria • Semantics should be used in segmentation • Colors are the most important features in artistic paintings

  34. Track 3 Track requirements: • Possibility of moving through 3D galleries • Automatic generation of 3D galleries based on user’s query • Manual generation of 3D galleries • User interface for image zooming • Application for image processing

  35. Track 3 Underlying algorithms: • Dynamic creation of gallery • Creation of static galleries • Algorithm for picture zooming • Algorithm for picture processing

  36. Track 3 Creation of galleries: • Validation of the created gallery • Forming VRML files depending on users query • Determining the number of pictures in the gallery • Drawing a 2D floorplan based on the 3D gallery • “Forest fire” algorithm for filling the floorplan with color

  37. Track 3 Picture processing: • Loading image into memory • Clone image into different-size copies • Filtering of copies • Parting of copies

  38. Track 3 Development tools: • C# in .NET Framework for programming image processing • Macromedia Dreamweaver for programming zoom tool • VRML Pad v2.0

  39. Track 3 Flowchart:

  40. Track 3 Creation of galleries: • Making files based on users data • Putting data on server so it can be available for artists • Artist chooses which gallery he/she will be using for exhibition • User can move through 3D world • Selecting the textures for gallery • Selecting the starting position of the user

  41. Track 3 NF filter details: • If the new picture is smaller, every pixel is one pixel of the old picture. • If the new picture is bigger, pixels are calculated based on the pixels surrounding the current.

  42. Track 3 Errors made: • Requests were not precise, so there was a gap at the end of the project between wanted and done • Better results could be done with better using of ASP and XML

  43. Track 3 Lessons learned: • Every member of the team gets a part where his experience is dominating • More planning at the start reduces a lot of work later • Good communication between programmers can save a lot of time

  44. Demo

  45. Future Plans • Improving the existing 3D dynamic gallery • Improving search engine capabilities • Improving feature extraction algorithms and objects recognition

  46. Future Track 1 • 3D Multimedia Showroom Environment: • Implementing a general Web based 3D Multimedia Showroom Environment • Exhibiting various MM data types: images, 3D objects, videos, audio, etc. • Set of MM data types should be extendable

  47. Future Track 2 • MM Object Feature Extraction: • Implementing algorithms and software components for extracting features from MM data types (images, videos, 3D objects), in order to enable content based search • System should be extendable (“plug-in”)

  48. Future Track 3 • Semantic Abstraction of MM Feature Spaces: • Developing methods and SW components which derive mapping from extracted features of MM objects to semantic concepts • Using intelligent classification algorithms (Neural Networks, Fuzzy Classifier) • Developing semantic query engine (answering questions, which could previously only be answered by humans)

  49. Usability • Art Galleries • Museums • Exhibition Fairs

  50. Instead of Conclusion IPSI Belgrade, office@ipsi.co.yuhttp://www.ipsi.co.yu

More Related