1 / 24

Inferring compositional style in the neo-plastic paintings of Piet Mondrian by machine learning

David Andrzejewski, Univ. of Wisconsin-Madison (USA) David G. Stork, Ricoh Innovations, Inc. and Stanford Univ. (USA) Xiaojin Zhu, Univ. of Wisconsin-Madison (USA) Ron Spronk, Queen's Univ. (Canada)

bernad
Télécharger la présentation

Inferring compositional style in the neo-plastic paintings of Piet Mondrian by machine learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. David Andrzejewski, Univ. of Wisconsin-Madison (USA) David G. Stork, Ricoh Innovations, Inc. and Stanford Univ. (USA) Xiaojin Zhu, Univ. of Wisconsin-Madison (USA) Ron Spronk, Queen's Univ. (Canada) Inferring compositional style in the neo-plastic paintings of Piet Mondrian by machine learning

  2. Stylometry: mathematics of “style” • Visual arts • Digital authentication of Bruegel, Perugino (Lyu et al, 2004) • Jackson Pollock(Taylor, 1999)(Irfan and Stork, 2009) • Writings • Authorship of the Federalist Papers(Mosteller and Wallace, 1964) • Ronald Reagan’s radio addresses (Airoldi et al, 2007)

  3. Piet Mondrian (1872-1944)Avond (Evening); Red Tree (1908) http://www.artchive.comHaags Gemeentemuseum, The Hague

  4. Piet Mondrian (1872-1944)Neo-plastic abstractions

  5. Goal of this work • Better understand compositional style • Develop a formal representation of the paintings • Extract these representations from paintings • Train a generative model • Learn relative visual weights of colors • Classify true Mondrians versus • “fakes” created by the generative model in step 3 • “earlier states” of the Transatlantic paintings

  6. Simplified representation • Vertical/horizontal lines • locations • extents • Rectangles • locations • sizes • colors • can span multiple lines

  7. Semi-automatic extraction tool

  8. Semi-automatic extraction tool 8

  9. Generative modeling • Hypothesize an underlying probabilistic model that generates observed data • Many uses in machine learning • Make predictions (Naïve Bayes) • Generate new examples (Markov model) • Interpret parameter values (Linear regression) • Given data, learn/train model parameters • Our approach: Maximum likelihood estimation (MLE)

  10. Our generative model Canvas aspect ratios (kernel density estimator)

  11. Our generative model Number of horiz/vert lines (Poisson) Horiz/vert line spacing (Dirichlet)

  12. Our generative model Segments are deleted / invisible / left alone (Polya)

  13. Our generative model Rectangle colors (Multinomial)

  14. Additional constraints Don’t allow unrealistic “hanging” lines Require ≥ 1 vertical line

  15. Learned model parameters

  16. Learning color weights • Calculate visual “center of mass” • Assume true Mondrians centered at [0.5,0.5] • Learn color weights via linear programming

  17. The Transatlantic paintings • Completed in Europe, but then altered after Mondrian’s arrival in the United States • A variety of techniques (x-ray, UV, etc) were used to recover the earlier states (Cooper & Spronk, 2001 )

  18. The Transatlantic paintings Composition with Red, Blue, and Yellow (1937-1942)

  19. The Transatlantic paintings Composition with Red, Yellow, and Blue (1935-1942)

  20. The Transatlantic paintings No. 9 (1939-1942)

  21. Decision tree classification • Very popular technique in machine learning • At each iteration, choose a rule to “split” on • Resulting partitions should be more “pure” with respect to target classification (true Mondrian or computer-generated fake?) • Key feature: resulting trees easy to interpret • Estimate accuracy with leave-one-out cross-validation • Control over-fitting with pruning

  22. Decision tree classification • 45 true Mondrians versus 45 generated “fakes” • 45 true Mondrians versus 11 “earlier states”

  23. Example learned decision tree • Transatlantic dataset • < 1% pixels blue • # horiz / # vert < 0.9 • Low visual “density” • THEN Transatlantic • Analysis of results

  24. Summary • Formal representation and feature extraction • Generative model • Fitting simple statistics of Mondrians cannot create realistic synthetic paintings • Color weights align well with our intuitions • Classification • Can reliably discriminate true Mondrians vs. computer-generated • Cannot do so for true Mondrians vs Transatlantic “earlier states” • Underlying images were “nearly complete” (!)

More Related