Lecture 12: Theory and History of Computation

Lecture 12: Theory and History of Computation IS246Multimedia Information (FILM 240, Section 4) Prof. Marc Davis UC Berkeley SIMS Monday and Wednesday 2:00 pm – 3:30 pm Spring 2003 http://www.sims.berkeley.edu/academics/courses/is246/s03/

Today’s Agenda • Review of Last Time • Movie Ideas Feedback • Theory and History of Computation • Turing and von Neumann • Ideas of Computation • Discussion Questions • Action Items for Next Time

Movie Ideas Feedback • Time constraints • Make sure your movie will be between 2:30 and 3:00 minutes long • Add shot durations to your annotated storyboards • POV • Who is making your message movie? • To whom is it intended? • What do you want your audience to do with it? • How do your cinematographic choices (especially shot POV) reflect these decisions?

Computation in Intellectual History • Computation as instrumentality • PCs, PDAs, embedded processors, etc. • Computation as ideas • Modeling process • Languages for modeling process • Primitives, combination, abstraction • Parameterization • Black boxing functionality • Optimization

Universal Turing Machine • An abstract representation of a computing device • It has a read/write/erase head that scans a (possibly infinite) one-dimensional (bi-directional) tape divided into squares, each of which can be inscribed with a symbol (e.g., 0 or 1) • Computation begins with the machine in a given "state” scanning a given square • It can erase what it finds on the square and print a symbol (e.g., 0 or 1), move to an adjacent square, and go into a new state • A table of instructions specifies, for each state, what the machine should write, which direction it should move in, and which state it should go into • The machine’s behavior is determined by three parameters • The state the machine is in • The square it is scanning • A table of instructions • The table can list only finitely many states, each of which becomes implicitly defined by the role it plays in the table of instructions

von Neumann Machine • Stored program • Program is no longer external to machine itself • A “conditional control transfer” permitted the program sequence to be interrupted and reinitiated at any point • Computer can modify both its data and programs • von Neumann machines allowed the construction of practical digital computers

Programming As Representation • There is a structure of formal symbols that can be manipulated according to a precisely defined and well-understood system of rules • There is a mapping through which the relevant properties of the domain can be represented by symbol structures • This mapping is systematic in that a community of programmers can agree as to what a given structure represents

Programming As Representation • There are operations that manipulate the symbols in such a way as to produce veridical results—to derive new structures that represent the domain in such a way that the programmers would find them accurate representations • Programs can be written that combine these operations to produce desired results

Levels of Representation • Physical machine • Wires, chips, disks, or pipes, valves, sinks, or tinker toys • Logical machine • And-gates, or Or-gates, inverters, etc. • Abstract machine • Machine instructions for manipulating stored symbols • High-level language • Java, C++, Lisp, etc. (operator, data structures, etc.) • Representation language • Represents primitives and operations about a domain

Algorithms and Programming • Algorithm • A step-by-step description of a procedure to achieve a desired result • Programming • Primitives • Means of combination • Means of abstraction

From Algorithms to Programs • Algorithm • A step-by-step description of a procedure to achieve a desired result • How can we walk a square? • Walk forward • Turn • Walk forward • Turn • Walk forward • Turn • Walk forward

LOGO Square Example • TO SQUARE • FORWARD 5 • RIGHT 90 • FORWARD 5 • RIGHT 90 • FORWARD 5 • RIGHT 90 • FORWARD 5 • END

LOGO Square Example • TO SQUARE :SIZE • FORWARD :SIZE • RIGHT 90 • FORWARD :SIZE • RIGHT 90 • FORWARD :SIZE • RIGHT 90 • FORWARD :SIZE • END

LOGO Square Example • TO WINDOW :SIZE • SQUARE :SIZE • SQUARE :SIZE • SQUARE :SIZE • SQUARE :SIZE • END

Computation for Designing Artifacts • Four computational ideas/techniques from Carlo Sequin • Procedural generation • Parameterization • Optimization • Evolutionary power

Procedural Generation • Rather than creating artifacts directly, the user may design a generating program that will then generate the desired artifact • The empowering aspect of this approach is that the generating procedure will not just create the one artifact originally desired, but, with minor variations to the program, it can produce many different artifacts that may all fit a specified set of constraints or usage

Parameterization • For classes of frequently needed artifacts, the procedural generation mentioned above can be captured in a robust and more general program that contains a modest number of parameters that can be easily adjusted by non-programming users • A judicious selection and coupling of such parameters can enhance the likelihood that any arbitrary combinations of parameters still produce a meaningful output, although it may be far from desirable or optimal with respect to some specific application • However, the ease of modifying the parameter values and previewing the expected outcome, would allow even novice users to achieve better than average results obtained by un-aided users

Optimization • Given that the tedium of creating individual artifacts can be greatly reduced by procedural generation, users can explore a far larger space of possibilities than they could if they had to craft each artifact individually • This allows them to home in on a more optimal solution than they could by building a few prototypes • If the constraints and goal functions are well understood, then the generating program may contain its own evaluation loop that allows it to explore many options on its own and gradually converge towards a local optimum

Evolutionary Power • The ease of exploration afforded by the use of procedural generation permits an informed user to more clearly see and locate the boundaries of the paradigm captured in a generating program • By making these boundaries more visible, it also becomes more obvious to ask what lies beyond • Often such questions can be answered with a modest re-programming effort that enlarges the scope of the generator

Discussion Questions • On Winograd and Flores (Ana Ramirez) • Winograd and Flores discuss the importance of abstraction and separation of concerns in computing. Increased levels of abstraction and separation of concerns result in an increase in programmer productivity and sometimes a decrease in performance. Moore's law has placed an importance on the programmer’s productivity, since computers double in speed every 18 months, most performance problems disappear once the newest processor comes out. We are quickly approaching a point where Moore's law will no longer hold. Will we find a new technology that continues this rapid increase in speed in processor technology or will the emphasis be moved from programmers' productivity back to program performance? How will the role of abstraction and separation of concerns change?

Discussion Questions • On Winograd and Flores (Ana Ramirez) • The abstraction boundaries must be crossed occasionally in order to increase performance. One such case is in systems that must operate in real-time. How does this show up in multimedia systems?

Discussion Questions • On History of Computation (Catherine Lai) • Look at the pervasive presence of computing in modern science and technology, has the history of computing established a significant presence in the history of science and technology? • Is technology the creator of demand or a response to it? • What role do governments play in fostering and directing technological innovation and development? • What does the term “computer” mean to you? Have we prematurely united its multiple historical resources into one thing? What about for computing? Is there a dual nature or tripartite structure to it?

Discussion Questions • How can we think about motion pictures and computation in terms of each other as “representational” systems? • How could we describe an editing process computationally? • How would we need to describe video to be able to operate on it computationally?

Readings for Next Week • Monday 03/10 “Computational Media Theory” • Required • Lev Manovich. The Language of New Media. The MIT Press, Cambridge, Massachusetts, 2001; pp. 19-61. • Gilles Bloch, G.R. From Concepts to Film Sequences, Yale University Department of Computer Science, 1987; pp. 1-8. • Chitra Dorai and Svetha Venkatesh. Computational Media Aesthetics: Finding Meaning Beautiful. IEEE Multimedia, 8 (4); pp. 10-12. • Optional • Marc Davis and David Levitt. Time-Based Media Processing System (US Patent 6,243,087), Interval Research Corporation, USA, 2001; pp. 1-20.

Readings for Next Week • Wednesday 03/12 “Automated Media Analysis: Audio” • Required • Jonathan Foote. An Overview of Audio Information Retrieval. Multimedia Systems, vol. 7, 1999; pp. 1-18. • Kenichi Minami, Akihito Akutsu, Hiroshi Hamada, and Yoshinobu Tomomura. Video Handling with Music and Speech Detection. IEEE MultiMedia, vol. 5, 1998; pp. 17-25. • Optional (On IS246 Course Web Site) • Christian Spevak and Emmanuel Favreau. Sound Spotter - A Prototype System for Content-Based Audio Retrieval. In: Proceedings of the 5th International Conference on Digital Audio Effects (DAFx-02). Hamburg, 2002. • George Tzanetakis, Perry Cook. Automatic Musical Genre Classification of Audio Signals. In Proc. International Symposium for Audio Information Retrieval (ISMIR 2001) Bloomington, USA, October 2001.

Lecture 12: Theory and History of Computation