1 / 29

Universal Semantic Communication

Universal Semantic Communication. Madhu Sudan Microsoft Research. Joint with Oded Goldreich ( Weizmann ) and Brendan Juba ( MIT ). TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A. The Meaning of Bits. Alice. Channel. Bob.

olin
Télécharger la présentation

Universal Semantic Communication

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Universal SemanticCommunication MadhuSudan Microsoft Research Joint withOdedGoldreich(Weizmann)and Brendan Juba(MIT). Semantic Communication @ Northwestern TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA

  2. The Meaning of Bits Alice Channel Bob • Is this perfect communication? • What if Alice is trying to send instructions? • Aka, an algorithm • Does Bob understand the correct algorithm? • What if Alice and Bob speak in different (programming) languages? • Question important theoretically, and in practice of computing/communication. 01001011 01001011 Freeze! Bob Semantic Communication @ Northwestern

  3. This talk • Meaning of information: Meaning via Goal-oriented communication • Example: Computational Goal • Going Beyond Example • General Goals • Efficiency via compatible beliefs • Semantics in general Semantic Communication @ Northwestern

  4. Meaning? A first attempt • Sender is sending instructions/algorithms • Can we understand/execute it? • Answer: NO! • Under sufficient richness of language (any finite length string means anything), can never achieve this state. • So what should we try to achieve? Semantic Communication @ Northwestern

  5. Communications as a means to an end • Communication is painful: • Unreliability of communication medium, misunderstanding, loss of privacy, secrecy. • So why do it? • Must be some compensating gain. • Communication should strive to achieve some goal. • “Understanding Meaning” is when we can achieve the goal in the absence of common language. Semantic Communication @ Northwestern

  6. Part II: Computational Motivation Semantic Communication @ Northwestern

  7. Computational Goal for Bob • Why does Bob want to learn algorithm? • Presumably to compute some function f (A is expected to compute this function.) • Lets focus on the function f. • Setting: • Bob is prob. poly time bounded. • Alice is computationally unbounded, does not speak same language as Bob, but is “helpful”. • What kind of functions f? • E.g., uncomputable, PSPACE, NP, P? Semantic Communication @ Northwestern

  8. Setup Server Alice Bob User f(x) = 0/1? R Ã $$$ qk q1 a1 ak Different from interactions in cryptography/security: There, User does not trustServer, while here he does not understand her. Computes P(x,R,a1,…,ak) Hopefully P(x,…) = f(x)! Semantic Communication @ Northwestern

  9. Intelligence & Cooperation? • For User to have a non-trivial interaction, Server must be: • Intelligent: Capable of computing f(x). • Cooperative: Must communicate this to User. • Formally: • Server S is helpful (for f) if 9some (other) user U’ s.t. 8 x, starting states ¾ of the server (U’(x) $ S(¾)) outputs f(x) Semantic Communication @ Northwestern

  10. Successful universal communication • Universality: Universal User U should be able to talk to any (every) helpful server S to compute f. • Formally: • U is f-universal, if 8 helpful S, 8¾, 8x (U(x) $ S(¾)) = f(x) (w.h.p.) • What happens if S is not helpful? • Paranoid view ) output “f(x)” or “?” • Benign view ) Don’t care (everyone is helpful) Semantic Communication @ Northwestern

  11. Main Theorems [Juba & S. ‘08] • If fis PSPACE-complete, then there exists a f-universal user who runs in probabilistic polynomial time. • Extends to checkable problems • (NP Å co-NP, breaking cryptosystems) • S not helpful ) output is safe • Conversely, if there exists a f-universal user, then fis PSPACE-computable. • Scope of computation by communication is limited by misunderstanding (alone). Semantic Communication @ Northwestern

  12. Implications • No universal communication protocol  • If there were, should have been able to solve every problem (not just (PSPACE) computable ones). • But there is gain in communication: • Can solve more complex problems than on one’s own, but not every such problem. • Resolving misunderstanding? Learning Language? • Formally No! No such guarantee. • Functionally Yes! If not, how can user solve such hard problems? Semantic Communication @ Northwestern

  13. Few words about the proof: Positive result • Positive result: Enumeration + Interactive Proofs • Guess: Interpreter; b2 {0,1} (value of f(x)) • Proof works )f(x) = b. • If it doesn’t ) {Interpreter or b} incorrect. Prover Server User Interpreter Semantic Communication @ Northwestern

  14. Proof of Negative Result • L not in PSPACE)Usermakes mistakes. • Suppose Serveranswers every question so as to minimize the conversation length. • (Reasonable effect of misunderstanding). • Conversation comes to end quickly. • Userhas to decide. • Conversation + Decision simulatable in PSPACE (since Server’s strategy can be computed in PSPACE). • f is not PSPACE-computable )User wrong. • Warning:Only leads to finitely many mistakes. Semantic Communication @ Northwestern

  15. Part III: Beyond ExampleIII.1 General Goals Semantic Communication @ Northwestern

  16. General Goals • Limitations of example: • Gain is computational • Gain possible only if Server more powerful than User (asymmetric). • Communication (presumably) serves many other goals • What are they? • Can we capture them all in single definition? • Usual definitions (via transcript of interaction) inadequate in “semantic” setting. Semantic Communication @ Northwestern

  17. Modelling User/Interacting agents User • (standard AI model) • User has state and input/output wires. • Defined by the map from current state and input signals to new state and output signals. Semantic Communication @ Northwestern

  18. Generic Goal? X X X User Server • Goal = function of ? • User? – But user wishes to change actions to achieve universality! • Server? – But server also may change behaviour to be helpful! • Transcript of interaction? – How do we account for the many different languages? Semantic Communication @ Northwestern

  19. Generic Goals User Server Referee/Environment • Key Idea: Introduce 3rd entity: Referee • Poses tasks to user. • Judges success. • Generic Goal specified by • Referee (just another agent) • Boolean Function determining if the state evolution of the referee reflects successful achievement of goal. • Class of users/servers. Semantic Communication @ Northwestern

  20. Results in “General Setting” • New concept: “Sensing” • Ability of User to predict Referee’s verdict. • Computational example shows this can be achieved in non-trivial ways. • Relatively straightforward generalization of computational example: • Sensing (is necessary and) sufficient for achieving goals in semantic setting. Semantic Communication @ Northwestern

  21. Part III: Beyond ExampleIII.2: Efficient Learning? Semantic Communication @ Northwestern

  22. The Enumeration Bottleneck • Enumeration of users seems inefficient, can we get around it? • Formally, in k time, User can only explore O(k) other users. • Bad News: • Provable bottleneck: Server could use passwords (of length log k). • Good News: • Can formalize this as only bottleneck … • using “Beliefs, Compatibility” Semantic Communication @ Northwestern

  23. Broadmindedness, Compatible beliefs: • Beliefs of server S: • Expects users chosen from distribution X. • Allows “typical” user to reach goal in time T. • # such users may be exponential • Beliefs of user U: • Anticipates some distribution Y on users that the server is trying to serve. • Compatibility: K = (1 - |X – Y|TV) • Theorem[JS]: U can achieve goal in time poly(T/K). Semantic Communication @ Waterloo

  24. Part III: Beyond ExampleIII.3: Semantics? Semantic Communication @ Northwestern

  25. Semantic Communication • Origins: The Gap between Turing and Shannon • Turing counts on reliable communication • Shannon counts on general computation • Separating theories was essential to initial progress. • Modern technology: • Communication & Computation deeply intertwined. • Unreasonable to separate the two. • Semantic Communication: Prime example Semantic Communication @ Northwestern

  26. A new model Semantic Communication Model A1 B1 B2 A2 Channel A B B3 A3 New Class of Problems New challenges Needs more attention! Bj Ak Classical Shannon Model Semantic Communication @ Northwestern

  27. Compression in semantic setting • Human-Human communication: • Robust, ambiguous, redundant. • Explored in [Juba,Kalai,Khanna,S. ICS ‘11] • Thesis: Reason is diversity of audiences/their priors. • Leads to compression for “uncertain” priors. • Reveals same phenomena as natural languages: • Novel redundancy (increases with uncertainty), still ambiguous, but robust. Semantic Communication @ Northwestern

  28. References • Juba & S. • ECCC TR07-084: http://eccc.uni-trier.de/report/2007/084/ • Goldreich, Juba & S. • ECCC TR09-075: http://eccc.uni-trier.de/report/2009/075/ • Juba & S. • ICS ‘11: http://people.csail.mit.edu/madhu/papers/beliefs.pdf • Juba, Kalai, Khanna & S. • ICS ‘11: http://people.csail.mit.edu/madhu/papers/ambiguity.pdf Semantic Communication @ Northwestern

  29. Thank You! Semantic Communication @ Northwestern

More Related