Peter J. Braspenning p.braspenning@CRS.unimaas.nl Local I-MASS group

1. Peter J. Braspenning p.braspenning@CRS.unimaas.nl Local I-MASS group Peter-Paul Kruijsen, Gabriel Hopmans & Peter J. Braspenning Communications Research & Semiotics University of Maastricht Communications Research & Semiotics (CR&S) The Knowledge Technology of I-MASS (EC IST Research)Communication, Computing and Interactive Networks Assoc. Prof. Peter J. Braspenning, CR&S, UM

2. Signs & Meanings Assoc. Prof. Peter J. Braspenning, CR&S, UM

3. Communications Research & Semiotics (CR&S) Content • Introduction • What is (Computational) Semiotics? • Semiotic Framework • Logo of CR&S • Communication Process among People • Perceive-Act Pathways • Multi-Modal User Interface: • Semiotic Engineering • Cognitive Engineering • Agent-Oriented Modeling • System’s Modeling via Multi-Agent Societies • Computational Semiotics for Agent Technology • Knowledge Landscape & VRR • Conclusions Assoc. Prof. Peter J. Braspenning, CR&S, UM

4. Communications Research & Semiotics (CR&S) What is Semiotics? Semiotics: discipline of combining the theory of signs (representa-tions), symbols (categories), and meaning extraction. It is an in-clusive discipline which incorporates all aspects of dealing with symbols and symbolic systems, starting with encoding and ending with the extraction of meaning. Mathematical tools of semiotics include those used in control scien-ces, pattern recognition, neural networks, artificial intelligence, and cybernetics. Unified use within a computational semiotic framework leads to better treatments of the complexities (com-munication and computation) inherent in advanced intelligent systems. Semiotics is a strongly emerging multi-disciplinary field of study around a new paradigm for surpassing the classical mind-body dichotomy by focussing on all processes in which the triad object-sign-interpreter plays an essential role. The pervasive use of icons in the interaction with communicative virtual environments (CVEs), is also part of Semiotics. A lot of different kinds of signs are exchanged while communication takes place. Assoc. Prof. Peter J. Braspenning, CR&S, UM

5. What is Semiotics? (continued) • Semiotics is devoted to studying COM-MUNICATION: representations, their interpretation and usage • It investigates SIGNS and the processes by which we take them to mean something to us and expect them to mean something to others • It investigates the resolution of meanings in conversations, collective discourse and culture in general • Semiotics also covers non-human commu-nication processes such as that of animals and machines/artificial systems Assoc. Prof. Peter J. Braspenning, CR&S, UM

6. Peirce: sign = “something standing for something else for somebody in one or more respects” interpretant sign object Components of Symbol System [Schuyt] interpretation system groups acts & events Semiotic framework Assoc. Prof. Peter J. Braspenning, CR&S, UM

7. Logo of CR&S Assoc. Prof. Peter J. Braspenning, CR&S, UM

8. Communication Process among People interpretant Unlimited semiosis interpretant interpretant decoding coding “I like guys with dark hair” message (signs) medium Assoc. Prof. Peter J. Braspenning, CR&S, UM

9. Perceive-Act Pathways Assoc. Prof. Peter J. Braspenning, CR&S, UM

10. message system designer user Semiotic Engineering perspective of HCI • design intention • interaction principles Assoc. Prof. Peter J. Braspenning, CR&S, UM

11. task model + user model design model = Cognitive Engineering usage model interaction system image user designer Assoc. Prof. Peter J. Braspenning, CR&S, UM

12. context context Cognitive Engineering xSemiotic Engineering Cognitive Engineering Semiotic Engineering context medium designer user Assoc. Prof. Peter J. Braspenning, CR&S, UM

13. Semiotic Engineering and Interface Evaluation Communicability Concept • Communicability is the property of software that efficiently and effectively conveys to users its underlying design intent and interactive principles • The communicability evaluation method allows designers to appreciate how well users are getting the intended messages across the interface and to identify communication breakdowns that may take place during interaction Assoc. Prof. Peter J. Braspenning, CR&S, UM

14. Assessing Communicability Assoc. Prof. Peter J. Braspenning, CR&S, UM

15. Intelligent Agent: an assistant that takes care of many gory details of many mundane tasks Additional properties are autonomy, sociability, and a human-like communication Often able to adapt to user's interests, habits and preferences Enabled to communicate with other agents it is potentially entering role-taking behavior and social commitments with other agents that allow it to function in a society of agents Multi-Agent System: bring such agents together in a kind of abstract society, wherein coordination, cooperation and/or collaboration are of paramount importance in order to solve problems that no single agent could handle on its own FIPA specifications represent a collection of standards, which are intended to promote the interoperation of heterogeneous agents and the services that they can represent. Agent-Oriented Modeling (issues) Assoc. Prof. Peter J. Braspenning, CR&S, UM

16. System’s Modelingvia Multi-Agent Societies • One has to decide how to provide efficient inter-agent communication support, what language should the agents talk, should the agents be stationary or mobile, and what technology should be used to build the architecture • At present, there are not much experience reports • Architecture of a multi-agent system can naturally be viewed as a computational organization • Additional organizational concepts • organizational rules, • organizational structures, and • organizational patterns Assoc. Prof. Peter J. Braspenning, CR&S, UM

17. System’s Modelingvia Multi-Agent Societies (continued) • I-MASS uses MAS’s perspective not just as a framework for inter-action, but more as forming abstract societies consisting of agencies (comparable to societal institutions), complex agents (in the sense of consisting of simpler agents), and agents (roughly comparable to individuals in a societal context). • We try to deal with contentinter-operability issues at different abstraction layers of syntactics, semantics, pragmatics and social world. These layers fit into a coherent semiotics framework. Assoc. Prof. Peter J. Braspenning, CR&S, UM

18. Coordination is cen-tral to building MASs Coordinating behav-iors in MASs are often realized by forming groups in which both control and data are distri-buted. Therefore, agents have to have some auto-nomy in performing their actions. However, this autono-my may lead to un-coordinated activities due to uncertainty about the actions of each of the agents. Computational Semiotics for Agent Technology Assoc. Prof. Peter J. Braspenning, CR&S, UM

19. The relationship between uncertainty and the situation that the agents have to handle The uncertainty lowers as the familiarity of the situation that needs to be handled increases! Therefore, it makes sense to develop a framework in which agents know how to handle routine, familiar, and unfamiliar situations Assoc. Prof. Peter J. Braspenning, CR&S, UM

20. Co-ordination among agents: guiding principles • Coordination among agents is easier to esta-blish in routine than in unfamiliar situations; • In general, communi-cation between agents will be more needed in unfamiliar situations than in routine situations. Assoc. Prof. Peter J. Braspenning, CR&S, UM

21. Needed: an agent-architecturein which three kinds of interaction are adressed • Conceptual models[J. Rasmussen, Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering, 1986] • skills • rules • knowledge • The knowledge representation should be adapted to these kinds of interactions. Assoc. Prof. Peter J. Braspenning, CR&S, UM

22. Computational scenario • First, perceived information from the environment leads the agent to execute an action if the correspond-ing situation is perceived in terms of action. • If this is not the case, the agent tries to recognize the situation. It can recognize the considered situation in terms of an action or in terms of a goal. In the first case, it tries to execute the corresponding action, and in the second case it invokes the planning module. • Finally, if the agent faces an ambiguity and cannot come to a decision, or faces many alternatives, then it invokes the decision­making module (based on a Cognitive Map) to make a decision in order to commit to achieve a goal or an action. A goal leads an agent to plan, that is to produce a sequence of actions that achieve the chosen goal. Assoc. Prof. Peter J. Braspenning, CR&S, UM

23. Three levels of control of human behavior Knowledge perception ­ recognition ­ decision ­ planning ­ execution perception ­ recognition ­ decision ­ execution perception ­ recognition ­ planning a ­ execution perception ­ recognition ­ planning b ­ execution perception ­ recognition ­ execution Rules Skills perception ­ execution B. Chaib-draa & P. Levesque, Hierarchical Model and Communication by Signs, Signals and Symbols in Multiagent Environments, 1998 a the planning process adapts old cases to the new situation, and the adaptation is significant b the planning process adapts old cases to the new situation, and the adaptation is generally minor Assoc. Prof. Peter J. Braspenning, CR&S, UM

24. Signals can be viewed as data representing time­space variables from a dynamic, spatial configuration in the environment and they can be processed directly by the agents as continuous varia-bles. In communication by signals, the signal delivered by an agent i has the end of simply being a releaser for the receiving agent j -- of simply eliciting a reaction by j. That is, the signal generally invokes a stimulus or a reaction, without passing through the memory (a data base in this model). Signs indicate a state in the environ-ment with reference to certain norms for acts. In the case of communication by signs, the sender makes a sign which refers to some state of environment and which has the end of signifying, of letting the receiver knows the same reference. Of course, the sender and the receiver should share a set of signs with their references in order to communi-cate efficiently. For instance in urban traffic, communication between a driver and a policeman at a crossroad is generally done by signs. The policeman makes a specific sign which refers to a certain action and which is addressed to certain driver(s). The addressee(s) recognize(s) the reference of this sign and activate(s) stored patterns of behaviors. Signals, Signs, and Symbols Assoc. Prof. Peter J. Braspenning, CR&S, UM

25. Finally, agents can also communi-cate by symbols. Symbols repre-sent variables, relations and properties and can be formally processed. They are abstract con-structs related to and defined by a formal structure of relations and processes, which according to convention can be related to features of the external world. In urban traffic for instance, a dialogue between a policeman and a driver in natural language reflects a symbol­based communication. Another example of symbolic communication is ``honk the car horn'', etc. Information at knowledge and rule levels can act as symbols depending on the situation and the language used for com-munication. In familiar situa-tions corresponding to the rule level, agents can use a specific language (derived or not from a natural language). This lang-uage is generally constructed from repeated activities. When unfamiliar situations occur, agents do not dispose of any operative knowledge nor of any specialized language. They must then make use of a non specialized language (for example natural language), which is less concise but more flexible than their oper-ative language used in familiar situation. Signals, Signs, and Symbols Assoc. Prof. Peter J. Braspenning, CR&S, UM

26. Coordination: in summary With signals and signs, agents do not force their cognitive control to a higher level (i.e. the knowledge level) than the demands of the situation requires. In contrast, agents have a propensity for behaviors based on skills and rules. These behaviors are gene-rally fast, effortless and propitious to a better co-ordination between agents. Assoc. Prof. Peter J. Braspenning, CR&S, UM

27. Recap • Communication and Semiosis are two sides of the same coin; • Knowledge Representation has tradition-ally only be treated in the context of solip-sistic systems. However, communication is constitutive for Knowledge (Represent-ation) & Reasoning Reflection; • Agent Technology as a modeling metho-dology allows us to treat rather complex systems. Moreover, the semiotics perspec-tive sheds new light on issues concerning User Interfaces, Exploration Narratives, and all emerging kinds of New Media Assoc. Prof. Peter J. Braspenning, CR&S, UM

28. Knowledge Landscape & VRR • A systemic approach via Agent-Oriented Modeling & the development of agent-based tools, and • An operational elaboration of the new concept of the Virtual Reference Room, by means of which contextualized access to heterogeneous objects can be realized, and more knowledge-based navigation by means of these contexts becomes feasible. Assoc. Prof. Peter J. Braspenning, CR&S, UM

29. Explanations about how to explore the collections, pro-ducts and services of the institution; Delivery of information about actual services of the Reference Room that are in force; Pointers to where bibliographical services may be found; Pointers to relevant exhibitions and other relevant events in connection with their search questions and associated references; Orientation maps to more autonomously explore the facilities of the Reference Room and around the particular institutional collections maintained; Virtual Reference Room Assoc. Prof. Peter J. Braspenning, CR&S, UM

30. Virtual Reference Room (cont.) Assoc. Prof. Peter J. Braspenning, CR&S, UM

31. Virtual Reference Room (cont.) Assoc. Prof. Peter J. Braspenning, CR&S, UM

32. Assoc. Prof. Peter J. Braspenning, CR&S, UM

33. Assoc. Prof. Peter J. Braspenning, CR&S, UM

34. Assoc. Prof. Peter J. Braspenning, CR&S, UM

35. The big picture Assoc. Prof. Peter J. Braspenning, CR&S, UM

36. Conclusions w.r.t. I-MASS • Short term research will address the precise kinds of enabling technologies (e.g., from information retrieval, ontological engineering and knowledge engineering) that the I-MASS system should incorporate to synthesize (configuring and presenting) pieces of information that seem to fulfill an apparently existing need for information at the side of the users; • Also research about semantical (and pragmatical) inter-operability will require much effort, especially as it contributes quite a lot to the system’s ability to provide good answers; • Longer-term research must address how the cultural domain may be modeled by means of process models that capture the relevant insights of cultural processes (e.g., the rise and fall of the Roman Empire, or the Renais-sance). Assoc. Prof. Peter J. Braspenning, CR&S, UM

37. Communications Research & Semiotics (CR&S) Conclusions Leibniz’ (1646-1716) ambition was “to awake the sleeping child in us all” I-MASS forces us to use Knowledge Tech-nology to the utmost and to use societal metaphors as much as possible! Assoc. Prof. Peter J. Braspenning, CR&S, UM