From Semiotics to Computational Semiotics

1. From Semiotics to Computational Semiotics State University of Campinas UNICAMP - Brazil Ricardo Gudwin

2. Semiotics and Computational Semiotics • Semiotics • branch of human sciences, that studies the sciences of signification and representation, involving mainly the phenomena of cognition and communication on living systems. • Intelligent systems • systems that exhibits behavior that can be considered intelligent • some of the objectives are the study of the phenomena of cognition and communication, but now explicitly under the scope of artificial systems • Computational semiotics • proposition of a set of methodologies that in some way try to use the concepts and terminology of semiotics, but composing a framework suitable to be used in the construction of artificial systems, in this case, implementable within computers

3. Computational Semiotics • Computational semiotics • new inborn science, • but, there are currently some important contributions that despite still not complete and definitive, help us in understanding the nature of semiotic processes and allow their synthesis and implementation within computational platforms. • In this work • we explore one possible pathway along a set of ideas evolving around, which proposes one way of gathering the transition between traditional semiotics and computational semiotics, making it possible to synthesize semiotic systems by means of artificial computing devices.

4. Semiotic Analysis • Semiotics • Tool of Analysis - main goal is to understand the semiotic processing happening in nature • Semiotic Beings (interpreters) are already there • living organisms (biosemiotics) • human beings (human semiotics) • it is easier to create concepts and apply them to things that do already exist and are already working • Questions • should it be possible to use the same conceptual background in order to synthesize new beings (systems), performing the same semiotic behavior as living/human beings would do ? • What would be the challenges in such an endeavor ?

5. Semiotic Synthesis • Problem • things are not already working • so, we should put them to work ! • Hidden Problems • specify the basic entities involved within semiosis • in a way in which it can be produced within a computer • specify the mechanism by which signs are interpreted • there are a lot of intermediary steps that are generally not considered within the context of human semiotics • How from a scene given by a video camera we discover the objects involved into this scene ? • Can we talk about signs if the system is still not aware of objects ? • Are computational devices able to carry on all sign-processing that living/human beings are able to perform ?

6. Semiotic Synthesis • Basic Foundations • set up a generic scenario in which semiotic synthesis is going to be discussed • try to get clues on how semiotic processes really happens • allow the implementation of a computational version of semiotic processes • Terminology • related to standard semiotic terminology • but we don’t want to limit the meaning of terms to human/bio semiotics • Requirement • be careful when applying semiotic analysis to our synthesis scenario

7. Semiotic SynthesisBasic Foundations • Representation Spaces

8. Semiotic SynthesisBasic Foundations • Shareable and Non-shareable spaces

9. Semiotic SynthesisBasic Foundations • Interpreting Fields

10. Semiotic SynthesisBasic Foundations • Multiple Internal Spaces and Interpreting Fields

11. Semiotic SynthesisBasic Foundations • Interpreting Field • concept originated from field theory • function (energy function ?) that to each point in space and time determine a unique value • state • External Space • interpreting field is continuous (that’s the real world) • by definition, is not knowable in its entirety • Internal Spaces • accommodate a model of external interpreting field • internal interpreting fields are functions that depend on the type of semiotic synthesis we are trying to model

12. Semiotic SynthesisBasic Foundations • Sign • Everything under the interpreter’s focus of attention (internal or external) that would cause an interpreter action • Interpreter Possible Actions • Change in the focuses of attention (internal and/or external) • Determination, for the time t = t+1 of a new value for any interpreting field (internal or external), at a point (x,y,z) covered by the focus of attention in that space • Interpretant • any interpreter action caused by a sign • any change in internal and external interpreting fields for time t = t+1, caused by an interpreter action due to the effect of the sign

13. External Semiosis • Interpretant of signs • happens at the external space • Change in external interpreting field • change in environment • shareable with other interpreters • can act as a sign for the same interpreter or to other interpreters • Happens mainly on interpreters that do not have internal spaces • semiosis in molecules and chemical reactions • very simple biological organisms • Can be the final result of a chain of internal semiosis

14. Internal Semiosis • Interpretant of Signs • happens within any of the internal spaces • Signs can be • at the external space (semiotic transduction) • at the internal space • A typical semiosis chain • starts with an external sign • generates a set of internal interpretants, that • become internal signs • generating new internal interpretants, until • some of them become an internal sign • that generates an external interpretant

15. Information, Signs and Knowledge • Signals and Information • signals - values of parts of interpreting fields that can be differentiated (distinguished) from other values • information - meaning of signals • Example • suppose that E (x,y,z,t) has a counter-domain like [0,5] • but, due to sensor limitations, the interpreter is only able to sense values in {0,1,2,3,4,5} • then, values like 2.3 or 2.2 would equally be understand like 2 • so, the information that those signals convey is tied to only 6 discrete values • Signals only describe states • they do not cause any actions

16. Information, Signs and Knowledge • Once signals are able to cause actions • they become signs • The information they carry • associated to the actions they cause • is then called knowledge • Signals - Information • Signs - Knowledge • Region under a focus of attention of some space • sign • knowledge unit

17. Things to Remember • External Interpreting Field • is infinite, continuous and probably take values on continuous sets • can not be known as a whole • can be known in parts, with approximations • The only way we are able to know the external interpreting field is due to sensors • The most basic knowledge units that can be stored into internal interpreting field is of sensorial type • Internal Interpreting Field (Concrete Space) • our best model of external interpreting field

18. Things to Think About • Storing sensorial information is not efficient • We need better models • Basic mechanism • The notion of “Entity” • From sensorial knowledge units • the system must try to represent the same external interpreting field as a collection of entities • Entities • may have attributes that would change in time • Occurrences • model the change in entities attributes • Sensorial knowledge, entities and occurrences • grouped to represent situations

19. A Hierarchy for Knowledge Units

20. Simplification of the Model • Instead working in general spaces and interpreting fields • restrict ourselves to memories and places • assign sign processing to micro-interpreters

21. Micro-Interpreter • Micro-Interpreter’s Responsibilities • choose the knowledge units that will be used (focus of attention) • eventually destroy them after use • create new knowledge units using information contained in earlier ones

22. Building Intelligent Systems • Using multiple micro-interpreters • in cooperation to each other • processing knowledge units

23. Conclusions • Dyadic or Triadic ? • Some people would say that we are building a dyadic model for a sign • But, there is some kind of “mediation”, due to: • the focus of attention mechanism • the influence that some knowledge units may have over the processing of other knowledge units (catalytic knowledge units) • We still need to do further reflections in order to have a better position on this issue • This is not the final word regarding Computational Semiotics • it is only a first exercise in order to get insights to the problem of semiotic synthesis • computational implementations of such model indicate that, up to some point, it is worth the value of working on it