Semantic Models in Neural Networks

1. Semantic Models in Neural Networks Markus Werning Chair of Theoretical Philosophy Heinrich-Heine-University Düsseldorf Email: werning@phil-fak.uni-duesseldorf.de Co-operation: Alexander Maye Computer Science Department Technical University of Berlin

2. The place of meanings/concepts among language, mind and world

3. Compositionality: the constraint for both meaning and content functions • A syntax of some representational medium is a structure S=<T, {1, …, j}> • where T is the set of syntactical terms and • the i are syntactical operations, i.e., partial functions from some Cartesian product of T into T. Compositionality: • Let μ be a function that maps every syntactical term of a syntax S=<T, {1, …, j} on its semantic value. Then μ is called compositional if and only if, for every non-atomic term (t1, …, tn) there is a function μ such that • μ ((t1, …, tn)) = μ (μ(t1), …, μ(tn)).

4. The Turing model of cognition • According to the Turing model of cognition (cf. Fodor 1972, Pylyshyn 1984), internal representations are concatenations of symbols. • Virtues: • The Turing model provides a compositional semantics of “logical forms” for natural language. • The Turing model explains how truth-conducive processes on representations are possible. • Truth-conducive processes are understood as proof-theoretic derivations defined in terms of serial symbol manipulations. • A set of symbolic processes is truth-conducive if the set of underlying derivation rules is sound. • Vices: The Turing model is psychologically as well as neurobiologically hardly plausible. Well discussed deficits of the Turing model are: • no solution to the frame problem • no graceful degradation (cf. Horgan & Tienson, 1996) • no learning from examples • no content adressable memory • no content sensitivity of logical reasoning (cf. Gigerenzer & Hug, 1992) • etc.

5. The search for a connectionist alternative • Design a connectionist network that closely matches up with neurobiological data! • Show that the network realizes a certain algebra N of brain states! • Prove that N provides a compositional semantics of meanings for a sufficiently rich fragment of natural language! • Prove that N has a compositional semantics of contents! • Show that the network is designed in a way so the brain states of N reliably co-vary with the contents assigned to them by the compositional content function! • Provide a criterion of truth-conduciveness of state transitions for the network!

6. Semantic Models • The notion of a semantic model is borrowed from model-theoretic semantics: • Ind: The class of individuals. • Pr: The class of properties (sets of individuals). • Wr: The class of possible worlds. • (Wr): The class of propositions (sets of possible worlds) • Let S* be an extension of S such that S* is a compositional semantics of some language L. Then any algebra A is called an algebra of semantic models with respect to L iff • there is an isomorphism i from S* to A • and i preserves the constituent relations within S*, i.e.: If an entity x of some carrier set of S* is a constituent of an entity y of some carrier set of S*, then i(x) is a constituent of i(y). • The set of semantic models in A is the isomorphic counterpart of the set of propositions in S*.

7. Cells coding for like properties are organized in retinotopic clusters Optical image of orientation map of macaque V1. Colors code orientations as indicated by the colored bars (from Obermayer and Blasdel 1993) Response of a single orientation selective cell in macaque V1. The black bar illustrates the stimulus, the rectangel marks the receptive field borders (after Hubel & Wiesel 1968)

8. Cells coding properties of the same object synchronizeCells coding properties different objects de-synchronize Two cells from different columns of area 17 of cat visual cortex with overlapping receptive fields and broad orientation tuning are recorded. Orientation preferences differed by 45°. • (A and B) Stimulation with a moving light bar of intermediate orientation elicited oscillatory responses on both cells. The activities of (1) and (2) were synchronized to a significant degree. • (B and C) Due to their broad orientation tuning, the diagonally tuned cell (2) also responded to a horizontally oriented bar, which was optimal for cell (1). The activities of both cells were in synchrony. • (E and F) Adding the optimal stimulus for cell (2) eliminated synchrony within the pair of cells (from Engel et al. 1991).

9. Oscillatory networks • (a) A single oscillator consists of an excitatory (x) and an inhibitory (y) neuron. Each neuron represents the average activity of a cluster of biological cells. • (b) Synchronizing connections (solid) holds between oscillators within one layer and desynchronizing connections (dotted) between different layers. “R” and “G” denote the red and green channel. • (c) Oscillators are arranged in a 3D-topology. The shaded circles visualize the range of synchronizing (light gray) and desynchronizing (dark gray) connections of a neuron in the top layer (black pixel).

10. The network in action

11. Activity functions • Activity functions can treated as verctors in the Hilbert space L2[-T/2, +T/2] of in the interval [-T/2, +T/2] square-integrable functions. • In L2[-T/2, +T/2] the inner product in defined as • It has the countable orthonormal basis • The measure for the angle between two activity functions is:

12. The neuronal algebra Synchrony: Asynchrony: Pertaining to: Co-occurence: • Br: The set of all possible time-slice of brains over the interval I =[-T/2, +T/2]. • (Br): The set of all possible brain states (sets of time-slices) during I. • Osc: The set of activity functions L2[-T/2, +T/2]. • Cl: a subset of (Osc). Each element of Cl contains the activity functions of some property-indicating cluster of neurons. • The algebra N qualifies as a semantic model.

13. Semantic model of v1: [a1G  a1H  ã1R  ã1V  a1ã1]N red green vertical horizontal v1 v2 v3 v4 eigenmodes activity functions

14. Degree of neuronal realization = Degree of truth assignment • The fuzzy valuation functionc: (Br): [-1, +1]determines to which degree the subject’s brain realizes a neuronal semantic model. • The value can be interpreted as the degree of truth the subject assigns to the proposition that corresponds to the neuronal semantic model in question. • The fuzzy valuation function is modeled after Gödel’s min-max fuzzy logic. The degree of synchrony measures the identity of activity functions: Distinctness corresponds to the fuzzy negation of identity The best match between an activity function in the neuronal F-cluster determines to which degree an activity function pertains to the cluster The degree of co-occurrence is modeled after fuzzy conjunction:

15. Ambiguous representations Ambiguous Representations

16. Alternation • A network state is a dynamic superposition of eigenmodes. Since the strongest eigenmodes constitute semantic models, the superposition can be interpreted as a fuzzy alternation of semantic models. c([p  q]N= max{c([p]N), c([q]N}

17. Further logical tools • The existential quantifier according to fuzzy logic allows us to define: c([(x)p(x)]N)= sup{c([p(x)]N)| x  Osc}. • Example: The amount of the strongest activity function in the F-cluster corresponds to the assigned truth value of the proposition [(x)(x  F)]S. • The fuzzy subjunction allows us to define: c([pq]N) = sup{w | min{c([p]N), w} c([q])N)}. • Interpretation: c([pq]N) is a measure for the strength by which the brain state [p]N drives the brain state [q]N. • The universal quantifier according to fuzzy logic: c([(x)p(x)]N)= inf{c([p(x)]N)| x  Osc}. • Example: c([(x)(xF  xG)]N) is a measure for the weakest synchronizing connection from the F-neurons to the G-neurons.

18. Illusions: The Kanizsa-Triangle