The Logos Model as
Symbolic Neural Net
The graphic sequence in what follows illustrates an analysis of
a complex English
sentence, 55 words in length:
Let me also note that because of the relatively close
movement of the
Canadian dollar with the U.S. dollar, our currency has declined along with
the U.S. dollar against these other currencies this past year, removing much
of the exchange rate distortion that was hampering the ability of Canadian
firms to compete with producers overseas.
The graphics below show the various steps actually taken as this
sentence processes
through the Logos Model pipeline. The graphic metaphor being used to illustrate the
process is that of a neural net. The Logos Model is claimed to have certain affinities
with biologically oriented symbolic neural nets.
In the graphic immediately below, the resemblance of the Logos
Model
to neural net architecture is immediately evident. Click on the hidden
layers R1, R2, P1, P2, P3, P4 to follow the sample sentence (shown above) as it
progresses through the sequence of hidden layers.
Neural Net Architecture of the Logos
System
The Logos Model is depicted here as a six-stage neural net.
V1-V7 are input/output vectors containing symbolic (SAL)
representation of the input sentence. (Earlier in this tutorial
this input/output vector is identified as an SWORK array.) The
V1 vector can be 70 cells in length. Input sentences longer
than this are broken up into two (or more) sentences
Shaded rectangles R1, R2, P1, P2, P3, P4 are hidden layers
(R=RES, P=PARSE). On average each of these hidden layers
comprises from two to five thousand units (rules), each
specialized for some symbolic SAL pattern at progressively
higher levels of abstraction. Large numbers of essentially
shallow processing units (that perform relatively minor
functions) constitutes another point of similarity between the
Logos Model and neural nets.
Units are often constrained by top-down, network-wide state
conditions which must be satisfied for the unit to fire. (Provision
for this is not shown in this graphic. See graphic for RES2).
The principal work of the neural net is to incrementally disam-
biguate and decomplexify the input stream entering via V1. ( In
the diagram, ambiguity is expressed by unit shading in vectors
V1-V7, which is gradually change to white as ambiguities are
resolved. Complexity is expressed by the number of units in
these vectors, gradually lessening as the string becomes more
abstract.
Note features of the Logos Model that differ from standard
artificial neural nets (ANN): (a) not all units interconnect, indi-
cating that unit interconnectivity is selective, based on unit
specialization; (b) although this is not apparent in the present
diagram, the network employs recurrent circuitry, allowing a
hidden layer to feed-back both to its input vector and to itself.
This explains why some units fire but do not project to the
next layer.
The various circuit types used in hidden layers of the
Logos Model are
illustrated below.
Variety of Circuit Types in the Logos
Model
back
_______________________
At the beginning of the process, after lexical look-up,
the input sentence is
re-presented as an input vector (V1) comprising a chain of SAL entities (simplified
here merely as parts-of-speech). This vector is then submitted to RES1-RES2 for
macro-parsing.
Some of the words in the sentence were found to have more than
one part-of-
speech in the lexicon, and are thus ambiguous. For example, "close" (9th
lexical
match) can be a transitive verb, an intransitive verb, or an adjective. It is
RES1-RES2's task to create a path through the input vector, and in doing so to
resolve "close" and every other ambiguous element to a single part of speech,
resolving "close" in this case to an adjective (j).
There are over two million possible paths in this rather longish
sentence and so
the opportunity for error is high. Nevertheless, RES must do its work with a high
degree of accuracy since even minor errors in RES can propagate into major
errors down the pipeline, affecting the quality of translation. On average, RES
parses with no more than a 2% error rate in part-of-speech assignments. (A
small percentage of these are corrected by subsequent PARSE modules.)
To achieve this accuracy, RES must effectively perform a
macro-parse of the
sentence, recognizing, e.g., all clausal types and transitions. Such top-down
information regarding clause types and transitions exercises constraints on
the bottom-up process.
For example, when a new clause is entered, this fact is
reflected in the top-down
picture being maintained of the sentence. The top-down picture in turn enables
certain rules whose function is to tag the verb of a clause, and inhibits other rules
whose function is to resolve N/V homographs to N (the rule is inhibited
because its presupposition that the current clause already has a verb is false).
This interaction between top-down information and bottom up processes is
effected entirely through the mechanism of constraint satisfaction, another
reason why the Logos Model is sometimes said to resemble a neural net.
back
_______________________
The Logos Model includes so-called recurrent
circuitry whereby the
output of a unit (cell or rule) is fed back to all the other units, affecting
their ability to become active or not. The above graphic is a metaphor of
this top-down control process.
When a cell (unit or rule) fires, it typically does two things: (a) resolves
one or more ambiguities in the input vector (V2) and sends the result to
the output vector (V3), thus defining a path through the unresolved input
structure; (b) sends a signal about its actions to a top-down control array.
This control array maintains a picture of what has happened in the current
clause and in previous clauses of the sentence. For example, when a rule
fires defining a verb, the top-down control array is told that fact along with
the SAL code of the rule.
A subsequent rule in that clause that might want to
resolve an element to
a verb would only be allowed to do so if the SAL code of the first verb, stored
in the top-down array, was pre-verbal (i.e, has a verbal complement).
The V3 output vector of RES2 now becomes the input vector to PARSE1.
The
top-down information is also passed to the PARSE modules.
back
_______________________
The resolved vector V3 output from RES2 here becomes the
input vector to PARSE1.
PARSE1 initiates a micro-parse of the sentence. Note how the long, complex
noun phrase "the relatively close movement of the... " in V3 is reduced in the
output vector V4.
For a general discussion of PARSE1 functions, click here.
back
_______________________
Vector V4 from PARSE1 here becomes the input
vector to PARSE2. Note how
the highlighted complex NP in V4 is reduced to a single NP in output vector V5.
PARSE2 also extracts nested clausal elements, such as relative clauses, and
places them at the end of the sentence, treating them as a separate sentence
with a dummy subject. This is done to keep clauses as simple as possible.
Parentheticals and absolute constructions are also extracted in this fashion.
In all such cases, a place marker (trace) is left behind to allow for reinsertion
of extracted material in the transfer stage.
In the present example, the relative clause itself contains an embedded
verb complement to the noun "ability". This complement is extracted
from the relative clause and also treated separately.
(For a general discussion of PARSE2 functions,
click here.)
back
_______________________
Vector V5 from PARSE2 here becomes the input vector to
PARSE3. Note how
the NP ("movement") in V4 is now head element of a prepositional phrase (PP)
in output vector V6. All other prepositional phrases are similarly reduced to
PP.
(For a general discussion of PARSE3 functions, click here.)
back
_______________________
PARSE4 completes the micro-parse, reducing the input
sentence to its
clausal constituents in output vector V7. The constituents in V7 then connect
to the S node (not shown).
PARSE4 thus completes input sentence analysis. Now begins the transfer
phase (TRAN). TRAN exactly mirrors the structure of PARSE, consisting of four TRAN
modules, beginning with TRAN1. Architecturally, the series of vectors and hidden
layers shown above for PARSE applies equally to TRAN.
The TRANs rebuild the source parse tree, following the
guidelines now provided by
PARSE. (A relatively pro forma function since all constituents have now been
disambiguated and decomplexified.) But the TRANs now also build a target tree.
As the TRAN modules progress from TRAN1 to TRAN4, the source parse tree is being
transferred, node by node, to a target language equivalent, both syntactically and
semantically (within the ken of the knowledge base).
TRAN may also do additional micro-analysis of the source sentence in the case of
a unique target requirement that PARSE did not attend to.
The major function of the TRANs, to be sure, is the syntactic
and semantic
transfer of the source elements to a target language equivalent. In effect,
the TRAN modules create two output vectors, one for the source, and a second
and more critical one for the target language. As the sentence progresses
through the TRAN modules,
Click here for a general discussion of PARSE4
functions and also of the
Transfer Phase.
back
_______________________
In FIG. 1, we see short-term memory cells A, B, C in the input
vector
attempting to match on long-term memory cells L, M, N in the
hidden layer. Only L(bac) is specialized for the A, B, C properties
of the input vector. M and N have only partial matches and cannot
become active therefore. (L may have to compete with other matched
cells of the hidden layer, not shown. The connection with the greatest
weight gets to fire. Weight is a function of pattern length and
semantic
specificity, among other things.
In FIG. 2, we see a blow-up of synapse occurring between cell A
in the
input vector and the corresponding receptor membrane of stored
cell L. Shown is the axon of cell A in synapse with receptor site
(dendrite a) of cell L.
Synapse occurs because the SAL chain transmitted from A matches
that of L(a). To stretch the biological metaphor here, the matching of
SAL chains is likened to the matching of amino-acid chains
(neurotransmitter) in synapse.
Note that the dendritic receptor site does not have to
accommodate
all the signals sent to it by the axon. However, another receptor site
that accommodates more of these signals will have greater
connectivity weight and will more likely get to fire.
(back)
