Semantic Engine
(Part I)

Natural Language Processing and Semantics

To process a text analysis, the Semantic Engine operates in 6 stages:

1. Sentence and Proposition Hashing,
2. Ambiguity Solving (with respect to the words of the text),
3. Identification of Equivalent classes (senses),
4. Statistics, detection of Bundles and Episodes,
5. Detection of the Most Characteristic Parts of text,
6. Layout and display of the result.

Words are grouped together in several main Word categories. Among these, six are of interest to us:

• Verbs,
• Connectors (conjunctions, conjunctive phrases),
• Modalities (adverbs or adverbial phrases),
• Qualifying Adjectives,
• Personal pronouns,
• Substantives and Proper Nouns.

An analysis is highly complex. During the process, the software will:

• assign all the significant words to the above categories;
• analyze their distribution into subcategories (Word categories, Equivalent classes, see below);
• examine their occurrence order, both within the propositions (Relations, Actants and Acted) and throughout the text (Distribution graph, Bundles, Episodes, Most Characteristic Parts of text, see below).

Propositional hashing (Lexical Analysis)

To simplify the analysis, the Semantic Engine divides the text into propositions (simple sentences). This first stage is based on a scrutiny of the punctuation, and on a complex process of syntax analysis, which will not be detailed here. This yields highly reliable co-occurrence statistics (Relations), since it is not possible for two words to fit into the same grammatical proposition if they are not closely connected. Propositional hashing is bound to involve errors (propositions that are either too short or too long), but this does not affect the results.

Ambiguity solving (Semantic and Lexical Analysis)

The automatic interpretation of words in any living language, either written or spoken, requires the solving of numerous ambiguities:

• grammatical and syntactic (the word "pound", for example, corresponds to a noun in "a pound of fish", and to a verb in "Pound the spices into a paste"),
• semantic (the noun "pound", for example, can refer to a weight or to a currency). On average, one word out of four is deemed ambiguous.

One of the main functions of this software is to solve these ambiguities by means of a set of Artificial Intelligence problem-solving algorithms. Though a perfect result is impossible to achieve, the error rate is low enough to guarantee an accurate analysis of your text.

Read the next part:
Word Categories and Semantics

 

---