Concept network
This article by Dr Piotr Wozniak is part of SuperMemo Guru series on memory, learning, creativity, and problem solving.
Definition
Concept network is a network of knowledge in the brain. The network maps to neural connectivity. It also maps to the parallel active network of ideas in the mind. Concept networks provide a simple illustration of how the brain works. They also show how future artificial intelligence can be built. Many branches of science provide their own formalizations and their own terminology of concept networks. For the sake of clarity, I will try to stick to a single well-defined term as in this text. Hereby I also hint how concept network relates to similar terms used in various areas of science.
More formally, a concept network is a directed graph representing conceptual connectivity in the brain. The term is analogous to a semantic network used in linguistics, concept map used in knowledge representation, mind map used as a mnemonic prop, or a connectome used in brain anatomy. A cognitive map that represents a spatial environment can be seen as the subset of the concept network of the brain.
Concept network is also analogous to other more or less formalized ontologies. In addition to its structural properties, a concept network adds the functionality of learning and reasoning. The network structure can expand via learning. Concept networks can also perform conceptual computation (reasoning) by means of spreading activation.
The brain follows a developmental trajectory from a naïve concept network to a mature rich sparse concept network in the process of conceptualization, which involves generalization via learning and selective forgetting.
Structure
Structurally, a concept network is made of concepts and connections that link the concepts. Natural neural networks correspond with, and implement concept networks in the brain.
Concepts
Concepts in a concept network can be interpreted as:
- vertices/nodes in a directed graph
- neurons in a neural network
- concepts in a thinking brain
- concept cells representing ideas in the brain
- concepts in a semantic network
- concepts in a concept map, etc.
Connections
Connections are characterized primarily by:
- two concepts that are associated by the connection
- direction (e.g. from Concept A to Concept B)
- valence (e.g. strong activation, weak inhibition, etc.)
Connections in a concept network can be interpreted as:
- association of ideas in learning and creativity
- connections between brain cells (from an axon to a dendritic spine)
- synapses whose weight or valance is determined by molecular properties of the synapse
- weights/connections in an artificial neural network
- directed edges in graphs, concept maps, semantic networks, etc.
- conditional dependencies in a Bayesian network
- probabilistic value influence in a knowledge valuation network
Function
Concept networks are capable of reasoning if they are provided with simple mechanisms known from neuroscience. These are:
- activation of individual concepts
- activation threshold based on the valance of input connections
- spreading activation from concept to concept
- memory (plasticity of connections)
- reinforcement of useful memories
- memory optimization
Activation
Concepts can be subject to activation like firing neurons. Activation of neurons or concepts is based on summation that may integrate signals in space and in time. Each neuron or concept becomes a pattern recognition device capable of recognizing input patterns (for review see: Hawkins 2016). Depolarization leading to an action potential occurs when input signals sum up to surpass the threshold potential.
Spreading activation
Activation of a concept may lead to activation of other concepts. The activation in a thinking brain can propagate throughout the concept network. This process is called spreading activation. When spreading activation is used in creative problem solving, I call it conceptual computation (see: How to solve any problem?). Creativity arises from stochastic search algorithms based on spreading activation. Conceptual computation in concept networks is the mechanism underlying human intelligence.
Memory and reinforcement
Patterns of activity in a concept network leave traces of memory. Dedicated sub-network may serve reinforcement learning, which occurs via signal valuation (see: knowledge valuation network). For example, if a decision in a network leads to a tasty meal, reinforcement will strengthen connections that led to a reward. If reasoning leads to satisfactory creative outcomes, activation pathways in the network will be reinforced as well. Memory strenghtening is based on the rules determined by the two component model of long-term memory
Learning
Memory makes it possible for concept networks to grow via the acquisition of knowledge. The expansion is dendritic in nature in that new concepts are always established by creating new connections to existing concepts. For example, new brain cells may establish connections with existing active brain cells. Effective activations help concept networks stabilize their long-term knowledge, and provide generalization by means of selective forgetting. Conceptualization is the emergence of new concepts in a developing concept network.
Memory optimization
For their efficient function, concept networks should undergo regular memory optimization, which provides streamlined connectivity. In the brain, this process occurs in sleep (see: Neural optimization in sleep). Memory optimization can metaphorically be compared to disk defragmentation.
Evolution of connectivity
The history of neural network development in artificial intelligence is complex and meandering. One of the central themes was a debate on the pros and cons of neural network architectures and algorithms that would best capitalize on individual structural approaches.
Under a microscope, the 2.5 mm thick neocortex seems to exhibit a simple, 6-layered, repetitive architecture that follows similar patterns across the entire cortical surface. That repetitiveness might be a sign of universally versatile functionality evolved to perfection. It might equally well be a reflection of the evolving brain adding computational power at a low cost by repeating an architectural solution that works on a small scale. What is less visible and harder to study is that the brain exhibits wiring on demand. Its microarchitecture is determined by functionality, which is determined while exploring the environment. Such evolving architecture may be implemented in artificial networks by setting weight of connections to zero. However, considering the number of permutations on all possible connections in a structure that reaches a trillion of synapses, architectural connectivity determined by functionality is the simplest answer to a streamlined operation of the entire concept network.
Connectivity optimization
At the level of a neuron we can see several potential routes towards the optimization of connectivity. Those routes are dendritic sprouting on the input, axonal growth on the output, and arborization thereof. Most importantly, dendritic branches may scout the proximal tissue environment by sending out filopodia than may turn out into dendritic spines that establish new synapses. Those synapses are an example of concept network connections established on the basis of experience. See: Dendritic arbors undergo branching followed by stabilization.
The brain's concept network connectivity never stops evolving (see: Conceptualization). The evolution of connectivity has the following prime components:
- developmental connectivity of a growing brain (underlying future brain architecture)
- structural connectivity of a learning brain
- memory stabilization in response to activity (see: Neurostatistical model of memory)
- generalization as a result of selective forgetting
Connectivity in development
The evolution of connectivity in development follows a simple plan that initially relies on the mechanisms known from laying out the outlines of other body organs. However, at birth, a baby is equipped with only a small set of basic reflexes such as the rooting reflex necessary for feeding. Beyond those simple reflexive needs, the high-level functionality of the brain at birth is largely undetermined. Only the very basic architecture is laid in place, for example, to ensure that the input from the optic nerve is wired to reach the areas of the cortex that are predestined to become the visual cortex.
Functional connectivity
Once the general outline of the brain is established, the detailed wiring is the reflection of the functional needs, which in turn are determined by the exposure to the environment and by exploration. Sensory deprivation will result in corresponding cortical areas taken over by different functionality. In rich environments, individual areas of the cortex will compete for optimum wiring. This will result in individuals with various skills and predispositions. Genetic and environmental diversity will have a contribution to neurodiversity.
Memory
With passing years, the demand for neurogenesis will diminish, the rate of sprouting will diminish, and the rate of synaptic pruning will increase. The total number of synapses will decline in teen years leading to fast thinking, and specialization of interests. However, the core processes underlying the evolving connectivity will never stop: neurogenesis, neuronal process sprouting, synaptic pruning, creativity, learning, memory optimization, and memory stabilization.
Smart neurites
From the architectural point of view, the most important part of the connectivity evolution is the ability of axons, dendrites, and dendritic filopodia to respond to signals in the environment. Of those signals, activity of the nearby neurons and neurites is central. Those control signals are not fully understood (esp. in ref. to target specificity), however, we can safely bet that the effect of the control will reflect functional needs of the brain. An undifferentiated neuron will usually develop a few neurites of which one will dominate to become an axon within a week or so. New neurons or poorly utilized old neurons will tend to seek new experience by dendritic sprouting towards active axonal targets. In contrast, well-established concept cells will gladly share their knowledge of the world by directing axonal growth towards friendly receptive dendritic targets (see pictures: mechanism, outcome). This process may result in white matter connectivity that may connect disparate areas of the cortex, including areas in the opposite hemisphere. Finally, dendritic filopodia seem to have their contribution to the spacing effect and memory stabilization by establishing and stabilizing connections along axonal routes of interest (see: Two component model of memory stability).
Example: devouring apples
When we see an ripe apple hanging on a tree, we are tempted to reach and consume. For this act to happen, a few years of concept network connectivity evolution are necessary. The evolving architecture of the brain makes it possible to use convolutional networks to generalize the visual concept of an apple. That concept may be armed with the connectivity needed to establish the attribute of ripeness. In its early exploration, the toddler will condition and stabilize the knowledge on the edibility of ripe apples. That knowledge will later grow richer with details of nutritional value, risk factors and little nugget of wisdom such as: Do not gorge on apples before a marathon. For a toddler to establish the knowledge of the suitability of ripe apples for consumption, it needs to connect (1) the ripe apple concept, or the concept of apple activated in conjunction with the attribute of ripeness, or a larger number of concept cells, with (2) the attribute of edibility represented by a separate concept cell, or even a concept sub-network. Before all that edibility learning can take place, the motor program for moving the apple in the direction of the mouth must be perfected. This may involve a trigger concept "consume" with a set of lesser concepts that determine the necessary parameters of the motor program and its reference frames: location of the apple, position of the arm, relative coordinates of the mouth, etc. In the final act, the activated concept of a ripe apple, its proximity attributes, signals from the appetite control system, and other participating concept may trigger a semi-autonomous motor program for consuming the apple.
Semantic web
Tim Berners-Lee's semantic web can be seen as a prelude to implementing a concept network that will be able to undertake artificial reasoning. Semantic web will likely power future artificial intelligence. This intelligence will be a meta-network of concepts that will interface with a diversity of human brains and distributed intelligences as part of its processing power.
Artificial intelligence
I hypothesize that the fastest pathway towards building artificial intelligence is to build artificially intelligent baby brain with 3 essential structural sub-components:
- artificial concept network with the ability to (1) expand via learning and (2) spread activation (an equivalent of the neocortex)
- input networks with feature extraction (an equivalent of the sensory system in the brain, e.g. the visual system)
- reinforcement networks with reward valuation (an equivalent of reward systems in the brain underlying the learn drive)(see: Pleasure of learning)
With good learning algorithms, such a baby brain connected to an autonomous robot might explore the world and acquire knowledge in the same way as human babies do. Alternatively, the baby brain could just explore and integrate with the semantic web. See: Artificial intelligence needs to sleep
My cursory interest in artificial intelligence seems to indicate that there are key areas that are still poorly appreciated:
- power of grandmother cells and concept networks
- power of the spacing effect in the two component model of long-term memory
- network-level reward in contrast to the output-level reinforcement
Only the mechanics of the learn drive seems non-trivial in this equation. It is the network-level reward that converts a skeleton of a concept network into a self-learning structure with emergent intelligence and meaningful knowledge. Output-level reinforcement makes concept networks computationally implausible. By analogy, passive schooling fails because extrinsic rewards have minimal impact on long-term learning. For that reason, educators and artificial intelligence researchers should study the pleasure of learning.
Terminology
In my texts, depending on the context, I used to employ better known terms such as semantic network when speaking about knowledge or learning, concept map when speaking about creativity, reasoning or SuperMemo, or neural network when speaking about biological implementation of concept networks.
Other similar terms with similar definitions and/or properties are used in the context of human and artificial intelligence. I will try to avoid those terms to minimize ambiguity. Those include: cognitive maps, frame networks, conceptual diagrams, Bayesian networks, Markov networks, belief networks, decision networks, knowledge networks, symbolic networks, knowledge spaces, semantic spaces, topic maps, semantic maps, semantic graphs, conceptual graphs, taxonomies, topologies, hierarchies, schemas, conceptual maps, neural maps, cortical maps, developmental networks (Juyang Weng), and more.
In my own knowledge collection, I first noticed the term "concept network" used loosely in a paper by Robert French in 2000. The term concept network is not popular, but it is pretty unique, which may help unify the terminology used to describe a thinking brain.
Further reading
- a hypothetical mechanism by which a concept network learns about zebras is presented in: The truth about grandmother cells
- a model of conceptual computation based on a concept network is presented in: How to solve any problem?
- a model of childhood amnesia based on a 'concept network is suggested in: Conceptualization theory of childhood amnesia
- molecular and structural aspects of evolving connectivity are listed in: Neurostatistical model of memory
- self-organization of societies can be enhanced by following the functional principles of a concept network: Society as a concept network