\chapter{Design of the Simulator Kernel}
\label{KernelDesign}
\thispagestyle{empty}

\section{Network Model of the Simulator}

The network model, on which the simulator kernel is based, was
designed to accommodate many network types. An important feature of the
kernel is the complete encapsulation of all internal data structures
and the efficient memory management.

The network model of the simulator kernel has the following properties:

%\renewcommand{\labelitemi}{\bullet}

\begin{itemize}

\item each unit has an arbitrary, user definable 
 \begin{itemize}

 \item {\em activation function} which takes the network input, the
  present activation, and the threshold to compute the new activation.

 \item {\em output function} which computes the output value from the
  activation of the unit.
 \end{itemize}

\item each unit has an arbitrary number (possibly zero) of inputs which
can be defined by the user. These so called {\em sites} have
separate user definable input functions.

\item sites may have an arbitrary number of connections from other
units. A recurrent connection to the unit itself is possible.
\end{itemize}


\section{Design Factors} 

The goal in designing the simulator kernel was to meet the following
specifications:

\begin{itemize}

\item representation of a universal network model for small to medium
sized networks ($10^4$ units, $10^6$ connections)
\item interactive manipulation of the net by the user
\item encapsulation of all internal data structures with proper interfaces
\item extendability
\item very high efficiency
\item modularity
\item portability
\end{itemize}

The size of the networks that can be handled by the kernel is limited
only by the size of (virtual) memory and the address space of the
computer used. The simulator memory management relieves the operating
system (UNIX), especially with larger nets. In interactive mode, the
user has some powerful commands at hand to create and manipulate
networks. These interface functions reduce the complex
internal representation of the data to a representation at the logical
network level.

Naturally, the demands of encapsulation and efficiency contradict each
other. Nevertheless, a good compromise has been found: the functions,
that can be defined by the user (activation and site functions)may be
used with a macro library to access the kernel structures. This
principle makes the combination of tight encapsulation and high
execution speed possible. Measurements on several computer systems
yielded the following propagation rates:


\begin{table}[h]
\small{
\begin{tabular}{|l|l|r|r|r|}     \hline
{\bf Machine type}      & {\bf Operating System} 
                                        & {\bf Propagations}
                                        & {\bf Weight Updates}  \\
                        &               & connections/sec. 
                                        & conn. updates/sec.    \\
                        &               & (CPS) 
                                        & (CUPS)                \\ \hline
Intel 80486-33          & Linux         &   453.000 &   194.000 \\ \hline
DECserver 5400          & Ultrix V4.2   &   398.000 &   175.000 \\ \hline
DECstation 2100         & Ultrix V4.2   &   494.000 &   211.000 \\ \hline
DECstation 3100         & Ultrix V4.2   &   689.000 &   287.000 \\ \hline
DECstation 5000/200     & Ultrix V4.2   &   733.000 &   248.000 \\ \hline
SUN 3/60                & SUN-OS 4.1.1  &   120.000 &    42.000 \\ \hline
SUN SPARCstation ELC    & SUN-OS 4.1.1  &   791.000 &   334.000 \\ \hline
SUN SPARCstation 2      & SUN-OS 4.1.1  &   950.000 &   415.000 \\ \hline
SUN SPARCstation 10     & SUN-OS 4.1.1  &   950.000 &   415.000 \\ \hline
IBM RS 6000/320         & AIX V.3.1     & 1.998.000 &   773.000 \\ \hline
IBM RS 6000/320H        & AIX V.3.1     & 2.207.000 &   814.000 \\ \hline
HP 9000/720             & HP-UX 8.07    & 1.707.000 &   764.000 \\ \hline
HP 9000/730             & HP-UX 8.07    & 2.147.000 & 1.093.000 \\ \hline
\end{tabular}}
\end{table}

\index{Performance Data}
\index{Propagation!Rate}
The propagation rate measures the speed of the simulator in
recall mode, i.e. the forward propagation rate. In this mode no weight
updates take place. The usual measurement unit is connections/sec
(CPS).
\index{CPS}

\index{Weight!Update Rate}
The weight update rate measures the speed of the training of a
fully connected feedforward network trained with 'vanilla` (on line)
backpropagation. Because there is a forward propagation, a backward
propagation and a weight update phase for every pattern in each cycle,
the weight update rates, measured in connection updates/sec. (CUPS)
\index{CUPS}
are usually lower by a factor of between 2 and 3 than the propagation
rates.

These performance numbers have been obtained on machines running in
our lab during normal use, with other users on the machines, with
different main memory sizes and with the SNNS home directory mounted
via NFS over the ethernet. Therefore, these numbers should only be
regarded as performance indicators of SNNS but may not be quoted as
machine architecture benchmarks. 

The use of simulators for neural nets almost demands the use of
parallel computers like the Connection Machine CM-2~\cite
{hil85,hil86,gab86} or the MasPar MP-1, because of the inherent
parallelism of the algorithm. The use of parallel computers will
therefore surely increase the above values.


\section{Layer Model of the Simulator Kernel}


The simulator kernel is internally structured in three layers. Each
higher layer represents a higher level of abstraction and is based on
the layers below.

\begin{itemize}
\item The bottom layer is the SNNS memory management. It sits on top
of the Unix memory management and offers functions to allocate and
free data structures for the layer above.

\item The next layer contains all functions to operate on, and to
modify the network as well as propagation functions.

\item The interface functions for the graphical user interface
are located in the topmost layer.

\item In the same layer as the function interface, there exists a file
interface to the Nessus compiler.
\end{itemize}

In chapter~\ref{KernelStruct} the data structures are described in
detail. The description of the compiler interface can be found in
appendix A of the SNNS user manual.
