\renewcommand{\topfraction}[0]{1.0}
\renewcommand{\bottomfraction}[0]{1.0}
\renewcommand{\textfraction}[0]{0.0}
\renewcommand{\floatpagefraction}[0]{1.0}

\chapter{Network Creation Tools}
\label{Bignet}
\index{BigNet}
\thispagestyle{empty}


SNNS provides five tools for easy creation of large, regular networks.
All these tools carry the common name BigNet. They are called by
clicking the button \fbox{\tt BigNet} in the manager panel. This
invokes the selection menu of figure~\ref{Bignetmenu}, where the
individual tools can be selected. This chapter gives a short
indroduction to the handling of each of them.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{xgui_bignet/bn_menu.ps}
  \end{minipage}
\caption{The BigNet Menu Window}
\label{Bignetmenu}
\end{center}
\end{figure}



\section{BigNet for Feed-Forward Networks}
\subsection{Terminology of the Tool BigNet}
\label{BigNetTerminology}

BigNet subdivides a net into several planes.  The input layer, the
output layer and every hidden layer are called a {\bf plane} in the
\index{Plane}
notation of BigNet\@. A plane is a two-dimensional array of units.
Every single unit within a plane can be addressed by its coordinates.
The unit in the upper left corner of every plane has the coordinates
(1,1).  A group of units within a plane, ordered in the shape of a
\index{Cluster}
square, is called a {\bf cluster}. The position of a cluster is
determined by the coordinates of its upper left corner and its
expansion in the x direction (width) and y direction (height)
(fig.~\ref{ClustersUnits}).

\index{BigNet!Window}
\index{Window!BigNet Window}
\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.75}{xgui_bignet/bn_window.ps}
  \end{minipage}
\caption{The BigNet window for Feed-Forward Networks}
\label{BigNetWindow}
\end{center}
\end{figure}

\clearpage

BigNet creates a net in two steps:
\begin{enumerate}
   \item Edit net: This generates internal data structures in BigNet
         which describe the network but doesn't generate the network yet.
        This allows for easy modification of the network parameters before
        creation of the net. \\
         The net editor consists of two parts:
         \begin{enumerate}
            \item The plane editing part for editing planes. The input data is
                  stored in the {\bf plane list}. 
            \item The link editing part for editing links between planes. The 
                  input data is stored in the {\bf link list}. 
         \end{enumerate}
   \item Generate net in SNNS: This generates the network from the
         internal data structures in BigNet\@.
\end{enumerate}

Both editor parts are subdivided into an input part (Edit plane, Edit
link) and into a display part for control purposes (Current plane,
Current link). The input data of both editors is stored, as described
above, in the plane list and in the link list. After pressing
\fbox{ENTER}, \fbox{INSERT}, or \fbox{OVERWRITE} the input data is
added to the corresponding editor list. In the control part one list
element is always visible.  The buttons
\epsfbox{xgui_bignet/bn_button_first.ps},
\epsfbox{xgui_bignet/bn_button_prev.ps},
\epsfbox{xgui_bignet/bn_button_next.ps}, and
\epsfbox{xgui_bignet/bn_button_last.ps}\ \ enable moving around in the
list.  The operations DELETE, INSERT, OVERWRITE, CURRENT PLANE TO
EDITOR and CURRENT LINK TO EDITOR refer to the current element.
Input data is only entered in the editor list if it is correct,
otherwise nothing happens.


\subsection{Buttons of BigNet}

\fbox{\tt ENTER} : 
Input data is entered at the end of the plane or the link list.

\fbox{\tt INSERT} : 
Input data is inserted in the plane list in front of the current plane.

\fbox{\tt OVERWRITE} : 
The current element is replaced by the input data.

\fbox{\tt DELETE} : 
The current element is deleted.

\fbox{\tt PLANE TO EDIT} : 
The data of the current plane is written to the edit plane.

\fbox{\tt LINK TO EDIT} : 
The data of the current link is written to the edit link.

\fbox{\tt TYPE} :  
The type (input, hidden, output) of the units of a plane is
determined.

\fbox{\tt POS} : The position of a plane is always described relative
(left, right, below) to the position of the previous plane. The upper
left corner of the first plane is positioned at the coordinates (1,1)
as described in Figure~\ref{PlanePositioning}. BigNet then
automatically generates the coordinates of the units.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.9}{xgui_bignet/bn_clusters_units.ps}
  \end{minipage}
\caption{Clusters and units in BigNet}
\label{ClustersUnits}
\end{center}
\end{figure}
%\vspace{1cm}
\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{1.0}{xgui_bignet/bn_plane_pos.ps}
  \end{minipage}
\caption{Positioning of the planes}
\label{PlanePositioning}
\end{center}
\end{figure}


\fbox{\tt FULL CONNECTION} : 
A fully connected feed forward net is generated. If there are n planes
numbered $ 1..n $ then every unit in plane $i$ with $ i>0 $ is
connected with every unit in plane $i+1$ for all $ 1 \leq i \leq n-1 $.


\fbox{\tt SHORTCUT CONNECTION} :
If there exist $n$~planes $1 \dots n$ then every unit in plane~$i$ with
$ 1 \leq i<n $ is connected with every unit in all planes~$j$ with 
$ i < j \leq n $.

\fbox{\tt CREATE NET} :
The net described by the two editors is generated by SNNS\@. The default
name of the net is {\bf SNNS\_NET.net}. If a net with this name
already exists a warning is issued before it is replaced.

\fbox{\tt CANCEL} : 
All internal data of the editors is deleted.

\fbox{\tt DONE} :
Exit BigNet and return to the simulator windows.


\subsection{Plane Editor}
\index{BigNet!Plane Editor}
\label{BN_planeEdit}

Every plane is characterized by the number of units in x and y
direction.  The unit type of a plane can be defined and changed by
\fbox{\tt TYPE}. The position of the planes is determined relative to the
previous plane.  The upper left corner of plane no.~1 is always
positioned at the coordinates~$(1,1)$. Pressing \fbox{\tt POS}, one can
choose between `left', `right' and `below'. Figure
\ref{PlanePositioning} shows the layout of a network with 6 planes
which were positioned relative to their predecessors as indicated
starting with plane 1.

\index{Plane!Number}
Every plane is associated with a plane number. This number is
introduced to address the planes in a clear way. The number is
important for the link editor. The user cannot change this number.

In the current implementation the z coordinate is not used by BIGNET\@.
It has been implemented for future use with the 3D visualization
component.


\subsection{Link Editor}
\index{BigNet!Link Editor}

A link always leads from a source to a target. To generate a fully
connected net (connections from each layer to its succeeding layer, no
shortcut connections), it is only sufficient to press the button
\fbox{\tt FULL CONNECTION} after the planes of the net are defined.
Scrolling through the link list, one can see that every plane $i$ is
connected with the plane $i+1$.  The plane number shown in the link editor
is the same as the plane number given by the plane editor.

If one wants more complicated links between the planes one can edit them 
directly. There are nine different combinations to specify link
connectivity patterns:

\begin{math}
\mbox{\rm Links from} \left\{ \begin{array}{c}
                        \mbox{\rm all units of a plane} \\
                        \mbox{\rm all units of a cluster} \\
                        \mbox{\rm a single unit}
                   \end{array} \right\}
\mbox{\rm to}      \left\{ \begin{array}{c}
                        \mbox{\rm all units of a plane} \\
                        \mbox{\rm all units of a cluster} \\
                        \mbox{\rm a single unit}
                   \end{array} \right\}.
\end{math}

Figure~\ref{possibleInput} shows the display for the three possible input
combinations with (all units of) a plane as source. The other
combinations are similar. Note that both source plane and target plane
must be specified in all cases, even if source or target consists of a
cluster of units or a single unit.  If the input data is inconsistent
with the above rules it is rejected with a warning and not entered
into the link list after pressing \fbox{\tt ENTER} or \fbox{\tt OVERWRITE}.


\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{xgui_bignet/bn_link_input.ps}
  \end{minipage}
\caption{possible input combinations with (all units of) a plane as
source, between 1) a plane and a plane, 2) a plane and a cluster, 3) a
plane and a unit. Note that the target plane is specified in all three
cases since it is necessary to indicate the target cluster or target unit.}
\label{possibleInput}
\end{center}
\end{figure}


With the Move parameters one can declare how many steps a cluster or a
unit will be moved in x or y direction within a plane after the
cluster or the unit is connected with a target or a source. This
facilitates the construction of receptive fields where all units of a
cluster feed into a single target unit and this connectivity pattern
is repeated in both directions with a displacement of one unit.

The parameter dx (delta-x) defines the step width in the x direction
and dy (delta-y) defines the step width in the y direction. If there is no
entry in dx or dy there is no movement in this direction.
Movements within the source plane and the target plane is independent
from each other. Since this feature is very powerful and versatile it
will be illustrated with some examples.

%\clearpage

\subsubsection*{Example 1: Receptive Fields in Two Dimensions}

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{1.0}{xgui_bignet/bn_example_net_1.ps}
  \end{minipage}
\caption{The net of example 1}
\label{figarrow4}
\end{center}
\end{figure}


There are two planes given (fig.~\ref{figarrow4}). To realize the links 

\begin{center}
   source: plane 1 (1,1), (1,2), (2,1) (2,2) $\longrightarrow$  
   target: plane 2 (1,1) \\
   source: plane 1 (1,2), (1,3), (2,2) (2,3) $\longrightarrow$  
   target: plane 2 (1,2) \\
   source: plane 1 (2,1), (2,2), (3,1) (3,2) $\longrightarrow$  
   target: plane 2 (2,1) \\
   source: plane 1 (2,2), (2,3), (3,2) (3,3) $\longrightarrow$  
   target: plane 2 (2,2) \\
\end{center}

between the two planes, the move data shown in
figure~\ref{tablearrow4} must be inserted in the link editor.

%\begin{table}
%\center{\input{xgui_bignet/bn_example_disp_1}}
%\caption{Example 1}
%\label{tablearrow4}
%\vspace{0.5cm}
%\center{\mbox{\input{xgui_bignet/bn_example_disp_2}\hfill %\hspace{5mm}
%             \input{xgui_bignet/bn_example_disp_3}}}
%\caption{Example 2}
%\label{tablearrow5}
%\end{table}

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{xgui_bignet/bn_example_disp_1.ps}
  \end{minipage}
\caption{Example 1}
\label{tablearrow4}
\end{center}
\end{figure}

%\vspace{0.5cm}

First, the cluster (1,1), (1,2), (2,1) (2,2) is connected with the
unit (1,1). After this step the source cluster and the target unit are
moved right one step (this corresponds to dx = 1 for the source plane
and the target plane). The new cluster is now connected with the new
unit. The movement and connection building is repeated until either
the source cluster or the target unit has reached the greatest
possible x value.  Then the internal unit pointer moves moves down one
unit (this corresponds to dy = 1 for both planes) and back
to the beginning of the planes.  The ``moving'' continues in both
directions until the boundaries of the two planes are reached.


\subsubsection*{Example 2: Moving in Different Dimensions}

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{xgui_bignet/bn_example_disp_2.ps}
  \end{minipage}
\caption{Example 2}
\label{tablearrow5}
\end{center}
\end{figure}

This time the net consists of three planes (fig. \ref{E2}). To create 
the links

\begin{center}
   source: plane1 (1,1), (1,2), (1,3) $\longrightarrow$  
   target: plane 2 (1,1) \\
   source: plane1 (2,1), (2,2), (2,3) $\longrightarrow$  
   target: plane 2 (1,2) \\
   source: plane1 (3,1), (3,2), (3,3) $\longrightarrow$  
   target: plane 2 (1,3) \\
   source: plane1 (1,1), (2,1), (3,1) $\longrightarrow$  
   target: plane 3 (1,1) \\
   source: plane1 (1,2), (2,2), (3,2) $\longrightarrow$  
   target: plane 3 (1,2) \\
   source: plane1 (1,3), (2,3), (3,3) $\longrightarrow$  
   target: plane 3 (1,3) \\
\end{center}


between the units one must insert the move data shown in figure
\ref{tablearrow5}.  Every line of plane 1 is a cluster of width 3 and
height 1 and is connected with a unit of plane 2, and every column of
plane 1 is a cluster of width 1 and height 3 and is connected with a
unit of plane 3. In this special case one can fill the empty input
fields of ``move'' with any data because a movement in this
directions is not possible and therefore these data is neglected.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{1.0}{xgui_bignet/bn_example_net_2.ps}
  \end{minipage}
\caption{The net of example 2}
\label{E2}
\end{center}
\end{figure}


\subsection{Create Net}
\index{BigNet!Create Net}
\index{Create Net}

After one has described the net one must press \fbox{\tt CREATE NET} to
generate the net in SNNS\@. The weights of the links are set to the
\index{Link!Weight Default Value}
default value $0.5$. Therefore one must initialize the net before one
starts learning. The net created has the default name {\bf
SNNS\_NET.net}. If a net already exists in SNNS a warning is issued
before it is replaced. If the network generated happens to have two
units with more than one connection in the same direction between them
then SNNS sends the error message ``Invalid Target''.

\clearpage

\section{BigNet for Time-Delay Networks}
\index{BigNet\_td}
\index{BigNet!Time Delay}

The BigNet window for Time Delay networks
(figure~\ref{BigNetWindowTD}) consists of three parts: The Plane
editor where the number, placement, and type of the units are defined,
the link editor, where the connectivity between the layer is defined,
and three control buttons at the bottom, to create the network, cancel
editing, and close the window.


\index{BigNet\_td!Window}
\index{Window!BigNet\_td Window}
\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{xgui_bignet/bn_window_td.ps}
  \end{minipage}
\caption{The BigNet window for Time Delay Networks}
\label{BigNetWindowTD}
\end{center}
\end{figure}

Since the buttons of this window carry mostly the same functionality
as in the feed-forward case, refer to the previous section for a
description of their use.


\subsection{Terminology of Time-Delay BigNet}

The following naming conventions have been adopted for the BigNet
window. Their meaning may be clarified by figure~\ref{TD_naming}.

\begin{itemize}

\item{Receptive Field:} The cluster of units in a layer totally
connected to one row of units in the next layer.

\item{1st feature unit:} The starting row of the receptive field.

\item{width:} The width of the receptive field.

\item{delay length:} The number of significant delay steps of the
receptive field. Must be the same value for all receptive fields in
this layer.

\item{No. of feature units:} The width of the current layer

\item{Total delay length:} The length of the current layer. Total delay
length times the number of feature units equals the number of units in
this layer. Note that the total delay length must be the same as the
delay length plus the total delay length of the next layer minus one!

\item{z-coordinates of the plane:} gives the placing of the plane in
space. This value may be omitted (default = 0).

\end{itemize}


\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.5}{xgui_bignet/bn_td_naming2.ps}
  \end{minipage}
\caption{The naming conventions}
\label{TD_naming}
\end{center}
\end{figure}

 
\subsection{Plane Editor}
Just as in BigNet for feed-forward networks, the net is divided into
several planes. The input layer, the output layer and every hidden
layer are called a {\bf plane} in the \index{Plane} notation of
BigNet\@. A plane is a two-dimensional array of units.  Every single unit
within a plane can be addressed by its coordinates.  The unit in the
upper left corner of every plane has the coordinates (1,1).

See~\ref{BN_planeEdit} for a detailed description.

\subsection{Link Editor}

In the link panel the connections special to TDNNs can be defined. In
TDNNs links always lead from the receptive field in a source plane to
one or more units of a target plane. Note, that a receptive field
has to be specified only once for each plane and is automatically
applied to all possible delay steps in that plane.
figure~\ref{TD_const1} gives an example of a
receptive field specification and the network created thereby.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{10cm}
    \screendump{0.5}{xgui_bignet/bn_td_construct1.ps}
  \end{minipage}
  \hfill
  \begin{minipage}{10cm}
    \screendump{0.4}{xgui_bignet/bn_td_construct2.ps}
  \end{minipage}
\caption{An example TDNN construction and the resulting network}
\label{TD_const1}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.5}{xgui_bignet/bn_td_construct3.ps}
  \end{minipage}
\caption{Two receptive fields in one layer}
\label{TD_const3}
\end{center}
\end{figure}

It is possible to specify seperate receptive fields for different
feature units. With only one receptive field for all feature units, a
"1" has to be specified in the input window for "1st feature unit:".
For a second receptive field, the first feature unit should be the
width of the first receptive field plus one. Of course, for all number
of receptive fields, the sum of their width has to equal the number of
feature units! An example network with two receptive fields is
depicted in figure~\ref{TD_const3}



\section{BigNet for ART-Networks}
\index{BigNet!ART}
\label{BigNet art}

The creation of the ART networks is based on just a few parameters.
Although the network topology for these models is rather complex, only
four parameters for ART1 and ART2, and eight parameters for ARTMAP,
have to be specified.

If you have selected the \fbox{\tt ART 1}, \fbox{\tt ART 2} or the
\fbox{\tt ARTMAP} button in the BigNet menu, one of the windows
shown in figure~\ref{BigNet art windows} appears on the screen.

\begin{figure}[hbtp]
\begin{center}
  \begin{minipage}{10cm}
    \screendump{0.65}{xgui_bignet/bn_art1window.ps}
  \end{minipage}
  \hfill
  \begin{minipage}{10cm}
    \screendump{0.65}{xgui_bignet/bn_art2window.ps}
  \end{minipage}

  \vspace*{5mm}

  \begin{minipage}{\textwidth}
    \screendump{0.65}{xgui_bignet/bn_artmapwindow.ps}
  \end{minipage}
\caption{The BigNet windows for the ART models}
\label{BigNet art windows}
\end{center}
\end{figure}



\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.5}{xgui_bignet/bn_art1param.ps}
  \end{minipage}

  \vspace*{5mm}

  \begin{minipage}{\textwidth}
    \screendump{0.35}{xgui_bignet/bn_art1net.ps}
  \end{minipage}

\caption{Example for the generation of an ART1 network. First the
         BigNet (ART1) panel is shown with the specified parameters.
         Next you see the created net as you can see it when using
         an SNNS display.}
\label{BigNet ART1 example}
\end{center}
\end{figure}

The four parameters you have to specify for ART1 and ART2
are simple to choose. First
you have to tell BigNet the number of units ($N$) the F$_1$ layer 
consists of.
Since the F$_0$ layer has the same number of units, BigNet takes only
the value for F$_1$.

Next the way how these $N$ units to be displayed has to be specified.
For this purpose enter the number of rows.  An example for ART1 is
shown in figure~\ref{BigNet ART1 example}.


The same procedure is to be done for the F$_2$ layer. Again you have
to specify the number of units $M$ for the recognition 
part\footnote{The F$_2$ layer consists of three internal layers. See
chapter~\ref{art models in snns}.} of the F$_2$ layer and the number
of rows. 


Pressing the \fbox{\tt CREATE NET} button will generate a network with
the specified parameters. If a network exists when pressing \fbox{\tt
CREATE NET} you will be prompted to assure that you really want to
destroy the current network. A message tells you if the generation
terminated successfully. Finally press the \fbox{\tt DONE} button to
close the BigNet panel.

For ARTMAP things are slightly different. Since an ARTMAP network
exists of two ART1 subnets (ART$^a$ and ART$^b$), for both of them the
parameters described above have to be specified. This is the reason,
why BigNet (ARTMAP) takes eight instead of four parameters.  For the
MAP field the number of units and the number of rows is taken from the
repective values for the F$_2^b$ layer.


\section{BigNet for Self-Organizing Maps}

As described in chapter \ref{SOMchap}, it is recommended to
create Kohonen-Self-Organizing Maps only by using either the {\bf
BigNet} network creation tool or {\bf convert2snns} outside the
graphical user interface. The SOM architecture consists of the
component layer (input layer) and a two-dimensional map, called
competitive layer. Since component layer and competitive layer are
fully connected, each unit of the competitive layer represents a
vector with the same dimension as the component layer.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.3}{nn_models/bn_kohonen_panel.ps}
  \end{minipage}
\caption{The BigNet window for the SOM architecture}
\end{center}
\end{figure}

To create a SOM, only 3 parameters have to be specified:

\begin{itemize}
\item{\sl Components:} The dimension of each weight vector. It equals
the number of input units.

\item{\sl X-size:} The width of the competitive layer. When learning
is performed, the x-size value must be specified by the fifth learning
parameter.

\item{\sl Y-size:} The length of the competitive layer. The number of
hidden (competitive) units equals X-size $\ast$ Y-size.
\end{itemize}

If the parameters are correct (positive integers), pressing the 
\fbox{CREATE NET} button will create the specified network. If the
creation of the network was successful a confirming message is
issued. The parameters of the above example would create the network
of figure \ref{KohEx}. Eventually close the BigNet panel by pressing
the \fbox{DONE} button.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{nn_models/bn_kohonen_example.ps}
  \end{minipage}
\caption{An Example SOM}
\label{KohEx}
\end{center}
\end{figure}

%\clearpage

\section{BigNet for Autoassociative Memory Networks}

The easiest way to create an autoassociative memory network is with
the help of this bignet panel, although this type of network may also
be constructed interactively with the graphical network editor.  The
architecture consists of the world layer (input layer) and a layer of
hidden units, identical in size and shape to the world layer, called
learning layer. Each unit of the world layer has a link to the
corresponding unit in the learning layer. The learning layer is
connected as a clique.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.7}{nn_models/auto_assoz_panel.ps}
  \end{minipage}
\caption{The BigNet Window for Autoassociative Memory Networks}
\end{center}
\end{figure}

To create an autoassociative memory, only 2 parameters have to be specified:

\begin{itemize}
\item{\sl X-size:} The width of the world and learning layers.

\item{\sl Y-size:} The length of the world and learning layers. The number of
units in the network equals 2 $\ast$ X-size $\ast$ Y-size.
\end{itemize}

If the parameters are correct (positive integers), pressing the 
\fbox{CREATE NET} button will create the specified network. If the
creation of the network was successful a confirming message is
issued. The parameters of the above example would create the network
of figure \ref{AAEx}. Eventually close the BigNet panel by pressing
the \fbox{DONE} button.

\begin{figure}[hbtp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{nn_models/auto_assoz_example.ps}
  \end{minipage}
\caption{An Example Autoassociative Memory }
\label{AAEx}
\end{center}
\end{figure}

%\clearpage
\section{BigNet for Partial Recurrent Networks}

\subsection{BigNet for Jordan Networks}

The BigNet window for Jordan networks is shown in figure
\ref{bn_jordan}. 

\begin{figure}[htb]
 \begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{nn_models/bn_jordan.ps}
  \end{minipage}
  \caption{The BigNet Window for Jordan Networks} 
  \label {bn_jordan}
 \end{center}
\end{figure}

In the column {\tt No.~of Units} the number of units in the input,
hidden and output layer have to be specified. The number of context
units equals the number of output units. The units of a layer are
displayed in several columns. The number of these columns is given by
the value in the column {\tt No.~of Col.}.  The network will be
generated by pressing the \fbox{\tt CREATE NET} button:

\begin{itemize}
 \item The input layer is fully connected to the hidden layer,i.e. 
       every input unit is connected to every unit of the hidden layer.   
       The hidden layer is fully connected to the output layer.
 \item Output units are connected to context units by recurrent
       1-to-1-connections. Every context unit is connected to itself
       and to every hidden unit. 
 \item Default activation function for input and context units is the
       identity function, for hidden and output units the logistic function.
 \item Default output function for all units is the identity function
\end{itemize}

To close the BigNet window for Jordan networks click on the
\fbox{\tt DONE} button.


\subsection{BigNet for Elman Networks}

By clicking on the \fbox{\tt ELMAN} button in the BigNet menu,
the BigNet window for Elman networks (see fig.\ref{bn_elman}) is opened.

\begin{figure}[htb]
 \begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.6}{nn_models/bn_elman.ps}
  \end{minipage}
  \caption{The BigNet Window for Elman Networks} 
  \label{bn_elman}
 \end{center}
\end{figure}

The number of units of each layer has to be specified in the column 
{\tt No.~of Units}. Each hidden layer is assigned a context layer
having the same size. The values in the column {\tt No.~of Col.}~have 
the same meaning as the corresponding values in the BigNet window 
for Jordan networks.
  
The number of hidden layers can be changed with the buttons 
\fbox{\tt INSERT} and \fbox{\tt DELETE}. \fbox{\tt INSERT} adds a new
hidden layer just before the output layer. The hidden layer with the
highest layer number can be deleted by pressing the \fbox{\tt DELETE}
button. The current implementation requires at least one and at most
eight hidden layers. If the network is supposed to also contain a
context layer for the output layer, the \framebox[3em][c]{\tt YES}
button has to be toggled, else the \framebox[3em][c]{\tt
NO} button.  Press the \fbox{\tt CREATE NET} button to create the
net. The generated network has the following properties:

\begin{itemize}
 \item The layer $i$ is fully connected to the layer $i+1$.
 \item Each context layer is fully connected to its hidden layer. 
       A hidden layer is connected to its context layer with
       recurrent 1-to-1-connections. 
 \item Each context unit is connected to itself. 
 \item If there is a context layer assigned to the output layer, the
       same connection rules as for hidden layers are used.
 \item Default activation function for input and context units is the
       identity function, for hidden and output units the logistic function.
 \item Default output function for all units is the identity function
\end{itemize}

Click on the \fbox{\tt DONE} button to close the BigNet window for
Elman networks.




