\chapter{Handling Patterns with SNNS}
\label{PatHandle}
\index{Patterns ! Handling}
\thispagestyle{empty}

The general case of using patterns with a neural network is an exact
fit of the patterns onto the network. The set of activations of all
input units is called input pattern, the set of activations of all
output units is called output pattern. The input pattern and its
corresponding output pattern is simply called a pattern. This definition
implies that all patterns for a particular network have the same
size. These patterns will be called {\sl regular} or {\sl fixed
sized}.

SNNS also offers another, much more flexible type of pattern.  These
patterns will be called {\sl variable sized}. Here, the patterns are
usually larger than the input/output layers of the network. To train
and recall these patterns small portions (subsequently called {\sl
subpatterns}) are systematically cut out from the large pattern and
propagated through the net, one at a time. Only the smaller
subpatterns do have to have the fixed size fitting the network. The
pattern itself may have an arbitrary size and different patterns
within one pattern set may have differing sizes.  The number of
variable dimensions is also variable. Example applications for one and
two variable dimensions include time series patterns for TDNNs and
picture patterns.

Both of these types of patterns are loaded into SNNS from the same
kind of pattern file. For a detailed description of the structure of
this file see sections \ref{fixPat} and \ref{varPat}. The grammar is
given in appendix \ref{patGrammar}


\section{Handling Pattern Sets}

Although activations can be propagated through the network without
patterns defined, learning can be performed only with patterns
present. A set of patterns belonging to the same task is called a
pattern set. Normally there are two dedicated pattern sets when
dealing with a neural network. One for training the network (training
pattern set), and one for testing purposes to see what the network has
learned (test pattern set). In SNNS both of these (and more) can be
kept in the simulator at the same time. They are loaded with the file
browser (see chapter \ref{FilePanel}). The pattern set loaded last is
made the current pattern set. All actions performed with the simulator
refer only to, and affect only the current pattern set. To switch
between pattern sets press the button \fbox{USE} in the control panel
(see figure \ref{ControlPanel} on page \pageref{ControlPanel}). It
opens up a list of loaded pattern sets from which a new one can be
selected. The name of the current pattern set is displayed to the
right of the button. The name equals the name body of the loaded
pattern file. If no pattern set is loaded,  ``Patternfile ?'' is given
as indication that no associated pattern file is defined.

Loaded pattern sets can be removed from main memory with the \fbox{DELETE}
button in the control panel. Just like the \fbox{USE} button it opens a
list of loaded pattern sets, from which any set can be deleted. When a
pattern set is deleted, the corresponding memory is freed, and again
available for other uses. This is especially important with larger
pattern sets, where memory might get scarce. 


\section{Fixed Size Patterns}
\label{fixPat}
\index{Patterns ! fixed}

When using fixed size patterns, the number of input and output values
has to match the number of input and output units in the network
respectively for training purposes. Patterns without output
activations can be defined for networks without output units (e.g. ART
networks), or for test/recall purposes for networks with output
units. It is possible, for example, to load two sets of patterns into
SNNS: A training pattern set with output values for the training of
the network, and a test pattern set without output values for
recall. The switch between the pattern sets is performed with the
\fbox{USE} button as described above.

%The pattern definition files V3.0 require both input {\bf and} output
%activations to match the number of units in the network. Patterns
%without output values are possible only if the network also has no
%output units{$^1$}.
%
Pattern definition files prior to V3.0 require output activations.
Networks or patterns without output are not possible\footnote{
SNNSv4.0 reads all pattern file formats, but writes only the new,
flexible format. This way SNNS itself can be used as a conversion utility.}

{\bf Note:} All Pattern definition files prior to V3.2 now correspond
to the type fixed size!

\section{Variable Size Patterns}
\label{varPat}
\index{Patterns ! variable}

Variable patterns are much more difficult to define and handle.
Example applications for variable pattern set include TDNN patterns
for one variable dimension and picture processing for two variable
dimensions. The SNNS pattern definition is very flexible and allows a
great degree of freedom. Unfortunately this also renders the writing of
correct pattern files more difficult and promotes mistakes. 

To make the user acquainted with the pattern file format we describe the
format with the help of an example pattern file. The beginning of the
pattern file describing a bitmap picture is given below. For easier
reference, line numbers have been added on the left.

{\scriptsize \begin{verbatim}
0001  SNNS pattern definition file V3.2
0002  generated at Fri Aug  3 00:00:44 1999
0003
0004  No. of patterns : 10
0005  No. of input units : 1
0006  No. of output units : 1
0007  No. of variable input dimensions : 2
0008  Maximum input dimensions : [ 200 200 ]
0009  No. of variable output dimensions : 2
0010  Maximum output dimensions : [ 200 200 ]
0011 
0012  # Input pattern 1: pic1
0013  [ 200 190 ]
0014  1 1 1 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 0 1
      .
      .
      .
0214  1 1 1 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 0 1
0215  # Output pattern 1: pic1
0216  [ 200 190 ]
0217  1 1 1 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 0 1
      .
      .
      .

\end{verbatim}}

Some of the comments identifying parameter names make no sense when
the file describes a variable pattern. They are kept, however, for
reasons of compatibility with the regular fixed size pattern
definitions.

The meaning of the various lines is:

%{\renewcommand{\labelenumi}{Line 000\arabic{enumi}}
\begin{list}{}{
\labelwidth17mm
\labelsep3mm
\leftmargin20mm}
\item[Line 0001] gives the version number of the grammar this file follows. For
variable size pattern files the version V3.2 is mandatory!

\item[Line 0002] is information for the book keeping of the user only. Usually
the time of the generation of the pattern file is given here. The string
'generated at' is mandatory !

%\setcounter{enumi}{3}

\item[Line 0004] gives the number of patterns defined in this file. The number
of subpatterns is not specified, since it depends on the size of the network.
Remember: The same pattern may be used by different sized networks
resulting in varying numbers of subpatterns!

\item[Line 0005] {\bf CAUTION !} This variable does {\bf NOT} give the number
of input units but the size $C$ of the fixed dimension. For TDNNs this would
be the (invariant) number of features, for a picture it would be the
number of values per pixel (i.e. a bitmap picture would have size 1,
an RGB picture size 3).

\item[Line 0006] corresponds to line 0005 for the output pattern

\item[Line 0007] this line specifies the number of variable input
dimensions $I$. With fixed size patterns 0 has to be specified.

\item[Line 0008] this line specifies the size of the largest pattern in this
pattern set. It is required for parsing and storage allocation
purposes. The number of entries in the list has to match the number
given in line 0007, if 0 was specified there an empty list (i.e. ``[
]'')has to be given here.

{\bf Note:} The lines 0007 and 0008 are pairwise mandatory, i.e. if
one is given, the other has to be specified as well. Old pattern files
do have neither one and can therefore still be read correctly.

\item[Line 0009] corresponds to line 0007 for the output pattern. It specifies
the number of variable output dimensions $O$. 

%\renewcommand{\labelenumi}{Line 00\arabic{enumi}}

\item[Line 0010] corresponds to line 0008 for the output pattern.

{\bf Note:} The lines 0009 and 0010 are again pairwise mandatory, i.e. if
one is given, the other has to be specified as well. Old pattern files
do have neither one and can therefore still be read correctly.

\setcounter{enumi}{11}

\item[Line 0012] an arbitrary comment. All Text following the sign
`\#' in the same line is ignored.

\item[Line 0013] this line has to be specified whenever $I$ in line 0007 is $\neq
0$. It specifies the size of the following input pattern and is given
as a list of integers separated by blanks and enclosed in [ ]. The
values have to be given by descending dimensions row, i.e. [ dimension\_3
dimension\_2 dimension\_1 ] (here: [200 190]). Note that [200 190] is
less than the maximum, which is specified in line 0008.

\item[Line 0014] the first line of $ \prod^I_{i=1} dimension_i * C$
activation values\footnote{C is the value read from line 0005}
(i.e. here 1*190 = 190 integer values). The values are expected to be
stored as:\\
$dimension_{I-1}$ times\\
\hspace*{5mm}....\\
\hspace*{10mm}$dimension_1$ times\\
\hspace*{15mm}$C$ values.

%\renewcommand{\labelenumi}{Line 0\arabic{enumi}}
%\setcounter{enumi}{213}

\item[Line 0214] the last line of $ \prod^I_{i=1} dimension_i * C $
activation values (i.e. here the $200^{th}$ line)

\item[Line 0215] corresponds to line 0012 for the output pattern.

\item[Line 0216] corresponds to line 0013 for the output pattern.

\item[Line 0217] corresponds to line 0014 for the output pattern.

\end{list}

Once the patterns are loaded into the simulator, their handling can be
controlled by using the control panel. For the handling of variable size
patterns an additional subpattern panel is provided. The handling of
patterns is described in conjunction with the control panel description
in section \ref{remPanel}. All these explanations are intended for
fixed sized patterns, but also hold true for variable size patterns
so they are not repeated here.

The additional functionality necessary for dealing with variable size
patterns is provided by the subpat panel depicted in figure
\ref{subpatpan}.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.7}{xgui_figs/subPatPanel.ps}
  \end{minipage}
\caption{The handling panel for variable pattern sizes.}
\label{subpatpan}
\end{center}
\end{figure}

\textfloatsep 4pt plus 2pt minus 4pt

A subpattern is defined as the number of input and output activations
that match the number of input and output units of the network. The
size and shape of the subpattern must be defined in the subpat panel.

{\bf Note:} A correct subpattern must be defined before any learning,
propagation or recall function can be executed.

{\bf Note:} For a network with 30 input units input subpatterns of
size 1x30, 2x15, 3x10, 5x6, 6x5, 10x3, 2x15, and 30x1 would all be
valid and would be propagated correctly if $C = 1$. Is the position of
the various input units important, however, (as in pictures) both size
{\bf and shape} have to match the network. Shape is not checked
automatically, but has to be taken care of by the user! In the case of
a color picture, where each pixel is represented by three values (RGB)
$C$ would be set to three and the set of possible combinations
would shrink to 1x10, 2x5, 5x2, and 10x1.

{\bf Note:} When loading a new pattern set, the list of activations is
assigned to the units in order of ascending unit number. The user is
responsible for the correct positioning of the units. When creating
and deleting units their order is easily mixed up. This leads to
unwanted graphical representations and the impression of the patterns
being wrong. To avoid this behavior always make sure to have the lowest
unit number in the upper left corner and the highest in the lower
right. To avoid these problems use BIGNET for network creation.

Once a subpattern is defined, the user can scroll through the pattern
along every dimension using the buttons 
\epsfbox{xgui_figs/xgui_button_first.ps},
\epsfbox{xgui_figs/xgui_button_prev.ps},
\epsfbox{xgui_figs/xgui_button_next.ps}, and
\epsfbox{xgui_figs/xgui_button_last.ps}. The step size used for
scrolling when pressing the buttons
\epsfbox{xgui_figs/xgui_button_prev.ps}~~and
\epsfbox{xgui_figs/xgui_button_next.ps}~~is determined by the input and
output step fields for the various dimensions. The user can still as well
browse through the pattern set using the arrow buttons of the control
panel.

It is possible to load various pattern sets with a varying number of
variable dimensions. The user is free to use any of them with the same
network alternatively. When switching between these pattern sets the
subpattern panel will automatically adapt to show the correct number
of variable dimensions. 

When stepping through the subpatterns, in learning, testing, or simply
displaying, the resulting behavior is always governed by the input
pattern. If the last possible subpattern within the current pattern is
reached, the request for the next subpattern will automatically yield
the first subpattern of the next pattern in both input and output
layer. Therefore it is not possible to handle all subpatterns for
training when there are not the same number of subpatterns in input
and output layer available. By adjusting the step width accordingly,
it should always be possible to achieve correct network behavior.

\begin{figure}[htbp]
\begin{center}
  \begin{minipage}{\textwidth}
    \screendump{0.9}{xgui_figs/subPatTiling.ps}
  \end{minipage}
\caption{Tiling versus shifting of subpatterns.}
\label{patTile}
\end{center}
\end{figure}

\textfloatsep 20pt plus 2pt minus 4pt

The last possible subpattern in the above description is dependent
from the settings of the subpattern panel. An example for a
one-dimensional case would be: In a pattern of size 22 is the last
subpattern with size = 3 and step = 5 the position 15. Changing the
step width to 2 would lead to a last position of 18.
In figure \ref{patTile}, the left pattern would have only 9
subpatterns, whereas the right one would have 49 !

The next reachable position with the current step width will always be
a multiple of this step width. That is, if the step width is 4 and
pattern position 8 is reached, a change of step width to 5 and a
subsequent press of \epsfbox{xgui_figs/xgui_button_next.ps}
\hspace*{1mm} would result in position 10 (and not 13 as some might expect).

When selecting a step width, the user also has to remember whether the
pattern should be divided in tiles or overlapping pieces. When
implementing filter e.g., whether picture or others, a tiling stile
will always be more appropriate, since different units are treated not
concordantly. 

It is the sole responsibility of the user to define the step width and
the size of the subpattern correctly for both input and output. The
user has to take care for the subpatterns to be correspondent. A wrong
specification can lead to unpredictable learning behavior. The best
way to check the settings is to press the \fbox{TEST} button, since
exactly those subpatterns are thereby generated that will also be used
for the training. By observing the reported position in the subpattern
panel it can be verified whether meaningful 
values have been specified.