\chapter{Transfer Functions}
\label{TransferFunctions}
\index{Transfer Function!Predefined}
\thispagestyle{empty}

\section{Predefined Transfer Functions}

The following site, activation, and output functions are already
predefined. Future releases of the kernel will have additional
transfer functions.

\large{
\vspace{0.5cm}

Site Functions:
\index{Site!Function!Linear}
\index{Site!Function!Produkt}
\index{Site!Function!PI}
\index{Site!Function!Max}
\index{Site!Function!Min}

\begin{table}[h]
\begin{center}
\begin{tabular}{|l|l|r|}     \hline
{\bf Function} & {\bf Formula}\\ \hline \hline
{\tt Linear}      & $net_j(t) = \sum_{i} {w_{ij}o_i}$ \\ \hline
{\tt Produkt}     & $net_j(t) = \prod_{i} {w_{ij}o_i}$ \\ \hline
{\tt PI}          & $net_j(t) = \prod_{i} {o_i}$ \\ \hline
{\tt Max}         & $net_j(t) = \max_{i} {w_{ij}o_i}$ \\ \hline
{\tt Min}         & $net_j(t) = \min_{i} {w_{ij}o_i}$ \\ \hline
{\tt At\_least\_2}  & $net_j(t) = \left\{ \begin{array}{ll}
                                     0 & \mbox{if } \sum_{i} {w_{ij}o_i} < 2 \\
                                     1 & \mbox{else.}
                                  \end{array}
                          \right.$ \\ \hline
\end{tabular}
\end{center}
\end{table}
}
Several other site functions have been implemented for the ART models
in SNNS: {\tt At\_least\_2, At\_least\_1, At\_most\_0, Reciprocal}.
These functions normally are not useful for other networks. So they
are mentioned here, but not described in detail. For more information
please refer to the section about the corresponding ART model in SNNS.

%\vspace{0.5cm}
\clearpage

\large{
Activation functions:
\index{Activation Function!Predefined!BAM}
\index{Activation Function!Predefined!BSB}
\index{Activation Function!Predefined!Identity}
\index{Activation Function!Predefined!IdentityPlusBias}
\index{Activation Function!Predefined!Logistic}
\index{Activation Function!Predefined!Logistic\_notInhibit}
\index{Activation Function!Predefined!Logistic\_Tbl.}
\index{Activation Function!Predefined!MinOutPlusWeight}
\index{Activation Function!Predefined!Perceptron}
\index{Activation Function!Predefined!RBF\_Gaussian}
\index{Activation Function!Predefined!RBF\_MultiQuadratic}
\index{Activation Function!Predefined!RBF\_ThinPlateSpline}
\index{Activation Function!Predefined!Signum}
\index{Activation Function!Predefined!Signum0}
\index{Activation Function!Predefined!Step}
\index{Activation Function!Predefined!TanH}
\index{Activation Function!Predefined!TanHPlusBias}
\index{Activation Function!Predefined!TanH\_Xdiv2}
\index{Activation Function!Predefined!Elliott}
%\begin{table}[htbp]
\begin{center}
\begin{tabular}{|l|l|}     \hline
{\bf Function} & {\bf Formula}\\ \hline \hline
{\tt BAM} & $a_j(t) = \left\{ \begin{array}{ll}
                               1  & \mbox{if $net_j(t) > 0$}\\
                               a_j(t-1)  & \mbox{if $net_j(t) = 0$}\\
                               -1 & \mbox{if $net_j(t) < 0$}\\
                               \end{array}
                       \right. $\\ \hline
{\tt BSB} & $a_j(t) = net_j(t) * \theta_j$  \\ \hline
{\tt Elliott} & $a_j(t) = \frac{net_j(t) + \theta_j}{1 + |net_j(t)+ \theta_j|}$ \\ \hline
{\tt Identity}   & $a_j(t) = net_j(t)$ \\ \hline
{\tt IdentityPlusBias}   & $a_j(t) = net_j(t) + \theta_j $ \\ \hline
{\tt Logistic}   & $a_j(t) = \frac{1}{1 + e^{-(net_j(t) + \theta_j)}}$ \\ \hline
{\tt Logistic\_notInhibit}   & like Logistic, but skip input from units\\
                   & named "Inhibit"\\ \hline
{\tt Logistic\_Tbl}   & like Logistic, but with table lookup \\ 
                  & instead of computation \\ \hline
{\tt MinOutPlusWeight} & $a_j(t) = \min_{i} (w_{ij} + o_i)$ \\ \hline
{\tt Perceptron} & $a_j(t) = \left\{ \begin{array}{ll}
                               1  & \mbox{if $net_j(t) \geq \theta_j$}\\
                               0  & \mbox{if $net_j(t) < \theta_j$}\\
                               \end{array}
                       \right. $\\ \hline
{\tt Produc}t & $a_j(t) = \prod_i w_{ij}o_i$ \\ \hline
{\tt RBF\_Gaussian} & see the chapter about RBFs in the user manual \\ \hline
{\tt RBF\_MultiQuadratic} & see the chapter about RBFs in the user manual \\ \hline
{\tt RBF\_ThinPlateSpline} & see the chapter about RBFs in the user manual \\ \hline
{\tt RM}         & $a_j(t) = \left\{ \begin{array}{ll}
                        0.85 * a_j(t-1) +\\ 
                        \vspace{1ex}
                        0.15*net_j(t)*\\ (1-a_j(t-1))& \mbox{if $net_j(t) > 0$}\\
                        0.85 * a_j(t-1) + \\
                        0.15*net_j(t)*\\ (1+a_j(t-1))& \mbox{if $net_j(t) \leq0$}\\
                               \end{array}
                       \right. $\\ \hline
{\tt Signum}     & $a_j(t) = \left\{ \begin{array}{ll}
                               1  & \mbox{if $net_j(t) > 0$}\\
                               -1 & \mbox{if $net_j(t) \leq 0$}\\
                               \end{array}
                       \right. $\\ \hline
{\tt Signum0}    & $a_j(t) = \left\{ \begin{array}{ll}
                               1  & \mbox{if $net_j(t) > 0$}\\
                               0  & \mbox{if $net_j(t) = 0$}\\
                               -1 & \mbox{if $net_j(t) < 0$}\\
                               \end{array}
                       \right. $\\ \hline
{\tt StepFunc}   & $a_j(t) = \left\{ \begin{array}{ll}
                               1  & \mbox{if $net_j(t) > 0$}\\
                               0 & \mbox{if $net_j(t) \leq 0$}\\
                               \end{array}
                       \right. $\\ \hline
{\tt TanH} & $a_j(t) = \tanh(net_j(t) + \theta_j)$ \\ \hline
{\tt TanH\_Xdiv2} & $a_j(t) = \tanh(net_j(t)/2 + \theta_j)$ \\ \hline
\end{tabular}
\end{center}
%\end{table}
}

\vspace*{1ex}

Several other activation functions have been implemented for the ART
models in SNNS: \hspace{0.5cm}{\tt Less\_than\_0, At\_most\_0,
At\_least\_2, At\_least\_1, Exactly\_1,\\ ART1\_NC, ART2\_Identity,
ART2\_Rec, ART2\_NormP, ART2\_NormV, ART2\_NormW,\\ ART2\_NormIP,
ART2\_Rst, ARTMAP\_NCa, ARTMAP\_NCb, ARTMAP\_DRho}.

\clearpage

These functions normally are not useful for other networks. So they
are mentioned here, but not described in detail. For cascade
correlation and time delay networks the following modified versions of
regular activation functions have been implemented: {\tt
RCC\_logistic, RCC\_LogisticSym, RCC\_Tanh, RCC\_Linear,
TD\_Logistic,\\ TD\_Elliott}. They behave like the ordinary functions
with the same name body.


\large{
Output Functions:
\index{Output Function!Clip\_0\_1}
\index{Output Function!Clip\_1\_1}
\index{Output Function!Threshold\_0.5}

\begin{table}[htbp]
\begin{center}
\begin{tabular}{|l|l|r|}     \hline
{\bf Function} & {\bf Formula}\\ \hline \hline
{\tt Identity} & $o_j(t)=a_j(t)$ \\ \hline
{\tt Clip\_0\_1}    & $o_j(t)= \left\{ \begin{array}{ll}
                               0      & \mbox{if $a_j(t) \leq 0$}\\
                               a_j(t) & \mbox{if $a_j(t) < 1$}\\
                               1 & \mbox{if $a_j(t) \geq 1$}
                        \end{array}
                       \right. $\\ \hline
{\tt Clip\_1\_1}    & $o_j(t)= \left\{ \begin{array}{ll}
                               -1     & \mbox{if $a_j(t) \leq -1$}\\
                               a_j(t) & \mbox{if $-1 < a_j(t) < 1$}\\
                               1      & \mbox{if $a_j(t) \geq 1$}
                               \end{array}
                       \right. $\\ \hline
{\tt Threshold\_0.5} & $o_j(t)=  \left\{ \begin{array}{ll}
                               0      & \mbox{if $a_j(t) \leq 0.5$}\\
                               1      & \mbox{if $a_j(t) > 0.5$}
                               \end{array}
                       \right. $\\ \hline
\end{tabular}
\end{center}
\end{table}
}
\normalsize


Two other output functions have been implemented for ART2
in SNNS: {\tt ART2\_Noise\_PLin} and {\tt ART2\_Noise\_ContDiff}
These functions are only useful for the ART2 implementation
So they are mentioned here, but not described in detail.


\section{User Defined Transfer Functions}
\index{Transfer Function!User Defined}
\index{Activation Function!User Defined}
The group of transfer functions can be exented arbitrarily by the
user. In order to make them available inside SNNS the following steps
have to be performed:

\begin{enumerate}

\item In file ``{\tt .../SNNSv4.0/kernel/func\_tbl.c} the name of the
function has to be inserted in the function table. An example entry
for an activation function would be

\centerline{"Act\_MyFunction",ACT\_FUNC, 0, 0, (FunctionPtr) MyFancyFunction,}

Notice that the second entry defines the type (activation,
initialization, etc.) of the new function!

If the new function is an activation function, the corresponding
derivation function has also to be inserted in the function table.
E.g.:\\
"Act\_MyFunction",ACT\_DERIV\_FUNC, 0, 0, (FunctionPtr)ACT\_DERIV\_MFF,

This entry has to be given, even if no such derivation function exists
in the mathematical sense. In that case {\tt ``ACT\_DERIV\_Dummy''} has
to be specified as name of the derivation function.

If the function exists, it has to be declared and implemented just as
the activation function.

Please note, that activation and derivation function have to have the
same name suffix (here: ``MyFunction'')!

\item The functions must be implemented as C programs in following files:

\begin{quote}
activation functions in ``{\tt .../SNNSv4.0/kernel/trans\_f.c}'' \\
output functions in ``{\tt .../SNNSv4.0/kernel/trans\_f.c}'' \\
site functions in ``{\tt .../SNNSv4.0/kernel/trans\_f.c}'' \\
initialization functions in ``{\tt .../SNNSv4.0/kernel/init\_f.c}'' \\
learning functions in ``{\tt .../SNNSv4.0/kernel/learn\_f.c}'' \\
update functions in ``{\tt .../SNNSv4.0/kernel/update\_f.c}'' 
\end{quote}

The name of the implemented function has to match the name specified
in the function table!


\item Function prototypes have to be defined in the corresponding
header (.h) and private header files (.ph).

\item Go to the  SNNSv4.0 root directory and build (using the command
'build') new executables. Remember that if you don't recompile, the
modification to the kernel won't take any effect.


\end{enumerate}

The new function should be available now in the user interface together
with all predefined functions.
