Back | Home


Trainer help

 
Trainer is a simple tool for generating a multilayer feedforward network and training it with backpropagation algorithm. It uses two configuration files and produces one output file.

Using Trainer is very easy:

If you don't know much about computers and you are currently using Windows, then:
  • Edit configuration files with Notepad (just double-click on configuration.txt or trainingfile.txt). Don't forget to save your changes before moving on.
  • Start trainer.exe by double-clicking on it.
  • Read results with Notepad from config_generated.txt.


    • Otherwise:

    If you call trainer.exe without any command line parameters, it will search for configuration.txt and trainingfile.txt, and it will produce config_generated.txt. If you want to use other filenames, then call trainer.exe configfile trainingfile outputfile, where you replace each filename with the names of your own files (outputfile parameter is optional).


    Note that training time is different every time you start the trainer. That's because connection weights are initialized randomly every time. So when training was not successful in the first attempt, it may be successful next time.

    Currently it is not possible to stop the training procedure at your will. Well, at least not in a clean way -- Ctrl+Break or "close the window" button (in Windows) will usually work.

    The configuration files are pretty much self-explanatory. File configuration.txt looks like this:
      

    In this file you can describe your network.

*********************
* Network structure *
*********************

Without this section your NeuralNetwork object will have
NO neurons (unless you modify source code) and is therefore
quite useless!

Currently it is possible to create only a fully connected
multilayer feedforward network. To do that, just answer
some easy questions below.

First layer in network is input layer. It consists of simple
buffer nodes that do NOT compute anything, but only hold
input values. How many input nodes your network must have?

  input_layer: 2

Other layers consist of computing neurons. How many nodes
should be in computational layers starting from the input
layer side of the network? For adding a layer just add a
line with the keyword "comp_layer:". The last computational
layer will also act as the output layer. Layers without
any neurons will be ignored.

  comp_layer: 4
  comp_layer: 4
  comp_layer: 1


**********************
* Neuron parameters  *
**********************

Work of an artificial neural network depends on neuron parameters.
If you have no idea what they should be, then you may set them
to default values (which are given in help texts before
each parameter). Or just leave them empty (delete the
numbers after each parameter identifier), then they will be
automatically set to default values. Actually it is even allowed
to delete this section "Neuron parameters" completely from this
file and leave only the "Network structure" section. Then
everything will be set to default as well. But make a backup
of this file first ;)

Currently it is possible to use only one type of neuron.
Its activation function is sigmoidal (logistic function
1 / (1 + exp(-av)) where "a" is slope parameter and "v"
is internal activity level).


- Parameter: treshold. Type: double. Default value: 0 - -
Treshold (or bias) value is used for lowering or increasing
the sum (internal activity level) before applying the activation
function.

  treshold: 0

- Parameter: slope_parameter. Type: double. Default value: 0.5 - -
It is the slope parameter of the sigmoid function. When slope
parameter approaches infinity, sigmoid function becomes simply a
treshold function (do not confuse with treshold parameter, which was
described above).

  slope_parameter: 0.5


********************
* Additional info  *
********************

It is also possible in this file to set max_weight parameter
(normally you should do it in training configuration file) and
all weights of connections. However, in normal circumstances
you shouldn't do it here. If you want to set all weights manually,
then first generate a network with needed structure, then edit this
generated file according to your needs. You can then use the
generated and edited file as a config file for creating
a new network object.

Note: you will see that generated configuration file contains
max_weight parameter. In most cases it is not used during the
normal work of neural network, but some rare applications may
use it (like my neural network visualizer). Therefore when you
change the weights manually, consider if it is also necessary
to change max_weight value.


    Do not forget to add space between any keywords and numbers after them!
    For example comp_layer: 5  is correct, but comp_layer:5  is not.
     

    File trainingfile.txt looks like this:
     

    Typical backpropagation algorithm is used for training.
    In this file you can configure its parameters.

    *********************
    * Training patterns *
    *********************

    If you are going to train your network, you MUST first
    define all training patterns. The easiest way is to describe
    them is to do it in this file.

    Give as much training patterns as necessary, using format:
    "pattern: inputs -> desired outputs". Note that number of
    inputs and outputs in patterns must match your network
    structure given in configuration file.

      pattern: 0 0 -> 0
      pattern: 0 1 -> 1
      pattern: 1 0 -> 1
      pattern: 1 1 -> 0
                           

    *******************************
    * Training related parameters *
    *******************************

    Process of training depends on several parameters. If you
    have no idea what they should be, then you may set them
    to default values (which are given in help texts before
    each parameter). Or just leave them empty (delete the
    numbers after each parameter identifier), then they will be
    automatically set to default values. Actually it is even allowed
    to delete this section "Training related parameters" completely
    from this file and leave only the "Training patterns" section.
    Then everything will be set to default as well. But make a
    backup of this file first ;)


    - Parameter: error_limit. Type: double. Default value: 0.1 - -
    Error limit is used for evaluating the success of training.
    If all outputs are in a range from (desired_output - error_limit)
    to (desired_output + error_limit), then training is completed.

      error_limit: 0.1

    - Parameter: max_iterations. Type: int. Default value: 20000 - -
    When this number of iterations is reached, training will be
    stopped and is failed (i.e. outputs were still outside the
    given error limits).

      max_iterations: 20000

    - Parameter: learning_rate. Type: double. Default value: 5 - -
    How fast should network learn? Too small values make
    learning too slow. Too big may halt the learning process.

      learning_rate: 5

    - Parameter: max_weight. Type: double. Default value: 10 - -
    Maximal allowed weight for connections. Weights will be
    in range -max_weight..+max_weight.

      max_weight: 10


    Be reasonable with your patterns. Not everything is possible to achieve. For example, if you require:  0 0 -> 0 and 0 0 -> 1 at the same time, it is just impossible.

    Patterns can also contain fractions, like 0.5 0.8 -> 0.7.
     

    The result of training, config_generated.txt, looks like this: 

    input_layer: 2

comp_layer: 4
  weights:  -9.9055592274468722  -8.6426648049884633
  weights:   4.2954430717673731 -10.0000000000000000
  weights:  -5.2333839447251931   8.6645811142879534
  weights: -10.0000000000000000   5.4667244897148892

comp_layer: 4
  weights:   7.3365444984805759   7.4310744304231662  -0.2002578717261061   7.9767345222931514
  weights:   9.9999903345722743  -8.4489966768275337  10.0000000000000000   6.1203844957474400
  weights:   9.9999109055190250   6.5702854017823036   4.1924238186179448  -9.9997423320067149
  weights:  -5.0713789883692932   5.2303987739264022   3.8133885383571613   1.0846559577464676

comp_layer: 1
  weights:   9.9944199304957611  -9.9860383437970999  -9.9937227905149673   6.2474381765918432

treshold: 0.0000000000000000
slope_parameter: 0.5000000000000000
max_weight: 10.0000000000000000

    It follows the same format as configuration.txt, so it can be used as a configuration file as well. There is one weights: keyword for each neuron. First number after keyword is the weight of a connection between this neuron and the first neuron in previous layer; second number is the weight of a connection between this neuron and the second neuron in previous layer; etc.

    Back | Home