NAME: Prune

SYNOPSIS: Prune [options] Net | Prune -g crit [options] Net Input

DESCRIPTION

Connection pruning. This command is used to reduce the number of connections in a network to increase simulation speed and possibly also increase the network's generalization ability by reducing the number of free parameters. The command can be run with or without training data. When it is used without data, the magnitude of the connection strength determines if the connection is pruned. When training data is used (see the -g option), the pruning criterion is based on the error gradient, i.e., connections that contribute little to the total error are pruned. This is done by computing the first order Taylor approximation from the gradient information of the back propagation algorithm. For the Taylor approximation to be valid, only connections with small strength are considered, i.e., the same criterion as in the simple case without data is used together with a criterion on the error gradient with respect to the connection strength.
Because connections with plasticty 0.0 often are special purpose connections, they can not be pruned.

OPTIONS

-S This option changes the interpretation of the argument "Input". With the -S option specified, the training data is taken from external files with the basenames taken from the rows of the script file "Input". Otherwize "Input" is itself a basename. In any case, the streams of the network read/write data from files specified by the basename together with the information about file extension and directory in each stream (see the Streams section).

-M g1 g2 Mark the connections from group g1 to group g2 for pruning. If no -M option is specified, all connections except those with plasticity 0.0 are selected. Multiple -M options are allowed to select more complex sets.

-C g N M This option introduces a constraint on the number of connections flowing into any unit. If ensures that after pruning, each unit in group g has at least N and at most M inflowing connections.

-w crit This option controls the most important parameter -- the minimum allowed weight strength for pruning. No weights with a higher strength will be pruned (except when the -C option is used which may force connections to be pruned anyway).

-g crit The -g option selects "2nd order mode". It means that the main criterion for pruning connections is not simply the connection strength, but the partial derivative of the error with respect to the connection strength. Thus, the more a change from the current value to zero affects the error, the more important the connection is. 'crit' is the maximum change in promille of the total error that are tolerated for removing a connection. To compute the derivatives the training data is needed, therefore the syntax of the command is changed when this option is used (see above). Note that the bound on the connction strength is still active. This is necessary for the 2nd order approximation to be useful, i.e., the effect on the total error can only be computed from the derivative if the strength is small.

-S	This option changes the interpretation of the argument "Input". With the -S option specified, the training data is taken from external files with the basenames taken from the rows of the script file "Input". Otherwize "Input" is itself a basename. In any case, the streams of the network read/write data from files specified by the basename together with the information about file extension and directory in each stream (see the Streams section).
-M g1 g2	Mark the connections from group g1 to group g2 for pruning. If no -M option is specified, all connections except those with plasticity 0.0 are selected. Multiple -M options are allowed to select more complex sets.
-C g N M	This option introduces a constraint on the number of connections flowing into any unit. If ensures that after pruning, each unit in group g has at least N and at most M inflowing connections.
-w crit	This option controls the most important parameter -- the minimum allowed weight strength for pruning. No weights with a higher strength will be pruned (except when the -C option is used which may force connections to be pruned anyway).
-g crit	The -g option selects "2nd order mode". It means that the main criterion for pruning connections is not simply the connection strength, but the partial derivative of the error with respect to the connection strength. Thus, the more a change from the current value to zero affects the error, the more important the connection is. 'crit' is the maximum change in promille of the total error that are tolerated for removing a connection. To compute the derivatives the training data is needed, therefore the syntax of the command is changed when this option is used (see above). Note that the bound on the connction strength is still active. This is necessary for the 2nd order approximation to be useful, i.e., the effect on the total error can only be computed from the derivative if the strength is small.