pylissom.optim

Submodules

pylissom.optim.optimizers module

Extends the torch.optim.Optimizer class with Lissom optimizers, mainly Hebbian Learning

class pylissom.optim.optimizers.CortexOptimizer(cortex)

Bases: torch.optim.Optimizer

Abstract torch.optim.Optimizer that can only be used with pylissom.nn.modules.Cortex

class pylissom.optim.optimizers.SequentialOptimizer(*optimizers)

Bases: object

Similar to torch.nn.Sequential but for optimizers, used to contain pylissom.optim.optimizers.CortexHebbian for ReducedLissom modules

step()

zero_grad()

class pylissom.optim.optimizers.CortexHebbian(cortex, learning_rate)

Bases: pylissom.optim.optimizers.CortexOptimizer

Implements hebbian learning over a pylissom.nn.modules.Cortex weights

The formula is as follows:

..math:

\begin{equation*}
\text{w\'}_pq,ij = \text{w\'}_pq,ij + \alpha X_pq n_ij
\end{equation*}

:param - cortex - pylissom.nn.modules.Cortex map to apply formula: :param - learning_rate -:

step(**kwargs)

class pylissom.optim.optimizers.ReducedLissomHebbian(afferent_mod, inhibitory_mod, excitatory_mod, aff_params, inhib_params, excit_params)

Bases: pylissom.optim.optimizers.SequentialOptimizer

class pylissom.optim.optimizers.CortexPruner(cortex, pruning_step=2000)

Bases: pylissom.optim.optimizers.CortexOptimizer

Abstract class that prunes the weights in each step, subclasses must implement pylissom.optim.optimizers.CortexPruner._prune()

:param - cortex - pylissom.nn.modules.Cortex map to apply formula: :param - pruning_step -:

step(**kwargs)

class pylissom.optim.optimizers.ConnectionDeath(cortex, pruning_step=2000, connection_death_threshold=0.0025)

Bases: pylissom.optim.optimizers.CortexPruner

Prunes the weights that are less than a threshold

:param - cortex - pylissom.nn.modules.Cortex map to apply formula: :param - pruning_step -: :param - connection_death_threshold -:

class pylissom.optim.optimizers.NeighborsDecay(cortex, pruning_step=2000, decay_fn=<function linear_decay>, final_epoch=8.0)

Bases: pylissom.optim.optimizers.CortexPruner

Reduces the connective radius of each neuron

:param - cortex - pylissom.nn.modules.Cortex map to apply formula: :param - pruning_step -: :param - decay_fn - Default = linear_decay: :param - final_epoch - necessary for the linear function: