aihwkit_lightning.simulator.parameters.pre_post module

Pre-post processing related parameters for resistive processing units.

class aihwkit_lightning.simulator.parameters.pre_post.InputRangeParameter(fast_mode=False, act_range_momentum=0.95, enable=<factory>, dynamic=False, learn_input_range=True, init_value=3.0, init_from_data=100, init_std_alpha=3.0, decay=0.001, input_min_percentage=0.95)[source]

Bases: _PrintableMixin

Parameter related to input range learning

Parameters:

  • fast_mode (bool) –

  • act_range_momentum (float) –

  • enable (bool) –

  • dynamic (bool) –

  • learn_input_range (bool) –

  • init_value (float) –

  • init_from_data (int) –

  • init_std_alpha (float) –

  • decay (float) –

  • input_min_percentage (float) –

act_range_momentum: float = 0.95

Momentum for the activation range.

decay: float = 0.001

Decay rate for input range learning.

dynamic: bool = False

Dynamically compute the abs-max per input vector and use this as the input range.

enable: bool

Whether to enable to learn the input range. Note that if enable is False then no clip is applied.

Note

The input bound (forward.inp_bound) is assumed to be 1 if enabled as the input range already scales the input into to the range \((-1, 1)\) by dividing the input to the type by itself and multiplying the output accordingly.

Typically, noise and bound management should be set to NONE for the input range learning as it replaces the dynamic managements with a static but learned input bound. However, in some exceptional experimental cases one might want to enable the management techniques on top of the input range learning, so that no error is raised if they are not set to NONE.

fast_mode: bool = False

Whether to use fast mode for input range learning.

init_from_data: int = 100

Number of batches to use for initialization from data. Set 0 to turn off.

init_std_alpha: float = 3.0

Standard deviation multiplier for initialization from data.

init_value: float = 3.0

Initial setting of the input range in case of input range learning.

input_min_percentage: float = 0.95

Decay is only applied if percentage of non-clipped values is above this value.

Note

The added gradient is (in case of non-clipped input percentage percentage > input_min_percentage):

grad += decay * input_range
learn_input_range: bool = True

Whether to learn the input range when enabled.

Note

If not learned, the input range should in general be set with some calibration method before training the DNN.

class aihwkit_lightning.simulator.parameters.pre_post.PrePostProcessingParameter(input_range=<factory>)[source]

Bases: _PrintableMixin

Parameter related to digital input and output processing, such as input clip learning.

Parameters:

input_range (InputRangeParameter) –

input_range: InputRangeParameter
class aihwkit_lightning.simulator.parameters.pre_post.PrePostProcessingRPU(pre_post=<factory>)[source]

Bases: _PrintableMixin

Defines the pre-post parameters and utility factories

Parameters:

pre_post (PrePostProcessingParameter) –

pre_post: PrePostProcessingParameter

Parameter related digital pre and post processing.