nupic.tensorflow.layers

class KWinners(percent_on, k_inference_factor=1.0, boost_strength=1.0, boost_strength_factor=1.0, duty_cycle_period=1000, name=None, **kwargs)

Bases: nupic.tensorflow.layers.k_winners.KWinnersBase

Applies K-Winner function to the input tensor.

Parameters

• percent_on (float) – The activity of the top k = percent_on * n will be allowed to remain, the rest are set to zero.

• k_inference_factor (float) – During inference (training=False) we increase percent_on by this factor. percent_on * k_inference_factor must be strictly less than 1.0, ideally much lower than 1.0

• boost_strength (float) – boost strength (0.0 implies no boosting).

• boost_strength_factor (float) – Boost strength factor to use [0..1]

• duty_cycle_period (int) – The period used to calculate duty cycles

• kwargs (dict) – Additional args passed to keras.layers.Layer

build(input_shape)

call(inputs, training=None, **kwargs)

compute_duty_cycle(inputs)

\[dutyCycle = \frac{dutyCycle \times \left( period - batchSize \right) + newValue}{period}\]

class KWinners2d(percent_on=0.1, k_inference_factor=1.5, boost_strength=1.0, boost_strength_factor=0.9, duty_cycle_period=1000, data_format=tensorflow.python.keras.backend.image_data_format, name=None, **kwargs)

Bases: nupic.tensorflow.layers.k_winners.KWinnersBase

Applies K-Winner function to Conv2D input tensor.

Parameters

• percent_on (float) – The activity of the top k = percent_on * number of input units will be allowed to remain, the rest are set to zero.

• k_inference_factor (float) – During inference (training=False) we increase percent_on by this factor. percent_on * k_inference_factor must be strictly less than 1.0, ideally much lower than 1.0

• boost_strength (float) – boost strength (0.0 implies no boosting).

• boost_strength_factor (float) – Boost strength factor to use [0..1]

• duty_cycle_period (int) – The period used to calculate duty cycles

• data_format (str) – one of channels_first or channels_last. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, height, width, channels) while channels_first corresponds to inputs with shape (batch, channels, height, width). Similar to data_format argument in keras.layers.Conv2D.

• kwargs (dict) – Additional args passed to keras.layers.Layer

build(input_shape)

call(inputs, training=None, **kwargs)

compute_duty_cycle(x)

Compute our duty cycle estimates with the new value. Duty cycles are updated according to the following formula:

\[dutyCycle = \frac{dutyCycle \times \left( period - batchSize \right) + newValue}{period}\]

Parameters

x – Current activity of each unit

get_config()

class KWinnersBase(percent_on, k_inference_factor, boost_strength, boost_strength_factor, duty_cycle_period, name=None, **kwargs)

Bases: tensorflow.keras.layers.Layer

Base KWinners class.

Parameters

• percent_on (float) – The activity of the top k = percent_on * number of input units will be allowed to remain, the rest are set to zero.

• k_inference_factor (float) – During inference (training=False) we increase percent_on by this factor. percent_on * k_inference_factor must be strictly less than 1.0, ideally much lower than 1.0

• boost_strength (float) – boost strength (0.0 implies no boosting). Must be >= 0.0

• boost_strength_factor (float) – Boost strength factor to use [0..1]

• duty_cycle_period (int) – The period used to calculate duty cycles

• kwargs (dict) – Additional args passed to keras.layers.Layer

build(input_shape)

call(inputs, **kwargs)

abstract compute_duty_cycle(x)

Compute our duty cycle estimates with the new value. Duty cycles are updated according to the following formula:

\[dutyCycle = \frac{dutyCycle \times \left( period - batchSize \right) + newValue}{period}\]

Parameters

x – Current activity of each unit

compute_output_shape(input_shape)

get_config()

update_boost_strength()

Update boost strength using given strength factor during training.

compute_kwinners(x, k, duty_cycles, boost_strength)

Use the boost strength to compute a boost factor for each unit represented in x. These factors are used to increase the impact of each unit to improve their chances of being chosen. This encourages participation of more columns in the learning process.

The boosting function is a curve defined as: boost_factors = exp[ - boost_strength * (dutyCycle - target_density)] Intuitively this means that units that have been active (i.e. in the top-k) at the target activation level have a boost factor of 1, meaning their activity is not boosted. Columns whose duty cycle drops too much below that of their neighbors are boosted depending on how infrequently they have been active. Unit that has been active more than the target activation level have a boost factor below 1, meaning their activity is suppressed and they are less likely to be in the top-k.

Note that we do not transmit the boosted values. We only use boosting to determine the winning units.

The target activation density for each unit is k / number of units. The boostFactor depends on the dutyCycle via an exponential function:

  boostFactor
      ^
      |
      |\
      | \
1  _  |  \
      |    _
      |      _ _
      |          _ _ _ _
      +--------------------> dutyCycle
         |
    target_density

Parameters

• x – Current activity of each unit.

• k – The activity of the top k units will be allowed to remain, the rest are set to zero.

• duty_cycles – The averaged duty cycle of each unit.

• boost_strength – A boost strength of 0.0 has no effect on x.

Returns

A tensor representing the activity of x after k-winner take all.