pyrcn.base

The pyrcn.base base functionalities for PyRCN.

It contains activation functions and simple object-oriented implementations of the building blocks for Reservoir Computing Networks 1.

References

1

16. Steiner et al., ‘PyRCN: A Toolbox for Exploration and Application

of Reservoir Computing Networks’, under review.

pyrcn.base.blocks.InputToNode

class pyrcn.base.blocks.InputToNode(*, hidden_layer_size: int = 500, sparsity: float = 1.0, input_activation: Literal['tanh', 'identity', 'logistic', 'relu', 'bounded_relu'] = 'tanh', input_scaling: float = 1.0, input_shift: float = 0.0, bias_scaling: float = 1.0, bias_shift: float = 0.0, k_in: Optional[int] = None, random_state: Optional[Union[int, RandomState]] = 42, predefined_input_weights: Optional[ndarray] = None, predefined_bias_weights: Optional[ndarray] = None)

Bases: BaseEstimator, TransformerMixin

InputToNode class for reservoir computing modules.

Parameters

• hidden_layer_size (int, default=500) – Sets the number of nodes in hidden layer. Equals number of output features.

• sparsity (float, default = 1.) – Quotient of input weights per node (k_in) and number of input features (n_features)

• input_activation (Literal['tanh', 'identity', 'logistic', 'relu',) –

• 'bounded_relu'] –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• 'tanh' (default =) –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• input_scaling (float, default = 1.) – Scales the input weight matrix.

• input_shift (float, default = 0.) – Shifts the input weight matrix.

• bias_scaling (float, default = 1.) – Scales the input bias of the activation.

• bias_shift (float, default = 0.) – Shifts the input bias of the activation.

• k_in (Union[int, np.integer, None], default = None.) – input weights per node. By default, it is None. If set, it overrides sparsity.

• random_state (Union[int, np.random.RandomState, None], default = 42) – Determines random number generation for centroid initialization. Use an int to make the randomness deterministic.

• predefined_input_weights (Optional[np.ndarray], default = None) – A set of predefined input weights.

• predefined_bias_weights (Optional[np.ndarray], default = None) – A set of predefined bias weights.

property bias_weights: ndarray

Return the bias.

Returns

bias

Return type

ndarray of size (hidden_layer_size)

fit(X: ndarray, y: None = None) → InputToNode

Fit the InputToNode. Initialize input weights and bias.

Parameters

• X (ndarray of shape (n_samples, n_features)) – The input data.

• y (None) – Not used, present here for API consistency by convention.

Returns

self

Return type

returns a fitted InputToNode.

property input_weights: Union[ndarray, csr_matrix]

Return the input weights.

Returns

input_weights – of size (n_features, hidden_layer_size)

Return type

Union[np.ndarray, scipy.sparse.csr_matrix]

transform(X: ndarray, y: None = None) → ndarray

Transform the input matrix X.

Parameters

• X (ndarray of shape (n_samples, n_features)) – The input data.

• y (None) – Not used, present here for API consistency by convention.

Returns

X_new – X transformed in the new space.

Return type

ndarray of size (n_samples, hidden_layer_size)

class pyrcn.base.blocks.PredefinedWeightsInputToNode(*args, **kwargs)

Bases: InputToNode

PredefinedWeightsInputToNode class for reservoir computing modules.

Parameters

• predefined_input_weights (np.ndarray,shape=(n_features,) –

• hidden_layer_size) – A set of predefined input weights.

• input_activation (Literal['tanh', 'identity', 'logistic', 'relu',) –

• 'bounded_relu'] –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• 'tanh' (default =) –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• input_scaling (float, default = 1.) – Scales the input weight matrix.

• input_shift (float, default = 0.) – Shifts the input weight matrix.

• bias_scaling (float, default = 1.) – Scales the input bias of the activation.

• bias_shift (float, default = 0.) – Shifts the input bias of the activation.

• random_state (Union[int, np.random.RandomState, None], default = 42) –

fit(X: ndarray, y: None = None) → InputToNode

Fit the PredefinedWeightsInputToNode. Set input weights and initialize bias.

Parameters

• X (ndarray of shape (n_samples, n_features)) –

• y (ignored) –

Returns

self

Return type

returns a trained PredefinedWeightsInputToNode.

class pyrcn.base.blocks.BatchIntrinsicPlasticity(*, distribution: Literal['exponential', 'uniform', 'normal'] = 'normal', algorithm: Literal['neumann', 'dresden'] = 'dresden', input_activation: Literal['tanh', 'identity', 'logistic', 'relu', 'bounded_relu'] = 'tanh', hidden_layer_size: int = 500, sparsity: float = 1.0, random_state: Optional[Union[int, RandomState]] = 42)

Bases: InputToNode

BatchIntrinsicPlasticity class for reservoir computing modules.

Parameters

• distribution (Literal['exponential', 'uniform', 'normal'],) –

• 'normal' (default =) – This element represents the desired target distribution.

• algorithm (Literal['neumann', 'dresden'], default = 'dresden') – This element represents the transformation algorithm to be used.

• input_activation (Literal['tanh', 'identity', 'logistic', 'relu',) –

• 'bounded_relu'] –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• 'tanh' (default =) –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• hidden_layer_size (int, default=500) – Sets the number of nodes in hidden layer. Equals number of output features.

• sparsity (float, default = 1.) – Quotient of input weights per node (k_in) and number of input features (n_features)

• random_state (Union[int, np.random.RandomState, None], default = 42) –

IN_DISTRIBUTION_PARAMS = {'exponential': (-0.5, -0.5), 'normal': (0.3, 0.0), 'uniform': (0.7, 0.0)}

OUT_DISTRIBUTION = {'exponential': <function BatchIntrinsicPlasticity.<lambda>>, 'normal': <function BatchIntrinsicPlasticity.<lambda>>, 'uniform': <function BatchIntrinsicPlasticity.<lambda>>}

fit(X: ndarray, y: None = None) → InputToNode

Fit the BatchIntrinsicPlasticity.

Parameters

• X (ndarray of shape (n_samples, n_features)) –

• y (ignored) –

Returns

self

Return type

returns a trained BatchIntrinsicPlasticity.

transform(X: ndarray, y: None = None) → ndarray

Transform the input matrix X.

Parameters

• X (ndarray of shape (n_samples, n_features)) – The input data.

• y (None) – Not used, present here for API consistency by convention.

Returns

Y

Return type

ndarray of size (n_samples, hidden_layer_size)

pyrcn.base.blocks.NodeToNode

class pyrcn.base.blocks.NodeToNode(*, hidden_layer_size: int = 500, sparsity: float = 1.0, reservoir_activation: Literal['tanh', 'identity', 'logistic', 'relu', 'bounded_relu'] = 'tanh', spectral_radius: float = 1.0, leakage: float = 1.0, bidirectional: bool = False, k_rec: Optional[Union[int, integer]] = None, random_state: Optional[Union[int, RandomState]] = 42, predefined_recurrent_weights: Optional[ndarray] = None)

Bases: BaseEstimator, TransformerMixin

NodeToNode class for reservoir computing modules.

Parameters

• hidden_layer_size (int, default=500) – Sets the number of nodes in hidden layer. Equals number of output features.

• sparsity (float, default = 1.) – Quotient of recurrent weights per node (k_rec) and number of input features (n_features)

• reservoir_activation (Literal['tanh', 'identity', 'logistic', 'relu',) –

• 'bounded_relu'] –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• 'tanh' (default =) –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• spectral_radius (float, default = 1.) – Scales the recurrent weight matrix.

• leakage (float, default = 1.) – parameter to determine the degree of leaky integration.

• bidirectional (bool, default = False.) – Whether to work bidirectional.

• k_rec (Union[int, np.integer, None], default = None.) – recurrent weights per node. By default, it is None. If set, it overrides sparsity.

• random_state (Union[int, np.random.RandomState, None], default = 42) – Determines random number generation for centroid initialization. Use an int to make the randomness deterministic.

• predefined_recurrent_weights (Optional[np.ndarray], default = None) – A set of predefined recurrent weights.

fit(X: ndarray, y: None = None) → NodeToNode

Fit the NodeToNode. Initialize recurrent weights.

Parameters

• X (ndarray of shape (n_samples, n_features)) – The input data.

• y (None) – Not used, present here for API consistency by convention.

Returns

self

Return type

returns a trained NodeToNode.

property recurrent_weights: Union[ndarray, csr_matrix]

Return the recurrent weights.

Returns

• recurrent_weights (Union[np.ndarray, scipy.sparse.csr.csr_matrix])

• of size (hidden_layer_size, hidden_layer_size)

transform(X: ndarray, y: None = None) → ndarray

Transform the input matrix X.

Parameters

• X (ndarray of shape (n_samples, n_features)) – The input data.

• y (None) – Not used, present here for API consistency by convention.

Returns

Y

Return type

ndarray of size (n_samples, hidden_layer_size)

class pyrcn.base.blocks.PredefinedWeightsNodeToNode(*args, **kwargs)

Bases: NodeToNode

PredefinedWeightsNodeToNode class for reservoir computing modules.

Parameters

• predefined_recurrent_weights (np.ndarray) – A set of predefined recurrent weights. Sets the number of nodes in hidden layer. Equals number of output features.

• reservoir_activation (Literal['tanh', 'identity', 'logistic', 'relu',) –

• 'bounded_relu'] –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• 'tanh' (default =) –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• spectral_radius (float, default = 1.) – Scales the recurrent weight matrix.

• leakage (float, default = 1.) – parameter to determine the degree of leaky integration.

• bidirectional (bool, default = False.) – Whether to work bidirectional.

fit(X: ndarray, y: None = None) → NodeToNode

Fit the PredefinedWeightsNodeToNode. Sets the recurrent weights.

Parameters

• X (ndarray of shape (n_samples, n_features)) –

• y (ignored) –

Returns

self

Return type

returns a trained PredefinedWeightsNodeToNode.

class pyrcn.base.blocks.HebbianNodeToNode(*, hidden_layer_size: int = 500, sparsity: float = 1.0, reservoir_activation: Literal['tanh', 'identity', 'logistic', 'relu', 'bounded_relu'] = 'tanh', spectral_radius: float = 1.0, leakage: float = 1.0, bidirectional: bool = False, k_rec: Optional[Union[int, integer]] = None, random_state: Optional[Union[int, RandomState]] = 42, learning_rate: float = 0.01, epochs: Union[int, integer] = 100, training_method: Literal['hebbian', 'anti_hebbian', 'oja', 'anti_oja'] = 'hebbian')

Bases: NodeToNode

HebbianNodeToNode for reservoir computing modules (e.g. ESN).

Applies the hebbian rule to a given set of randomly initialized reservoir weights.

Parameters

• hidden_layer_size (int, default=500) – Sets the number of nodes in hidden layer. Equals number of output features.

• sparsity (float, default = 1.) – Quotient of recurrent weights per node (k_rec) and number of input features (n_features)

• reservoir_activation (Literal['tanh', 'identity', 'logistic', 'relu',) –

• 'bounded_relu'] –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• 'tanh' (default =) –

  This element represents the activation function in the hidden layer.

  • ’identity’, no-op activation, useful to implement linear

  bottleneck, returns f(x) = x - ‘logistic’, the logistic sigmoid function, returns f(x) = 1/(1+exp(-x)). - ‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x). - ‘relu’, the rectified linear unit function, returns f(x) = max(0, x) - ‘bounded_relu’, the bounded rectified linear unit function, returns f(x) = min(max(x, 0),1)

• spectral_radius (float, default = 1.) – Scales the recurrent weight matrix.

• leakage (float, default = 1.) – parameter to determine the degree of leaky integration.

• bidirectional (bool, default = False.) – Whether to work bidirectional.

• k_rec (Union[int, np.integer, None], default = None.) – recurrent weights per node. By default, it is None. If set, it overrides sparsity.

• random_state (Union[int, np.random.RandomState, None], default = 42) –

• learning_rate (float, default = 0.01.) – Determines how fast the weight values are updated.

• epochs (Union[int, np.integer], default=100) – Number of training epochs

• training_method (Literal['hebbian', 'anti_hebbian', 'oja', 'anti_oja'],) –

• "hebbian" (default =) – Method used to fit the recurrent weights.

fit(X: ndarray, y: None = None) → NodeToNode

Fit the HebbianNodeToNode. Initialize recurrent weights and do Hebbian learning.

Parameters

• X (ndarray of shape (n_samples, n_features)) –

• y (None) – ignored

Return type

self