syntropy.mixed package

syntropy.mixed.shannon_entropy(discrete_vars, continuous_vars, continuous_estimator='gaussian', k=5)

For discrete \(X\) and continuous \(Y\), leverages the identity

\[H(X,Y) = H(X) + H(Y | X)\]

Parameters:

• discrete_vars (NDArray[np.integer]) – Numpy array of the discrete variables, of shape (n_variables, n_samples)

• continuous_vars (NDArray[np.floating]) – Numpy array of the continuous variables, of shape (n_variables, n_samples)

• continuous_estimator (str) – Whether to use a Gaussian or KNN-based estimator of the continuous entropy. Options are “gaussian” or “knn”.

• k (int (optional)) – If a KNN-based estimator is selected, sets the k-value. Defaults to 5.

Returns:

• NDArray[np.floating] – The local entropy for each sample.

• floating – The expected entropy over all samples.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

syntropy.mixed.conditional_entropy(discrete_vars, continuous_vars, conditional='discrete', continuous_estimator='gaussian', k=5)

For discrete \(X\) and continuous \(Y\), leverages the identity

\[H(X | Y) = H(X,Y) - H(Y)\]

Either the discrete variables or the continuous variables can be conditioning variable (in which case the other is conditioned).

Parameters:

• discrete_vars (NDArray[np.integer]) – Numpy array of the discrete variables, of shape (n_variables, n_samples)

• continuous_vars (NDArray[np.floating]) – Numpy array of the continuous variables, of shape (n_variables, n_samples)

• conditional (str) – Wheter to condition the discrete variables on the continuous, or vice versa.

• continuous_estimator (str) – Whether to use a Gaussian or KNN-based estimator of the continuous entropy. Options are “gaussian” or “knn”.

• k (int (optional)) – If a KNN-based estimator is selected, sets the k-value. Defaults to 5.

Returns:

• NDArray[np.floating] – The local entropy for each sample.

• floating – The expected entropy over all samples.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

syntropy.mixed.mutual_information(discrete_vars, continuous_vars)

Returns the local and average mutual information between two (potentially multivariate) discrete and continuous random variables.

Parameters:

• discrete_vars (NDArray[np.integer]) – Numpy array of the discrete variables, of shape (n_variables, n_samples)

• continuous_vars (NDArray[np.floating]) – Numpy array of the continuous variables, of shape (n_variables, n_samples)

Returns:

• NDArray[np.floating] – The local entropy for each sample.

• floating – The expected entropy over all samples.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

Submodules

syntropy.mixed.shannon module

syntropy.mixed.shannon.shannon_entropy(discrete_vars, continuous_vars, continuous_estimator='gaussian', k=5)

For discrete \(X\) and continuous \(Y\), leverages the identity

\[H(X,Y) = H(X) + H(Y | X)\]

Parameters:

• discrete_vars (NDArray[np.integer]) – Numpy array of the discrete variables, of shape (n_variables, n_samples)

• continuous_vars (NDArray[np.floating]) – Numpy array of the continuous variables, of shape (n_variables, n_samples)

• continuous_estimator (str) – Whether to use a Gaussian or KNN-based estimator of the continuous entropy. Options are “gaussian” or “knn”.

• k (int (optional)) – If a KNN-based estimator is selected, sets the k-value. Defaults to 5.

Returns:

• NDArray[np.floating] – The local entropy for each sample.

• floating – The expected entropy over all samples.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

syntropy.mixed.shannon.conditional_entropy(discrete_vars, continuous_vars, conditional='discrete', continuous_estimator='gaussian', k=5)

For discrete \(X\) and continuous \(Y\), leverages the identity

\[H(X | Y) = H(X,Y) - H(Y)\]

Either the discrete variables or the continuous variables can be conditioning variable (in which case the other is conditioned).

Parameters:

• discrete_vars (NDArray[np.integer]) – Numpy array of the discrete variables, of shape (n_variables, n_samples)

• continuous_vars (NDArray[np.floating]) – Numpy array of the continuous variables, of shape (n_variables, n_samples)

• conditional (str) – Wheter to condition the discrete variables on the continuous, or vice versa.

• continuous_estimator (str) – Whether to use a Gaussian or KNN-based estimator of the continuous entropy. Options are “gaussian” or “knn”.

• k (int (optional)) – If a KNN-based estimator is selected, sets the k-value. Defaults to 5.

Returns:

• NDArray[np.floating] – The local entropy for each sample.

• floating – The expected entropy over all samples.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

syntropy.mixed.shannon.mutual_information(discrete_vars, continuous_vars)

Returns the local and average mutual information between two (potentially multivariate) discrete and continuous random variables.

Parameters:

• discrete_vars (NDArray[np.integer]) – Numpy array of the discrete variables, of shape (n_variables, n_samples)

• continuous_vars (NDArray[np.floating]) – Numpy array of the continuous variables, of shape (n_variables, n_samples)

Returns:

• NDArray[np.floating] – The local entropy for each sample.

• floating – The expected entropy over all samples.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]