syntropy.gaussian package

syntropy.gaussian.differential_entropy(cov, idxs=None)

Computes the expected differential entropy of a multivariate distribution parameterized by a covariance matrix using a Gaussian estimator via direct computation.

The differential entropy is given by:

\[H(X) = \int dx P(x)\log P(x)\]

And if \(X\) is drawn from a k-dimensional Gaussian, it is equal to

\[H(X) = \frac{k}{2}\log 2\pi\textnormal{e} + \frac{1}{2}\log|\Sigma|\]

Where \(|\Sigma|\) is the determinant of the covariance matrix.

Parameters:

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

• idxs (tuple[int, ...], optional) – The specific subset of variables to compute the entropy of. Defaults to using the entire covariance matrix.

Return type:

float

syntropy.gaussian.local_differential_entropy(data, cov=None, idxs=None)

Computes the framewise differential entropy for a set of variables.

\[h(x) = -\log P(x)\]

For data drawn from a k-dimensional Gaussian

\[P(x) = (2\pi)^{-k/2}|\Sigma|^{-1/2}\textnormal{e}^{\frac{-(x - \mu)^\mathrm{T} \Sigma^{-1}(x - \mu)}{2}}\]

Parameters:

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If none is provided, it is computed from the data object.

• idxs (tuple[int, ...], optional) – The specific subset of variables to compute the entropy of. Defaults to using all channels.

Returns:

The series of pointwise entropies.

Return type:

NDArray[np.floating]

syntropy.gaussian.conditional_entropy(idxs_x, idxs_y, cov)

Computes the conditional entropy of X given Y using Gaussian estimation.

\[H(X|Y) = H(X,Y) - H(Y)\]

Parameters:

• idxs_x (tuple) – The indices of the variables to compute the conditional entropy on.

• idxs_y (tuple) – The indices of the conditioning set.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If none is provided, it is computed from the data object.

Return type:

float

syntropy.gaussian.local_conditional_entropy(idxs_x, idxs_y, data, cov=None)

Computes the local condition entropy for every sample in data using Gaussian estimation.

\[h(x|y) = h(x,y) - h(y)\]

Parameters:

• idxs_x (tuple) – The indices of the variables to compute the conditional entropy on.

• idxs_y (tuple) – The indices of the conditioning set.

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If none is provided, it is computed from the data object.

Return type:

NDArray[np.floating]

syntropy.gaussian.mutual_information(idxs_x, idxs_y, cov)

Computes the mutual information between two (potentially multivariate) sets of elements.

\[\begin{split}I(X;Y) &= H(X) + H(Y) - H(X,Y) \\ &= H(X) - H(X|Y) \\ &= H(Y) - H(Y|X) \\ &= H(X,Y) - H(X|Y) - H(Y|X)\end{split}\]

For Gaussian random variables:

\[I(X;Y) = \frac{1}{2}\log\frac{|\Sigma_{X}||\Sigma_{Y}|}{|\Sigma_{XY}|}\]

In the particular case where \(X\) and \(Y\) are univariate, the mutual information can be computed directly from the Pearson correlation coefficient \(r\):

\[I(X;Y) = \frac{-\log(1-r^{2})}{2}\]

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• idxs_x (tuple) – The indices of the source variables. Can be multivariate.

• idxs_y (tuple) – The indices of the idxs_y variable. Can be multivariate.

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

Return type:

float

syntropy.gaussian.local_mutual_information(idxs_x, idxs_y, data, cov=None)

Computes the local mutual information between X and Y for every sample in data using Gaussian estimation. Note that the local mutual information can be negative.

\[\begin{split}i(x;y) &= h(x) + h(y) - h(x,y) \\ &= \log\frac{p(x|y)}{p(x)} \\ &= \log\frac{p(y|x)}{p(y)} \\ &= \log\frac{p(x,y)}{p(x)p(y)} \\\end{split}\]

If you wish to use a Gaussian copula estimator, use the transformed data and the correlation matrix returned by the function utils.copula_transform().

Parameters:

• idxs_x (tuple) – The indices of the source variables. Can be multivariate.

• idxs_y (tuple) – The indices of the idxs_y variable. Can be multivariate.

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If none is provided, it is computed from the data object.

Return type:

NDArray[np.floating]

syntropy.gaussian.conditional_mutual_information(idxs_x, idxs_y, idxs_z, cov)

Computes the expected mutual information between a set of variables X and Y, conditional on a third set Z.

\[\begin{split}I(X,Y|Z) &= H(X|Z) + H(Y|Z) - H(X,Y|Z) \\ &= I(X;Y,Z) - I(X;Z)\end{split}\]

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• idxs_x (tuple) – The indices of the X variables. Can be multivariate.

• idxs_y (tuple) – The indices of the Y variable. Can be multivariate.

• idxs_z (tuple) – The indices of the conditioning set. Can be multivariate.

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

Return type:

float

syntropy.gaussian.local_conditional_mutual_information(idxs_x, idxs_y, idxs_z, data, cov=None)

Computes the local conditional mutual information between two sets of variables X and Y, conditional on another set Z.

\[\begin{split}i(x,y|z) &= h(x|z) + h(y|z) - h(x,y|z) \\ &= i(x;y,z) - i(x;z)\end{split}\]

If you wish to use a Gaussian copula estimator, use the transformed data and the correlation matrix returned by the function utils.copula_transform().

Parameters:

• idxs_x (tuple) – The indices of the X variables. Can be multivariate.

• idxs_y (tuple) – The indices of the Y variable. Can be multivariate.

• idxs_z (tuple) – The indices of the conditioning set. Can be multivariate.

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If none is provided, it is computed from the data object.

Return type:

NDArray[np.floating]

syntropy.gaussian.kullback_leibler_divergence(cov_posterior, cov_prior, mu_posterior=0, mu_prior=0)

Computes the Gaussian Kullback-Leibler divergence between two \(k\)-dimensional multivariate Gaussians parameterized by covariance matrices.

\[D_{KL}(\mathcal{N}_0 || \mathcal{N}_1) = \frac{1}{2}[ \operatorname{tr}(\Sigma_{1}^{-1}\Sigma_{0}) - k + (\mu_1 - \mu_0)^\mathsf{T} \Sigma_{1}^{-1}(\mu_1 - \mu_0) + \log\frac{|\Sigma_{1}|}{|\Sigma_{0}|}]\]

Parameters:

• cov_posterior (NDArray[np.floating]) – The covariance matrix that defines the posterior distribution.

• cov_prior (NDArray[np.floating]) – The covariance matrix that defines the prior distribution .

• mu_posterior (ndarray[tuple[Any, ...], dtype[floating]] | int)

• mu_prior (ndarray[tuple[Any, ...], dtype[floating]] | int)

Return type:

float

syntropy.gaussian.local_kullback_leibler_divergence(cov_posterior, cov_prior, data)

Computes the local Kullback-Leibler divergence between the posterior and the prior for every sample in the data. The local KL divergence is a rarely used measure.

\[d_{kl}^{P||Q}(x) = h^{Q}(x) - h^{P}(x)\]

Parameters:

• cov_posterior (NDArray[np.floating]) – The covariance matrix that defines the posterior distribution.

• cov_prior (NDArray[np.floating]) – The covariance matrix that defines the prior distribution .

• data (NDArray[np.floating]) – The data, assumed to be in channels x samples format.

Returns:

The local Kullback-Leibler divergence.

Return type:

NDArray[np.floating]

References

Varley, T. F. (2024). Generalized decomposition of multivariate information. PLOS ONE, 19(2), e0297128. https://doi.org/10.1371/journal.pone.0297128

syntropy.gaussian.local_total_correlation(data, cov=None, idxs=None)

The local total correlation. Note that this measure can be negative.

\[tc(x) = \sum_{i=1}^{N}h(x_i) - h(x)\]

If you wish to use a Gaussian copula estimator, use the transformed data and the correlation matrix returned by the function utils.copula_transform().

Parameters:

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating] | None) – The covariance matrix used to compute the local entropies. If None is provided, it is inferred from the data. The default is None.

• inputs (tuple | None) – The indices of the channels to include. If None is provided, all channels are used. The default is None.

• idxs (tuple[int, ...] | None)

Returns:

The local total correaltion for each frame.

Return type:

NDArray[np.floating]

References

Scagliarini, T., Marinazzo, D., Guo, Y., Stramaglia, S., & Rosas, F. E. (2022). Quantifying high-order interdependencies on individual patterns via the local O-information: Theory and applications to music analysis. Physical Review Research, 4(1), 013184. https://doi.org/10.1103/PhysRevResearch.4.013184

Pope, M., Varley, T. F., Grazia Puxeddu, M., Faskowitz, J., & Sporns, O. (2025). Time-varying synergy/redundancy dominance in the human cerebral cortex. Journal of Physics: Complexity, 6(1), 015015. https://doi.org/10.1088/2632-072X/adbaa9

syntropy.gaussian.total_correlation(cov, idxs=None)

The expected total correlation.

\[\begin{split}TC(X) &= D_{KL}(P(X) || \prod_{i=1}^{N}P(X_i) \\ &= \sum_{i=1}^{N}H(X_i) - H(X)\end{split}\]

For Gaussian random variables, the estimator is:

\[\hat{TC}(X) = \frac{-\log R}{2}\]

Where \(R\) is the Pearson correlation matrix.

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• cov (NDArray[np.floating]) – The covariance matrix.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

Returns:

The expected total correlation.

Return type:

float

References

Watanabe, S. (1960). Information Theoretical Analysis of Multivariate Correlation. IBM Journal of Research and Development, 4(1), Article 1. https://doi.org/10.1147/rd.41.0066

Tononi, G., Sporns, O., & Edelman, G. M. (1994). A measure for brain complexity: Relating functional segregation and integration in the nervous system. Proceedings of the National Academy of Sciences, 91(11), Article 11. https://doi.org/10.1073/pnas.91.11.5033

Pascual-Marqui, R. D., Kochi, K., & Kinoshita, T. (2025). Total/dual correlation/coherence, redundancy/synergy, complexity, and O-information for real and complex valued multivariate data (No. arXiv:2507.08773). arXiv. https://doi.org/10.48550/arXiv.2507.08773

syntropy.gaussian.local_dual_total_correlation(data, cov=None, idxs=None)

Computes the dual total correlation using Gaussian estimation. Note that this measure can be negative.

\[\begin{split}dtc(x) &= h(x) - \sum_{i=1}^{N}h(x_i|x^{-i}) \\ &= (N-1)\times tc(x) - \sum_{i=1}^{N}tc(x^{-i})\end{split}\]

If you wish to use a Gaussian copula estimator, use the transformed data and the correlation matrix returned by the function utils.copula_transform().

Parameters:

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If unspecified it is computed directly from the data.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

Returns:

The series of local dual total correlations.

Return type:

NDArray[np.floating].

syntropy.gaussian.dual_total_correlation(cov, idxs=None)

Computes the dual total correlation using Gaussian estimation.

\[\begin{split}DTC(X) &= H(X) - \sum_{i=1}^{N}H(X_i|X^{-i}) \\ &= (N-1)\times TC(X) - \sum_{i=1}^{N}TC(X^{-i})\end{split}\]

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform(). :param data: The data in channels x samples format. :type data: NDArray[np.floating] :param cov: The covariance matrix that defines the distribution. . :type cov: NDArray[np.floating] :param idxs: The specific subset of variables to compute the total correlation of.

Defaults to computing the TC of the entire covariance matrix.

Returns:

The expected dual total correlation.

Return type:

float

Parameters:

• cov (ndarray[tuple[Any, ...], dtype[floating]])

• idxs (tuple, optional)

References

Abdallah, S. A., & Plumbley, M. D. (2012). A measure of statistical complexity based on predictive information with application to finite spin systems. Physics Letters A, 376(4), 275–281. https://doi.org/10.1016/j.physleta.2011.10.066

Rosas, F., Mediano, P. A. M., Gastpar, M., & Jensen, H. J. (2019). Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3), Article 3. https://doi.org/10.1103/PhysRevE.100.032305

syntropy.gaussian.local_s_information(data, cov=None, idxs=None)

Compute local S-information using Gaussian estimation.

\[\begin{split}\sigma(X) &= \sum_{i=1}^{N}i(x_i;x^{-i}) \\ &= N\times tc(x) - \sum_{i=1}^{N}tc(x^{-i}) \\ &= tc(x) + dtc(x)\end{split}\]

If you wish to use a Gaussian copula estimator, use the transformed data and the correlation matrix returned by the function utils.copula_transform().

Parameters:

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If unspecified it is computed directly from the data.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

Returns:

The series of local S-information.

Return type:

NDArray[np.floating].

syntropy.gaussian.s_information(cov, idxs=None)

Compute S-information using Gaussian estimation.

\[\begin{split}\Sigma(X) &= \sum_{i=1}^{N}I(X_i;X^{-i}) \\ &= N\times TC(X) - \sum_{i=1}^{N}TC(X^{-i}) \\ &= TC(X) + DTC(X)\end{split}\]

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

• Returns

• -------

• float – The expected S-information.

Return type:

float

References

Rosas, F., Mediano, P. A. M., Gastpar, M., & Jensen, H. J. (2019). Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3), Article 3. https://doi.org/10.1103/PhysRevE.100.032305

Varley, T. F., Pope, M., Faskowitz, J., & Sporns, O. (2023). Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex. Communications Biology, 6(1), Article 1. https://doi.org/10.1038/s42003-023-04843-w

syntropy.gaussian.local_o_information(data, cov=None, idxs=None)

Computes the local O-information for each sample using Gaussian estimation.

\[\begin{split}\omega(x) &= (2-N)tc(x) + \sum_{i=1}^{N}tc(x^{-i}) \\ &= tc(x) - dtc(x)\end{split}\]

If you wish to use a Gaussian copula estimator, use the transformed data and the correlation matrix returned by the function utils.copula_transform().

Parameters:

• data (NDArray[np.floating]) – The data in channels x samples format.

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If unspecified it is computed directly from the data.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

Returns:

The series of local O-informations.

Return type:

NDArray[np.floating].

References

Scagliarini, T., Marinazzo, D., Guo, Y., Stramaglia, S., & Rosas, F. E. (2022). Quantifying high-order interdependencies on individual patterns via the local O-information: Theory and applications to music analysis. Physical Review Research, 4(1), 013184. https://doi.org/10.1103/PhysRevResearch.4.013184

Pope, M., Varley, T. F., Grazia Puxeddu, M., Faskowitz, J., & Sporns, O. (2025). Time-varying synergy/redundancy dominance in the human cerebral cortex. Journal of Physics: Complexity, 6(1), 015015. https://doi.org/10.1088/2632-072X/adbaa9

syntropy.gaussian.o_information(cov, idxs=None)

Compute O-information using Gaussian estimation. O-information quantifies the balance between redundancy (positive values) and synergy (negative values) in multivariate information.

\[\begin{split}\Omega(X) &= (2-N)TC(X) + \sum_{i=1}^{N}TC(X^{-i}) \\ &= TC(X) - DTC(X)\end{split}\]

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• cov (NDArray[np.floating], optional) – The covariance matrix that defines the distribution. If unspecified it is computed directly from the data.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

Returns:

The expected O-information.

Return type:

float

References

Rosas, F., Mediano, P. A. M., Gastpar, M., & Jensen, H. J. (2019). Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3), Article 3. https://doi.org/10.1103/PhysRevE.100.032305

Varley, T. F., Pope, M., Faskowitz, J., & Sporns, O. (2023). Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex. Communications Biology, 6(1), Article 1. https://doi.org/10.1038/s42003-023-04843-w

syntropy.gaussian.tse_complexity(num_samples, cov)

Computes the Tononi-Sporns-Edelman complexity using Gaussian estimators.

\[\begin{split}TSE(X) &= \sum_{k=1}^{\lfloor N/2\rfloor} \bigg\langle I(X^{k}_j;X^{-k}_j) \bigg\rangle_{j} \\ &= \sum_{k=2}^{N}\bigg[\bigg(\frac{k}{N}\bigg)TC(X) - \langle TC(X^{k}_{j}) \rangle_{j} \bigg]\end{split}\]

Runtimes scale very badly with system size (as it requires brute-forcing) all possible bipartitions of the system. If the system is too large, a sub-sampling approach is taken: at each scale, num_samples are drawn from the space of bipartitions.

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• num_samples (int) – The number of sample subsets to compute.

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

Returns:

The TSE complexity.

Return type:

float

References

Tononi, G., Sporns, O., & Edelman, G. M. (1994). A measure for brain complexity: Relating functional segregation and integration in the nervous system. Proceedings of the National Academy of Sciences, 91(11), Article 11. https://doi.org/10.1073/pnas.91.11.5033

syntropy.gaussian.description_complexity(cov, idxs=None)

\[C(X) = \frac{DTC(X)}{N}\]

If you wish to use a Gaussian copula estimator, use the correlation matrix returned by the function utils.copula_transform().

Parameters:

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

• idxs (tuple, optional) – The specific subset of variables to compute the total correlation of. Defaults to computing the TC of the entire covariance matrix.

Returns:

The expected description complexity.

Return type:

float

References

Varley, T. F., Pope, M., Faskowitz, J., & Sporns, O. (2023). Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex. Communications Biology, 6(1), Article 1. https://doi.org/10.1038/s42003-023-04843-w

syntropy.gaussian.partial_entropy_decomposition(data, inputs=None, cov=None)

Computes the partial entropy decomposition of a joint distribution with up to four elements. Uses the Gaussian hmin estimator. See:

Parameters:

• inputs (tuple[int, ...]) – The indices of the elements to analyze.

• data (NDArray[np.floating]) – The numpy array, assumed to be in channels x time format.

• cov (NDArray[np.floating], optional) – The covariance matrix. If none is not provided, it is computed from the data directly.

Returns:

• ptw (dict) – The dictionary of local values for each partial entropy atom. The local values are represented by a numpy array of the same length as the data array.

• avg (dict) – The dictionary of expected values for each partial entropy atom.

Return type:

tuple[dict[tuple[tuple[int, …], …], ndarray[tuple[Any, …], dtype[floating]]], dict[tuple[tuple[int, …], …], float]]

References

Finn, C., & Lizier, J. T. (2020). Generalised Measures of Multivariate Information Content. Entropy, 22(2), Article 2. https://doi.org/10.3390/e22020216

syntropy.gaussian.partial_information_decomposition(inputs, target, data, cov=None, redundancy_function='ipm')

Computes the partial information decomposition for up to four input variables onto one (potentially joint) target variable.

The available redundancy functions are \(MMI\) and \(i_{\min}\) from Finn and Lizier.

\[\begin{split}i_{pm}(\alpha;t) &= \min h(\alpha_i) - \min h(\alpha_i|t) \\ i_{MMI}(\alpha;t) &= \min_i I(\alpha_i;T)\end{split}\]

Parameters:

• inputs (tuple[int, ...]) – The indices of the input elements.

• target (tuple[int, ...]) – The indices of the target elements.

• data (NDArray[np.floating]) – The numpy array, assumed to be in channels x time format.

• cov (NDArray[np.floating], optional) – The covariance matrix. If none is not provided, it is computed from the data directly.

• redundancy_function (str) – The redundancy function. Options are ipm and mmi

Return type:

Any

References

Williams, P. L., & Beer, R. D. (2010). Nonnegative Decomposition of Multivariate Information. arXiv:1004.2515 [Math-Ph, Physics:Physics, q-Bio]. http://arxiv.org/abs/1004.2515

Finn, C., & Lizier, J. T. (2018). Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices. Entropy, 20(4), Article 4. https://doi.org/10.3390/e20040297

syntropy.gaussian.generalized_information_decomposition(inputs, data, cov_posterior, cov_prior)

Computes the generalized information decomposition from Varley et al. The GID is a decomposition of the Kullback-Leibler divergence of a posterior distribution from a prior distribution.

The available redundancy function is the Gaussian hmin.

Parameters:

• inputs (tuple[int, ...]) – The indices of the elements to analyze.

• data (NDArray[np.floating]) – The numpy array, assumed to be in channels x time format.

• cov_posterior (NDArray[np.floating]) – The covariance matrix that defines the prior distribution.

• cov_prior (NDArray[np.floating]) – The covariance matrix that defines the posterior distribution.

Returns:

• ptw (dict) – The dictionary of local values for each partial entropy atom. The local values are represented by a numpy array of the same length as the data array.

• avg (dict) – The dictionary of expected values for each partial entropy atom.

Return type:

tuple[dict[tuple[tuple[int, …], …], ndarray[tuple[Any, …], dtype[floating]]], dict[tuple[tuple[int, …], …], float]]

References

Varley, T. F. (2024). Generalized decomposition of multivariate information. PLOS ONE, 19(2), e0297128. https://doi.org/10.1371/journal.pone.0297128

syntropy.gaussian.integrated_information_decomposition(inputs, target, data, cov=None, redundancy_function='ipm')

Computes the integrated information decomposition introduced by Rosas, Mediano, et al. The PhiID relaxes the requirement of only having a single target, and instead allows for redundant-redundant, synergistic-synergistic, etc interactions.

Available redundancy functions are:

\[\begin{split}i_{pm}(\alpha;\beta) &= \min_i h(\alpha_i) + \min_i h(\beta_i) - \min h(\alpha_i, \beta_i) \\ i_{MMI}(\alpha;\beta) &= \min_{ij} I(\alpha_i;\beta_j)\end{split}\]

Parameters:

• inputs (tuple[int, ...]) – The indices of the input elements.

• target (tuple[int, ...]) – The indices of the target elements.

• data (NDArray[np.floating]) – The numpy array, assumed to be in channels x time format.

• cov (NDArray[np.floating], optional) – The covariance matrix. If none is not provided, it is computed from the data directly.

• redundancy_function (str) – The redundancy function. Options are ipm and mmi

Return type:

Any

References

Mediano, P. A. M., Rosas, F. E., Luppi, A. I., Carhart-Harris, R. L., Bor, D., Seth, A. K., & Barrett, A. B. (2025). Toward a unified taxonomy of information dynamics via Integrated Information Decomposition. Proceedings of the National Academy of Sciences, 122(39), e2423297122. https://doi.org/10.1073/pnas.2423297122

Rosas, F. E., Mediano, P. A. M., Jensen, H. J., Seth, A. K., Barrett, A. B., Carhart-Harris, R. L., & Bor, D. (2020). Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data. PLOS Computational Biology, 16(12), Article 12. https://doi.org/10.1371/journal.pcbi.1008289

syntropy.gaussian.representational_complexity(avg, comparator=<built-in function min>)

Computes the representational complexity of a given partial information or entropy lattice. The representational complexity is a measure of how much partial information atoms of a given degree of synergy contribute to the overall mutual information or entropy.

Parameters:

• avg (dict) – The dictionary of partial information/entropy atoms. Returned from any of the above functions.

• comparator (function, optional) – Whether to consider the minimum complexity of an atom. or the maximum complexity of an atom. Options are: min, max, np.min, np.max. The default is min, following the original work by Ehrlich et al.,.

Returns:

The representational complexity.

Return type:

float

References

Ehrlich, D. A., Schneider, A. C., Priesemann, V., Wibral, M., & Makkeh, A. (2023). A Measure of the Complexity of Neural Representations based on Partial Information Decomposition. Transactions on Machine Learning Research. https://openreview.net/forum?id=R8TU3pfzFr

syntropy.gaussian.idep_partial_information_decomposition(inputs, target, cov, data=None)

Computes the I_dep partial information decomposition for Gaussian systems using the dependency constraint method from Kay & Ince (2018).

Currently only supports 2 predictors (univariate or multivariate).

Adapted from:

https://github.com/robince/partial-info-decomp/blob/master/calc_pi_Idep_mvn.m

Parameters:

• inputs (tuple[tuple[int, ...], tuple[int,...]],) – The indices of the two predictor variables/sets. Must have length 2.

• target (tuple[int, ...]) – The indices of the target variable(s).

• data (NDArray[np.floating] | None) – The data in channels x samples format. Optional if cov provided.

• cov (NDArray[np.floating] | None) – The covariance matrix. If None, computed from data.

Returns:

Dictionary with keys: ‘unq0’, ‘unq1’, ‘red’, ‘syn’

Return type:

dict[str, float]

References

Kay, J. W., & Ince, R. A. A. (2018). Exact Partial Information Decompositions for Gaussian Systems Based on Dependency Constraints. Entropy, 20(4), 240. https://doi.org/10.3390/e20040240

syntropy.gaussian.differential_entropy_rate(idxs, data, fs=1, nperseg=1024)

Computes the differential entropy rate of a potentially multivariate stochastic process.

\[H(X)=\frac{1}{4\pi} \int_{-\pi}^{\pi} \log \left( (2\pi e)^N |S_X(\omega)| \right) \, d\omega\]

Parameters:

• idxs (tuple[int, ...]) – The indices of the channels to include in the analysis.

• data (NDArray[np.floating]) – The data in channels x samples format.

• fs (int) – The sampling rate of the time series data in Hz. The default is 1.

• nperseg (int) – The number of samples to include in each segment. Passed to the various scipy.signal functions that compute the spectral analyses. The default is 1024

Returns:

• NDArray[np.floating] – The local entropies for each frequency band.

• float – The average entropy across the whole spectrum.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

syntropy.gaussian.mutual_information_rate(idxs_x, idxs_y, data, fs=1, nperseg=1024)

Computes the mutual information rate between two (potentially multivariate) Gaussian processes.

\[I(X; Y) = \frac{1}{4\pi} \int_{-\pi}^{\pi} \log \left( \frac{ |S_X(\omega)||S_Y(\omega)| }{ |S_{XY}(\omega)| } \right) d\omega\]

Parameters:

• idxs_x (tuple[int, ...]) – The indices of the channels to include in the inputs.

• idxs_y (tuple[int, ...]) – The indices of the channels to include in the target.

• data (NDArray[np.floating]) – The data in channels x samples format.

• fs (int) – The sampling rate of the time series data in Hz. The default is 1.

• nperseg (int) – The number of samples to include in each segment. Passed to the various scipy.signal functions that compute the spectral analyses. The default is 1024

Returns:

• NDArray[np.floating] – The local mutual informations for each frequency band.

• float – The average mutual information across the whole spectrum.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

References

Faes, L., Sparacino, L., Mijatovic, G., Antonacci, Y., Ricci, L., Marinazzo, D., & Stramaglia, S. (2025). Partial Information Rate Decomposition (No. arXiv:2502.04550). arXiv. https://doi.org/10.48550/arXiv.2502.04550

Faes, L., Pernice, R., Mijatovic, G., Antonacci, Y., Krohova, J. C., Javorka, M., & Porta, A. (2021). Information decomposition in the frequency domain: A new framework to study cardiovascular and cardiorespiratory oscillations. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 379(2212), 20200250. https://doi.org/10.1098/rsta.2020.0250

syntropy.gaussian.total_correlation_rate(idxs, data, fs=1, nperseg=1024)

A straightforward extension of the mutual information rate to the total correlation.

\[TC(X,Y,\ldots,Z) = \frac{1}{4\pi} \int_{-\pi}^{\pi} \log \left( \frac{ |S_X(\omega)||S_Y(\omega)|\ldots|S_Z(\omega)| }{ |S_{XY\ldots Z}(\omega)| } \right) d\omega\]

WARNING: As far as I know this TC rate idea has never been formally explored before. It should work fine as a natural generalization of the MI, but it hasn’t ever been published or peer reviewed.

Parameters:

• idxs (tuple[int, ...]) – The indices of the channels to include in the total correlation calculation.

• data (NDArray[np.floating]) – The data in channels x samples format.

• fs (int) – The sampling rate of the time series data in Hz. The default is 1.

• nperseg (int) – The number of samples to include in each segment. Passed to the various scipy.signal functions that compute the spectral analyses. The default is 1024

Returns:

• NDArray[np.floating] – The local total correlation for each frequency band.

• float – The average total correlation across the whole spectrum.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

syntropy.gaussian.dual_total_correlation_rate(idxs, data, fs=1, nperseg=1024, verbose=False)

A straightforward extension of the dual total correlation rate from the total correlation rate.

The dual total correlation is given alternately by:

\[DTC(X) = H(X) - \sum_{i=1}^{N}H(X_i|X^{-i})\]

\[DTC(X) = (N-1)TC(X) - \sum_{i=1}^{N}TC(X^{-i})\]

WARNING: As far as I know this rate idea has never been formally explored before. It should work fine as a natural generalization of the MI, but it hasn’t even been published or peer reviewed.

Parameters:

• idxs (tuple[int, ...]) – The indices of the channels to include in the calculation.

• data (NDArray[np.floating]) – The data in channels x samples format.

• fs (int) – The sampling rate of the time series data in Hz. The default is 1.

• nperseg (int) – The number of samples to include in each segment. Passed to the various scipy.signal functions that compute the spectral analyses. The default is 1024

• verbose (bool)

Returns:

• NDArray[np.floating] – The local dual total correlation rate for each frequency band.

• float – The average dual total correlation across the whole spectrum.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

References

Abdallah, S. A., & Plumbley, M. D. (2012). A measure of statistical complexity based on predictive information with application to finite spin systems. Physics Letters A, 376(4), 275–281. https://doi.org/10.1016/j.physleta.2011.10.066

Rosas, F., Mediano, P. A. M., Gastpar, M., & Jensen, H. J. (2019). Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3), Article 3. https://doi.org/10.1103/PhysRevE.100.032305

syntropy.gaussian.s_information_rate(idxs, data, fs=1, nperseg=1024, verbose=False)

A straightforward extension of the S-information rate from the total correlation rate.

The S-information is equivalant to:

\[\Sigma(X) = \sum_{i=1}^{N}I(X_i;X^{-i})\]

\[\Sigma(X) = TC(X) + DTC(X)\]

WARNING: As far as I know this rate idea has never been formally explored before. It should work fine as a natural generalization of the MI, but it hasn’t even been published or peer reviewed.

Parameters:

• idxs (tuple[int, ...]) – The indices of the channels to include in the calculation.

• data (NDArray[np.floating]) – The data in channels x samples format.

• fs (int) – The sampling rate of the time series data in Hz. The default is 1.

• nperseg (int) – The number of samples to include in each segment. Passed to the various scipy.signal functions that compute the spectral analyses. The default is 1024

• verbose (bool)

Returns:

• NDArray[np.floating] – The local S-information rate for each frequency band.

• float – The average S-information across the whole spectrum.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

References

Rosas, F., Mediano, P. A. M., Gastpar, M., & Jensen, H. J. (2019). Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3), Article 3. https://doi.org/10.1103/PhysRevE.100.032305

Varley, T. F., Pope, M., Faskowitz, J., & Sporns, O. (2023). Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex. Communications Biology, 6(1), Article 1. https://doi.org/10.1038/s42003-023-04843-w

syntropy.gaussian.o_information_rate(idxs, data, fs=1, nperseg=1024, verbose=False)

A straightforward extension of the O-information rate from the total correlation rate.

\[\Omega(X) = (2-N)TC(X) + \sum_{i=1}^{N}TC(X^{-i})\]

\[\Omega(X) = TC(X) - DTC(X)\]

WARNING: As far as I know this rate idea has never been formally explored before. It should work fine as a natural generalization of the MI, but it hasn’t even been published or peer reviewed.

Parameters:

• idxs (tuple[int, ...]) – The indices of the channels to include in the calculation.

• data (NDArray[np.floating]) – The data in channels x samples format.

• fs (int) – The sampling rate of the time series data in Hz. The default is 1.

• nperseg (int) – The number of samples to include in each segment. Passed to the various scipy.signal functions that compute the spectral analyses. The default is 1024

• verbose (bool)

Returns:

• NDArray[np.floating] – The local O-information rate for each frequency band.

• float – The average O-information across the whole spectrum.

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], float]

References

Rosas, F., Mediano, P. A. M., Gastpar, M., & Jensen, H. J. (2019). Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3), Article 3. https://doi.org/10.1103/PhysRevE.100.032305

Varley, T. F., Pope, M., Faskowitz, J., & Sporns, O. (2023). Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex. Communications Biology, 6(1), Article 1. https://doi.org/10.1038/s42003-023-04843-w

Submodules

syntropy.gaussian.optimization module

Created on Sun Feb 16 13:01:12 2025

@author: thosvarley

syntropy.gaussian.optimization.neg_o_information(x)

A utility function for computing the negative of the O-information for a hill-climbing optimizer.

Parameters:

x (tuple[NDArray[np.floating], set[int]]) – A tuple of the arguments to the O-information function. The covariance matrix (NDArray) and indices (tuple of ints)

Returns:

The negative O-information.

Return type:

float

syntropy.gaussian.optimization.simulated_annealing(cov, function, size, temperature=1.0, cooling_rate=0.999, min_temperature=1e-05, iters_per_temperature=10, convergence_window=100, convergence_threshold=1e-06, verbose=False)

Implements a simulated annealing algorithm for optimizing multivariate information measures (O-info, DTC, etc) from a covariance matrix. Modified from;

Varley, T. F., Pope, M., Faskowitz, J., & Sporns, O. (2023). Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex. Communications Biology, 6(1), Article 1. https://doi.org/10.1038/s42003-023-04843-w

Parameters:

• cov (NDArray[np.floating]) – The covariance matrix that defines the full distribution.

• function (function) – The objective function. Should be taken from the syntropy.gaussian.multivariate_mi module.

• size (int) – The size of the ensemble of elements.

• temperature (float, optional) – The initial temperature to initialize the optimization with. The default is 1.0.

• cooling_rate (float, optional) – The rate at which the annealing cools. Temperature, T, decreases with T(t+1) = T(t)*cooling_rate The default is 0.999.

• min_temperature (float, optional) – The stopping temperature. The default is 1e-3.

• iters_per_temperature (int, optional) – How many times to iterate the annealing algorithim before cooling one step. The default is 10.

• convergence_window (int, optional) – How many steps back in time to consider for assessing convergence. The default is 100.

• convergence_threshold (float, optional) – The standard deviation of the window below which convergence is said to have been achieved.. The default is 1e-6.

• verbose (bool, optional) – Whether to print progress messages during optimization. The default is False.

Returns:

• set[int] – The indicies of the optimal set.

• float – The value of the optimal set.

• list – The time series of values.

Return type:

tuple[set[int], float, ndarray[tuple[Any, …], dtype[floating]]]

syntropy.gaussian.optimization.irreducible_synergy(cov, inputs)

Computes whether it is possible to remove an element from a synergistic sysem and increase the synergy.

Parameters:

• cov (NDArray[np.floating]) – The covariance matrix that defines the distribution.

• inputs (tuple[int, ...]) – The specific elements to test.

Returns:

Returns True if the synergy is irreducible (i.e. there is no element that, when removed, increaes the synergy). Returns False otherwise.

Return type:

bool

syntropy.gaussian.utils module

syntropy.gaussian.utils.check_cov(cov, data)

Parameters:

• cov (ndarray[tuple[Any, ...], dtype[floating]] | None)

• data (ndarray[tuple[Any, ...], dtype[floating]])

Return type:

ndarray[tuple[Any, …], dtype[floating]]

syntropy.gaussian.utils.correlation_to_mutual_information(cov)

Converts a Pearson correlation matrix to a Guassian mutual information matrix based on the identity:

\(I(X;Y) = \frac{-\log(1-(r_{XY})^2)}{2}\)

Where \(r_{XY}\) is the Pearson correlation coefficient between \(X\) and \(Y\).

Also works for a covariance matrix if the processes have 0 mean and unit variance.

Parameters:

cov (NDArray[np.floating]) – A covariance matrix.

Returns:

The equivalent mutual information matrix.

Return type:

NDArray[np.floating]

syntropy.gaussian.utils.copula_transform(X)

Transform data to Gaussian copula space and compute the correlation matrix. Useful for copula-based information estimators.

The resulting data can be plugged into any local or expected information estimator.

Parameters:

X (ndarray, shape (n_channels, n_samples)) – Input data (channels x samples)

Returns:

• Z (NDArray[np.floating]) – Gaussianized copula data (zero-mean, unit-variance)

• R (NDArray[np.floating]) – Copula correlation matrix

Return type:

tuple[ndarray[tuple[Any, …], dtype[floating]], …]

References

Ince, R. A. A., Giordano, B. L., Kayser, C., Rousselet, G. A., Gross, J., & Schyns, P. G. (2016). A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula. Human Brain Mapping, 38(3), 1541–1573. https://doi.org/10.1002/hbm.23471

syntropy.gaussian.utils.covariance_to_correlation(cov)

Converts a non-standardized covariance matrix to a correlation matrix.

Parameters:

cov (NDArray[np.floating]) – The covariance matrix.

Returns:

The correlation matrix.

Return type:

NDArray[np.floating]