Shannon entropy, KL & Jensen-Shannon divergence, mutual information, Rényi and
Tsallis entropy — mirrors scipy.stats.entropy and related utilities.
entropy(pk) computes H(p) = −∑ pᵢ log pᵢ.
Use base=2 for bits, omit base for nats.
entropy(pk, qk) computes D_KL(p‖q). klDivergence is an explicit alias.
crossEntropy(p, q) = H(p) + D_KL(p‖q).
JSD is a symmetric, bounded (0 ≤ JSD ≤ log 2) measure of similarity between distributions.
jsDistance is the square root — a proper metric satisfying the triangle inequality.
mutualInformation(xy) accepts an array of [x, y] pairs and estimates
I(X;Y) = H(X) + H(Y) − H(X,Y). Related: jointEntropy, conditionalEntropy.
normalizedMI scales I(X;Y) to [0, 1]. Four normalisation methods available.
variationOfInformation is a metric on partitions: VI = H(X|Y) + H(Y|X).
Generalisations of Shannon entropy. Rényi: H_α(X) = (1/(1−α)) log(∑ pᵢ^α). Tsallis: S_q(X) = (1/(q−1))(1 − ∑ pᵢ^q). Both converge to Shannon at α/q → 1.