emb_diversity.measures package

Submodules

emb_diversity.measures.bins_entropy.bins_entropy(data, n_bins_x=5, n_bins_y=5, normalize=True, normalization='effective', projection='umap', pca_kwargs=None, umap_kwargs=None, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: [0, 1] with the default "effective" normalization; with "bins" it may be < 1.

Compute bins-based entropy diversity from a 2D projection of a vector set.

  1. Project the input vectors to 2D with UMAP or PCA.

  2. Bin points into a n_bins_x × n_bins_y grid.

  3. Compute Shannon entropy over bin occupancies.

  4. Optionally normalize.

References

Cox, Samuel Rhys, Yunlong Wang, Ashraf Abdul, Christian von der Weth, and Brian Y. Lim. “Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation.” Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, May 6, 2021, 1–35. https://doi.org/10.1145/3411764.3445782. Yang, Yuming, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, and Xuan-Jing Huang. “Measuring data diversity for instruction tuning: A systematic analysis and a reliable metric.” In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 18530-18549. 2025.

Parameters:

  • data – Iterable/array-like of (embedding) vectors with shape (n, d). Must contain at least 2 samples. Accepts numpy arrays and (optionally) torch tensors.

  • n_bins_x (int) – Number of bins along x-axis. Must be > 0.

  • n_bins_y (int) – Number of bins along y-axis. Must be > 0.

  • normalize (bool) – If True, normalize entropy by a log factor.

  • normalization (str) –

    Normalization denominator:

    • ”effective”: (default) divide by log(min(n, B)); ensures result in [0, 1].

    • ”bins”: divide by log(B) (paper-style; max < 1 when n < B).

  • projection (str) – “umap” or “pca”. Defaults to “umap”.

  • pca_kwargs – Extra kwargs passed to PCA(…). Defaults to None (treated as {}). PCA is deterministic for full SVD solver.

  • umap_kwargs – Extra kwargs passed to UMAP(…). Defaults to None (treated as {}). If random_state is not provided, random_state=42 is used.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the bins-based entropy (normalized to [0, 1] if normalize=True with the default effective normalization) and parameters records the configuration used.

Raises:

  • ImportError – If projection is “umap” but UMAP is not installed.

  • ValueError – If input shape, bins, projection, or normalization is invalid.

emb_diversity.measures.bottleneck.bottleneck(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute the minimum pairwise distance in a vector set.

  1. Compute all unique pairwise distances between datapoints.

  2. Return the smallest distance (the bottleneck distance).

References

Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556. Xie, Yutong, Ziqiao Xu, Jiaqi Ma, and Qiaozhu Mei. “How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules.” arXiv:2112.12542. Preprint, arXiv, March 6, 2023. https://doi.org/10.48550/arXiv.2112.12542.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of embedding vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the minimum distance across all unique pairs and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.chamfer_dist.chamfer_dist(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute the average nearest-neighbour distance across all datapoints.

  1. Compute all unique pairwise distances between datapoints.

  2. For each point, find the minimum distance to any other point (excluding itself).

  3. Return the mean of those nearest-neighbour distances.

References

Cox, Samuel Rhys, Yunlong Wang, Ashraf Abdul, Christian von der Weth, and Brian Y. Lim. “Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation.” Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, May 6, 2021, 1–35. https://doi.org/10.1145/3411764.3445782. Zhang, Tianhui, Bei Peng, and Danushka Bollegala. “Evaluating the Evaluation of Diversity in Commonsense Generation.” In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), edited by Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar. Association for Computational Linguistics, 2025. https://aclanthology.org/2025.acl-long.1181/.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the mean nearest-neighbour distance (higher = more dispersed) and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.cluster_inertia.cluster_inertia(data, n_clusters=200, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, unbounded above.

Compute diversity as k-means inertia over (embedding) vectors.

  1. Fit k-means with n clusters on the data.

  2. Return the inertia (sum of squared Euclidean distances to cluster centres).

References

Yang, Yuming, Yang Nan, Junjie Ye, et al. “Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric.” arXiv:2502.17184. Preprint, arXiv, February 28, 2025. https://doi.org/10.48550/arXiv.2502.17184. Du, Wenchao, and Alan W. Black. “Boosting Dialog Response Generation.” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, edited by Anna Korhonen, David Traum, and Lluís Màrquez. Association for Computational Linguistics, 2019. https://doi.org/10.18653/v1/P19-1005.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • n_clusters (int) – Number of clusters for k-means. For 10k samples, 200 clusters is a common default in the literature (e.g. Yang et al 2025) Automatically reduced to n-1 if fewer datapoints than clusters are provided.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the sum of squared Euclidean distances from each point to its assigned cluster centre (higher = greater spread) and parameters records the configuration used.

Raises:

ValueError – If data is empty or contains fewer than 2 datapoints.

emb_diversity.measures.convex_hull_volume_3d.convex_hull_volume_3d(data, random_state=42, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, unbounded above (0 if the projected points are coplanar).

Compute diversity as the volume of the convex hull of the data, after first projecting the input to 3 dimensions.

  1. Project the input to 3D (UMAP, or use as-is if already 3D).

  2. Compute the convex hull of the projected points.

  3. Return its volume (0.0 if the points are coplanar).

References

Yu, Yu, Shahram Khadivi, and Jia Xu. “Can data diversity enhance learning generalization?.” Proceedings of the 29th international conference on computational linguistics. 2022.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable of vectors (lists/tuples/np.ndarrays), shape (n, d), or raw text strings.

  • random_state (int) – Seed passed to UMAP. Defaults to 42.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the volume of the 3D convex hull of the projected points (0.0 if coplanar) and parameters records the configuration used. This value is not normalized (it is not in [0, 1]).

Raises:

ValueError – If data is empty or has fewer than 4 points.

Note

Why project to 3D first: scipy’s ConvexHull (Qhull) becomes intractable past ~10 dimensions: the number of facets grows as O(n^floor(d/2)). For typical embedding dimensions (e.g. 4096), the raw convex hull is both infeasible to compute and numerically meaningless (the “volume” is a product of many small lengths and underflows). Projecting to 3D yields a stable, comparable “volume” measure.

Dimension reduction: if the input already has 3 columns, it is used as-is (no projection). Otherwise the input is projected with umap.UMAP(n_components=3, random_state=random_state). If umap-learn is not installed, or if UMAP fails to fit (e.g. on very small inputs where n_neighbors exceeds n), the function falls back to taking the first 3 columns of the input and emits a UserWarning. This fallback is only sensible for toy data — for real embeddings, install umap-learn.

Comparability caveat: the returned value is a volume in the UMAP-projected space. UMAP is non-linear and its scale is data-dependent, so values are NOT comparable across separate UMAP fits (i.e. across different datasets, or across runs with different random_state). Fix random_state and compare within a single experiment.

emb_diversity.measures.dcscore.dcscore(data, kernel_type='cs', tau=1.0, normalize=True, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: [1, n] (1 when all points are identical; approaches n when each row’s softmax is concentrated on the diagonal). The attainable maximum depends on kernel_type / normalize and tau; for the default kernel_type="cs" with normalize=True and tau=1, the maximum for well-separated points approaches e ≈ 2.718 as n grows.

Compute DCScore, a diversity metric based on the self-similarity of the samples under a row-wise softmax (as in the original DCScore implementation).

  1. Build a similarity matrix K ∈ R^{n×n} from the input vectors.

  2. Apply a row-wise softmax to K to obtain a probability matrix P.

  3. Return the sum of the diagonal of P: DCScore = sum_i P_ii.

References

Zhu, Yuchang, Huizhe Zhang, Bingzhe Wu, Jintang Li, Zibin Zheng, Peilin Zhao, Liang Chen, and Yatao Bian. “Measuring diversity in synthetic datasets.” arXiv preprint arXiv:2502.08512 (2025).

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • kernel_type (str) –

    Type of similarity/kernel:

    • "cs": cosine-similarity-like (X Xᵀ / tau, optionally normalized).

    • "rbf": RBF kernel (rbf_kernel with gamma=tau).

    • "lap": Laplacian kernel (laplacian_kernel with gamma=tau).

    • "poly": Polynomial kernel (polynomial_kernel with degree=tau).

  • tau (float) – Temperature / kernel parameter. For “cs”, the similarity matrix is (X Xᵀ) / tau. For RBF / Laplacian it is passed as gamma=tau, for polynomial as degree=tau.

  • normalize (bool) – If True and kernel_type==”cs”, L2-normalize the input vectors row-wise before computing X Xᵀ, as in the original text-based DCScore.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the scalar DCScore and parameters records the configuration used.

Raises:

  • ValueError – If there are fewer than 2 datapoints or tau <= 0.

  • NotImplementedError – For unknown kernel_type.

Warns:

UserWarning – If kernel_type="cs" and normalize=True and the input contains an all-zero row (cosine similarity is undefined for it). The score is still returned, treating the zero row as near-orthogonal to every other point.

emb_diversity.measures.diameter.diameter(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute the maximum pairwise distance in a set of vectors (the diameter).

  1. Compute all unique pairwise distances between datapoints.

  2. Return the largest distance (the diameter).

References

Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556. Xie, Yutong, Ziqiao Xu, Jiaqi Ma, and Qiaozhu Mei. “How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules.” arXiv:2112.12542. Preprint, arXiv, March 6, 2023. https://doi.org/10.48550/arXiv.2112.12542.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the maximum distance across all unique pairs and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.energy.energy(data, metric='cosine', gamma=1.0, epsilon=1e-12, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value (closer to 0) = more diverse. Range: (-inf, 0]; always <= 0.

Compute the energy-based diversity of a set of vector representations.

  1. Compute all unique pairwise distances between datapoints (floored at epsilon for numerical stability).

  2. Raise each distance to the power gamma and take its reciprocal (the pairwise energy).

  3. Return the negative mean of these pairwise energies.

References

Velikonivtsev, Fedor, Mikhail Mironov, and Liudmila Prokhorenkova. “Challenges of generating structurally diverse graphs.” Advances in Neural Information Processing Systems 37 (2024): 57993-58022. Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • gamma (float) – Exponent applied to each pairwise distance. Defaults to 1.0 (as in the paper).

  • epsilon (float) – Lower bound applied to each distance, so zero distances (e.g. duplicates) do not blow up the reciprocal. Defaults to 1e-12.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the energy of the dataset and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

Note

When gamma is 1, the value can be interpreted as the average pairwise energy of a system of equally charged particles. The result is multiplied by -1 so that larger values correspond to more diverse datasets.

emb_diversity.measures.geo_mean_std.geo_mean_std(data, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, unbounded above.

Compute diversity as the geometric mean of the per-dimension standard deviations along each (embedding) dimension.

  1. Compute the standard deviation σi along each (embedding) dimension i.

  2. Return their geometric mean: (σ1 * σ2 * … * σH) ** (1/H).

References

Lai, Yi-An, et al. “Diversity, density, and homogeneity: Quantitative characteristic metrics for text collections.” Proceedings of the Twelfth Language Resources and Evaluation Conference. 2020.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the geometric mean of standard deviations across all embedding dimensions and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

Note

Because this is a geometric mean, it is very sensitive to low-variance dimensions: a single near-constant dimension (std clipped to 1e-12 to avoid log(0)) drags the whole value toward 0, even if every other dimension is well spread. The value also scales with the magnitude of the input vectors, so it is not comparable across differently scaled embeddings.

emb_diversity.measures.graph_entropy.graph_entropy(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, grows with n (a sum of per-node entropies, each <= log(n-1)).

Compute graph entropy over a complete weighted graph whose vertices are the data samples and whose edge weights are pairwise distances.

  1. Build a complete weighted graph: the weight of edge (i, j) is the pairwise distance d_ij under metric.

  2. Turn each node’s edge weights into a local probability distribution: f_ij = d_ij / sum_k d_ik, i.e. normalize node i’s distances to all other nodes so they sum to 1.

  3. Compute each node’s local Shannon entropy: H_i = -sum_j f_ij * log(f_ij).

  4. Return the total graph entropy: the sum of all local node entropies.

References

Yu, Yu, Shahram Khadivi, and Jia Xu. “Can data diversity enhance learning generalization?.” Proceedings of the 29th international conference on computational linguistics. 2022.

Parameters:

  • data (TensorLike) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 3 samples: with only 2 each node has a single neighbour, so the entropy is degenerately 0.

  • metric (DistanceMetric) – Distance metric name or callable accepted by scipy.spatial.distance.pdist, used as edge weights. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

MeasureResult

Returns:

A dict {"value": float, "parameters": {...}} where value is the total graph entropy (sum of all local node entropies) and parameters records the configuration used.

Raises:

ValueError – If input is not 2D, empty, or has fewer than 3 datapoints.

emb_diversity.measures.hamdiv.hamdiv(data, metric='cosine', solver='christofides', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, grows with n; the upper bound depends on metric (unbounded for an unbounded metric).

Compute geometric diversity as the length of the shortest Hamiltonian circuit (Traveling Salesman Problem tour) through all points.

  1. Build a complete weighted graph: the weight of edge (i, j) is the pairwise distance d_ij under metric.

  2. Find an (approximately) shortest Hamiltonian circuit through all points with the chosen TSP solver.

  3. Return the total length of that tour.

References

Hu, Xiuyuan, et al. “Hamiltonian diversity: effectively measuring molecular diversity by shortest hamiltonian circuits.” Journal of Cheminformatics 16.1 (2024): 94. Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable of vectors (lists/tuples/np.ndarrays), shape (n, d), or raw text strings.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable, as accepted by scipy.spatial.distance.pdist. Default is "cosine".

  • solver (Literal['greedy', 'christofides']) –

    NetworkX TSP solver strategy. Options:

    • "greedy": Greedy nearest-neighbour heuristic.

    • "christofides": Christofides algorithm (default).

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the length of the Hamiltonian circuit and parameters records the configuration used.

Raises:

ValueError – If data is empty or contains fewer than 2 datapoints, or if solver is invalid.

emb_diversity.measures.knn.knn(data, k=2, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute the average k-th-nearest-neighbour distance across all datapoints.

A generalisation of chamfer_dist() (which is the special case k=1): instead of each point’s nearest neighbour, it looks at each point’s k-th nearest neighbour.

  1. Compute all unique pairwise distances between datapoints.

  2. For each point, find the distance to its k-th nearest neighbour (excluding itself; k=1 is the nearest neighbour, k=2 the second-nearest, and so on).

  3. Return the mean of those k-th-nearest-neighbour distances.

References

Yang, Yuming, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, and Xuan-Jing Huang. “Measuring data diversity for instruction tuning: A systematic analysis and a reliable metric.” In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 18530-18549. 2025.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least k + 1 samples.

  • k (int) – Which nearest neighbour to use, counted from 1 (the closest other point). Defaults to 2 (the second-nearest neighbour).

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the mean k-th-nearest-neighbour distance (higher = more dispersed) and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, k is not positive, or there are fewer than k + 1 datapoints.

emb_diversity.measures.log_determinant.log_determinant(data, kernel_type='cs', tau=1.0, normalize=True, eps=1e-06, use_cholesky=True, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse (can be negative; less negative = more diverse). Range: unbounded in general, (-inf, +inf), depending on kernel choice. For the default cosine kernel (kernel_type="cs", tau=1.0, normalize=True), trace(K) = n bounds the value above by n * log(1 + eps) (approximately 0), so scores are negative in practice, with magnitude driven by zero or near-zero eigenvalues (e.g. when n > d).

Compute Log-Determinant Diversity (LDD): the log-determinant of a kernel matrix built from the input vectors, LDD = log det(K + eps * I). For a positive definite kernel matrix this measures the “volume” spanned by the data in the feature space, so larger values indicate greater diversity.

  1. Build a similarity/kernel matrix K from the input vectors (per kernel_type).

  2. Add jitter to the diagonal for numerical stability: A = K + eps * I.

  3. Return log det(A) (via Cholesky, falling back to slogdet).

References

Kulesza, A., & Taskar, B. (2012). Determinantal Point Processes for Machine Learning. Found. Trends Mach. Learn., 5, 123-286. Wang, Peiqi, Yikang Shen, Zhen Guo, Matthew Stallone, Yoon Kim, Polina Golland, and Rameswar Panda. “Diversity measurement and subset selection for instruction tuning datasets.” arXiv preprint arXiv:2402.02318 (2024).

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • kernel_type (str) –

    Type of similarity/kernel:

    • "cs": cosine-similarity-like (X Xᵀ / tau, optionally normalized).

    • "rbf": RBF kernel (rbf_kernel with gamma=tau).

    • "lap": Laplacian kernel (laplacian_kernel with gamma=tau).

    • "poly": Polynomial kernel (polynomial_kernel with degree=tau).

  • tau (float) – Temperature / kernel parameter. For “cs”, the similarity matrix is (X Xᵀ) / tau. For RBF / Laplacian it is passed as gamma=tau, for polynomial as degree=tau.

  • normalize (bool) – If True and kernel_type==”cs”, L2-normalize the input vectors row-wise before computing X Xᵀ, so dot product equals cosine similarity.

  • eps (float) – Jitter term added to the diagonal (eps * I) for numerical stability. Makes the matrix more positive definite and prevents singular matrices.

  • use_cholesky (bool) – If True, use Cholesky decomposition for efficient logdet computation when the matrix is positive definite. Falls back to slogdet if Cholesky fails.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the log-determinant of (K + eps * I) (higher = more diverse) and parameters records the configuration used.

Raises:

  • ValueError – If there are fewer than 2 datapoints, or if tau <= 0, or if eps <= 0.

  • NotImplementedError – For unknown kernel_type.

  • np.linalg.LinAlgError – If the matrix determinant is not positive (sign <= 0) after adding eps. Try increasing eps or re-checking kernel choice.

Warns:

UserWarning – If kernel_type="cs" and normalize=True and the input contains an all-zero row (cosine similarity is undefined for it). The score is still returned, treating the zero row as near-orthogonal to every other point.

emb_diversity.measures.mean_pw_dist.mean_pw_dist(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute the average of all pairwise distances between datapoints.

  1. Compute all unique pairwise distances between datapoints.

  2. Return their mean.

References

Guy Tevet and Jonathan Berant. 2021. Evaluating the Evaluation of Diversity in Natural Language Generation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 326–346, Online. Association for Computational Linguistics. Tianhui Zhang, Bei Peng, and Danushka Bollegala. 2024. Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9226–9242, Miami, Florida, USA. Association for Computational Linguistics. Miranda, Brando, Alycia Lee, Sudharsan Sundar, Allison Casasola, Rylan Schaeffer, Elyas Obbad, and Sanmi Koyejo. “Beyond scale: The diversity coefficient as a data quality metric for variability in natural language data.” arXiv preprint arXiv:2306.13840 (2023). Cox, Samuel Rhys, et al. “Directed diversity: Leveraging language embedding distances for collective creativity in crowd ideation.” Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the average pairwise distance across all unique pairs and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.mst_dispersion.mst_dispersion(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, grows with n; bounded by (n-1) times the largest edge under metric.

Compute the total edge weight of the minimum spanning tree (MST).

  1. Build a complete weighted graph: the weight of edge (i, j) is the pairwise distance d_ij under metric.

  2. Compute the minimum spanning tree of that graph.

  3. Return the total weight of the MST edges.

References

Cox, Samuel Rhys, et al. “Directed diversity: Leveraging language embedding distances for collective creativity in crowd ideation.” Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021. Atwal, Tevin, Chan Nam Tieu, Yefeng Yuan, Zhan Shi, Yuhong Liu, and Liang Cheng. “Privacy-Preserving Synthetic Review Generation with Diverse Writing Styles Using LLMs.” arXiv preprint arXiv:2507.18055 (2025).

Parameters:

  • data (TensorLike) – (Embedding) vectors of shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (DistanceMetric) – Distance metric name or callable accepted by scipy.spatial.distance.pdist, used as edge weights. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

MeasureResult

Returns:

A dict {"value": float, "parameters": {...}} where value is the total MST edge weight and parameters records the configuration used.

Raises:

ValueError – If input is not 2D, empty, or has fewer than 2 datapoints.

emb_diversity.measures.renyi_entropy.renyi_entropy(data, alpha=2.0, kernel_type='cs', tau=1.0, normalize=True, eps=1e-12, use_eigendecomp=None, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: [0, log(n)] (natural log / nats).

Compute Rényi Kernel Entropy (RKE), a matrix-based Rényi entropy of the eigenvalue spectrum of a kernel matrix built from the input vectors. A more spread-out spectrum (more modes in the vector space) gives higher entropy.

  1. Build a PSD kernel/similarity matrix K (n x n) from the input vectors.

  2. Normalize to A = K / tr(K) so its eigenvalues lambda_i sum to 1.

  3. Compute the order-alpha Rényi entropy of the eigenvalues of A: RKE = (1 / (1 - alpha)) * log(sum_i lambda_i ** alpha).

Special cases (computed without a full eigendecomposition):

  • alpha = 2: RKE = -log(tr(A^2)) = -log(||A||_F^2).

  • alpha = 1: von Neumann entropy, -sum_i lambda_i * log(lambda_i).

References

Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556. Jalali, Mohammad, Cheuk Ting Li, and Farzan Farnia. “An information-theoretic evaluation of generative models in learning multi-modal distributions.” Advances in Neural Information Processing Systems 36 (2023): 9931-9943.

Parameters:

  • data (Sequence[Sequence[float]]) – (Embedding) vectors of shape (n, d), or raw text strings. Must contain at least 2 samples.

  • alpha (float) – Order of the Rényi entropy. Must be > 0. Defaults to 2.0.

  • kernel_type (str) –

    Type of similarity/kernel:

    • "cs": linear kernel on (optionally) L2-normalized vectors (PSD).

    • "rbf": RBF kernel with gamma=tau.

    • "lap": Laplacian kernel with gamma=tau.

    • "poly": Polynomial kernel with degree=int(tau).

  • tau (float) –

    Kernel parameter:

    • "cs": temperature scaling via division by tau (K = (X Xᵀ) / tau).

    • "rbf" / "lap": passed as gamma=tau.

    • "poly": degree=tau (must be an integer).

  • normalize (bool) – If True and kernel_type==”cs”, L2-normalize rows so the dot product equals cosine similarity.

  • eps (float) – Jitter for numerical stability (clips eigenvalues, avoids division by zero).

  • use_eigendecomp (bool | None) – If None, choose automatically (False for alpha in {1, 2}, True otherwise). If True, force eigenvalue computation even for alpha==2. If False, restrict alpha to {1, 2} (errors for other alpha values).

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the RKE score and parameters records the configuration used.

Raises:

  • ValueError – If there are fewer than 2 datapoints, if tau <= 0, if alpha <= 0, or if use_eigendecomp=False with alpha not in {1, 2}.

  • NotImplementedError – For unknown kernel_type.

Warns:

UserWarning – If kernel_type="cs" and normalize=True and the input contains an all-zero row (cosine similarity is undefined for it). The score is still returned, treating the zero row as near-orthogonal to every other point.

emb_diversity.measures.span_centroid.span_centroid(data, metric='cosine', percentile=90.0, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute Span with Centroid diversity: a percentile of the distances from each datapoint to the dataset centroid.

  1. Compute the centroid (mean) of all input vectors.

  2. Compute each point’s distance to the centroid under metric.

  3. Return the given percentile of those distances.

References

Cox, Samuel Rhys, Yunlong Wang, Ashraf Abdul, Christian von der Weth, and Brian Y. Lim. “Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation.” Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, May 6, 2021, 1–35. https://doi.org/10.1145/3411764.3445782.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.cdist. Defaults to “cosine”.

  • percentile (float) – Percentile (0–100) of the centroid distances to return. Defaults to 90.0.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to cdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the specified percentile of distances from datapoints to the centroid and parameters records the configuration used.

Raises:

ValueError – If input is not 2D, empty, or has fewer than 2 datapoints — or, under the cosine metric, if a datapoint or the centroid is the zero vector (cosine distance is undefined there).

emb_diversity.measures.span_medoid.span_medoid(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; the upper bound depends on metric (e.g. [0, 2] for cosine distance).

Compute Span with Medoid diversity: the mean distance from all datapoints to the medoid.

  1. Compute all pairwise distances between datapoints.

  2. Find the medoid: the point with the smallest sum of distances to all others.

  3. Return the mean distance from all points to the medoid.

References

Cox, Samuel Rhys, Yunlong Wang, Ashraf Abdul, Christian von der Weth, and Brian Y. Lim. “Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation.” Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, May 6, 2021, 1–35. https://doi.org/10.1145/3411764.3445782.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the mean distance to the medoid and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.sum_bottleneck.sum_bottleneck(data, metric='cosine', normalize_by_n=False, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; grows with n unless normalize_by_n; the upper bound depends on metric (e.g. <= 2n for cosine distance).

Compute SumBottleneck: the sum over samples of each sample’s distance to its nearest other sample, SumBottleneck(X) = sum_i min_{j != i} d(x_i, x_j).

  1. Compute all pairwise distances between datapoints.

  2. For each sample, take the distance to its nearest other sample.

  3. Return the sum of those per-sample minima (or their average if normalize_by_n).

References

Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556. Xie, Yutong, Ziqiao Xu, Jiaqi Ma, and Qiaozhu Mei. “How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules.” arXiv:2112.12542. Preprint, arXiv, March 6, 2023. https://doi.org/10.48550/arXiv.2112.12542.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • normalize_by_n (bool) – If True, return the average per-sample minimum distance (sum / n) instead of the sum. Defaults to False.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the sum (or average if normalized) of per-sample minimum distances and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.sum_diameter.sum_diameter(data, metric='cosine', normalize_by_n=False, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0; grows with n unless normalize_by_n; the upper bound depends on metric (e.g. <= 2n for cosine distance).

Compute SumDiameter: the sum over samples of each sample’s distance to its farthest other sample, SumDiameter(X) = sum_i max_{j != i} d(x_i, x_j).

  1. Compute all pairwise distances between datapoints.

  2. For each sample, take the distance to its farthest other sample.

  3. Return the sum of those per-sample maxima (or their average if normalize_by_n).

References

Mironov, Mikhail, and Liudmila Prokhorenkova. “Measuring Diversity: Axioms and Challenges.” arXiv:2410.14556. Preprint, arXiv, June 14, 2025. https://doi.org/10.48550/arXiv.2410.14556. Xie, Yutong, Ziqiao Xu, Jiaqi Ma, and Qiaozhu Mei. “How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules.” arXiv:2112.12542. Preprint, arXiv, March 6, 2023. https://doi.org/10.48550/arXiv.2112.12542.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • normalize_by_n (bool) – If True, return the average per-sample maximum distance (sum / n) instead of the sum. Defaults to False.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the sum (or average if normalized) of per-sample maximum distances and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

emb_diversity.measures.sum_pairwise_dist.sum_pairwise_dist(data, metric='cosine', *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None, **metric_kwargs)[source]

Interpretation of values: larger value = more diverse. Range: >= 0, grows with n; the upper bound depends on metric (e.g. <= n(n-1) for cosine distance).

Compute the sum of all pairwise distances between datapoints.

  1. Compute all unique pairwise distances between datapoints.

  2. Return their sum.

References

Yu, Yu, Shahram Khadivi, and Jia Xu. “Can data diversity enhance learning generalization?.” Proceedings of the 29th international conference on computational linguistics. 2022. Yang, Yuming, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, and Xuan-Jing Huang. “Measuring data diversity for instruction tuning: A systematic analysis and a reliable metric.” In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 18530-18549. 2025.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name or callable accepted by scipy.spatial.distance.pdist. Defaults to “cosine”.

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to pdist for the selected metric.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the sum of all pairwise distances across all unique pairs and parameters records the configuration used.

Raises:

ValueError – If input is invalid, empty, or has fewer than 2 datapoints.

Shared type aliases for emb_diversity.

Two-level cached pairwise distance computation.

scipy.pdist is the bottleneck when several measures are run on the same embedding matrix. This module wraps it with:

Level 1 — in-process memory: a small bounded LRU dict keyed by full

content fingerprint of the matrix plus the metric / kwargs. Up to _MEMORY_MAX entries are kept; oldest is evicted on overflow.

Level 2 — disk: condensed distance array stored under .cache/pdist/

as safetensors, keyed the same way. Survives across processes — a SLURM job that finished yesterday leaves a cache that today’s job can pick up.

Level 3 — compute: scipy.pdist + write through both layers.

The cache key folds in the metric and any metric_kwargs, so different metrics on the same data do not collide.

emb_diversity.measures.utils.clear_distance_cache(cache_dir=PosixPath('.cache/pdist'))[source]

Clear both memory and disk caches.

Return type:

None

emb_diversity.measures.utils.compute_pairwise_distances(data, metric='cosine', cache_dir=PosixPath('.cache/pdist'), **metric_kwargs)[source]

Compute pairwise distances with two-level (memory + disk) caching.

Parameters:

  • data (Sequence[Sequence[float]]) – 2D array-like of shape (n_samples, n_features).

  • metric (str | Callable[[ndarray, ndarray], float]) – Distance metric name (e.g. “cosine”, “euclidean”) or callable.

  • cache_dir (Path) – Root directory for the disk cache.

  • **metric_kwargs (Any) – Extra keyword arguments forwarded to scipy.pdist. Included in the cache key so different kwargs do not collide.

Return type:

ndarray

Returns:

Condensed distance array (upper triangle from scipy.pdist).

Raises:

ValueError – If data is not numeric (strings are rejected, not coerced), has fewer than 2 samples, is not 2-dimensional, contains non-finite values (nan or inf), or contains all-zero vectors while metric is "cosine" (cosine distance is undefined for zero vectors).

emb_diversity.measures.utils.distance_cache_info(cache_dir=PosixPath('.cache/pdist'))[source]

Return memory and disk cache statistics.

Return type:

dict

emb_diversity.measures.vendi_score.vendi_score(data, q=1.0, normalize=True, use_dual=True, *, diversity_axis='semantic', embedding_model=None, chunking_kwargs=None)[source]

Interpretation of values: larger value = more diverse. Range: [1, n] (the effective number of unique elements).

Compute diversity using the Vendi Score: the “effective number of unique elements” in a set, derived from the entropy of the eigenvalues of a similarity matrix over the samples.

  1. Build a similarity matrix over the samples (a dot-product / cosine kernel on the input vectors; the dual formulation avoids forming it explicitly).

  2. Take the eigenvalues of the similarity matrix, normalized to sum to 1.

  3. Return the exponential of their order-q (Rényi) entropy — the effective number of unique elements.

References

Friedman, Dan, and Adji Bousso Dieng. “The Vendi Score: A Diversity Evaluation Metric for Machine Learning.” Transactions on Machine Learning Research (2023). Pasarkar, A. P., & Dieng, A. B. (2023). Cousins of the vendi score: A family of similarity-based diversity metrics for science and machine learning. arXiv preprint arXiv:2310.12952.

Parameters:

  • data (Sequence[Sequence[float]]) – Iterable/array-like of (embedding) vectors with shape (n, d), or raw text strings. Must contain at least 2 samples.

  • q (float) – Order of the Vendi score (Renyi-style generalization). q = 1.0 corresponds to the original Vendi Score.

  • normalize (bool) – Whether to L2-normalize rows of X when using dot-product similarity. For normalized vectors, the dot product corresponds to cosine similarity.

  • use_dual (bool) – If True, use vendi.score_dual(X, …) which is efficient when d < n. If False, build a Gram matrix K and call vendi.score_K(K, …).

  • diversity_axis (str) – Registered axis used to embed text input (default “semantic”).

  • embedding_model (str | None) – Explicit embedding model id; overrides diversity_axis.

Return type:

Dict[str, Any]

Returns:

A dict {"value": float, "parameters": {...}} where value is the Vendi Score and parameters records the configuration used.

Raises:

  • ImportError – If vendi_score is not installed.

  • ValueError – If input is not 2D, or has fewer than 2 datapoints.

Warns:

UserWarning – If normalize=True and the input contains an all-zero row (cosine similarity is undefined for it). The score is still returned, treating the zero row as near-orthogonal to every other point. Applies to both the dual and explicit paths.

Note

Wraps the official vendi_score implementation (https://github.com/vertaix/Vendi-Score).

Module contents