Metrics (xcolumns.metrics)

xcolumns.metrics module implements a set of methods for calculating the metrics based on both the confusion matrix and the true and predicted labels. The methods calculating the metrics on the entries of the confusion matrix can be also used as arguments for the methods in the xcolumns.block_coordinate and xcolumns.frank_wolfe modules.

xcolumns.metrics.binary_0_1_loss(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True)

Calculates 0/1 loss matric from the given true and predicted labels.

See binary_0_1_loss_on_conf_matrix() for definition.

xcolumns.metrics.binary_0_1_loss_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, normalize: bool = True) → int | float | number | ndarray[source]

Calculates binary 0/1 from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{accuracy} = \frac{FP + FN}{TP + FP + FN + TN}\]

If normalize is False, returns the sum of false positives and false negatives.

xcolumns.metrics.binary_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True)

Calculates accuracy matric from the given true and predicted labels.

See binary_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.binary_accuracy_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, normalize: bool = True) → int | float | number | ndarray[source]

Calculates binary accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{accuracy} = \frac{TP + TN}{TP + FP + FN + TN}\]

If normalize is False, returns the sum of true positives and true negatives.

xcolumns.metrics.binary_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates binary balanced accuracy matric from the given true and predicted labels.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.binary_balanced_accuracy_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates ballanced accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Balanced accuracy is the average of true positive rate and true negative rate.

\[\text{balanced accuracy} = \frac{TP}{TP + FN + \epsilon} + \frac{TN}{TN + FP + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-06)

Calculates binary F1 score matric from the given true and predicted labels.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.binary_f1_score_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates binary F1 score, also known as balanced F-score or F-measure from the given true positives, false positives, false negatives and true negatives. This is an alias for binary_fbeta_score_on_conf_matrix() with beta=1.0.

xcolumns.metrics.binary_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-06)

Calculates binary F-beta score matric from the given true and predicted labels.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.binary_fbeta_score_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, beta: float = 1.0, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Compute the binary F-beta score. from the given true positives, false positives, false negatives and true negatives.

The F-beta score is the weighted harmonic mean of precision and recall.

\[F_\beta = (1 + \beta^2) \cdot \frac{TP}{\beta^2 \cdot (TP + FP) + TP + FN + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates binary G-mean matric from the given true and predicted labels.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.binary_gmean_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates G-mean (geometric mean) from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

G-mean is the square root of the product of true positive rate and true negative rate.

\[\text{G-mean} = \sqrt{\frac{TP}{TP + FN + \epsilon} \times \text{TNR} = \frac{TN}{TN + FP + \epsilon}}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates binary H-mean matric from the given true and predicted labels.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.binary_hmean_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates H-mean (harmonic mean) from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

H-mean is the harmonic mean of true positive rate and true negative rate.

\[\text{H-mean} = \frac{2 \times \text{TPR} \times \text{TNR}}{\text{TPR} + \text{TNR}}\]

where \(\text{TPR} = \frac{TP}{TP + FN + \epsilon}\) and \(\text{TNR} = \frac{TN}{TN + FP + \epsilon}\) and \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_jaccard_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates binary Jaccard score matric from the given true and predicted labels.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.binary_jaccard_score_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates Jaccard score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Jaccard score is the intersection over union of predicted and true labels.

\[J = \frac{TP}{TP + FP + FN + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates binary precision matric from the given true and predicted labels.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.binary_precision_at_k_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray | None, fn: int | float | number | ndarray | None, tn: int | float | number | ndarray | None, k: int) → int | float | number | ndarray[source]

Calculates binary precision at k based on the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{precision@k} = \frac{TP}{k}\]

where \(k\) is the number of positive labels per instance.

xcolumns.metrics.binary_precision_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray | None, tn: int | float | number | ndarray | None, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates binary precision from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Precision is the fraction of predicted positives that are actually positive.

\[\text{precision} = \frac{TP}{TP + FP + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates binary recall matric from the given true and predicted labels.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.binary_recall_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray | None, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, epsilon: float = 1e-06) → int | float | number | ndarray[source]

Calculates binary recall from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Recall is the fraction of positives that were correctly predicted.

\[\text{recall} = \frac{TP}{TP + FN + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.check_if_y_pred_at_k(y_pred: ndarray | csr_matrix, k: int) → bool[source]

Checks if y_pred is a binary matrix with exactly k positive labels per instance.

Parameters:

  • y_pred – The binary labels matrix.

  • k – Number of positive labels required per instance.

Returns:

True if y_pred is a binary matrix with exactly k positive labels per instance.

xcolumns.metrics.coverage(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix)

Calculates coverage matric from the given true and predicted labels.

See coverage_on_conf_matrix() for definition.

xcolumns.metrics.coverage_on_conf_matrix(tp: ndarray, fp: ndarray | None, fn: ndarray | None, tn: ndarray | None) → int | float | number[source]

Calculates coverage from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Coverage is the fraction of instances that have at least one positive label.

\[\text{coverage} = \sum_{i=1}^m \frac{TP_i > 0}{m}\]

where \(m\) is the number of labels.

xcolumns.metrics.hamming_loss(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True)

Calculates Hamming loss matric from the given true and predicted labels.

See hamming_loss_on_conf_matrix() for definition.

xcolumns.metrics.hamming_loss_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, normalize: bool = True) → int | float | number[source]

Calculates Hamming loss from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Hamming loss is the fraction of labels that are incorrectly predicted. If normalize is False, returns the mean of false positives and false negatives across all labels.

xcolumns.metrics.hamming_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True)

Calculates Hamming score matric from the given true and predicted labels.

See hamming_score_on_conf_matrix() for definition.

xcolumns.metrics.hamming_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, normalize: bool = True) → int | float | number[source]

Calculates Hamming score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Hamming score is the fraction of correctly predicted labels. If normalize is False, returns the mean of true positives and true negatives across all labels.

xcolumns.metrics.inverse_label_priors(y: ndarray | csr_matrix) → ndarray[source]

Calculates inverse of prior probablity of each label.

Parameters:

y – The binary labels matrix.

Returns:

An vector the inverse label priors.

xcolumns.metrics.jpv_inverse_propensities(y: ndarray | csr_matrix, a: float = 0.55, b: float = 1.5) → ndarray[source]

Calculate inverse propensities of labels an empirical model proposed in Jain et al. 2016. Inverse propensity \(q_j\) of label \(j\) is calculated as:

\[c = (\log n - 1)(b + 1)^a \,, \ q_j = 1 + c(n_j + b)^{-A} \,,\]

where \(n\) is total number of data points, \(n_j\) is total number of data points for and \(a\) and \(b\) are dataset specific parameters.

Parameters:

  • y – The binary labels matrix.

  • a – A dataset specific parameter, typical values: - 0.5: WikiLSHTC-325K and WikipediaLarge-500K - 0.6: Amazon-670K and Amazon-3M - 0.55: otherwise

  • b – Dataset specific parameter, typical values: - 0.4: WikiLSHTC-325K and WikipediaLarge-500K - 2.6: Amazon-670K and Amazon-3M - 1.5: otherwise

Returns:

An vector the inverse label propensities.

xcolumns.metrics.jpv_propensities(y: ndarray | csr_matrix, a: float = 0.55, b: float = 1.5) → ndarray[source]

Calculates propensity of labels using an empirical model proposed in Jain et al. 2016. Propensity \(p_j\) of label \(j\) is calculated as:

\[c = (\log n - 1)(b + 1)^a \,, \ p_j = \frac{1}{1 + c(n_l + b)^{-a}} \,,\]

where \(n\) is total number of data points, \(n_j\) is total number of data points for and \(a\) and \(b\) are dataset specific parameters.

Parameters:

  • y – The binary labels matrix.

  • a – A dataset specific parameter, typical values: - 0.5: WikiLSHTC-325K and WikipediaLarge-500K - 0.6: Amazon-670K and Amazon-3M - 0.55: otherwise

  • b – Dataset specific parameter, typical values: - 0.4: WikiLSHTC-325K and WikipediaLarge-500K - 2.6: Amazon-670K and Amazon-3M - 1.5: otherwise

Returns:

An vector the label propensities.

xcolumns.metrics.label_counts(y: ndarray | csr_matrix) → ndarray[source]

Counts number of occurrences of each label.

Parameters:

y – The binary labels matrix.

Returns:

An vector the count of label occurrences.

xcolumns.metrics.label_priors(y: ndarray | csr_matrix) → ndarray[source]

Calculates prior probablity of each label.

Parameters:

y – The binary labels matrix.

Returns:

An vector the label priors.

xcolumns.metrics.macro_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates macro-averaged balanced accuracy matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_balanced_accuracy_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged balanced accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.macro_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-06)

Calculates macro-averaged F1 score matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_f1_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged F1 score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.macro_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-06)

Calculates macro-averaged F-beta score matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_fbeta_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged F-beta score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.macro_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates macro-averaged G-mean matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_gmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged G-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.macro_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates macro-averaged H-mean matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_hmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged H-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.macro_jaccard_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates macro-averaged Jaccard score matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_jaccard_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged Jaccard score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.macro_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates macro-averaged precision matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_precision_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged precision from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.macro_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates macro-averaged recall matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_recall_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged recall from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.micro_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates micro-averaged balanced accuracy matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_balanced_accuracy_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged balanced accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.micro_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-06)

Calculates micro-averaged F1 score matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_f1_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged F1 score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.micro_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-06)

Calculates micro-averaged F-beta score matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_fbeta_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged F-beta score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.micro_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates micro-averaged G-mean matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_gmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged G-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.micro_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates micro-averaged H-mean matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_hmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged H-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.micro_jaccard_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates micro-averaged Jaccard score matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_jaccard_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged Jaccard score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.micro_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates micro-averaged precision matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_precision_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged precision from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.micro_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-06)

Calculates micro-averaged recall matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_recall_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-06) → int | float | number

Calculates macro-averaged recall from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.precision_at_k(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix)

Calculates precision at k matric from the given true and predicted labels.

See precision_at_k_on_conf_matrix() for definition.

xcolumns.metrics.precision_at_k_on_conf_matrix(tp: ndarray, fp: ndarray | None, fn: ndarray | None, tn: ndarray | None, k: int) → int | float | number[source]

Calculates precision at k from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Factory functions for specific metrics

The module provides the factory functions for creating the metric micro- and macro-averaged functions based on the binary metric function as well as the functions that calculate the metrics on true and predicted labels.

xcolumns.metrics.make_macro_metric_on_conf_matrix(binary_metric, name) → Callable[source]

This is a factory function to create a macro-averaged metric from a binary metric defined on confusion matrix: binary_metric(tp, fp, fn, tn).

Parameters:

  • binary_metric – binary metric defined on confusion matrix

  • name – name of the metric to be used in the docstring

Returns:

The function calculating the macro-averaged metric defined on a confusion matrix.

xcolumns.metrics.make_micro_metric_on_conf_matrix(binary_metric: Callable, metric_name: str) → Callable[source]

This is a factory function to create a micro-averaged metric from a binary metric defined on confusion matrix: binary_metric(tp, fp, fn, tn).

Parameters:

  • binary_metric – binary metric defined on confusion matrix

  • name – name of the metric to be used in the docstring

Returns:

The function calculating the micro-averaged metric defined on a confusion matrix.

xcolumns.metrics.make_metric_on_y_true_and_y_pred(metric_on_conf_matrix: Callable, metric_name: str, skip_tn: bool = False) → Callable[source]

This is a factory function to create a metric calculated on true and predicted labels: metric(y_true, y_pred) from metric calculated on confusion matrix: metric(tp, fp, fn, tn).

Parameters:

  • metric_on_conf_matrix – metric calculated on confusion matrix

  • metric_name – name of the metric to be used in the docstring

  • skip_tn – whether to skip true negatives

Returns:

Function etric calculated on true and predicted labels