Metrics (xcolumns.metrics)

xcolumns.metrics module implements a set of methods for calculating the metrics based on both the confusion matrix and the true and predicted labels. The methods calculating the metrics on the entries of the confusion matrix can be also used as arguments for the methods in the xcolumns.block_coordinate and xcolumns.frank_wolfe modules.

xcolumns.metrics.abandonment(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix) → ndarray

Calculates instance-averaged abandonment metric from the given true and predicted labels.

See coverage_on_conf_matrix() for definition.

xcolumns.metrics.binary_0_1_loss(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True) → int | float | number

Calculates 0/1 loss matric from the given true and predicted labels.

See binary_0_1_loss_on_conf_matrix() for definition.

xcolumns.metrics.binary_0_1_loss_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, normalize: bool = True) → int | float | number | ndarray

Calculates binary 0/1 from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{accuracy} = \frac{FP + FN}{TP + FP + FN + TN}\]

If normalize is False, returns the sum of false positives and false negatives.

xcolumns.metrics.binary_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True) → int | float | number

Calculates accuracy matric from the given true and predicted labels.

See binary_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.binary_accuracy_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, normalize: bool = True) → int | float | number | ndarray

Calculates binary accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{accuracy} = \frac{TP + TN}{TP + FP + FN + TN}\]

If normalize is False, returns the sum of true positives and true negatives.

xcolumns.metrics.binary_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates binary balanced accuracy matric from the given true and predicted labels.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.binary_balanced_accuracy_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates ballanced accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Balanced accuracy is the average of true positive rate and true negative rate.

\[\text{balanced accuracy} = \frac{TP}{TP + FN + \epsilon} + \frac{TN}{TN + FP + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates binary F1 score matric from the given true and predicted labels.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.binary_f1_score_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates binary F1 score, also known as balanced F-score or F-measure from the given true positives, false positives, false negatives and true negatives. This is an alias for binary_fbeta_score_on_conf_matrix() with beta=1.0.

xcolumns.metrics.binary_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates binary F-beta score matric from the given true and predicted labels.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.binary_fbeta_score_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number | ndarray

Compute the binary F-beta score. from the given true positives, false positives, false negatives and true negatives.

The F-beta score is the weighted harmonic mean of precision and recall.

\[F_\beta = (1 + \beta^2) \cdot \frac{TP}{\beta^2 \cdot (TP + FP) + TP + FN + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates binary G-mean matric from the given true and predicted labels.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.binary_gmean_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates G-mean (geometric mean) from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

G-mean is the square root of the product of true positive rate and true negative rate.

\[\text{G-mean} = \sqrt{\frac{TP}{TP + FN + \epsilon} \times \text{TNR} = \frac{TN}{TN + FP + \epsilon}}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates binary H-mean matric from the given true and predicted labels.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.binary_hmean_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates H-mean (harmonic mean) from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

H-mean is the harmonic mean of true positive rate and true negative rate.

\[\text{H-mean} = \frac{2 \times \text{TPR} \times \text{TNR}}{\text{TPR} + \text{TNR}}\]

where \(\text{TPR} = \frac{TP}{TP + FN + \epsilon}\) and \(\text{TNR} = \frac{TN}{TN + FP + \epsilon}\) and \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_jaccard_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates binary Jaccard score matric from the given true and predicted labels.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.binary_jaccard_score_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates Jaccard score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Jaccard score is the intersection over union of predicted and true labels.

\[J = \frac{TP}{TP + FP + FN + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates binary precision matric from the given true and predicted labels.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.binary_precision_at_k_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray | None, fn: int | float | number | ndarray | None, tn: int | float | number | ndarray | None, k: int) → int | float | number | ndarray

Calculates binary precision at k based on the given entries of normalized confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{precision@k} = \frac{TP}{k}\]

where \(k\) is the number of positive labels per instance.

xcolumns.metrics.binary_precision_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray, fn: int | float | number | ndarray | None, tn: int | float | number | ndarray | None, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates binary precision from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Precision is the fraction of predicted positives that are actually positive.

\[\text{precision} = \frac{TP}{TP + FP + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates binary recall matric from the given true and predicted labels.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.binary_recall_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray | None, fn: int | float | number | ndarray, tn: int | float | number | ndarray | None, epsilon: float = 1e-09) → int | float | number | ndarray

Calculates binary recall from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Recall is the fraction of positives that were correctly predicted.

\[\text{recall} = \frac{TP}{TP + FN + \epsilon}\]

where \(\epsilon\) is a very small number to avoid division by zero.

xcolumns.metrics.binary_weighted_precision_at_k_on_conf_matrix(tp: int | float | number | ndarray, fp: int | float | number | ndarray | None, fn: int | float | number | ndarray | None, tn: int | float | number | ndarray | None, k: int, w: int | float | number | ndarray) → int | float | number | ndarray

Calculates binary weighted precision at k based on the given entries of normalized confusion matrix: true positives, false positives, false negatives, and true negatives.

\[\text{weighted-precision@k} = \frac{w \odot TP}{k}\]

where \(k\) is the number of positive labels per instance.

xcolumns.metrics.check_if_y_pred_at_k(y_pred: ndarray | csr_matrix, k: int) → bool

Checks if y_pred is a binary matrix with exactly k positive labels per instance.

Parameters:

• y_pred – The binary labels matrix.

• k – Number of positive labels required per instance.

Returns:

True if y_pred is a binary matrix with exactly k positive labels per instance.

xcolumns.metrics.coverage(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix) → int | float | number

Calculates coverage matric from the given true and predicted labels.

See coverage_on_conf_matrix() for definition.

xcolumns.metrics.coverage_on_conf_matrix(tp: ndarray, fp: ndarray | None, fn: ndarray | None, tn: ndarray | None) → int | float | number

Calculates coverage from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Coverage is the fraction of instances that have at least one positive label.

\[\text{coverage} = \sum_{i=1}^m \frac{TP_i > 0}{m}\]

where \(m\) is the number of labels.

xcolumns.metrics.hamming_loss(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True) → int | float | number

Calculates Hamming loss matric from the given true and predicted labels.

See hamming_loss_on_conf_matrix() for definition.

xcolumns.metrics.hamming_loss_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, normalize: bool = True) → int | float | number

Calculates Hamming loss from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Hamming loss is the fraction of labels that are incorrectly predicted. If normalize is False, returns the mean of false positives and false negatives across all labels.

xcolumns.metrics.hamming_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, normalize: bool = True) → int | float | number

Calculates Hamming score matric from the given true and predicted labels.

See hamming_score_on_conf_matrix() for definition.

xcolumns.metrics.hamming_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, normalize: bool = True) → int | float | number

Calculates Hamming score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Hamming score is the fraction of correctly predicted labels. If normalize is False, returns the mean of true positives and true negatives across all labels.

xcolumns.metrics.instance_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged balanced accuracy metric from the given true and predicted labels.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.instance_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged F1 score metric from the given true and predicted labels.

See binary_f1_score_on_conf_matrix() for definition.

xcolumns.metrics.instance_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged F-beta score metric from the given true and predicted labels.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.instance_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged G-mean metric from the given true and predicted labels.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.instance_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged H-mean metric from the given true and predicted labels.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.instance_jaccard_score_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged Jaccard score metric from the given true and predicted labels.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.instance_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged precision metric from the given true and predicted labels.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.instance_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → ndarray

Calculates instance-averaged recall metric from the given true and predicted labels.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.instance_tail_metric(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, metric_on_conf_matrix_func: Callable, k: int, w: ndarray, percentile: float = 0.5, epsilon: float = 1e-09)

xcolumns.metrics.instance_tail_recall_at_k(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, k: int, w: ndarray, percentile: float = 0.5, epsilon: float = 1e-09)

xcolumns.metrics.inverse_label_priors(y: ndarray | csr_matrix) → ndarray

Calculates inverse of prior probablity of each label.

Parameters:

y – The binary labels matrix.

Returns:

An vector the inverse label priors.

xcolumns.metrics.jpv_inverse_propensities(y: ndarray | csr_matrix, a: float = 0.55, b: float = 1.5) → ndarray

Calculate inverse propensities of labels an empirical model proposed in Jain et al. 2016. Inverse propensity \(q_j\) of label \(j\) is calculated as:

\[c = (\log n - 1)(b + 1)^a \,, \ q_j = 1 + c(n_j + b)^{-A} \,,\]

where \(n\) is total number of data points, \(n_j\) is total number of data points for and \(a\) and \(b\) are dataset specific parameters.

Parameters:

• y – The binary labels matrix.

• a – A dataset specific parameter, typical values: - 0.5: WikiLSHTC-325K and WikipediaLarge-500K - 0.6: Amazon-670K and Amazon-3M - 0.55: otherwise

• b – Dataset specific parameter, typical values: - 0.4: WikiLSHTC-325K and WikipediaLarge-500K - 2.6: Amazon-670K and Amazon-3M - 1.5: otherwise

Returns:

An vector the inverse label propensities.

xcolumns.metrics.jpv_propensities(y: ndarray | csr_matrix, a: float = 0.55, b: float = 1.5) → ndarray

Calculates propensity of labels using an empirical model proposed in Jain et al. 2016. Propensity \(p_j\) of label \(j\) is calculated as:

\[c = (\log n - 1)(b + 1)^a \,, \ p_j = \frac{1}{1 + c(n_l + b)^{-a}} \,,\]

where \(n\) is total number of data points, \(n_j\) is total number of data points for and \(a\) and \(b\) are dataset specific parameters.

Parameters:

• y – The binary labels matrix.

• a – A dataset specific parameter, typical values: - 0.5: WikiLSHTC-325K and WikipediaLarge-500K - 0.6: Amazon-670K and Amazon-3M - 0.55: otherwise

• b – Dataset specific parameter, typical values: - 0.4: WikiLSHTC-325K and WikipediaLarge-500K - 2.6: Amazon-670K and Amazon-3M - 1.5: otherwise

Returns:

An vector the label propensities.

xcolumns.metrics.label_counts(y: ndarray | csr_matrix) → ndarray

Counts number of occurrences of each label.

Parameters:

y – The binary labels matrix.

Returns:

An vector the count of label occurrences.

xcolumns.metrics.label_priors(y: ndarray | csr_matrix) → ndarray

Calculates prior probablity of each label.

Parameters:

y – The binary labels matrix.

Returns:

An vector the label priors.

xcolumns.metrics.macro_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged balanced accuracy matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_balanced_accuracy_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged balanced accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.macro_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged F1 score matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_f1_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged F1 score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.macro_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged F-beta score matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_fbeta_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged F-beta score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.macro_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged G-mean matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_gmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged G-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.macro_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged H-mean matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_hmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged H-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.macro_jaccard_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged Jaccard score matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_jaccard_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged Jaccard score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.macro_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged precision matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_precision_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged precision from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.macro_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged recall matric from the given true and predicted labels.

See macro_metric_on_conf_matrix() for definition.

xcolumns.metrics.macro_recall_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates macro-averaged recall from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.make_instance_metric_on_y_true_and_y_pred(binary_metric: Callable, metric_name: str, skip_tn: bool = False) → Callable

This is a factory function to create an instanve-wise metric calculated on true and predicted labels: metric(y_true, y_pred, **kwargs) from a binary metric defined on confusion matrix: binary_metric(tp, fp, fn, tn, **kwargs).

Parameters:

• binary_metric – binary metric defined on confusion matrix

• metric_name – name of the metric to be used in the docstring

Returns:

Function calculating the metric on true and predicted labels

xcolumns.metrics.make_tail_instance_metric_on_y_true_and_y_pred(binary_metric: Callable, metric_name: str, skip_tn: bool = False) → Callable

This is a factory function to create a tail instance-wise metric calculated on true and predicted labels: metric(y_true, y_pred, **kwargs) from a binary metric defined on confusion matrix: binary_metric(tp, fp, fn, tn, **kwargs).

Parameters:

• binary_metric – binary metric defined on confusion matrix

• metric_name – name of the metric to be used in the docstring

• skip_tn – whether to skip true negatives

Returns:

Function calculating the tail instance-wise metric on true and predicted labels

xcolumns.metrics.micro_balanced_accuracy(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged balanced accuracy matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_balanced_accuracy_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged balanced accuracy from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_balanced_accuracy_on_conf_matrix() for definition.

xcolumns.metrics.micro_f1_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged F1 score matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_f1_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged F1 score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.micro_fbeta_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged F-beta score matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_fbeta_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, beta: float = 1.0, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged F-beta score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_fbeta_score_on_conf_matrix() for definition.

xcolumns.metrics.micro_gmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged G-mean matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_gmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged G-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_gmean_on_conf_matrix() for definition.

xcolumns.metrics.micro_hmean(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged H-mean matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_hmean_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged H-mean from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_hmean_on_conf_matrix() for definition.

xcolumns.metrics.micro_jaccard_score(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged Jaccard score matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_jaccard_score_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged Jaccard score from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_jaccard_score_on_conf_matrix() for definition.

xcolumns.metrics.micro_precision(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged precision matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_precision_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged precision from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_precision_on_conf_matrix() for definition.

xcolumns.metrics.micro_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged recall matric from the given true and predicted labels.

See micro_metric_on_conf_matrix() for definition.

xcolumns.metrics.micro_recall_on_conf_matrix(tp: ndarray, fp: ndarray, fn: ndarray, tn: ndarray, epsilon: float = 1e-09) → int | float | number

Calculates micro-averaged recall from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.precision_at_k(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix) → int | float | number

Calculates precision at k matric from the given true and predicted labels.

See precision_at_k_on_conf_matrix() for definition.

xcolumns.metrics.precision_at_k_on_conf_matrix(tp: ndarray, fp: ndarray | None, fn: ndarray | None, tn: ndarray | None, k: int) → int | float | number

Calculates precision at k from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

xcolumns.metrics.tail_abandonment(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, priors: ndarray, percentile: float = 0.5) → ndarray

Calculates tail instance-averaged tail abandonment metric from the given true and predicted labels. Labels above provied frequancy percentile are ignored when calculating the metric.

See coverage_on_conf_matrix() for definition.

xcolumns.metrics.tail_recall(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix, priors: ndarray, percentile: float = 0.5, epsilon: float = 1e-09) → ndarray

Calculates tail instance-averaged tail recall metric from the given true and predicted labels. Labels above provied frequancy percentile are ignored when calculating the metric.

See binary_recall_on_conf_matrix() for definition.

xcolumns.metrics.weighted_precision_at_k(y_true: ndarray | csr_matrix, y_pred: ndarray | csr_matrix) → int | float | number

Calculates weighted precision at k matric from the given true and predicted labels.

See weighted_precision_at_k_on_conf_matrix() for definition.

xcolumns.metrics.weighted_precision_at_k_on_conf_matrix(tp: ndarray, fp: ndarray | None, fn: ndarray | None, tn: ndarray | None, k: int, w: ndarray) → int | float | number

Calculates weighted precision at k from the given entries of confusion matrix: true positives, false positives, false negatives, and true negatives.

Factory functions for specific metrics

The module provides the factory functions for creating the metric micro- and macro-averaged functions based on the binary metric function as well as the functions that calculate the metrics on true and predicted labels.

xcolumns.metrics.make_macro_metric_on_conf_matrix(binary_metric: Callable, name: str) → Callable

This is a factory function to create a macro-averaged metric from a binary metric defined on confusion matrix: binary_metric(tp, fp, fn, tn, **kwargs).

Parameters:

• binary_metric – binary metric defined on confusion matrix

• name – name of the metric to be used in the docstring

Returns:

The function calculating the macro-averaged metric defined on a confusion matrix.

xcolumns.metrics.make_micro_metric_on_conf_matrix(binary_metric: Callable, metric_name: str) → Callable

This is a factory function to create a micro-averaged metric from a binary metric defined on confusion matrix: binary_metric(tp, fp, fn, tn, **kwargs).

Parameters:

• binary_metric – binary metric defined on confusion matrix

• name – name of the metric to be used in the docstring

Returns:

The function calculating the micro-averaged metric defined on a confusion matrix.

xcolumns.metrics.make_metric_on_y_true_and_y_pred(metric_on_conf_matrix: Callable, metric_name: str, skip_tn: bool = False) → Callable

This is a factory function to create a metric calculated on true and predicted labels: metric(y_true, y_pred, **kwargs) from metric calculated on confusion matrix: metric(tp, fp, fn, tn, **kwargs).

Parameters:

• metric_on_conf_matrix – metric calculated on confusion matrix

• metric_name – name of the metric to be used in the docstring

• skip_tn – whether to skip true negatives

Returns:

Function etric calculated on true and predicted labels