`trw.train.metrics`¶

Module Contents¶

Classes¶

`Metric`	A metric base class
`MetricLoss`	Extract the loss from the outputs
`MetricClassificationBinaryAUC`	Calculate the Area under the Receiver operating characteristic (ROC) curve.
`MetricClassificationError`	Calculate the `1 - accuracy` using the output_truth and output
`MetricSegmentationDice`	Calculate the average dice score of a segmentation map 'output_truth' and class
`MetricClassificationF1`	A metric base class
`MetricClassificationBinarySensitivitySpecificity`	Calculate the sensitivity and specificity for a binary classification using the output_truth and output

Functions¶

`fast_confusion_matrix`(y: torch.Tensor, y_pred: torch.Tensor, num_classes: int, ignore_y_out_of_range: bool = False, device=torch.device('cpu')) → torch.Tensor	Compute confusion matrix to evaluate the accuracy of a classification.
`default_classification_metrics`()	"
`default_regression_metrics`()	"
`default_segmentation_metrics`()	"
`default_generic_metrics`()	"

trw.train.metrics.fast_confusion_matrix(y: torch.Tensor, y_pred: torch.Tensor, num_classes: int, ignore_y_out_of_range: bool = False, device=torch.device('cpu')) → torch.Tensor¶

Compute confusion matrix to evaluate the accuracy of a classification.

By definition a confusion matrix \(C\) is such that \(C_{i, j}\) is equal to the number of observations known to be in group \(i\) and predicted to be in group \(j\).

Thus in binary classification, the count of true negatives is \(C_{0,0}\), false negatives is \(C_{1,0}\), true positives is \(C_{1,1}\) and false positives is \(C_{0,1}\).

Similar to sklearn.metrics.confusion_matrix()

Parameters

y_pred – prediction (tensor of integers)
y – tensor of integers
num_classes – the number of classes
ignore_y_out_of_range – if True, indices of y greater than num_classes will be ignored
device – device where to perform the calculation

class trw.train.metrics.Metric¶

Bases: abc.ABC

A metric base class

Calculate interesting metric

abstract __call__(self, outputs: Dict) → Optional[Dict]¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

abstract aggregate_metrics(self, metric_by_batch: List[Dict]) → Dict[str, float]¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

class trw.train.metrics.MetricLoss¶

Bases: Metric

Extract the loss from the outputs

__call__(self, outputs)¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

aggregate_metrics(self, metric_by_batch)¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

class trw.train.metrics.MetricClassificationBinaryAUC¶

Bases: Metric

Calculate the Area under the Receiver operating characteristic (ROC) curve.

For this, the output needs to provide an output_raw of shape [N, 2] (i.e., binary classification framed as a multi-class classification) or of shape [N, 1] (binary classification)

__call__(self, outputs)¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

aggregate_metrics(self, metric_by_batch)¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

class trw.train.metrics.MetricClassificationError¶

Bases: Metric

Calculate the 1 - accuracy using the output_truth and output

__call__(self, outputs)¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

aggregate_metrics(self, metric_by_batch)¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

class trw.train.metrics.MetricSegmentationDice(dice_fn=losses.LossDiceMulticlass(normalization_fn=None, return_dice_by_class=True, smooth=0), aggregate_by: Optional[str] = None)¶

Bases: Metric

Calculate the average dice score of a segmentation map ‘output_truth’ and class segmentation logits ‘output_raw’.

Notes

by default, nn.Sigmoid function will be applied on the output to force a range [0..1] of the output
the aggregation will aggregate all the foregrounds/backgrounds THEN calculate the dice (but NOT average
of dices). Using this aggregation, it is possible to calculate the true dice on a partitioned input (e.g., 3D segmentations, we often use sub-volumes)

__call__(self, outputs)¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

static _aggregate_dices(metric_by_batch)¶

static _aggregate_dices_by_uid(metric_by_batch)¶

aggregate_metrics(self, metric_by_batch)¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

class trw.train.metrics.MetricClassificationF1(average=None)¶

Bases: Metric

A metric base class

Calculate interesting metric

__call__(self, outputs)¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

aggregate_metrics(self, metric_by_batch)¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

class trw.train.metrics.MetricClassificationBinarySensitivitySpecificity¶

Bases: Metric

Calculate the sensitivity and specificity for a binary classification using the output_truth and output

__call__(self, outputs)¶

Parameters: outputs – the outputs of a batch
Returns: a dictionary of metric names/values or None

aggregate_metrics(self, metric_by_batch)¶

Aggregate all the metrics into a consolidated metric.

Parameters: metric_by_batch – a list of metrics, one for each batch
Returns: a dictionary of result name and value

trw.train.metrics.default_classification_metrics()¶: ” Default list of metrics used for classification

trw.train.metrics.default_regression_metrics()¶: ” Default list of metrics used for regression

trw.train.metrics.default_segmentation_metrics()¶: ” Default list of metrics used for segmentation

trw.train.metrics.default_generic_metrics()¶: ” Default list of metrics

trw.train.metrics¶

Module Contents¶

Classes¶

Functions¶

`trw.train.metrics`¶