mbrs.metrics.cometkiwi module

mbrs.metrics.cometkiwi module#

class mbrs.metrics.cometkiwi.MetricCOMETkiwi(cfg: Config)[source]#

Bases: MetricReferenceless

COMETkiwi metric class.

class Config(model: str = 'Unbabel/wmt22-cometkiwi-da', batch_size: int = 64, fp16: bool = False, bf16: bool = False, cpu: bool = False)[source]#

Bases: Config

COMETkiwi metric configuration.

  • model (str): Model name or path.

  • batch_size (int): Batch size.

  • fp16 (bool): Use float16 for the forward computation.

  • bf16 (bool): Use bfloat16 for the forward computation.

  • cpu (bool): Use CPU for the forward computation.

batch_size: int = 64#
bf16: bool = False#
cpu: bool = False#
fp16: bool = False#
model: str = 'Unbabel/wmt22-cometkiwi-da'#
corpus_score(hypotheses: list[str], sources: list[str]) → float[source]#

Calculate the corpus-level score.

Parameters:

  • hypotheses (list[str]) – Hypotheses.

  • source (list[str]) – Sources.

Returns:

The corpus score.

Return type:

float

property device: device#

Returns the device of the model.

score(hypothesis: str, source: str) → float[source]#

Calculate the score of the given hypothesis.

Parameters:

  • hypothesis (str) – A hypothesis.

  • source (str) – A source.

Returns:

The score of the given hypothesis.

Return type:

float

scores(hypotheses: list[str], sources: list[str]) → Tensor[source]#

Calculate the scores of hypotheses.

Parameters:

  • hypotheses (list[str]) – N hypotheses.

  • source (list[str]) – N sources.

Returns:

N scores of the given hypotheses.

Return type:

torch.Tensor

Contents