• NAME
• DESCRIPTION
• METHODS

NAME

MEAD::Evaluation

DESCRIPTION

The DUC::Evaluation is the main class for MEADeval, an evaluation tool for summarizers, both abstractive and extractive.

TODO: Change this class's name from DUC::Evaluation to MEAD::Evaluation.

METHODS

$p = precision($extract, $standard);

p = match ($extract, $standard) / num ($extract)

$r = recall($extract, $standard);

r = match ($extract, $standard) / num ($standard)

$pw = precisionw($extract, $standard)

$rw = recallw($extract, $standard)

$k = kappa($num_sentences, @extracts);

k = ( P(A) - P(E) ) / ( 1 - P(E) )

P(A) is the precision between the extract and the standard. P(E) is the percent of the time that the exract and the standard would be expected to agree (randomly?)

Also need to get # sentences to compute P(E). That's why we take $cluster.

$ru = relative_utility($extract, $sentjudge);

$nru = normalized_relative_utility($extract, $sentjudge)

relevance_correlation

Haven't implemented this yet...

$uo = unigram_overlap($text1, $text2);

$bg = bigram_overlap($text1, $text2);

$c = simple_cosine($text1, $text2);

$c = cosine($text1, $text2, $idf_filename);