Summarization

Models

Model	Lang
mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization	EN
nandakishormpai/t5-small-machine-articles-tag-generation	EN
mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization	EN
UrukHan/t5-russian-summarization	RU
dmitry-vorobiev/rubert_ria_headlines	RU

Datasets

1. ccdv/govreport-summarization

   1. Lang: EN

   2. Rows: 973

   3. Preprocess:

      1. Select test split.

      2. Rename column report to source.

      3. Rename column summary to target.

      4. Reset indexes.

2. cnn_dailymail

   1. Lang: EN

   2. Rows: 11490

   3. Preprocess:

      1. Select 1.0.0 subset.

      2. Select test split.

      3. Drop columns id.

      4. Rename column article to source.

      5. Rename column highlights to target.

      6. Delete duplicates in dataset.

      7. Remove substring (CNN) for each source row.

      8. Reset indexes.

3. ccdv/pubmed-summarization

   1. Lang: EN

   2. Rows: 6658

   3. Preprocess:

      1. Select test split.

      2. Rename column article to source.

      3. Rename column abstract to target.

      4. Reset indexes.

4. IlyaGusev/gazeta

   1. Lang: RU

   2. Rows: 6793

   3. Preprocess:

      1. Select test split.

      2. Drop columns title, date, url.

      3. Rename column text to source.

      4. Rename column summary to target.

      5. Reset indexes.

5. d0rj/curation-corpus-ru

   1. Lang: RU

   2. Rows: 30454

   3. Preprocess:

      1. Select train split.

      2. Drop columns title, date, url.

      3. Rename column article_content to source.

      4. Rename column summary to target.

      5. Reset indexes.

6. CarlBrendt/Summ_Dialog_News

   1. Lang: RU

   2. Rows: 7609

   3. Preprocess:

      1. Select test split.

      2. Rename column info to source.

      3. Rename column summary to target.

      4. Reset indexes.

7. trixdade/reviews_russian

   1. Lang: RU

   2. Rows: 95

   3. Preprocess:

      1. Select train split.

      2. Rename column Reviews to source.

      3. Rename column Summary to target.

      4. Reset indexes.

Supervised Fine-Tuning (SFT) Parameters

Note

Set the parameter target_modules as ["query", "key", "value", "dense"] for the mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization, mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization and dmitry-vorobiev/rubert_ria_headlines models, and as ["q", "k", "wi", "wo"] for the UrukHan/t5-russian-summarization model.

Note

Set the parameters fine_tuning_steps=150, rank=24, alpha=36 for the mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization as SFT parameters. In pair with cnn_dailymail dataset, set fine_tuning_steps=150, rank=24, alpha=48, learning_rate=1e-5

Metrics

• BLEU

• ROUGE

Note

Use the rougeL metric and set seed=77 parameter when loading the rouge metric.