Medical Text Simplification

Repository on GitHub·Course report (HTML)

Medical T5 simplification

We fine-tuned T5 on MultiCochrane (EN) for medical simplification. We added a general-domain step first (WikiLarge), then medical data, with our best sequential T5-small run hitting SARI 44.78, above the ~39 to 43 band we compared against from the literature.

We built a Hugging Face training pipeline aroundT5(small and large in experiments) on English MultiCochrane sentence pairs. We ranOptunasweeps over LR, schedulers, warmup, weight decay, and label smoothing, and we scored outputs withSARI(primary for simplification) plus BLEU/ROUGE andJaccardoverlap to monitor copying ("parroting") versus real rewriting.

We compared training only on medical data withsequential fine-tuning: WikiLarge first, then MultiCochrane. That general-then-medical schedule was our way to learn broad simplification before specialising on jargon-heavy Cochrane-style sentences. With that sequential strategy,our strongest T5-small setup topped out at SARI 44.78(first-step WikiLarge → medical), ahead of a large-model sweep that did not use the same pipeline, so the step order mattered more than only scaling hyperparameters on medical data alone.

We also tried an anti-parrot objective (SARI combined with similarity penalties). It hurt scores in our runs, so we kept SARI as the main selection metric and discussed decoding (temperature vs. beam) qualitatively in the report.

Keywords & references

Short pointers tied to the tech stack; full bibliography is inthe HTML report.

T5: Raffel et al. (2020)— unified text-to-text transfer transformer, the encoder-decoder backbone we use.
Hugging Face Transformers: Model API and training loopwe used for T5 checkpoints.
MultiCochrane: Joseph et al. (2023)— multilingual medical simplification; we trained on English pairs from the public split.
WikiLarge: Zhang & Lapata (2017)— general-domain parallel simplification for the first-stage fine-tune.
SARI: Simplification metric(add/keep/delete).
Optuna: Optuna— hyperparameter search over LR, schedulers, warmup, weight decay, label smoothing.

Stack

Python, PyTorch, Hugging Face Transformers, T5, Optuna, MultiCochrane, WikiLarge, SARI.