Medical Text Simplification

Course report (HTML)

We fine-tuned T5 on MultiCochrane (EN) for medical simplification. We added a general-domain step first (WikiLarge), then medical data, with our best sequential T5-small run hitting SARI 44.78, above the ~39 to 43 band we compared against from the literature.

We built a Hugging Face training pipeline around T5 (small and large in experiments) on English MultiCochrane sentence pairs. We ran Optuna sweeps over LR, schedulers, warmup, weight decay, and label smoothing, and we scored outputs with SARI (primary for simplification) plus BLEU/ROUGE and Jaccard overlap to monitor copying (“parroting”) versus real rewriting.

We compared training only on medical data with sequential fine-tuning: WikiLarge first, then MultiCochrane. That general-then-medical schedule was our way to learn broad simplification before specializing on jargon-heavy Cochrane-style sentences. In the report we relate our numbers to multilingual medical-simplification work that reported SARI roughly in the high 30s to low 40s. With that sequential strategy, our strongest T5-small setup topped out at SARI 44.78 (first-step WikiLarge → medical), ahead of a large-model sweep that did not use the same pipeline, so the step order mattered more than only scaling hyperparameters on medical data alone.

We also tried an anti-parrot objective (SARI combined with similarity penalties). It hurt scores in our runs, so we kept SARI as the main selection metric and discussed decoding (temperature vs. beam) qualitatively in the report. We published the training code and experiment scripts on GitHub so the setup stays reproducible.