Skip to content

About

No description, website, or topics provided.

Resources

Stars

8 stars

Watchers

0 watching

Forks

Latest commit

 

History

38 Commits

Folders and files

Repository files navigation

Multilingual Handwritten Numeral Recognition with Attention-driven Transfer Learning

Code for the paper Advancing Multilingual Handwritten Numeral Recognition with Attention-driven Transfer Learning by Amirreza Fateh, Reza Tahmasbi Birgani, Mansoor Fateh and Vahid Abolghasemi, published in IEEE Access, 2024.

Dataset: the generated DIGITal dataset (100,000 printed digit images, 128 x 128) is available on Zenodo: doi.org/10.5281/zenodo.24181292.

Overview of the method

Method

Handwritten digits from twelve scripts are recognised in three stages (see the figure above):

  1. Restoration. An MRA U-Net, a U-Net with a multi-resolution attention (MRA) module on each skip connection, is trained on printed digits (DIGITal). Its input is a degraded copy of the image (shrunk and enlarged again) and its target is the original. Handwritten images are low-resolution and blurry once they are enlarged to 128 x 128, so the network is then used, frozen, to enhance them.
  2. Language recognition. A CNN with MRA modules, working on the restored image, predicts which of the twelve scripts a digit is written in.
  3. Digit recognition. For each language, the convolutional part of the language classifier is reused and frozen, and a new dense head is trained to classify the digit.

Repository layout

mranet/             models (MRA module, MRA U-Net, classifiers, baselines), data loading, helpers
scripts/            train_restoration, train_language, train_digit, evaluate
digital_dataset/    generator, fonts and documentation of the printed DIGITal dataset
datasets/           the twelve handwritten datasets (see datasets/README.md)
assets/             figures

Installation

The code was written for Python 3.10 or newer with TensorFlow 2.16.1 and Keras 3.1.1.

python -m venv .venv
source .venv/bin/activate        # on Windows: .venv\Scripts\activate
pip install -r requirements.txt

Reproducing the pipeline

All commands are run from the repository root.

1. Generate the printed digits (the DIGITal dataset of the paper: 100,000 images of 128 x 128):

python digital_dataset/generate_paper_dataset.py --out generated

2. Train the restoration U-Net:

python -m scripts.train_restoration --printed generated --out checkpoints/mra_unet.keras

3. Train the language classifier:

python -m scripts.train_language --unet checkpoints/mra_unet.keras --out checkpoints/language.keras

4. Train and evaluate the digit classifier of a language:

python -m scripts.train_digit --language Persian
python -m scripts.evaluate --language Persian --model checkpoints/digit_Persian.keras

Available languages: Arabic, ARDIS, BanglaLekha, Chinese, English, Gujarati, Gurmukhi, ISI_Bangla, Kannada, Persian, Tibetan, Urdu. The training set of each language is augmented with rotations (mranet/data.py, ROTATIONS).

Baselines (trained on the raw images, for comparison):

python -m scripts.train_digit --language Persian --baseline lenet5
python -m scripts.train_digit --language Persian --baseline vgg16

Settings that the original release did not record, such as the degradation size of stage 1, the loss and the learning rates, are command-line options (--help lists them).

Data

Each handwritten dataset keeps the license of its original authors. Please cite them when you use them.

Citation

If you find this work useful, please cite our paper:

@article{fateh2024advancing,
  title   = {Advancing Multilingual Handwritten Numeral Recognition with Attention-driven Transfer Learning},
  author  = {Fateh, Amirreza and Tahmasbi Birgani, Reza and Fateh, Mansoor and Abolghasemi, Vahid},
  journal = {IEEE Access},
  year    = {2024}
}

If you use the DIGITal dataset, please also cite the paper above and the dataset record:

@dataset{fateh2026digital,
  title     = {{DIGITal}: a synthetic multilingual dataset of 100,000 printed digit images},
  author    = {Fateh, Amirreza and Tahmasbi Birgani, Reza and Fateh, Mansoor and Abolghasemi, Vahid},
  year      = {2026},
  version   = {1.0},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.24181292}
}

License

The code is released under the MIT License. The DIGITal dataset is for non-commercial research and educational use; see digital_dataset/README.md for the terms and the note on fonts.

About

No description, website, or topics provided.

Resources

Stars

8 stars

Watchers

0 watching

Forks

Contributors

Languages