From 9d2dbb838845cdf15663fb611f5d8f477b469774 Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Mon, 20 Oct 2025 14:47:55 +0200 Subject: [PATCH 01/10] updating model based reading orde detection --- docs/models.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/models.md b/docs/models.md index 3d296d5..40b23ae 100644 --- a/docs/models.md +++ b/docs/models.md @@ -151,7 +151,7 @@ This model is used for the task of illustration detection only. Model card: [Reading Order Detection]() -TODO +The model extracts the reading order of text regions from the layout by classifying pairwise relationships between them. A sorting algorithm then determines the overall reading sequence. ## Heuristic methods From 3ec5ceb22e317fbe5234f625412898232277ab68 Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Mon, 20 Oct 2025 14:55:14 +0200 Subject: [PATCH 02/10] Update flowchart --- docs/models.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/models.md b/docs/models.md index 40b23ae..50ef726 100644 --- a/docs/models.md +++ b/docs/models.md @@ -18,7 +18,8 @@ Two Arabic/Persian terms form the name of the model suite: عين الله, whic See the flowchart below for the different stages and how they interact: -![](https://user-images.githubusercontent.com/952378/100619946-1936f680-331e-11eb-9297-6e8b4cab3c16.png) +eynollah_flowchart + ## Models From c8455370a9dfde698ee91125d3400d8a313ede5a Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Mon, 20 Oct 2025 15:13:45 +0200 Subject: [PATCH 03/10] updating heuristics and ocr documentation --- docs/models.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/models.md b/docs/models.md index 50ef726..7f83b33 100644 --- a/docs/models.md +++ b/docs/models.md @@ -154,13 +154,17 @@ Model card: [Reading Order Detection]() The model extracts the reading order of text regions from the layout by classifying pairwise relationships between them. A sorting algorithm then determines the overall reading sequence. +### OCR + +We have trained three OCR models: two CNN-RNN–based models and one transformer-based TrOCR model. The CNN-RNN models are generally faster and provide better results in most cases, though their performance decreases with heavily degraded images. The TrOCR model, on the other hand, is computationally expensive and slower during inference, but it can possibly produce better results on strongly degraded images. ## Heuristic methods Additionally, some heuristic methods are employed to further improve the model predictions: * After border detection, the largest contour is determined by a bounding box, and the image cropped to these coordinates. -* For text region detection, the image is scaled up to make it easier for the model to detect background space between text regions. +* Unlike the non-light version, where the image is scaled up to help the model better detect the background spaces between text regions, the light version uses down-scaled images. In this case, introducing an artificial class along the boundaries of text regions and text lines has helped to isolate and separate the text regions more effectively. * A minimum area is defined for text regions in relation to the overall image dimensions, so that very small regions that are noise can be filtered out. -* Deskewing is applied on the text region level (due to regions having different degrees of skew) in order to improve the textline segmentation result. -* After deskewing, a calculation of the pixel distribution on the X-axis allows the separation of textlines (foreground) and background pixels. -* Finally, using the derived coordinates, bounding boxes are determined for each textline. +* In the non-light version, deskewing is applied at the text-region level (since regions may have different degrees of skew) to improve text-line segmentation results. In contrast, the light version performs deskewing only at the page level to enhance margin detection and heuristic reading-order estimation. +* After deskewing, a calculation of the pixel distribution on the X-axis allows the separation of textlines (foreground) and background pixels (only in non-light version). +* Finally, using the derived coordinates, bounding boxes are determined for each textline (only in non-light version). +* As mentioned above, the reading order can be determined using a model; however, this approach is computationally expensive, time-consuming, and less accurate due to the limited amount of ground-truth data available for training. Therefore, our tool uses a heuristic reading-order detection method as the default. The heuristic approach relies on headers and separators to determine the reading order of text regions. From 2fc723d292093cdfb263e2d6681e478d7018b953 Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Wed, 22 Oct 2025 18:29:14 +0200 Subject: [PATCH 04/10] extend README --- README.md | 66 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 46 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 3ba5086..c6ba0e4 100644 --- a/README.md +++ b/README.md @@ -91,24 +91,35 @@ eynollah layout \ The following options can be used to further configure the processing: -| option | description | -|-------------------|:-------------------------------------------------------------------------------| -| `-fl` | full layout analysis including all steps and segmentation classes | -| `-light` | lighter and faster but simpler method for main region detection and deskewing | -| `-tll` | this indicates the light textline and should be passed with light version | -| `-tab` | apply table detection | -| `-ae` | apply enhancement (the resulting image is saved to the output directory) | -| `-as` | apply scaling | -| `-cl` | apply contour detection for curved text lines instead of bounding boxes | -| `-ib` | apply binarization (the resulting image is saved to the output directory) | -| `-ep` | enable plotting (MUST always be used with `-sl`, `-sd`, `-sa`, `-si` or `-ae`) | -| `-eoi` | extract only images to output directory (other processing will not be done) | -| `-ho` | ignore headers for reading order dectection | -| `-si ` | save image regions detected to this directory | -| `-sd ` | save deskewed image to this directory | -| `-sl ` | save layout prediction as plot to this directory | -| `-sp ` | save cropped page image to this directory | -| `-sa ` | save all (plot, enhanced/binary image, layout) to this directory | +| option | description | +|-------------------|:------------------------------------------------------------------------------- | +| `-fl` | full layout analysis including all steps and segmentation classes (recommended) | +| `-light` | lighter and faster but simpler method for main region detection and deskewing (recommended) | +| `-tll` | this indicates the light textline and should be passed with light version (recommended) | +| `-tab` | apply table detection | +| `-ae` | apply enhancement (the resulting image is saved to the output directory) | +| `-as` | apply scaling | +| `-cl` | apply contour detection for curved text lines instead of bounding boxes | +| `-ib` | apply binarization (the resulting image is saved to the output directory) | +| `-ep` | enable plotting (MUST always be used with `-sl`, `-sd`, `-sa`, `-si` or `-ae`) | +| `-eoi` | extract only images to output directory (other processing will not be done) | +| `-ho` | ignore headers for reading order dectection | +| `-si ` | save image regions detected to this directory | +| `-sd ` | save deskewed image to this directory | +| `-sl ` | save layout prediction as plot to this directory | +| `-sp ` | save cropped page image to this directory | +| `-sa ` | save all (plot, enhanced/binary image, layout) to this directory | +| `-thart` | threshold of artifical class in the case of textline detection. The default value is 0.1 | +| `-tharl` | threshold of artifical class in the case of layout detection. The default value is 0.1 | +| `-ocr` | do ocr | +| `-tr` | apply transformer ocr. Default model is a CNN-RNN model | +| `-bs_ocr` | ocr inference batch size. Default bs for trocr and cnn_rnn models are 2 and 8 respectively | +| `-ncu` | upper limit of columns in document image | +| `-ncl` | lower limit of columns in document image | +| `-slro` | skip layout detection and reading order | +| `-romb` | apply machine based reading order detection | +| `-ipe` | ignore page extraction | + If no further option is set, the tool performs layout detection of main regions (background, text, images, separators and marginals). @@ -124,7 +135,7 @@ The command-line interface for binarization can be called like this: eynollah binarization \ -i | -di \ -o \ - -m \ + -m ``` ### OCR @@ -138,9 +149,24 @@ eynollah ocr \ -i | -di \ -dx \ -o \ - -m | --model_name \ + -m | --model_name ``` +The following options can be used to further configure the ocr processing: + +| option | description | +|-------------------|:------------------------------------------------------------------------------- | +| `-dib` | directory of bins(files type must be '.png'). Prediction with both RGB and bins. | +| `-doit` | Directory containing output images rendered with the predicted text | +| `--model_name` | Specific model file path to use for OCR | +| `-trocr` | transformer ocr will be applied, otherwise cnn_rnn model | +| `-etit` | textlines images and text in xml will be exported into output dir (OCR training data) | +| `-nmtc` | cropped textline images will not be masked with textline contour | +| `-bs` | ocr inference batch size. Default bs for trocr and cnn_rnn models are 2 and 8 respectively | +| `-ds_pref` | add an abbrevation of dataset name to generated training data | +| `-min_conf` | minimum OCR confidence value. OCRs with textline conf lower than this will be ignored | + + ### Machine-based-reading-order The machine-based reading-order module employs a pretrained model to identify the reading order from layouts represented in PAGE-XML files. From ab9ddd5214f4161038a48193df94b4cd363729f8 Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Wed, 22 Oct 2025 18:41:15 +0200 Subject: [PATCH 05/10] OCR examples are added to README --- README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/README.md b/README.md index c6ba0e4..405cab4 100644 --- a/README.md +++ b/README.md @@ -140,6 +140,16 @@ eynollah binarization \ ### OCR +

+ Input Image + Output Image +

+ +

+ Input Image + Output Image +

+ The OCR module performs text recognition using either a CNN-RNN model or a Transformer model. The command-line interface for OCR can be called like this: From 59eb4fd3bee8199155998cffc75b47931dc8bb33 Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Wed, 22 Oct 2025 19:04:01 +0200 Subject: [PATCH 06/10] images with ro are added to readme --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index 405cab4..e8a2721 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,11 @@ ![](https://user-images.githubusercontent.com/952378/102350683-8a74db80-3fa5-11eb-8c7e-f743f7d6eae2.jpg) +

+ Input Image + Output Image +

+ ## Features * Support for 10 distinct segmentation classes: * background, [page border](https://ocr-d.de/en/gt-guidelines/trans/lyRand.html), [text region](https://ocr-d.de/en/gt-guidelines/trans/lytextregion.html#textregionen__textregion_), [text line](https://ocr-d.de/en/gt-guidelines/pagexml/pagecontent_xsd_Complex_Type_pc_TextLineType.html), [header](https://ocr-d.de/en/gt-guidelines/trans/lyUeberschrift.html), [image](https://ocr-d.de/en/gt-guidelines/trans/lyBildbereiche.html), [separator](https://ocr-d.de/en/gt-guidelines/trans/lySeparatoren.html), [marginalia](https://ocr-d.de/en/gt-guidelines/trans/lyMarginalie.html), [initial](https://ocr-d.de/en/gt-guidelines/trans/lyInitiale.html), [table](https://ocr-d.de/en/gt-guidelines/trans/lyTabellen.html) From b56bb4428444aa67d43d759f319704393214921e Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Wed, 22 Oct 2025 21:30:06 +0200 Subject: [PATCH 07/10] providing ocr model evaluation metrics --- docs/models.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/models.md b/docs/models.md index 7f83b33..a42cdb2 100644 --- a/docs/models.md +++ b/docs/models.md @@ -157,6 +157,38 @@ The model extracts the reading order of text regions from the layout by classify ### OCR We have trained three OCR models: two CNN-RNN–based models and one transformer-based TrOCR model. The CNN-RNN models are generally faster and provide better results in most cases, though their performance decreases with heavily degraded images. The TrOCR model, on the other hand, is computationally expensive and slower during inference, but it can possibly produce better results on strongly degraded images. + +#### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250805 + +This model is trained on data where most of the samples are in Fraktur german script. + +| Dataset | Input | CER | WER | +|-----------------------|:-------|:-----------|:----------| +| OCR-D-GT-Archiveform | BIN | 0.02147 | 0.05685 | +| OCR-D-GT-Archiveform | RGB | | | + +#### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250904 (Default) + +Compared to the model_eynollah_ocr_cnnrnn_20250805 model, this model is trained on a larger proportion of Antiqua data and achieves superior performance. + +| Dataset | Input | CER | WER | +|-----------------------|:------------|:-----------|:----------| +| OCR-D-GT-Archiveform | BIN | 0.01635 | 0.05410 | +| OCR-D-GT-Archiveform | RGB | 0.01471 | 0.05813 | +| BLN600 | RGB | 0.04409 | 0.08879 | +| BLN600 | Enhanced | 0.03599 | 0.06244 | + + +#### Transformer OCR model: model_eynollah_ocr_trocr_20250919 + +This transformer OCR model is trained on the same data as model_eynollah_ocr_trocr_20250919. + +| Dataset | Input | CER | WER | +|-----------------------|:------------|:-----------|:----------| +| OCR-D-GT-Archiveform | BIN | 0.01841 | 0.05589 | +| OCR-D-GT-Archiveform | RGB | | | +| BLN600 | RGB | 0.06347 | 0.13853 | + ## Heuristic methods Additionally, some heuristic methods are employed to further improve the model predictions: From 7b7714af2e3a40d18448a5dda6e7f624016c9eac Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Wed, 22 Oct 2025 22:42:37 +0200 Subject: [PATCH 08/10] completing ocr evaluations metric --- docs/models.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/models.md b/docs/models.md index a42cdb2..7482043 100644 --- a/docs/models.md +++ b/docs/models.md @@ -165,7 +165,7 @@ This model is trained on data where most of the samples are in Fraktur german sc | Dataset | Input | CER | WER | |-----------------------|:-------|:-----------|:----------| | OCR-D-GT-Archiveform | BIN | 0.02147 | 0.05685 | -| OCR-D-GT-Archiveform | RGB | | | +| OCR-D-GT-Archiveform | RGB | 0.01636 | 0.06285 | #### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250904 (Default) @@ -186,7 +186,7 @@ This transformer OCR model is trained on the same data as model_eynollah_ocr_tro | Dataset | Input | CER | WER | |-----------------------|:------------|:-----------|:----------| | OCR-D-GT-Archiveform | BIN | 0.01841 | 0.05589 | -| OCR-D-GT-Archiveform | RGB | | | +| OCR-D-GT-Archiveform | RGB | 0.01552 | 0.06177 | | BLN600 | RGB | 0.06347 | 0.13853 | ## Heuristic methods From d0ad7a98b723ba494eee107e8fef388c444768bf Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Wed, 22 Oct 2025 22:45:22 +0200 Subject: [PATCH 09/10] starting qualitative ocr evaluation --- docs/models.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/models.md b/docs/models.md index 7482043..741fc67 100644 --- a/docs/models.md +++ b/docs/models.md @@ -189,6 +189,16 @@ This transformer OCR model is trained on the same data as model_eynollah_ocr_tro | OCR-D-GT-Archiveform | RGB | 0.01552 | 0.06177 | | BLN600 | RGB | 0.06347 | 0.13853 | +##### Qualitative evaluation of the models + +###### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250805 + + +###### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250904 (Default) + + +###### Transformer OCR model: model_eynollah_ocr_trocr_20250919 + ## Heuristic methods Additionally, some heuristic methods are employed to further improve the model predictions: From 6192e5ba5c95f3b8b3ad21f2e23aed0fbdededad Mon Sep 17 00:00:00 2001 From: vahidrezanezhad Date: Thu, 23 Oct 2025 16:37:24 +0200 Subject: [PATCH 10/10] qualitative evaluation of ocr models are added to docs --- docs/models.md | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/docs/models.md b/docs/models.md index 741fc67..b858630 100644 --- a/docs/models.md +++ b/docs/models.md @@ -191,13 +191,27 @@ This transformer OCR model is trained on the same data as model_eynollah_ocr_tro ##### Qualitative evaluation of the models -###### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250805 +| | | | | +|:---:|:---:|:---:|:---:| +| Image | cnnrnn_20250805 | cnnrnn_20250904 | trocr_20250919 | -###### CNN-RNN model: model_eynollah_ocr_cnnrnn_20250904 (Default) + +| | | | | +|:---:|:---:|:---:|:---:| +| Image | cnnrnn_20250805 | cnnrnn_20250904 | trocr_20250919 | + + +| | | | | +|:---:|:---:|:---:|:---:| +| Image | cnnrnn_20250805 | cnnrnn_20250904 | trocr_20250919 | + + +| | | | | +|:---:|:---:|:---:|:---:| +| Image | cnnrnn_20250805 | cnnrnn_20250904 | trocr_20250919 | -###### Transformer OCR model: model_eynollah_ocr_trocr_20250919 ## Heuristic methods