📝 align former upstream train.md with wiki train.md syntactically

This commit is contained in:
kba 2025-09-29 15:11:02 +02:00
parent 52a7c93319
commit 6d379782ab
2 changed files with 116 additions and 47 deletions

View file

@ -1,7 +1,10 @@
# Documentation # Documentation
This repository assists users in preparing training datasets, training models, and performing inference with trained models. We cover various use cases including pixel-wise segmentation, image classification, image enhancement, and machine-based reading order. For each use case, we provide guidance on how to generate the corresponding training dataset. This repository assists users in preparing training datasets, training models, and performing inference with trained
All these use cases are now utilized in the Eynollah workflow. models. We cover various use cases including pixel-wise segmentation, image classification, image enhancement, and
machine-based reading order. For each use case, we provide guidance on how to generate the corresponding training
dataset.
All these use cases are now utilized in the Eynollah workflow.
As mentioned, the following three tasks can be accomplished using this repository: As mentioned, the following three tasks can be accomplished using this repository:
* Generate training dataset * Generate training dataset
@ -23,11 +26,15 @@ These three commands are:
### image-enhancement ### image-enhancement
Generating a training dataset for image enhancement is quite straightforward. All that is needed is a set of high-resolution images. The training dataset can then be generated using the following command: Generating a training dataset for image enhancement is quite straightforward. All that is needed is a set of
high-resolution images. The training dataset can then be generated using the following command:
`python generate_gt_for_training.py image-enhancement -dis "dir of high resolution images" -dois "dir where degraded images will be written" -dols "dir where the corresponding high resolution image will be written as label" -scs "degrading scales json file"` `python generate_gt_for_training.py image-enhancement -dis "dir of high resolution images" -dois "dir where degraded images will be written" -dols "dir where the corresponding high resolution image will be written as label" -scs "degrading scales json file"`
The scales JSON file is a dictionary with a key named 'scales' and values representing scales smaller than 1. Images are downscaled based on these scales and then upscaled again to their original size. This process causes the images to lose resolution at different scales. The degraded images are used as input images, and the original high-resolution images serve as labels. The enhancement model can be trained with this generated dataset. The scales JSON file looks like this: The scales JSON file is a dictionary with a key named 'scales' and values representing scales smaller than 1. Images are
downscaled based on these scales and then upscaled again to their original size. This process causes the images to lose
resolution at different scales. The degraded images are used as input images, and the original high-resolution images
serve as labels. The enhancement model can be trained with this generated dataset. The scales JSON file looks like this:
```yaml ```yaml
{ {
@ -37,21 +44,34 @@ The scales JSON file is a dictionary with a key named 'scales' and values repres
### machine-based-reading-order ### machine-based-reading-order
For machine-based reading order, we aim to determine the reading priority between two sets of text regions. The model's input is a three-channel image: the first and last channels contain information about each of the two text regions, while the middle channel encodes prominent layout elements necessary for reading order, such as separators and headers. To generate the training dataset, our script requires a page XML file that specifies the image layout with the correct reading order. For machine-based reading order, we aim to determine the reading priority between two sets of text regions. The model's
input is a three-channel image: the first and last channels contain information about each of the two text regions,
while the middle channel encodes prominent layout elements necessary for reading order, such as separators and headers.
To generate the training dataset, our script requires a page XML file that specifies the image layout with the correct
reading order.
For output images, it is necessary to specify the width and height. Additionally, a minimum text region size can be set to filter out regions smaller than this minimum size. This minimum size is defined as the ratio of the text region area to the image area, with a default value of zero. To run the dataset generator, use the following command: For output images, it is necessary to specify the width and height. Additionally, a minimum text region size can be set
to filter out regions smaller than this minimum size. This minimum size is defined as the ratio of the text region area
to the image area, with a default value of zero. To run the dataset generator, use the following command:
`python generate_gt_for_training.py machine-based-reading-order -dx "dir of GT xml files" -domi "dir where output images will be written" -docl "dir where the labels will be written" -ih "height" -iw "width" -min "min area ratio"` `python generate_gt_for_training.py machine-based-reading-order -dx "dir of GT xml files" -domi "dir where output images will be written" -docl "dir where the labels will be written" -ih "height" -iw "width" -min "min area ratio"`
### pagexml2label ### pagexml2label
pagexml2label is designed to generate labels from GT page XML files for various pixel-wise segmentation use cases, including 'layout,' 'textline,' 'printspace,' 'glyph,' and 'word' segmentation. pagexml2label is designed to generate labels from GT page XML files for various pixel-wise segmentation use cases,
To train a pixel-wise segmentation model, we require images along with their corresponding labels. Our training script expects a PNG image where each pixel corresponds to a label, represented by an integer. The background is always labeled as zero, while other elements are assigned different integers. For instance, if we have ground truth data with four elements including the background, the classes would be labeled as 0, 1, 2, and 3 respectively. including 'layout,' 'textline,' 'printspace,' 'glyph,' and 'word' segmentation.
In binary segmentation scenarios such as textline or page extraction, the background is encoded as 0, and the desired element is automatically encoded as 1 in the PNG label. To train a pixel-wise segmentation model, we require images along with their corresponding labels. Our training script
expects a PNG image where each pixel corresponds to a label, represented by an integer. The background is always labeled
as zero, while other elements are assigned different integers. For instance, if we have ground truth data with four
elements including the background, the classes would be labeled as 0, 1, 2, and 3 respectively.
To specify the desired use case and the elements to be extracted in the PNG labels, a custom JSON file can be passed. For example, in the case of 'textline' detection, the JSON file would resemble this: In binary segmentation scenarios such as textline or page extraction, the background is encoded as 0, and the desired
element is automatically encoded as 1 in the PNG label.
To specify the desired use case and the elements to be extracted in the PNG labels, a custom JSON file can be passed.
For example, in the case of 'textline' detection, the JSON file would resemble this:
```yaml ```yaml
{ {
@ -83,16 +103,32 @@ A possible custom config json file for layout segmentation where the "printspac
"printspace_as_class_in_layout" : 8 "printspace_as_class_in_layout" : 8
} }
``` ```
For the layout use case, it is beneficial to first understand the structure of the page XML file and its elements. In a given image, the annotations of elements are recorded in a page XML file, including their contours and classes. For an image document, the known regions are 'textregion', 'separatorregion', 'imageregion', 'graphicregion', 'noiseregion', and 'tableregion'.
Text regions and graphic regions also have their own specific types. The known types for us for text regions are 'paragraph', 'header', 'heading', 'marginalia', 'drop-capital', 'footnote', 'footnote-continued', 'signature-mark', 'page-number', and 'catch-word'. The known types for graphic regions are 'handwritten-annotation', 'decoration', 'stamp', and 'signature'. For the layout use case, it is beneficial to first understand the structure of the page XML file and its elements. In a
Since we don't know all types of text and graphic regions, unknown cases can arise. To handle these, we have defined two additional types: "rest_as_paragraph" and "rest_as_decoration" to ensure that no unknown types are missed. This way, users can extract all known types from the labels and be confident that no unknown types are overlooked. given image, the annotations of elements are recorded in a page XML file, including their contours and classes. For an
image document, the known regions are 'textregion', 'separatorregion', 'imageregion', 'graphicregion', 'noiseregion',
and 'tableregion'.
In the custom JSON file shown above, "header" and "heading" are extracted as the same class, while "marginalia" is shown as a different class. All other text region types, including "drop-capital," are grouped into the same class. For the graphic region, "stamp" has its own class, while all other types are classified together. "Image region" and "separator region" are also present in the label. However, other regions like "noise region" and "table region" will not be included in the label PNG file, even if they have information in the page XML files, as we chose not to include them. Text regions and graphic regions also have their own specific types. The known types for us for text regions are
'paragraph', 'header', 'heading', 'marginalia', 'drop-capital', 'footnote', 'footnote-continued', 'signature-mark',
'page-number', and 'catch-word'. The known types for graphic regions are 'handwritten-annotation', 'decoration',
'stamp', and 'signature'.
Since we don't know all types of text and graphic regions, unknown cases can arise. To handle these, we have defined two
additional types: "rest_as_paragraph" and "rest_as_decoration" to ensure that no unknown types are missed. This way,
users can extract all known types from the labels and be confident that no unknown types are overlooked.
In the custom JSON file shown above, "header" and "heading" are extracted as the same class, while "marginalia" is shown
as a different class. All other text region types, including "drop-capital," are grouped into the same class. For the
graphic region, "stamp" has its own class, while all other types are classified together. "Image region" and "separator
region" are also present in the label. However, other regions like "noise region" and "table region" will not be
included in the label PNG file, even if they have information in the page XML files, as we chose not to include them.
`python generate_gt_for_training.py pagexml2label -dx "dir of GT xml files" -do "dir where output label png files will be written" -cfg "custom config json file" -to "output type which has 2d and 3d. 2d is used for training and 3d is just to visualise the labels" "` `python generate_gt_for_training.py pagexml2label -dx "dir of GT xml files" -do "dir where output label png files will be written" -cfg "custom config json file" -to "output type which has 2d and 3d. 2d is used for training and 3d is just to visualise the labels" "`
We have also defined an artificial class that can be added to the boundary of text region types or text lines. This key is called "artificial_class_on_boundary." If users want to apply this to certain text regions in the layout use case, the example JSON config file should look like this: We have also defined an artificial class that can be added to the boundary of text region types or text lines. This key
is called "artificial_class_on_boundary." If users want to apply this to certain text regions in the layout use case,
the example JSON config file should look like this:
```yaml ```yaml
{ {
@ -114,9 +150,14 @@ We have also defined an artificial class that can be added to the boundary of te
} }
``` ```
This implies that the artificial class label, denoted by 7, will be present on PNG files and will only be added to the elements labeled as "paragraph," "header," "heading," and "marginalia." This implies that the artificial class label, denoted by 7, will be present on PNG files and will only be added to the
elements labeled as "paragraph," "header," "heading," and "marginalia."
For "textline," "word," and "glyph," the artificial class on the boundaries will be activated only if the "artificial_class_label" key is specified in the config file. Its value should be set as 2 since these elements represent binary cases. For example, if the background and textline are denoted as 0 and 1 respectively, then the artificial class should be assigned the value 2. The example JSON config file should look like this for "textline" use case: For "textline," "word," and "glyph," the artificial class on the boundaries will be activated only if the
"artificial_class_label" key is specified in the config file. Its value should be set as 2 since these elements
represent binary cases. For example, if the background and textline are denoted as 0 and 1 respectively, then the
artificial class should be assigned the value 2. The example JSON config file should look like this for "textline" use
case:
```yaml ```yaml
{ {
@ -125,7 +166,11 @@ For "textline," "word," and "glyph," the artificial class on the boundaries will
} }
``` ```
If the coordinates of "PrintSpace" or "Border" are present in the page XML ground truth files, and the user wishes to crop only the print space area, this can be achieved by activating the "-ps" argument. However, it should be noted that in this scenario, since cropping will be applied to the label files, the directory of the original images must be provided to ensure that they are cropped in sync with the labels. This ensures that the correct images and labels required for training are obtained. The command should resemble the following: If the coordinates of "PrintSpace" or "Border" are present in the page XML ground truth files, and the user wishes to
crop only the print space area, this can be achieved by activating the "-ps" argument. However, it should be noted that
in this scenario, since cropping will be applied to the label files, the directory of the original images must be
provided to ensure that they are cropped in sync with the labels. This ensures that the correct images and labels
required for training are obtained. The command should resemble the following:
`python generate_gt_for_training.py pagexml2label -dx "dir of GT xml files" -do "dir where output label png files will be written" -cfg "custom config json file" -to "output type which has 2d and 3d. 2d is used for training and 3d is just to visualise the labels" -ps -di "dir where the org images are located" -doi "dir where the cropped output images will be written" ` `python generate_gt_for_training.py pagexml2label -dx "dir of GT xml files" -do "dir where output label png files will be written" -cfg "custom config json file" -to "output type which has 2d and 3d. 2d is used for training and 3d is just to visualise the labels" -ps -di "dir where the org images are located" -doi "dir where the cropped output images will be written" `
@ -156,7 +201,7 @@ For the classification use case, we haven't provided a ground truth generator, a
The "dir_train" should be like this: The "dir_train" should be like this:
``` ```
. .
└── train # train directory └── train # train directory
├── apple # directory of images for apple class ├── apple # directory of images for apple class
@ -165,7 +210,7 @@ The "dir_train" should be like this:
And the "dir_eval" the same structure as train directory: And the "dir_eval" the same structure as train directory:
``` ```
. .
└── eval # evaluation directory └── eval # evaluation directory
├── apple # directory of images for apple class ├── apple # directory of images for apple class
@ -178,7 +223,10 @@ The classification model can be trained using the following command line:
`python train.py with config_classification.json` `python train.py with config_classification.json`
As evident in the example JSON file above, for classification, we utilize a "f1_threshold_classification" parameter. This parameter is employed to gather all models with an evaluation f1 score surpassing this threshold. Subsequently, an ensemble of these model weights is executed, and a model is saved in the output directory as "model_ens_avg". Additionally, the weight of the best model based on the evaluation f1 score is saved as "model_best". As evident in the example JSON file above, for classification, we utilize a "f1_threshold_classification" parameter.
This parameter is employed to gather all models with an evaluation f1 score surpassing this threshold. Subsequently,
an ensemble of these model weights is executed, and a model is saved in the output directory as "model_ens_avg".
Additionally, the weight of the best model based on the evaluation f1 score is saved as "model_best".
### reading order ### reading order
An example config json file for machine based reading order should be like this: An example config json file for machine based reading order should be like this:
@ -225,18 +273,25 @@ The classification model can be trained like the classification case command lin
#### Parameter configuration for segmentation or enhancement usecases #### Parameter configuration for segmentation or enhancement usecases
The following parameter configuration can be applied to all segmentation use cases and enhancements. The augmentation, its sub-parameters, and continued training are defined only for segmentation use cases and enhancements, not for classification and machine-based reading order, as you can see in their example config files. The following parameter configuration can be applied to all segmentation use cases and enhancements. The augmentation,
its sub-parameters, and continued training are defined only for segmentation use cases and enhancements, not for
classification and machine-based reading order, as you can see in their example config files.
* backbone_type: For segmentation tasks (such as text line, binarization, and layout detection) and enhancement, we offer two backbone options: a "nontransformer" and a "transformer" backbone. For the "transformer" backbone, we first apply a CNN followed by a transformer. In contrast, the "nontransformer" backbone utilizes only a CNN ResNet-50. * backbone_type: For segmentation tasks (such as text line, binarization, and layout detection) and enhancement, we
offer two backbone options: a "nontransformer" and a "transformer" backbone. For the "transformer" backbone, we first
apply a CNN followed by a transformer. In contrast, the "nontransformer" backbone utilizes only a CNN ResNet-50.
* task : The task parameter can have values such as "segmentation", "enhancement", "classification", and "reading_order". * task : The task parameter can have values such as "segmentation", "enhancement", "classification", and "reading_order".
* patches: If you want to break input images into smaller patches (input size of the model) you need to set this parameter to ``true``. In the case that the model should see the image once, like page extraction, patches should be set to ``false``. * patches: If you want to break input images into smaller patches (input size of the model) you need to set this
parameter to ``true``. In the case that the model should see the image once, like page extraction, patches should be
set to ``false``.
* n_batch: Number of batches at each iteration. * n_batch: Number of batches at each iteration.
* n_classes: Number of classes. In the case of binary classification this should be 2. In the case of reading_order it should set to 1. And for the case of layout detection just the unique number of classes should be given. * n_classes: Number of classes. In the case of binary classification this should be 2. In the case of reading_order it should set to 1. And for the case of layout detection just the unique number of classes should be given.
* n_epochs: Number of epochs. * n_epochs: Number of epochs.
* input_height: This indicates the height of model's input. * input_height: This indicates the height of model's input.
* input_width: This indicates the width of model's input. * input_width: This indicates the width of model's input.
* weight_decay: Weight decay of l2 regularization of model layers. * weight_decay: Weight decay of l2 regularization of model layers.
* pretraining: Set to ``true`` to load pretrained weights of ResNet50 encoder. The downloaded weights should be saved in a folder named "pretrained_model" in the same directory of "train.py" script. * pretraining: Set to ``true`` to load pretrained weights of ResNet50 encoder. The downloaded weights should be saved
in a folder named "pretrained_model" in the same directory of "train.py" script.
* augmentation: If you want to apply any kind of augmentation this parameter should first set to ``true``. * augmentation: If you want to apply any kind of augmentation this parameter should first set to ``true``.
* flip_aug: If ``true``, different types of filp will be applied on image. Type of flips is given with "flip_index" parameter. * flip_aug: If ``true``, different types of filp will be applied on image. Type of flips is given with "flip_index" parameter.
* blur_aug: If ``true``, different types of blurring will be applied on image. Type of blurrings is given with "blur_k" parameter. * blur_aug: If ``true``, different types of blurring will be applied on image. Type of blurrings is given with "blur_k" parameter.
@ -255,9 +310,14 @@ The following parameter configuration can be applied to all segmentation use cas
* brightness: The amount of brightenings. * brightness: The amount of brightenings.
* thetha: Rotation angles. * thetha: Rotation angles.
* degrade_scales: The amount of degradings. * degrade_scales: The amount of degradings.
* continue_training: If ``true``, it means that you have already trained a model and you would like to continue the training. So it is needed to provide the dir of trained model with "dir_of_start_model" and index for naming the models. For example if you have already trained for 3 epochs then your last index is 2 and if you want to continue from model_1.h5, you can set ``index_start`` to 3 to start naming model with index 3. * continue_training: If ``true``, it means that you have already trained a model and you would like to continue the
training. So it is needed to provide the dir of trained model with "dir_of_start_model" and index for naming the
models. For example if you have already trained for 3 epochs then your last index is 2 and if you want to continue from
model_1.h5, you can set ``index_start`` to 3 to start naming model with index 3.
* weighted_loss: If ``true``, this means that you want to apply weighted categorical_crossentropy as loss fucntion. Be carefull if you set to ``true``the parameter "is_loss_soft_dice" should be ``false`` * weighted_loss: If ``true``, this means that you want to apply weighted categorical_crossentropy as loss fucntion. Be carefull if you set to ``true``the parameter "is_loss_soft_dice" should be ``false``
* data_is_provided: If you have already provided the input data you can set this to ``true``. Be sure that the train and eval data are in "dir_output". Since when once we provide training data we resize and augment them and then we write them in sub-directories train and eval in "dir_output". * data_is_provided: If you have already provided the input data you can set this to ``true``. Be sure that the train
and eval data are in "dir_output". Since when once we provide training data we resize and augment them and then we
write them in sub-directories train and eval in "dir_output".
* dir_train: This is the directory of "images" and "labels" (dir_train should include two subdirectories with names of images and labels ) for raw images and labels. Namely they are not prepared (not resized and not augmented) yet for training the model. When we run this tool these raw data will be transformed to suitable size needed for the model and they will be written in "dir_output" in train and eval directories. Each of train and eval include "images" and "labels" sub-directories. * dir_train: This is the directory of "images" and "labels" (dir_train should include two subdirectories with names of images and labels ) for raw images and labels. Namely they are not prepared (not resized and not augmented) yet for training the model. When we run this tool these raw data will be transformed to suitable size needed for the model and they will be written in "dir_output" in train and eval directories. Each of train and eval include "images" and "labels" sub-directories.
* index_start: Starting index for saved models in the case that "continue_training" is ``true``. * index_start: Starting index for saved models in the case that "continue_training" is ``true``.
* dir_of_start_model: Directory containing pretrained model to continue training the model in the case that "continue_training" is ``true``. * dir_of_start_model: Directory containing pretrained model to continue training the model in the case that "continue_training" is ``true``.
@ -290,7 +350,8 @@ And the "dir_eval" the same structure as train directory:
└── labels # directory of labels └── labels # directory of labels
``` ```
After configuring the JSON file for segmentation or enhancement, training can be initiated by running the following command, similar to the process for classification and reading order: After configuring the JSON file for segmentation or enhancement, training can be initiated by running the following
command, similar to the process for classification and reading order:
`python train.py with config_classification.json` `python train.py with config_classification.json`
@ -339,7 +400,7 @@ An example config json file for binarization can be like this:
"thetha" : [10, -10], "thetha" : [10, -10],
"continue_training": false, "continue_training": false,
"index_start" : 0, "index_start" : 0,
"dir_of_start_model" : " ", "dir_of_start_model" : " ",
"weighted_loss": false, "weighted_loss": false,
"is_loss_soft_dice": false, "is_loss_soft_dice": false,
"data_is_provided": false, "data_is_provided": false,
@ -384,7 +445,7 @@ An example config json file for binarization can be like this:
"thetha" : [10, -10], "thetha" : [10, -10],
"continue_training": false, "continue_training": false,
"index_start" : 0, "index_start" : 0,
"dir_of_start_model" : " ", "dir_of_start_model" : " ",
"weighted_loss": false, "weighted_loss": false,
"is_loss_soft_dice": false, "is_loss_soft_dice": false,
"data_is_provided": false, "data_is_provided": false,
@ -429,7 +490,7 @@ An example config json file for binarization can be like this:
"thetha" : [10, -10], "thetha" : [10, -10],
"continue_training": false, "continue_training": false,
"index_start" : 0, "index_start" : 0,
"dir_of_start_model" : " ", "dir_of_start_model" : " ",
"weighted_loss": false, "weighted_loss": false,
"is_loss_soft_dice": false, "is_loss_soft_dice": false,
"data_is_provided": false, "data_is_provided": false,
@ -439,7 +500,8 @@ An example config json file for binarization can be like this:
} }
``` ```
It's important to mention that the value of n_classes for enhancement should be 3, as the model's output is a 3-channel image. It's important to mention that the value of n_classes for enhancement should be 3, as the model's output is a 3-channel
image.
#### Page extraction #### Page extraction
@ -476,7 +538,7 @@ It's important to mention that the value of n_classes for enhancement should be
"thetha" : [10, -10], "thetha" : [10, -10],
"continue_training": false, "continue_training": false,
"index_start" : 0, "index_start" : 0,
"dir_of_start_model" : " ", "dir_of_start_model" : " ",
"weighted_loss": false, "weighted_loss": false,
"is_loss_soft_dice": false, "is_loss_soft_dice": false,
"data_is_provided": false, "data_is_provided": false,
@ -486,7 +548,8 @@ It's important to mention that the value of n_classes for enhancement should be
} }
``` ```
For page segmentation (or print space or border segmentation), the model needs to view the input image in its entirety, hence the patches parameter should be set to false. For page segmentation (or print space or border segmentation), the model needs to view the input image in its entirety,
hence the patches parameter should be set to false.
#### layout segmentation #### layout segmentation
@ -533,7 +596,7 @@ An example config json file for layout segmentation with 5 classes (including ba
"thetha" : [10, -10], "thetha" : [10, -10],
"continue_training": false, "continue_training": false,
"index_start" : 0, "index_start" : 0,
"dir_of_start_model" : " ", "dir_of_start_model" : " ",
"weighted_loss": false, "weighted_loss": false,
"is_loss_soft_dice": false, "is_loss_soft_dice": false,
"data_is_provided": false, "data_is_provided": false,
@ -543,9 +606,11 @@ An example config json file for layout segmentation with 5 classes (including ba
} }
``` ```
## Inference with the trained model ## Inference with the trained model
### classification ### classification
For conducting inference with a trained model, you simply need to execute the following command line, specifying the directory of the model and the image on which to perform inference: For conducting inference with a trained model, you simply need to execute the following command line, specifying the
directory of the model and the image on which to perform inference:
`python inference.py -m "model dir" -i "image" ` `python inference.py -m "model dir" -i "image" `
@ -554,8 +619,9 @@ This will straightforwardly return the class of the image.
### machine based reading order ### machine based reading order
To infer the reading order using an reading order model, we need a page XML file containing layout information but
To infer the reading order using an reading order model, we need a page XML file containing layout information but without the reading order. We simply need to provide the model directory, the XML file, and the output directory. The new XML file with the added reading order will be written to the output directory with the same name. We need to run: without the reading order. We simply need to provide the model directory, the XML file, and the output directory. The
new XML file with the added reading order will be written to the output directory with the same name. We need to run:
`python inference.py -m "model dir" -xml "page xml file" -o "output dir to write new xml with reading order" ` `python inference.py -m "model dir" -xml "page xml file" -o "output dir to write new xml with reading order" `
@ -570,7 +636,8 @@ For conducting inference with a trained model for segmentation and enhancement y
Note that in the case of page extraction the -p flag is not needed. Note that in the case of page extraction the -p flag is not needed.
For segmentation or binarization tasks, if a ground truth (GT) label is available, the IOU evaluation metric can be calculated for the output. To do this, you need to provide the GT label using the argument -gt. For segmentation or binarization tasks, if a ground truth (GT) label is available, the IOU evaluation metric can be
calculated for the output. To do this, you need to provide the GT label using the argument -gt.

View file

@ -4,7 +4,7 @@ This repository assists users in preparing training datasets, training models, a
models. We cover various use cases including pixel-wise segmentation, image classification, image enhancement, and models. We cover various use cases including pixel-wise segmentation, image classification, image enhancement, and
machine-based reading order. For each use case, we provide guidance on how to generate the corresponding training machine-based reading order. For each use case, we provide guidance on how to generate the corresponding training
dataset. dataset.
All these use cases are now utilized in the Eynollah workflow. All these use cases are now utilized in the Eynollah workflow.
As mentioned, the following three tasks can be accomplished using this repository: As mentioned, the following three tasks can be accomplished using this repository:
* Generate training dataset * Generate training dataset
@ -61,6 +61,7 @@ to the image area, with a default value of zero. To run the dataset generator, u
pagexml2label is designed to generate labels from GT page XML files for various pixel-wise segmentation use cases, pagexml2label is designed to generate labels from GT page XML files for various pixel-wise segmentation use cases,
including 'layout,' 'textline,' 'printspace,' 'glyph,' and 'word' segmentation. including 'layout,' 'textline,' 'printspace,' 'glyph,' and 'word' segmentation.
To train a pixel-wise segmentation model, we require images along with their corresponding labels. Our training script To train a pixel-wise segmentation model, we require images along with their corresponding labels. Our training script
expects a PNG image where each pixel corresponds to a label, represented by an integer. The background is always labeled expects a PNG image where each pixel corresponds to a label, represented by an integer. The background is always labeled
as zero, while other elements are assigned different integers. For instance, if we have ground truth data with four as zero, while other elements are assigned different integers. For instance, if we have ground truth data with four
@ -102,6 +103,7 @@ A possible custom config json file for layout segmentation where the "printspac
"printspace_as_class_in_layout" : 8 "printspace_as_class_in_layout" : 8
} }
``` ```
For the layout use case, it is beneficial to first understand the structure of the page XML file and its elements. In a For the layout use case, it is beneficial to first understand the structure of the page XML file and its elements. In a
given image, the annotations of elements are recorded in a page XML file, including their contours and classes. For an given image, the annotations of elements are recorded in a page XML file, including their contours and classes. For an
image document, the known regions are 'textregion', 'separatorregion', 'imageregion', 'graphicregion', 'noiseregion', image document, the known regions are 'textregion', 'separatorregion', 'imageregion', 'graphicregion', 'noiseregion',
@ -199,7 +201,7 @@ For the classification use case, we haven't provided a ground truth generator, a
The "dir_train" should be like this: The "dir_train" should be like this:
``` ```
. .
└── train # train directory └── train # train directory
├── apple # directory of images for apple class ├── apple # directory of images for apple class
@ -208,7 +210,7 @@ The "dir_train" should be like this:
And the "dir_eval" the same structure as train directory: And the "dir_eval" the same structure as train directory:
``` ```
. .
└── eval # evaluation directory └── eval # evaluation directory
├── apple # directory of images for apple class ├── apple # directory of images for apple class
@ -277,11 +279,11 @@ classification and machine-based reading order, as you can see in their example
* backbone_type: For segmentation tasks (such as text line, binarization, and layout detection) and enhancement, we * backbone_type: For segmentation tasks (such as text line, binarization, and layout detection) and enhancement, we
offer two backbone options: a "nontransformer" and a "transformer" backbone. For the "transformer" backbone, we first offer two backbone options: a "nontransformer" and a "transformer" backbone. For the "transformer" backbone, we first
apply a CNN followed by a transformer. In contrast, the "nontransformer" backbone utilizes only a CNN ResNet-50. apply a CNN followed by a transformer. In contrast, the "nontransformer" backbone utilizes only a CNN ResNet-50.
* task : The task parameter can have values such as "segmentation", "enhancement", "classification", and "reading_order". * task : The task parameter can have values such as "segmentation", "enhancement", "classification", and "reading_order".
* patches: If you want to break input images into smaller patches (input size of the model) you need to set this * patches: If you want to break input images into smaller patches (input size of the model) you need to set this
parameter to ``true``. In the case that the model should see the image once, like page extraction, patches should be parameter to ``true``. In the case that the model should see the image once, like page extraction, patches should be
set to ``false``. set to ``false``.
* n_batch: Number of batches at each iteration. * n_batch: Number of batches at each iteration.
* n_classes: Number of classes. In the case of binary classification this should be 2. In the case of reading_order it should set to 1. And for the case of layout detection just the unique number of classes should be given. * n_classes: Number of classes. In the case of binary classification this should be 2. In the case of reading_order it should set to 1. And for the case of layout detection just the unique number of classes should be given.
* n_epochs: Number of epochs. * n_epochs: Number of epochs.
@ -311,11 +313,11 @@ set to ``false``.
* continue_training: If ``true``, it means that you have already trained a model and you would like to continue the * continue_training: If ``true``, it means that you have already trained a model and you would like to continue the
training. So it is needed to provide the dir of trained model with "dir_of_start_model" and index for naming the training. So it is needed to provide the dir of trained model with "dir_of_start_model" and index for naming the
models. For example if you have already trained for 3 epochs then your last index is 2 and if you want to continue from models. For example if you have already trained for 3 epochs then your last index is 2 and if you want to continue from
model_1.h5, you can set ``index_start`` to 3 to start naming model with index 3. model_1.h5, you can set ``index_start`` to 3 to start naming model with index 3.
* weighted_loss: If ``true``, this means that you want to apply weighted categorical_crossentropy as loss fucntion. Be carefull if you set to ``true``the parameter "is_loss_soft_dice" should be ``false`` * weighted_loss: If ``true``, this means that you want to apply weighted categorical_crossentropy as loss fucntion. Be carefull if you set to ``true``the parameter "is_loss_soft_dice" should be ``false``
* data_is_provided: If you have already provided the input data you can set this to ``true``. Be sure that the train * data_is_provided: If you have already provided the input data you can set this to ``true``. Be sure that the train
and eval data are in "dir_output". Since when once we provide training data we resize and augment them and then we and eval data are in "dir_output". Since when once we provide training data we resize and augment them and then we
write them in sub-directories train and eval in "dir_output". write them in sub-directories train and eval in "dir_output".
* dir_train: This is the directory of "images" and "labels" (dir_train should include two subdirectories with names of images and labels ) for raw images and labels. Namely they are not prepared (not resized and not augmented) yet for training the model. When we run this tool these raw data will be transformed to suitable size needed for the model and they will be written in "dir_output" in train and eval directories. Each of train and eval include "images" and "labels" sub-directories. * dir_train: This is the directory of "images" and "labels" (dir_train should include two subdirectories with names of images and labels ) for raw images and labels. Namely they are not prepared (not resized and not augmented) yet for training the model. When we run this tool these raw data will be transformed to suitable size needed for the model and they will be written in "dir_output" in train and eval directories. Each of train and eval include "images" and "labels" sub-directories.
* index_start: Starting index for saved models in the case that "continue_training" is ``true``. * index_start: Starting index for saved models in the case that "continue_training" is ``true``.
* dir_of_start_model: Directory containing pretrained model to continue training the model in the case that "continue_training" is ``true``. * dir_of_start_model: Directory containing pretrained model to continue training the model in the case that "continue_training" is ``true``.
@ -536,7 +538,7 @@ image.
"thetha" : [10, -10], "thetha" : [10, -10],
"continue_training": false, "continue_training": false,
"index_start" : 0, "index_start" : 0,
"dir_of_start_model" : " ", "dir_of_start_model" : " ",
"weighted_loss": false, "weighted_loss": false,
"is_loss_soft_dice": false, "is_loss_soft_dice": false,
"data_is_provided": false, "data_is_provided": false,