- `training.models.CTCDecoder`: switch back
from `tf.nn.ctc_beam_search_decoder()`
to `tf.nn.ctc_greedy_decoder()`
(because ONNX only implements `CTCGreedyDecoder`)
- `training.models.cnn_rnn_ocr_model(inference=True)` and
`training.models.cnn_rnn_ocr_model4inference`:
drop layer `tf.io.decode_raw()`
(because ONNX does not implement `DecodePaddedRaw`)
- `Eynollah_ocr.run_cnn()`: expect bytes arrays from predictor
instead of uint8
- `predictor`: to prevent segfaults when sending `tf.string` results
via `shared_memory`, convert `np.object` to `np.bytes_` directly
when rebuilding the inference model for cnn-rnn-ocr,
- open the old `characters_org.txt` file for the charset
- use it to pass the actual `n_classes` (overriding the config)
- use its path to pass the `characters_txt_file` (overriding the config)
Because transformers v4 and v5 API for image preprocessor differs,
and the model-internal image input sizes are actually irrelevant,
because the preprocessor will resize them anyway, and there is no
batch dimension (because the input images will have different shapes),
do not advertise this information in `.input_shape`.
- ModelZoo: drop `trocr_processor` model type
- `ModelZoo.load_models()`: use Predictor for `ocr_tr` models, too
- `ModelZoo.load_model()`: for `ocr_tr`, load processor and model,
then define a function object as stand-in for the common model
interface based on Keras (w/ `.predict_on_batch()`)
- Predictor: allow multi-input without actual batch dimension
for `ocr_tr` models (because the model takes a list of original
image arrays and resizes them to model shape internally)
- Eynollah_ocr: adapt (replacing preprocessing, prediction and
decoding steps by a single `.predict()` call)
- `min_conf_value_of_textline_text`: apply by skipping
lines below threshold (instead of writing empty text),
and delete their TextEquiv (if existing)
- `write_ocr()`: simplify, and ensure consistency between
line and region level text correctly
- drop `end_character` mechanics and `characters` model type
for decoding output probability (not needed)
- drop `decode_batch_predictions()` and `num_to_char` model type
(part of inference model)
- drop roughshot confidence estimation calculation
(returned precisely by inference model)
- adapt model prediction to inference model: just omit zeros,
map to bytes, filter OOV tokens and decode UTF-8 to str
- if no binarization input was provided, then compute it on the fly
using `binarization` model
- also apply `min_conf_value_of_textline_text` (as for TrOCR)
- batching over entire page instead of region-wise
(which underfilled batches)
- simplify and avoid copied redundant code
- rename `extracted_conf_value_merged` → `extracted_confs_merged`
- move `batched()` from `utils.utils_ocr` to `utils`
- drop `utils_ocr.distortion_free_resize()` (not needed)
- simplify `utils_ocr.break_curved_line_into_small_pieces_and_then_merge()`
- drop `utils_ocr.return_textline_contour_with_added_box_coordinate()`
and `utils_ocr.return_rnn_cnn_ocr_of_given_textlines()` (not needed)
- ModelZoo: drop `num_to_char` and `characters` model types,
also drop `_load_characters()` and `_load_num_to_char()` loaders
- `ModelZoo.load_models()`: use Predictor for `ocr` models, too
- `ModelZoo.load_model()`: delegate runtime/inference conversion of
OCR models to `eynollah.training.models.cnn_rnn_ocr_model4inference`
- `training.models`: add (purely functional) Keras layer `CTCDecoder`
for inference on top of softmax output, but using TF backend
function instead of (broken) `Keras.backend.ctc_decode()`, while
switching to beam search (instead of greedy) and also returning
decoded path probability
- `training.models.cnn_rnn_ocr_model()` w/ `inference=True`:
* add kwarg `characters_txt_file` for file path of character set
* configure secondary tensor path on OCR graph for binarized input
(additional input `image_bin`, averaging softmax outputs)
* use new `CTCDecoder` layer and inverse `StringLookup` layer to
decode from softmax output to tf.string; so inference models
now have 2 inputs (RGB, binarized) and 2 outputs (text, prob)
* since `np.dtype=object` cannot be handled by SharedMemory (as
needed by Predictor queues), also replace tf.string by tf.uint8
arrays
* use this for `training convert` for OCR models w/ `--rebuild`
- `training.models.cnn_rnn_ocr_model4inference`:
* new function which does the same but loads an existing OCR model
in training configuration (i.e. without prior `inference=True`)
* use this for `training convert` for OCR models w/o `--rebuild`
- comment out ad-hoc conversion/loading of autosized models
- refactor predictor backends for model types into separate functions
- only attempt inference conversion of cnn-rnn-ocr model
if applicable (`ctc_loss` layer still present)
- apply VRAM limits across model types
(Keras, TF-Serving, ONNX)
- apply TF device selection across model types
(Keras, TF-Serving)
- implement predictor backend for ONNX models:
- using onnxruntime
- covering CUDA and TensorRT providers
- trying to support manual device selection
- hiding session management details
- converting float32 to float16
