Commit graph

645 commits

Author SHA1 Message Date
Robert Sachunsky
62b55a3809 train params: drop reload_weights, re-use dir_of_start_model
- drop ad-hoc configuration parameter `reload_weights`
  (used for conversion/export of models for inference,
   to be replaced by extra CLI)
- re-interprete `dir_of_start_model` to also load weights
  if not `continue_training`
2026-05-28 17:42:55 +02:00
Robert Sachunsky
093030f503 train/models: move all model builders to models.get_model()
- models: add new `get_model()`, passing in Sacred config
  to capture builder function arguments
- train: fewer imports
- train: no need to pass `custom_objects` if loading with
  `compile=False` (and we custom-compile later, anyway)
2026-05-28 17:37:45 +02:00
Robert Sachunsky
faef1967f8 models.cnn_rnn_ocr_model: add inference option, drop model name 2026-05-28 17:33:57 +02:00
Robert Sachunsky
c4a7eec5b3 models: cosmetics
- using `Reshape`, do not pass `target_shape` as kwarg
- add a default `name` for `Patches` and `PatchEncoder`
2026-05-27 01:58:21 +02:00
Robert Sachunsky
9801129aa6 estimate_skew_contours: ensure retval is always float 2026-05-22 12:37:07 +02:00
Robert Sachunsky
26afc5ddab ModelZoo: ensure exported TensorShape is converted to plain tuple 2026-05-22 12:35:44 +02:00
Robert Sachunsky
0836230c6b utils_ocr: avoid module-level import of TF 2026-05-21 22:50:53 +02:00
Robert Sachunsky
f3a93983c0 ModelZoo: add ocr key for memory_limit 2026-05-21 22:50:13 +02:00
Robert Sachunsky
ea41dcae1d trocr: use beam search instead of greedy decoding 2026-05-21 17:52:27 +02:00
Robert Sachunsky
074753a98e ModelZoo: fix Torch device selection 2026-05-21 17:25:53 +02:00
Robert Sachunsky
000e4ac8d8 trocr: extract confidence, too 2026-05-21 17:25:39 +02:00
Robert Sachunsky
f3649adbf2 trocr: apply do_not_mask_with_textline_contour here, too 2026-05-21 17:23:11 +02:00
Robert Sachunsky
1d67e65f11 trocr: simplify, batch over entire page…
- batching over entire page instead of region-wise
  (underfilling batches)
- avoid copied redundant code
2026-05-21 15:48:21 +02:00
Robert Sachunsky
d50bd7c650 trocr: avoid warnings by passing clean_up_tokenization_spaces=False 2026-05-21 14:20:51 +02:00
Robert Sachunsky
f9f9130dbb do_order_of_regions: remove redundant+overcautious assertion 2026-05-21 03:21:36 +02:00
Robert Sachunsky
bf7ec0233d ModelZoo.load_model: use memory_limit instead of memory_growth
- growth strategy is more flexible, but uses much more VRAM
- limit strategy needs to be calibrated to models (currently fixed),
  and batch size, but needs much less VRAM and is faster
2026-05-21 02:43:34 +02:00
Robert Sachunsky
94a5e9da14 ModelZoo.load_model: avoid attempting to load exported models as Keras
models (which causes a warning), but switch to TF-Serving import right away
2026-05-21 02:41:19 +02:00
Robert Sachunsky
7f2bf715df ModelZoo.load_model: fix loading exported vs saved models 2026-05-21 02:39:59 +02:00
Robert Sachunsky
3de1407d18 drop unnecessary TF / Torch imports 2026-05-21 02:38:20 +02:00
Robert Sachunsky
bdfebd2c70 reload_weights: save()export() w/ serve() inference 2026-05-19 03:40:18 +02:00
Robert Sachunsky
86adaf299a training.models.transformer_block: tf.reshape → Keras Reshape layer 2026-05-19 03:40:16 +02:00
Robert Sachunsky
9efce5e9f2 Predictor.shutdown: use join() instead of terminate() 2026-05-19 03:40:07 +02:00
Robert Sachunsky
ffe5cdc519 ModelZoo.shutdown: drop extra del (already done by shutdown()) 2026-05-19 03:40:05 +02:00
Robert Sachunsky
481c286da9 ModelZoo.load_model: no XLA compilation 2026-05-19 03:40:05 +02:00
Robert Sachunsky
17b311441a model_zoo: also parse comma/colon syntax for device in Torch case 2026-05-19 03:40:03 +02:00
Robert Sachunsky
be4fe8c263 contour: drop unused functions depending on rotation_image_new() 2026-05-19 03:40:02 +02:00
Robert Sachunsky
21ecb043f7 CLIs: move --device option to group level 2026-05-19 03:39:59 +02:00
Robert Sachunsky
7ed1a1ebac CLIs: allow -h and show defaults uniformly, harmonise help, drop
remaining redundant negative options
2026-05-19 03:39:56 +02:00
Robert Sachunsky
cd62f13872 eynollah_ocr: make work again, re-use Eynollah base class…
- re-use Eynollah base class
- use `ModelZoo.load_models()` instead of `load_model()`
- pass in `device` init kwarg, delegate to `ModelZoo.load_models()`
- `device`: return Torch device at loaded model tensors
  instead of ad-hoc selection
- make numeric init kwargs non-optional (only numeric)
2026-05-19 03:39:55 +02:00
Robert Sachunsky
ded668a256 model_zoo: fix clash between Predictor and direct (OCR) use-cases…
- `load_models()`: uniformly handle arg types
- `load_model()`: move handling of non-model categories
  to `load_models()`
- `load_model()`: move SavedModel preference over HDF5 to `model_path()`
- `_load_ocr_model()`: add user-selected device handling and reporting
  for Torch (as for TF)
- `_load_ocr_model()`: move (TF-based) CNN-RNN case to `load_model()`
  (including Keras layer mapping)
- `shutdown()`: only apply `shutdown()` to Predictor model types
2026-05-19 03:39:53 +02:00
Robert Sachunsky
98e6fbbcbb mbreorder: make work again, re-use Eynollah base class 2026-05-19 03:39:52 +02:00
Robert Sachunsky
a1449da1d1 Revert "fix model loading in mb_ro and ocr"
This reverts commit 218a95e6a0.
2026-05-19 03:32:19 +02:00
kba
e612db2bb1 📦 v0.8.0 2026-05-11 13:16:30 +02:00
kba
218a95e6a0 fix model loading in mb_ro and ocr 2026-05-11 12:19:20 +02:00
kba
2035b07b55 Merge remote-tracking branch 'bertsky/ro-fixes-final' into prepare-release-v0.8.0
# Conflicts:
#	requirements-ocr.txt
2026-05-11 09:46:17 +02:00
Robert Sachunsky
e183937c5d separate_lines_new2: fix coord overflow by clipping, simplify…
- found positive and negative peaks, and even more so their
  relative offsets, may overflow in the cropped image,
  causing fake textlines; avoid that by clipping to the valid
  y coordinates
- calculation for number of tiles: sometimes one less
  tile is needed by making the previous last tile
  half-full on the right side
- add some (commented) plotting
- simplify (a lot, but only partially)
2026-05-11 03:09:02 +02:00
Robert Sachunsky
130f0aee42 do_work_of_slopes_curved: improve on d257869d
- relative images now need larger relative min_area
  (i.e. compensation factors)
- do not attempt (even) single-line skew estimation
  (via linear regression) if there is no (large enough)
  contour at all
- avoid re-computing `mask_parent`
- add some (commented) plotting
2026-05-11 03:03:04 +02:00
kba
03f3f9af17 update model zoo and docs to link to v0_8_0 model release on zenodo 2026-05-09 17:58:59 +02:00
Robert Sachunsky
4cd398bd0d standalone binarization: update, simplify…
- re-use Eynollah base class, drop copied code
- simplify `run()` and `run_single()`
- delegate to `do_prediction()`
  instead of custom (old) tiling loop
- drop `predict()`
- add `--device` option to CLI as well
2026-05-09 04:12:02 +02:00
Robert Sachunsky
c1b6a61301 standalone enhancer: make this work (at all)…
- re-use Eynollah base class, drop copied code
- write usable `run()` and `run_single()`
- delegate to `resize_image_with_column_classifier()`
  for column classifier, resizing and enhancement,
  instead of `resize_and_enhance_image_with_column_classifier()`
  (which does _not_ actually enhance)
- drop unused `predict_enhancement()`
- add defaults to `num_col` options (always numeric)
- add `--device` option to CLI as well
2026-05-09 02:55:01 +02:00
Robert Sachunsky
d63ce5538c resize_image_with_column_classifier(): apply num_col bounds here too
use rules from `resize_and_enhance_image_with_column_classifier()`
and apply them to `resize_image_with_column_classifier()` as well

(to be used by enhancer CLI)
2026-05-09 02:53:04 +02:00
Robert Sachunsky
6df2144c0f fix 2 typos in previous commits…
- becf031c65
- cefe596f8b
2026-05-09 02:31:22 +02:00
Robert Sachunsky
daf0c90d6e
Merge pull request #8 from bertsky/ro-fixes-training-reload
training: reload models
2026-05-08 18:46:43 +02:00
Robert Sachunsky
ea8f985ff1 apply cropping only after textline and early layout…
(because old models seem to fare better that way,
 despite training documentation)
2026-05-08 18:41:47 +02:00
Robert Sachunsky
58afdf5e87 do_prediction*(): ensure always returns dtype=uint8 2026-05-08 17:36:31 +02:00
Robert Sachunsky
68a26a5c3f do_prediction*(): smooth window transitions with sigmoid…
instead of hard cut-offs between overlapping window tiles,
apply sigmoid attenuation to slide from one to the next

(apply all postprocessing in the end)
2026-05-08 05:18:00 +02:00
Robert Sachunsky
cefe596f8b do_prediction*(): avoid unnecessary tiles, simplify…
- calculation for number of tiles: sometimes one less
  tile is needed by making the previous last tile
  half-full on the right side
- calculation of window margins: fix case if dimension
  extends to full image shape
- simplify (identifiers, slicing etc)
2026-05-08 00:55:18 +02:00
kba
a0bf1b51f4 makefile to reload models 2026-05-07 19:30:29 +02:00
kba
2747385f89 remove unused deprecating-warning-causing biopyton dependency 2026-05-07 17:15:15 +02:00
Robert Sachunsky
d8c83d6137 make_valid(): avoid oversimplification, improve parameter search 2026-05-05 15:00:16 +02:00