Commit graph

1348 commits

Author SHA1 Message Date
vahidrezanezhad
4651000191 debuging input shape + enable finetuning a model 2025-12-15 11:36:09 +01:00
vahidrezanezhad
4fc3ff33cb The cnn-rnn ocr model can be trained now 2025-12-09 17:22:12 +01:00
vahidrezanezhad
84a72a128b cnn-rnn model can be called - model input height and width are dynamic now - data generator is also callable 2025-12-09 15:30:19 +01:00
vahidrezanezhad
59e5a73654 adding cnn-rnn training script 2025-12-08 19:30:57 +01:00
vahidrezanezhad
7bf5e077d9 Restore correct execution of export_textline_images_and_text 2025-12-03 15:40:52 +01:00
vahidrezanezhad
6ac37af2f8 Fix eynollah ocr --help so it works again 2025-12-03 14:11:47 +01:00
vahidrezanezhad
d687d862d6 Restored correct functionality of the extract_only_images mode and cleaned up the argument handling 2025-12-03 12:01:42 +01:00
Robert Sachunsky
9fdae72e96 utils_ocr.return_textline_contour: gen cv2-like contours (w/ ndim=3, as in all other places) 2025-12-03 03:04:46 +01:00
Robert Sachunsky
ad8f8167c2 separate_lines/_vertical: gen cv2-like contours (w/ ndim=3, as in all other places) 2025-12-03 00:58:26 +01:00
Robert Sachunsky
43a95842bd writer: also ensure validity after scaling 2025-12-02 16:35:32 +01:00
kba
51abe9617a log to STDERR not STDOUT 2025-12-02 15:00:33 +01:00
Robert Sachunsky
56e73bf72f deskewing: add a 2nd stage for precision
after selecting the optimum angle on the original
search range, narrow down around in the vicinity
with half the range (adding computational costs,
but gaining precision)
2025-11-28 18:27:58 +01:00
Robert Sachunsky
adcea47bc0 return_boxes_of_images_by_order_of_reading_new: always erode
when passing the text region mask, do not apply erosion only
if there are more than 2 columns, but iff `not erosion_hurts`
(consistent with `find_num_col`'s expectations and making
 it as easy to find the column gaps on 1 and 2-column pages
 as on multi-column pages)
2025-11-28 18:23:59 +01:00
Robert Sachunsky
5a3de3b42d column detection: improve, aided by vseps whenever possible
- `find_number_of_columns_in_document`: retain vertical separators
  and pass to `find_num_col` for each vertical split
- `return_boxes_of_images_by_order_of_reading_new`: reconstruct
  the vertical separators from the segmentation mask and the separator
  bboxes; pass it on to `find_num_col` everywhere
- `return_boxes_of_images_by_order_of_reading_new`: no need to
  try-catch `find_num_col` anymore
- `return_boxes_of_images_by_order_of_reading_new`: when a vertical
  split has too few columns,
  * do not raise but lower the threshold `multiplier` responsible for
    allowing gaps as column boundaries
  * do not pass the `num_col_classifier` (i.e. expected number of
    resulting columns) of the entire page to the iterative
    `find_num_col` for each existing column, but only the portion
    of that span
2025-11-28 18:14:24 +01:00
Robert Sachunsky
4dd40c542b find_num_col: add optional criterion - sum of vertical separators
when searching for gaps between text regions, consider the vertical
separator mask (if given): add the vertical sum of vertical separators
to the peak scores (making column detection more robust if still slighly
skewed or partially obscured by multi-column regions, but fg seps are
present)
2025-11-28 18:07:15 +01:00
Robert Sachunsky
84d10962f3 return_boxes_of_images_by_order_of_reading_new: improve
- when searching for multi-col box makers, pick the right-most
  allowable column, not the left-most
2025-11-28 18:04:12 +01:00
Robert Sachunsky
5abf0c1097 return_boxes_of_images_by_order_of_reading_new: improve
- when analysing regions spanning across columns,
  disregard tiny regions (smaller than half the median size)
- if a region spans across columns just by a tiny fraction,
  and therefore is not good enough for a multi-col separator,
  then it should also not be good enough for a multi-col box
  maker
2025-11-28 17:58:44 +01:00
Robert Sachunsky
b71bb80e3a return_boxes_of_images_by_order_of_reading_new: fix 4abc2ff5
(forgot to also flip `regions_with_separators` if right2left)
2025-11-28 17:57:10 +01:00
Robert Sachunsky
a527d7a10d combine_hor_lines_and_delete_cross_points: improve
- avoid unnecessary `fillPoly` (we already have the mask)
- do not merge hseps if vseps interfere
- remove old criterion (based on total length of hseps)
- create new criterion (no x overlap and x close to each other)
- rename identifiers:
  * `sum_dis` → `sum_xspan`
  * `diff_max_min_uniques` → `tot_xspan`
  * np.std / np.mean → `dev_xspan`
- remove rule cutting around the center of crossing seps
  (which is unnecessary and creates small isolated seps
  at the center, unrelated to the actual crossing points)
- create rule cutting hseps by vseps _prior_ to merging
2025-11-28 17:34:11 +01:00
Robert Sachunsky
5c12b6a851 combine_hor_lines_and_delete_cross_points: simplify and rename
- `x_width_smaller_than_acolumn_width` →
  `avg_col_width`
- `len_lines_bigger_than_x_width_smaller_than_acolumn_width` →
  `nseps_wider_than_than_avg_col_width`
- `img_in_hor` → `img_p_in_hor` (analogous to vertical)
2025-11-28 17:27:12 +01:00
Robert Sachunsky
06cb9d1d31 combine_hor_lines_and_delete_cross_points: fix 1-off px bug
when eroding the vertical separator mask (by slicing),
avoid leaving 1px strips
2025-11-28 17:08:39 +01:00
Robert Sachunsky
38d91673b1 combine_hor_lines_and_delete_cross_points: get external contours
instead of tree without looking at the actual hierarchy

(to prevent retrieving holes as separators)
2025-11-28 16:50:08 +01:00
Robert Sachunsky
ee59a6809d contours_in_same_horizon: fix 5d15941b 2025-11-28 16:17:09 +01:00
kba
b161e33854 🔥 refactor eynollah ocr
.
2025-11-28 15:45:21 +01:00
kba
30f9c695dc move line-gt extraction out of ocr to eynollah-training 2025-11-28 15:12:31 +01:00
kba
951bd2fce6 CI: do not upgrade (now-unpineed) torch 2025-11-28 15:12:31 +01:00
kba
9bcfeab057 💀 remove dead code from eynollah.py 2025-11-28 12:52:28 +01:00
kba
5171e09c2d eynollah.py: fix kwargs to writer 2025-11-28 12:52:28 +01:00
kba
c24cf94bce enforce kwargs for writer.build_... 2025-11-28 12:52:28 +01:00
kba
4aa9543a7d remove more branches after textline_light default true 2025-11-27 11:30:00 +01:00
kba
177d555ded factor out extract_only_images as eynollah extract-images 2025-11-26 21:37:00 +01:00
kba
83e8b289da 🔥 drop light_version/textline_light (now default and implied) 2025-11-26 20:48:22 +01:00
kba
ca83cf934d fix imports from src/cli/cli_*/*_cli 2025-11-26 20:48:14 +01:00
kba
095b36c389 models: split into layout, extra and ocr
layout: Everything not OCR or extra
ocr: trocr/cnnrnn models
extra: obsolete or niche models
2025-11-26 19:49:59 +01:00
kba
000af16a47 🔥 remove torch pinning 2025-11-26 19:23:49 +01:00
kba
e503c1a0b7 drop obsolete multi-model binarization 2025-11-26 18:51:41 +01:00
kba
82266f8234 reorganize cli 2025-11-26 18:51:20 +01:00
kba
5a1900e664 🔥 remove OCR option from eynollah layout 2025-11-26 18:12:03 +01:00
kba
0f410c2e7c disable tf/keras logging on first import 2025-11-26 16:37:54 +01:00
kba
9d9d32daed update OCR-D bindings 2025-11-26 16:20:27 +01:00
kba
103c007368 . 2025-11-26 14:37:00 +01:00
kba
0149147e95 . 2025-11-25 13:45:47 +01:00
Robert Sachunsky
e428e7ad78 ensure separators stay within image bounds 2025-11-16 16:35:18 +01:00
Robert Sachunsky
406288b1fe fixup 72d059f3: forgot to update other writer calls 2025-11-16 16:32:45 +01:00
Robert Sachunsky
028ed16921 adapt ocrd-sbb-binarize 2025-11-15 17:17:37 +01:00
Robert Sachunsky
49ab269e08 fix typos found by ruff 2025-11-15 15:49:51 +01:00
Robert Sachunsky
72d059f3c9 reading order: simplify assignment / counting
- `do_order_of_regions`: simplify aggregating per-box orders
  for paragraphs and headings to overall order passed to
  `xml_reading_order`; no need for `order_and_id_of_texts`,
  no need to return `id_of_texts_tot`
- `do_order_of_regions_with_model`: no need to return `region_ids`
- writer: no need to pass `id_of_texts_tot` in `build_pagexml`
2025-11-15 14:34:12 +01:00
Robert Sachunsky
5a778003fd contour matching for deskewed image: ensure matches for both sides 2025-11-15 14:32:22 +01:00
Robert Sachunsky
3c15c4f7d4 back to rotate_image instead of rotation_image_new for deskewing
(because the latter does not preserve coordinates;
 it scales, even when resizing the image;
 this caused coordinate problems when matching deskewed contours)
2025-11-15 14:29:41 +01:00
Robert Sachunsky
4475183f08 improve rules governing column split
- reduce `sigma` for smoothing of input to `find_peaks`
  (so we get deeper gaps between columns)
- allow column boundaries closer to the margins
  (50 instead of 100 or 200 px, 170 instead of 370 px)
- allow column boundaries closer to each other
  (300 instead of 400 px)
- add a secondary `grenze` criterion for depth of gap
  (relative to lowest minimum, if that is smaller than
   the old criterion relative to lowest maximum)
- for calls to `find_num_col` within parts of a page,
  do allow unbalanced column boundaries
2025-11-14 13:15:09 +01:00