qurator-spk/neat
1
0
Fork 0
mirror of https://github.com/qurator-spk/neat.git synced 2025-06-09 11:49:54 +02:00
Code Issues Releases Wiki Activity

Update Preprocessing.md

Browse source
This commit is contained in:
Clemens Neudecker 2019-11-20 18:54:01 +01:00 • committed by GitHub
parent e3051b3108
commit 5ecc96864d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/Preprocessing.md
View file

@ -22,7 +22,7 @@ OCR is based on [OCR-D](https://github.com/OCR-D)'s [ocrd_tesserocr](https://git
### Tokenization ### Tokenization
A simple Python tool is used for the [transformation](https://github.com/qurator-spk/neath/tree/master/tools) of [PAGE-XML](https://github.com/PRImA-Research-Lab/PAGE-XML) to [TSV](https://github.com/qurator-spk/neath/blob/master/docs/User_Guide.md#data-format). A simple [Python tool](https://github.com/qurator-spk/neath/tree/master/tools) is used for the [transformation](https://github.com/qurator-spk/neath/tree/master/tools) of [PAGE-XML](https://github.com/PRImA-Research-Lab/PAGE-XML) to [TSV](https://github.com/qurator-spk/neath/blob/master/docs/User_Guide.md#data-format).
``INPUT ``: [PAGE-XML](https://github.com/PRImA-Research-Lab/PAGE-XML) file with bounding boxes for words and the contained text ``INPUT ``: [PAGE-XML](https://github.com/PRImA-Research-Lab/PAGE-XML) file with bounding boxes for words and the contained text
@ -30,8 +30,8 @@ A simple Python tool is used for the [transformation](https://github.com/qurator
Some postprocessing is then applied to the derived [TSV](https://github.com/qurator-spk/neath/blob/master/docs/User_Guide.md#data-format) file: Some postprocessing is then applied to the derived [TSV](https://github.com/qurator-spk/neath/blob/master/docs/User_Guide.md#data-format) file:
* replace ``„`` and ``“`` with ``"`` * replace ``„`` and ``“`` with ``"``
* detect sentence boundaries and mark them with a leading ``0`` * detect sentence boundaries and mark them with a newline and leading ``0``
* detect punctuation and split it from the adjacent string * detect punctuation (``.``, ``,``, ``;``, ``:``, ``!``, ``?``) and split it from the adjacent string
### Named Entity Recognition ### Named Entity Recognition