qurator-spk/dinglehopper
1
0
Fork 0
mirror of https://github.com/qurator-spk/dinglehopper.git synced 2025-07-27 11:19:55 +02:00
Code Issues Releases Wiki Activity

🐛 Fix word segmentation with uniseg 0.8.0

Browse source
This commit is contained in:
Mike Gerber 2024-03-26 19:34:22 +01:00
parent 4b64398cec
commit 4d4ead4cc8
2 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

2
requirements.txt
View file

@ -1,7 +1,7 @@
click click
jinja2 jinja2
lxml lxml
uniseg >= 0.7.2 uniseg >= 0.8.0
numpy numpy
colorama colorama
MarkupSafe MarkupSafe

2
src/dinglehopper/word_error_rate.py
View file

@ -24,7 +24,7 @@ def patch_word_break():
def new_word_break(c, index=0): def new_word_break(c, index=0):
if 0xE000 <= ord(c) <= 0xF8FF: # Private Use Area if 0xE000 <= ord(c) <= 0xF8FF: # Private Use Area
return "ALetter" return uniseg.wordbreak.WordBreak.ALETTER
else: else:
return old_word_break(c, index) return old_word_break(c, index)