You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Gerber, Mike 100b2a5e6c 🐛 Fix mods:relatedItem with mods:recordIdentifier source=dnb-ppn
mods:relatedItem may have a mods:recordIdentifier with source="dnb-ppn".
This may happen for digitized works that have their original work in
another library and use a DNB PPN there.

Fixes gh-22.
2 months ago
.circleci ✔️ CircleCI: - Python 3.5 + Python 3.10 1 year ago
.vscode ⚙️ Add VSCode settings 1 year ago
qurator 🐛 Fix mods:relatedItem with mods:recordIdentifier source=dnb-ppn 2 months ago
.editorconfig ⚙️ Add .editorconfig 1 year ago
.gitignore 🧹 .gitignore 1 year ago
LICENSE 📝 modstool: Add LICENSE 4 years ago 🎨 Fix link to CircleCI project 11 months ago
requirements-test.txt 🚧 Add support for ALTO Description 1 year ago
requirements.txt 🐛 Fix install on Python 3.10 1 year ago 🎨 Rename the Python package to mods4pandas 11 months ago

Extract the MODS/ALTO metadata of a bunch of METS/ALTO files into pandas DataFrames.

Build Status

mods4pandas converts the MODS metadata from METS files into a pandas DataFrame.

Column names are derived from the corresponding MODS elements. Some domain knowledge is used to convert elements to a useful column, e.g. produce sets instead of ordered lists for topics, etc. Parts of the tool are specific to our environment/needs at the State Library Berlin and may need to be changed for your library.

alto4pandas converts the metadata from ALTO files into a pandas DataFrame.

Column names are derived from the corresponding ALTO elements. Some columns contain descriptive statistics (e.g. counts or mean) of the corresponding ALTO elements or attributes.


mods4pandas /path/to/a/directory/containing/mets_files
alto4pandas /path/to/a/directory/full/of/alto_files


In this example we convert the MODS metadata contained in the METS files in /srv/data/digisam_mets-sample-300 to a pandas DataFrame under mods_info_df.pkl. This file can then be read by your data scientist using pd.read_pickle().

% mods4pandas /srv/data/digisam_mets-sample-300
INFO:root:Scanning directory /srv/data/digisam_mets-sample-300
301it [00:00, 19579.19it/s]
INFO:root:Processing METS files
100%|████████████████████████████████████████| 301/301 [00:01<00:00, 162.59it/s]
INFO:root:Writing DataFrame to mods_info_df.pkl

In the next example we convert the metadata from the ALTO files in the test data directory:

% alto4pandas qurator/mods4pandas/tests/data/alto
Scanning directory qurator/mods4pandas/tests/data/alto
Scanning directory qurator/mods4pandas/tests/data/alto/PPN636777308
Scanning directory qurator/mods4pandas/tests/data/alto/734008031
Scanning directory qurator/mods4pandas/tests/data/alto/PPN895016346
Scanning directory qurator/mods4pandas/tests/data/alto/PPN640992293
Scanning directory qurator/mods4pandas/tests/data/alto/alto-ner
Scanning directory qurator/mods4pandas/tests/data/alto/PPN767883624
Scanning directory qurator/mods4pandas/tests/data/alto/PPN715049151
Scanning directory qurator/mods4pandas/tests/data/alto/749782137
Scanning directory qurator/mods4pandas/tests/data/alto/weird-ns
INFO:alto4pandas:Processing ALTO files
INFO:alto4pandas:Writing DataFrame to alto_info_df.pkl