Lycophron

Lycophron is a CLI tool to support batch uploads of records to Zenodo.

The tool supports the upload through CSV files that describe each record to be uploaded.

"The" Quickstart Guide

1. Installation

   Install from GitHub using pip:

   pip install --user "lycophron @ git+https://github.com/plazi/lycophron@main"

   Alternatively, use pipx:

   pipx install "lycophron @ git+https://github.com/plazi/lycophron@main"

2. Initalize a local project named "monkeys"

   lycophron init monkeys

   List the project contents:

   cd monkeys/
   
   tree
   .
   ├── lycophron.cfg
   ├── lycophron.db
   ├── files
   │   └── Adam_Hubert_1976.pdf
   └── dev_logs.log

   You will be prompted to input the authentication token. You can leave it empty as this can be done afterward by directly editing the configuration file.

3. Configure the project (add or edit TOKEN and ZENODO_URL)

   cat lycophron.cfg
   
   # TOKEN = 'CHANGE_ME'
   # ZENODO_URL = 'https://zenodo.org/api'

4. Create a CSV file from a template

   Generate a template with all the fields:

   lycophron gen-template --filename output.csv --all

   Add custom fields (e.g. dwc and ac) to the template:

   lycophron gen-template --filename output.csv --custom "dwc,ac"

5. Fill in the metadata and load the file

   lycophron load --inputfile output.csv

6. Publish to Zenodo

   lycophron publish

Getting started

• Installation
• Commands
• Configuration
• Metadata
• How to create a CSV
• Development
• Known issues

Installation

Requirements

• Python v3.11+

Install from GitHub using pip or pipx:

pipx install "lycophron @ git+https://github.com/plazi/[email protected]

Note

In the future Lycophron will be published on PyPI and just require pip install lycophron.

Linux/macOS

It is recommended to use a virtual environment to install and run the application.

To create a virtual environment named lycophron, run the following command in your terminal:

python3 -m venv lycophron
source lycophron/bin/activate

To install the CLI tool for development, clone this repository and run:

# For local development
uv pip sync requirements-dev.txt
uv pip install -e .[dev]

Commands

init

This command initializes the project, creates necessary configuration files, and sets up a local database.

init [name] : initialize the app with the given name init --token : initialize the app with a token

⚠️ Adjust configurations in the generated files to meet specific upload requirements.

validate

This command validates the current project. I.e. it checks whether the application is properly configured, the directory structure is correct and the metadata is valid and ready to be loaded locally.

validate --filename : validate the project and the given file.

gen-template

This command generates a CSV file from the default template, containing all the required headers. The template can either be generated with all the fields or by explicitely adding custom fields on top of the required ones.

gen-template --filename : creates the output file in the given path and name. gen-template --all : creates a template using all fields (required and custom fields). gen-template --custom "x,y,z" : creates a template using the required fields plus the given custom fields.

load

This command loads records from a local file into the local Database, ensuring they are ready to upload to Zenodo.

load --inputfile: load records from a given file to a local DB

publish

Publishes the previously loaded records to Zenodo.

This command specifically targets records that are currently unpublished. Importantly, this operation is designed to be executed multiple times, allowing for a phased or incremental approach to publishing records as needed.

publish: publish records to Zenodo

Configuration

name	description
TOKEN	Token to authenticate with Zenodo
ZENODO_URL	URL where to publish records (e.g. https://zenodo.org/api/deposit/depositions)

Supported metadata

name	cardinality	data type	description
description	1	string	record's description
creators	1-N	list of strings	-
title	1	string	-
keywords	0-N	list of strings	-
access_right	0-1	string	-
upload_type	0-1	string	-
publication_type	0-1	string	-
publication_date	0-1	ISO8601-formatted date string	-
journal_title	0-1	string	-
journal_volume	0-1	string	-
journal_issue	0-1	string	-
journal_pages	0-1	string	-
communities	0-N	list of strings	-
doi	0-1	string	-
files	0-N	list of strings	name of the files to upload for the record

How to create a .csv

• Generate the template by running lycophron gen-template and fill in the metadata.
• File names must match the files under the directory /files/

Tip

We also provide a sample generated sheet import template with some pre-filled values.

Warning

When working with fields defined as a list in the CSV file, it is essential to separate each item with a new line ("\n"). This ensures proper formatting and accurate representation of the list structure in the CSV file, thus allowing Lycophron to parse the values correctly.

Example:

title	description	access_right	upload_type	communities	publication_type	publication_date	journal_title	journal_volume	journal_issue	journal_pages	doi	creators.name	creators.affiliation	creators.orcid	keywords	files	id	dwc:eventID	related_identifiers.identifier	related_identifiers.relation
LES NYCTERIDAE (CHIROPTERA) DU SÉNÉGAL: DISTRIBUTION, BIOMETRIE ET DIMORPHISME SEXUEL	Cinq especes de Nycteridae sont preserves au Sénegal	open	publication	bats_project	article	1976-12-31	Mammalia	40	4	597-613	10.1515/mamm.1976.40.4.597	Adam, F. Hubert, B.				Adam_Hubert_1976.pdf	specimen001		{doi:figure001} {doi:figure002} {doi:figure003}	Documents Documents Documents
New ecological data on the noctule bat (Nyctalus noctula Schreber, 1774) (Chiroptera, Vespertilionidae) in two towns of Spain	The Iberian Peninsula represents the South-western limit of distribution of thenoctule bat (Nyctalus noctula) in Europe.	open	publication	bats_project	article	1999-01-31	Mammalia	63	3	273-280	10.1515/mamm.1999.63.3.273	Alcalde, J. T.	Departamento de Zoología, Faculdad de Ciencias, Universidad de Navarra. Avda Irunlarrea s/n. 31080, Pamplona. Spain			Alcade_1999.pdf	figure001		{doi:speciment001}	isDocumentedBy
ROOSTING, VOCALIZATIONS, AND FORAGING BY THE AFRICAN BAT, NYCTERIS THEBAICA	There is no abstract	open	publication	bats_project	article	1990-05-21	Journal of Mammalogy	71	2	242-246	10.2307/1382175	Aldridge, H. D. J. N. Obrist, M. Merriam, H. G. Fenton, M. B.

Known issues

• DOIs are generated on demand, existing or external DOIs are not accepted. issue
• Metadata is compliant with legacy Zenodo, not RDM
• Output is not clear for the user. E.g. user does not know what to run next, how to fix issues with the data, etc.
• init does not create the needed file structure, still requires manual intervention (e.g. creation of ./files/)
• Linking between rows is not supported. E.g. row 1 references a record from row 2

Development

Dependency management

To manage Python dependencies, Lycophron uses uv.

Generate dependency files

You should not need to do this. More often, you will want to just bump all dependencies to their latest versions. See upgrading dependencies

# Generate requirements.txt
uv pip compile pyproject.toml > requirements.txt

# Generate requirements-dev.txt
uv pip compile pyproject.toml --extra tests > requirements-dev.txt

Add a new dependency

To add a new dependency, add it in the dependencies section of the pyproject.toml file. Add testing/development dependencies under [project.optional-dependencies] section's tests extra.

# Update the requirements.txt file
uv pip compile pyproject.toml --upgrade-package <new-package>

# Update the requirements-dev.txt file
uv pip compile pyproject.toml --extra tests --upgrade-package <new-package>

Upgrade all dependencies

To upgrade all dependencies in development, run the following command in your terminal:

uv pip compile pyproject.toml --extra tests --upgrade

and for production:

uv pip compile pyproject.toml --upgrade