We're happy to announce InvenioRDM v1.0!

VERY IMPORTANT, read the full release notes before you install and run the system! The v1.0 label does not mean the system is suitable for production use!

FAQ (do not skip!)

Can I use v1.0 in production?

No. Do not even try, or think it might work out!

When can I use InvenioRDM in production?

The first release of InvenioRDM that you can use in a production system will be the LTS (Long-Term Support) release due in end-July.

Why the v1.0 label?

We follow semantic versioning, and in the coming months we'll be releasing v2.0, v3.0, etc. We use the v1.0 label because we're now in a state where the project partners can start running demo systems to demonstrate the progress of the project to their stakeholders. Furthermore, the project partners will be able to start user testing more widely.

Can I upgrade from v1.0 to v2.0?

Read the upgrade contract below.

Upgrade contract (do not skip!)

The following "contract" is meant to align expectations on how and what you'll be able to upgrade from v1.0 to v2.0.

You MUST expect breaking changes to anything on subsequent releases until the LTS release! REST APIs, programmatic APIs, features, Jinja/React templates, data model, vocabularies, etc.

We will ONLY guarantee that you will be able to upgrade a database created with v1.0 to v2.0. With that, we mean that through a manual, labour intensive and offline process you'll be able to upgrade your database. Basically this boils down to, that we will document the steps you need to apply in order to move your data from v1.0 to v2.0 code. In no way do we promise it will be easy! You MAY need to apply manual changes to records. We of course plan to make this an easy and smooth process for the LTS release, but for now it's not.

What's new?

Now that you've hopefully read all the above and are very scared of installing InvenioRDM in production (you should be!), then let's share a bit of the excitement we have for the release this month.

We have reached a really major milestone for InvenioRDM. We now have a minimal working repository with a minimal acceptable user experience with a very minimal feature set. From now on, we are in a releaseable state! That means that in the coming months we'll be improving the UX, the feature set, the stability and production readiness, and thus we're in a very good position for accomplishing our production-ready LTS release in end-July.

Below you'll find the specific areas we've been working on this month.

Access control

The major new feature this month is the support for minimal access control.

The deposit form now has a new protection widget that allows you to deposit records in the following categories:

• Public records with public files
• Public records with restricted files (with or without embargo)
• Restricted records with restricted files (with or without embargo)
• Public metadata-only records
• Restricted metadata-only records (with or without embargo).

Below are some examples of how this now looks in the deposit form:

Restricted records and files

Restricted records and files are only visible for the owner (the user who deposited the material). This means that e.g. a restricted record will not show up in the search results of other users.

User without access (no files and no edit button visible):

User with access (files and edit button visible):

Note, that the message "Sorry, the files are restricted!" is a temporary message, until we have a solution for the access badge (see next section).

Limitation: Access status

We have temporarily removed the access status badge (see before/after screenshot below) until we have found a suitable solution for the badge. The current badge was not easily transferable to the new access control, and is mixing indications about if you can access a record, and/or if files are available.

Before:

After:

Limitation: Embargo feature

You can apply an embargo period to restricted files or records. Once an embargo period is over the record/files are supposed to be made automatically publicly accessible. In v1.0 however, we were not able to include the background job responsible for doing this automatically.

We'll ship this task in the March v2.0 release.

Limitation: Manage permissions

Currently, only the owner is able to see a restricted record/files. In future releases, we'll of course add the support for managing permissions, so that restricted records and files can be shared with colleagues, groups and entire institutions.

Access backend

Underneath the hood, we have prepared the entire access control system to be able to share access with other users, communities, IP networks, etc. Thus, the backend part for allowing management of permissions is almost done, while we are primarily missing the frontend that allows an end-user to interact with the permission system.

Minimal UX

Disabled unfinished features

We have disabled all unfinished features from the user interface. We have for instance disabled the communities module, the version flip-switch in search results, a button to the advanced search and so on.

The features will be added back, once they have reached sufficient quality to be running in a demo system.

Deposit form fields

We have also removed all deposit form fields from the user interface that were not ready. Each of these fields will be added back progressively as they are finished. Most notably we e.g. had to remove the subjects field for this release.

Field help text

We have added help text to as many fields as possible. Please provide feedback, if they are understandable.

Styling issues

We have fixed a large number of small styling and consistentcy issues. Still some remain, and you're feedback is very much appreciated on anything you find in this respect.

List uploads

The list uploads page now has a nice greeting for first time users:

Also, the overall styling of the page was improved:

Search guide

We have added a guide (taken from Zenodo) on how to perform advanced queries. See it on https://inveniordm.web.cern.ch/help/search.

Release v1.0

We have gone through a lot of quality checking to ensure that we'll be able to upgrade data created with the v1.0 release to the coming v2.0 release. This work has focused on ensuring, for example, proper validation and integrity of the data that makes it into the database.

We have also ensured that all database tables have proper upgrade recipes for our automatic schema upgrade system (Alembic).

Known issues

Landing page

• Preview: The image previewer is not using IIIF and only previews images up to 500kb.

• Preview: If you e.g. upload a .JPG extension file, you can set the default preview for the file, but the list of files will not have a preview button.

• Preview: The caption of the preview box does not change when selecting another file for preview.

Deposit form

• Validation error messages: The form validation error messages are not displaying correctly. Labels are system names, and sometimes [Object (Object)] is displayed for errors.

• Missing files: If files are missing to be uploaded, the publish button is disabled, but there no feedback to the end-user that files are missing.

• Performance: We have seen some slight performance issues when entering information into the form. We're looking into this, to make sure it is not noticeable.

• Affiliations vocabulary: We have not initialized the affiliation vocabulary, and thus there is no auto-completion for affiliations (creators and contributors).

• Publish button: It is not enabled when reloading a page with a valid record - you first have to save the record for the publish button to be enabled.

• Protection: The files restriction in the protection widget only become visible after uploading a file. It should be hidden only for metadata-only records.

• Languages: The language vocabulary search results are not ranked properly for the query. For instance searching for "english" will show less relevant results first.

• Creators: The list of creators in the deposit form does not display the role nor identifier if entered.

Search results

• HTML entities (e.g.  ) in the description fields are not removed from the descriptions displayed in search results, causing them to be printed verbatim.

List uploads

• Facet: The search facet distinguishing between published and drafts on the list uploads page is using system labels instead of human readable labels.

• The title of a search result item is not clickable even though it looks clickable.

Backend

• Embargo records are not automatically published due to the missing background job.

• The REST API JSON serialization has timestamps that are not in UTC, nor do they have timezone information.

Installation

• Some versions of NPM 7 are known to cause issues -- NPM 6.14.5 is recommended.

Install

If you previously installed InvenioRDM, make sure you have the latest Docker image of your choice according to the Python version:

docker pull inveniosoftware/centos7-python:3.6
docker pull inveniosoftware/centos8-python:3.7
docker pull inveniosoftware/centos8-python:3.8

To install:

pip install --upgrade invenio-cli
invenio-cli check-requirements
invenio-cli init rdm
cd my-site
invenio-cli containers start --lock --build --setup

This month's version of invenio-cli is v0.22.0.

To stop the instance without destroying the records that were created, you can just run:

cd my-site
invenio-cli containers stop

To destroy the Python virtualenv, and remove the docker containers, you can run:

cd my-site
invenio-cli destroy

Feedback

First, check if you've hit a known issue.

• Bugs: First check for known issues above, then report them on the Invenio-App-RDM repository on GitHub.
• Feedback: Suggestions for improvements, results of user testing, etc. - please reach out to Lars directly (chat or mail).

Credits

The development work in this release was done by:

• CERN (Lars & Zach)
• Northwestern University (Guillaume)
• TU Graz (Mojib)
• TU Wien (Max)

A special thanks goes to Max and Mojib as well as their institutions TU Graz and TU Wien for joining full-time during the February iteration. This made it possible to reach this month's milestone for the regular team from CERN & Northwestern University!

Take care and stay safe!

Congratulations! Your team has just joined the InvenioRDM open source project and is eager to contribute as much as possible. With a mixture of dedicated developers, resource curators, subject matter experts, and various leadership stakeholders, your team simply needs to know about the available resources and how to get started. Read on!

Developers

Whether you would like to install and customize InvenioRDM, contribute on the main project sprints, or take on development of a module locally to contribute back to the project, there are a few important points of entry to participation:

• Roadmap: Familiarize yourself with the overall project timeline and goals. Stable releases will be made through Summer 2021 on a monthly basis

• Development Roadmap: Get to know the specific milestones for 2021

• Sprintboard: See the planning board for the current sprint

• Documentation: Read the official project documentation

• Discord: Ask questions in the project Discord (chat/video platform) and introduce yourself in either the general or newcomers-help channels. A daily developer standup takes place on Discord at 15:15 UTC

• Discourse: Want to provide feedback, but in a less time-sensitive manner than a chat? Either join or start a thread in the Discourse discussion forum

• Development Setup: Install a development version

• Development Workflows: Learn about the development workflows used

Community Members

All stakeholders at your institution are welcome to contribute to InvenioRDM, whether their contribution is code-based or not. To create a tool to best serve the community, outlining stakeholder requirements will be vital:

• Demo site: Access the latest sandbox version of InvenioRDM here. Create an account to test record creation

• Discourse: Provide feedback on testing here. Use the Feedback category or start a new thread wherever you like

• Metadata Interest Group: This is the first community-focused group organized to help support the InvenioRDM development effort. This group meets about once/month and discusses issues related to the InvenioRDM Metadata Model. To join, either make a post on the Discourse thread or in this dedicated Discord channel

• Conference Calendar: Are you presenting on InvenioRDM at an upcoming conference? Make a note here so other interested members of the community can attend the session, or perhaps join your poster or presentation effort

• Welcome, Help and General Questions: Reach out to Community Manager Sara Gonzales via Discord for general questions or to join the Metadata Interest Group

• Schedule a Demo: Is your site, or a site you know, considering implementing InvenioRDM? Reach out to Community Manager Sara Gonzales via Discord to schedule a one-hour demo and information session about the repository

Everyone

• Project list-serv: Lars will have shared this email address with you upon joining the team. This is the frontline communication tool for the InvenioRDM community. Make this email address a safe sender to receive the latest updates on teleconferences (a.k.a. telecons) and community-based information such as invitations to the Metadata Interest Group meetings. Polls, forms, and additional community tools will be sent to this list-serv

• Telecons (see the list-serv for connection information): All partners of the InvenioRDM community are invited to bi-weekly telecons throughout the year. One telecon per month will focus on development updates, and the other will focus on community-based topics

• InvenioRDM Telecon Schedule 2021

• Blog: You are reading it! The blog will also alternate between development-based and community-based topics. Would you like to contribute a blog post? Sign up here or contact Sara to get on the schedule

• Code of Conduct

• Governance

Welcome to the team, and happy collaborating!

2020 didn't stop us from shipping new releases. 2021 won't either. We’re excited to announce the InvenioRDM Alpha 15 (January release)! Thank you to our team members for their efforts on this release.

What’s new?

This month was primarily dedicated to improving the user experience of the deposit form.

We have made significant improvements to the overall user experience of the deposit form. That said, we still have many smaller UX improvements to make. This work will continue throughout February.

Creators/Contributors

The creators/contributors fields have gotten a complete makeover that significantly reduces the clutter. We have optimized the new creators field for adding basic information (name/affiliation) as well as linking the affiliation field to vocabulary records (e.g. ROR). We postponed the more advanced use case of allowing the user to edit all fields such as identifier scheme and identifiers for affiliations, as this created an overly complex form for the majority of use cases.

The creator field we imagine can be extended in the future with support for importing authors via predefined files (e.g. BibTex, etc.).

One particular feature to highlight is that the name identifiers field now does automatic detection of the identifier scheme. You can now paste an ORCID (e.g. an ORCID URL), and the backend will automatically detect it as an ORCID and normalise the stored value.

Coming Soon

We have decided to hide certain fields that we know need further work. You'll see these as "Coming soon". Also, we have temporarily removed the language field from titles and descriptions as we had last minute issues with these fields.

Minor UX Improvements

• Sticky side bar: The side bar with the save/publish buttons, is now sticky so it will follow you when you scroll down the form.

• Resource type: The drop-down was reduced from two drop-downs to a single drop-down to reduce the number of needed clicks to select a resource type.

• Delete/discard button: There's now a delete button that you can use to delete your draft or discard edits to a published record.

• Button consistency: We have applied consistent styling, wording and and naming to most buttons.

• Errors for inline fields are now consistently displayed.

• Brand color: We have now made it easier to customize the styling of the field groups via a common brand color. Previously, if your header was purple, the field groups would remain blue.

• File warning: The warning displayed under the files section was updated to match Semantic UI styling.

• Field help text: We've added an example of field help text to the Dates field, which will expand to all fields needing it.

• Array fields styling: dates, related identifiers and titles are now properly using the entire form grid from side to side.

• Consistent naming of related work.

• Title: The deposit form now has a title.

Other Changes

• Vocabularies: The vocabularies REST API received some significant backend changes, which means that we can now properly show translated text in auto-completion fields such as languages, license, affiliations, and subjects.

• Extended Date Time Format indexing: We now have support for searching EDTF date ranges properly (contributed by TU Wien).

• Geospatial indexing: We are now able to properly index almost any GeoJSON data type (points, boxes, polygons, etc.) so that they can be searched properly at a later point. They are indexed via their centroid which is computed using the Shapely library (contributed by Cottage Labs).

• Core API methods needed for reindexing of vocabularies was added as well (contributed by EU JRC).

• Prettyprint: If you explore the REST API in a browser, you can now add ?prettyprint=1 to the URL, and the JSON will be nicely formatted for you.

Known Issues

• Discarding a change to an edited record doesn't work.
• Licenses cannot be saved.
• Languages cannot be saved and only Danish and English are currently available.
• License data is missing.
• Error messages at the top of the form are not displayed correctly.

Install

If you previously installed InvenioRDM, make sure you have the latest Docker image of your choice according to the Python version:

docker pull inveniosoftware/centos7-python:3.6
docker pull inveniosoftware/centos8-python:3.7
docker pull inveniosoftware/centos8-python:3.8

To install:

pip install --upgrade invenio-cli
invenio-cli init rdm
cd my-site
invenio-cli containerize --pre
invenio-cli demo --containers

This month's version of invenio-cli is 0.21.0 .

To stop the instance without destroying the records that were created, you can just run:

cd my-site
invenio-cli stop

To destroy the Python virtualenv, and remove the docker containers, you can now just run:

cd my-site
invenio-cli destroy

Feedback

As always, we welcome your feedback. When you provide feedback on Discourse your message should be pre-populated with the classic template (bugs, what worked well, what didn’t work well, wishes for documentation).

Here is the template to give feedback if it’s not automated:

## Bugs

## What worked well

## What didn't work well

## Wishes for documentation

Take care and stay safe!

We’re excited to announce the InvenioRDM Alpha 14 (December release)! Thank you to our team members for their efforts on this release.

What’s new?

Vocabularies support

The focus of the December release was to add support for linking bibliographic records with authority records such as subjects, grants, licenses, languages and more.

This includes:

• Auto-complete on deposit fields
• Facet searches based on vocabulary values
• Support for internationalization on deposit, on facets, etc.
• REST APIs for vocabularies
• Linking of records in the backend.

Deposit form autocompletion

The most visible way to see the vocabularies in action is via the deposit form fields.

The below components were added in the deposit form and they support vocabulary autocompletion:

Languages

You can see the autocomplete search help:

And an example of i18n support, when the instance is in Turkish:

Licenses (preview)

We have added support for both standard and custom licenses. Note that we were not able to fully complete this work, and there are known bugs in the license widget. Most notably you'll see:

• Search field does not allow spaces
• Link not working
• UX issues such as over-long page
• Descriptions for licenses not available.

Subjects (preview)

The subjects field autocompletes from a subjects vocabulary that can host multiple different vocabularies at the same time.

The current version is not yet connected with the API, thus it has some demonstration values instead. Only the values you see in the image below can be selected. Don't save a record with subjects yet either.

Invenio v3.4

Thanks to the hard work of the December Invenio sprint, Invenio v3.4 was released (see more here). The release of Invenio v3.4 was an important step for InvenioRDM. We now depend on stable releases of Semantic UI.

Partial save and error reporting

This release includes support for partial validation on draft creation. It also displays partial saves in the uploads page, for example with "No title" if the title was not included in the draft.

For example, saving an empty draft will tell you precisely what is missing at the top, along with pointers per field (see title at the bottom of the image).

The former error space will update to good (green) once you save a draft that is publication-ready. The publish button will then be enabled.

Install

$ invenio-cli init rdm
$ cd my-site
$ invenio-cli packages lock --pre
$ invenio-cli install --pre
$ invenio-cli services setup
$ pipenv run invenio vocabularies import languages licenses
$ invenio-cli run

Feedback

As always, we welcome your feedback. When you provide feedback on Discourse your message should be pre-populated with the classic template (bugs, what worked well, what didn’t work well, wishes for documentation).

Here is the template to give feedback if it’s not automated:

## Bugs

(note, that there's known issues in this release for license and subjects fields)

## What worked well

## What didn't work well

## Wishes for documentation

Take care and stay safe!

We are proud to announce the release of Invenio v3.4.0.

Head over to our Getting started to see it in action.

Read the full release notes.

What's new in Invenio v3.4?

New UI Framework: Semantic UI

The largest new feature in Invenio v3.4 is the release of Semantic UI as the new UI framework. This works has been under way since we released version 3.0 with a deprecation notice on AngularJS 2.5 years ago.

Semantic UI is now the new default UI framework for Invenio, replacing the existing Bootstrap v3 framework. Most importantly, we have added support in Invenio v3.4 for supporting multiple UI frameworks concurrently. This means that Bootstrap 3 is still supported in v3.4 together with Semantic UI, and that we in the future can support new frameworks.

UI Frameworks evolve very rapidly and we expect that Invenio eventually will have to go through multiple UI frameworks over it’s life-time. Allowing support for multiple frameworks, isolate future changes, and even allow third-parties to develop custom themes for Invenio without impacting the business logic Invenio.

Technically the support for multiple UI frameworks works by having specialised Jinja template loading and themable webassets build systems.

Read more about the new system in User interface styling.

Reorganised documentation

We have reorganised the Invenio documentation, to make it easier for new developers to get started with Invenio.

In particular we now have a documentation overview page on https://inveniosoftware.org/documentation/ that gathers both Framework, RDM and ILS documentation under one umbrella.

We have also reorganised the main Invenio Framework documentation on https://invenio.readthedocs.io/en/latest/

Records improvements

Invenio v3.4 marks the start of some larger improvements we want to make to the overall data flow for records in Invenio. With Invenio v3.4 we are releasing new features for Invenio-Records. These improvements are not being used by other Invenio modules yet, but allows us to ensure the backward compatibility of the changes.

Overall, the changes made to Invenio-Records focuses on having records stored in separate database tables/Elasticsearch indexes and making a simplified programmatic API that is less error prone to use.

The following only describes the new featuers at a high-level.

Dumping/Loading support

Records now supports dumping and loading of records. This will eventually replace the using signals for customising the JSON being indexed, as well as standardising the record being loaded from Elasticsearch, database or other sources.

Type support

Currently, a record only supports native JSON data types. In particular, date/time data types are not supported. We have now added the possiblity to support that certain fields inside your JSON have custom data types (such as datetime).

Extensions

We have added support record extensions as a mechanism for replacing signals. Overall the signals were causing issues, because you often needed signals per type of record you were dealing with. With the extensions, you can now more easily hook into

System fields

A typical issue you would deal with in Invenio is to create a record and a persistent identifier. System fields allows you to easily compose a record type with new features, so that e.g. Record.create(...) will also automatically create a persistent identifer.

Backward compatibility

All changes are backward compatible thus existing source code and stored records will continue to work with the latest release.

Testing improvements

GitHub Actions instead of Travis CI

We have moved almost all our 100+ repositories from using Travis CI as the continues integration system to GitHub Actions. This was due to Travis CI imposing a migration from .org to .com and lowering the total number of resources availble for open source projects. Overall, this meant very long waiting times for pull-request builds as well as PyPI releases. We have therefore migration to GitHub Actions. We would like to thank Travis CI for the service they have provided through out the years to the open source community.

Reprodicible tests

We have developed a new tool Docker-Services-CLI, that helps bring up/down required services such as databases, Elasticsearch, etc for testing. This makes it easier to investigate specific test failures locally, as well as making it easier to migrate between continuous integration systems.

Centralised test dependencies and Pytest-Invenio

With the release of Pytest-Invenio v1.4 we have centralised test dependencies to a single module, and can more easily control breaking changes from third-party packages. Previously new version of isort and pytest have been causing wide-spread test failures due to minor changes.

Minor changes in v3.4

• Invenio-Accounts:

  • Password length validation during login was removed.
  • "Log in" instead of "Sign in" is now being used consistently throughout the application. Previously both version could be found.

• Invenio-App:

  • The /ping endpoint (used by e.g. load balancers for aliveness checks) now also supports HEAD and OPTIONS HTTP verbs as recommended by e.g. HAProxy documentation.

• Invenio-Celery

  • Only Celery to 4.4.x to 5.0.x releases are supported. Note, Celery have declared Celery 4.4.x as a Long Term Support release. Note, that Celery changed command line arguments from celery worker -A <app> to celery -A <app> worker between version 4 and 5.

• Invenio-DB

  • Now support SQLAlchemy <1.4 which include e.g. PostgreSQL 12 support.
  • Hides the database password from CLI output when running db init or db create.

• Invenio-Indexer

  • Elasticsearch delete requests now uses optimistic concurrency control by providing the the version and version_type in delete requests. The previous behavior can restored by calling RecordIndexer().delete(record, version=None, version_type=None) instead. This fixes an issue where deleting and recreating a document right after would fail.

• Invenio-JSONSchemas

  • Fixes an issue related to nested allOf being ignored.

• Invenio-OAuthClient

  • Added support for CERN OpenID contrib.

• Invenio-Records-REST

  • The header Cache-Control: 'no-cache' is now sent on successful HTTP 200 responses to ensure that browsers will not cache responses client side and ETag will be evaluated.

• Invenio-REST

  • Fixes a bug with CSRF checking when the endpoint did not exist.

Deprecations in v3.4

Bootstrap 3

Bootstrap v3 has reached end-of-life and will no longer receive further updates from their maintainers. Migrating to Bootstrap v4 or v5 is as difficult as upgrading to Semantic UI and there are no easy migration paths.

We will continue to support the Bootstrap v3 integration throughout the maintenance period of Invenio v3.4. Invenio v3.5 may move all Bootstrap 3 templates to a separate Invenio package, that you could install, however v3.5 will not add new Bootstrap 3 templates.

You should plan allocating time for a migration from Bootstrap 3 to Semantic UI during 2021

Features removed in v3.4

Elasticsearch v2 and v5 support

We have removed support for Elasticsearch v2 and v5 as announced in v3.3. Elasticsearch v2 and v5 have reached end-of-life and is no longer maintained with security fixes, thus you should strongly consider upgrading in case you have not already done so.

AMD/RequireJS

The old AMD/RequireJS assets build system, that was deprecated and replaced with Webpack in Invenio v3.1 has now been completely removed from the code base.

Maintenance policy

Invenio v3.4 will be supported with bug and security fixes until the release of Invenio v3.6 and minimum until 2021-12-17.

See our Maintenance Policy.

On 17 and 24 November 2020 the InvenioRDM project partners met in two telecons to discuss the Metadata Reference document.

These conversations were a follow-up to the community's review of the document. Feedback and questions were collected in a Google Doc prior to the conversations. The conversations helped to confirm many answers to questions about the data model that had been posed in the Doc, such as: whether ‘role’ is required for Creators (no - only for Contributors), are Resource Types customizable (yes), and which file is shown in the previewer when a record has multiple files (the first, unless otherwise specified). See the Doc for responses to additional questions.

Use Cases Requested

For some fields, further feedback is needed in order to make them as useful for the community as possible. In particular, your use cases are requested for the following scenarios:

1. Adding a Language tag to Titles
2. Records with more than one Publisher (making Publisher a repeatable field)
3. Having a built-in “Unknown” option for Contributor type and/or role
4. Having more than one Identifier per Affiliation

Please post your use cases back to the top of the original Google Doc or to the new Metadata Interest Group topic at: https://invenio-talk.web.cern.ch (more on this below).

Subjects

In addition, the topic of Subjects was brought up for further discussion. The Subjects example in the Metadata Reference document shows the incorporation of one controlled vocabulary, the Medical Subject Headings, into the schema.

However, there are many controlled vocabularies that we can potentially map to the Subjects field. Which will be most useful to incorporate for the wider community, and what are potential use cases?

This is one of the first issues we would like to discuss in the Metadata Interest Group, which is set to launch in January 2021. This group welcomes the contributions and insights of developers, users, administrators, decision-makers, and anyone else at participating organizations with metadata experience, interest, and insights. Through the end of 2020 we will work to publicize the group and recruit members. If you’re interested in joining, please either sign up here: https://invenio-talk.web.cern.ch/t/metadata-interest-group/144 or DM me on Discord (see the Welcome channel).

Community Outreach

We are committed to serving the entire InvenioRDM community as our family of adopters continues to grow and thrive. This is the first, we hope, of many blog posts that will address topics of interest to the community at large, including developers, administrators, curators, and decision-makers. If there is a topic you would like us to explore, or perhaps even contribute a post on yourself, we’d like to know. If the topic would be best covered at a community-focused telecon (also on the horizon for the coming months!), we’d like to know that as well. Please contribute any and all ideas at my contact listed above.

Take care and stay safe!

We’re excited to announce the InvenioRDM Alpha 13 (November release)!

What’s new?

Files support

It is now possible to upload files on the draft form and publish them.

File uploader

When creating a new draft, you can now also upload files that will be included in the published record. The file uploader supports:

• Drag-n-dropping files to be uploaded
• Concurrent uploads (by default up to 4)
• Setting the default preview file
• Completely disabling files to allow for metadata-only records

New REST API

Big part of our efforts went into designing a new files REST API that will allow InvenioRDM to easily support upload workflows depending on third-party storage systems (e.g S3).

You can check out the documentation of these endpoints in our newly added REST API reference.

Files listing and preview

Uploaded files are also listed and can be previewed on the record's landing page, using a variety of extensible previewers. We currently support preview of PDFs, images, ZIP files, CSV tables, and some text-based formats like XML, JSON, and Markdown.

By default, files are listed in alphabetical order and the first previewable file is shown in the preview box. You can specifically set which file you want to be by default previewed in the record's draft form.

Reorganized CLI commands

Thanks to the community's feedback, in this release we have also updated the existing CLI commands, with the goal of improving the development and operational workflows of an InvenioRDM instance. Our newly added CLI reference documentation lists and describes in detail the command groups, their sub-commands, and the options they support.

Make sure to go through our documentation, since many old commands have now new options and different default behavior (e.g. service setup will now also create demo records).

Feedback

As always, we welcome your feedback. When you provide feedback on Discourse your message should be pre-populated with the classic template (bugs, what worked well, what didn’t work well, wishes for documentation).

Here is the template to give feedback if it’s not automated:

## Bugs

## What worked well

## What didn't work well

## Wishes for documentation

Take care and stay safe!

We're happy to announce the InvenioRDM October release. Thank you to our team members for their efforts on this release.

In early October we held a virtual project workshop. As part of the workshop we have updated the roadmap. The roadmap is now publicly available in two versions, one simplified and one for project tracking. See below:

1- Simplified

2- GitHub for project tracking

What's new?

Metadata Schema

The entire metadata schema for bibliographic records has been updated, and is now ready for a thorough review by all project partners. The update includes changes to the JSONSchema, Elasticsearch mappings, the REST API data validation layer as well as addition of many fields to the deposit form.

See (https://inveniordm.docs.cern.ch/reference/metadata/) for for a full reference of the new metadata schema.

User Experience

We have done a lot of work on improving the UX of the primary pages such as the frontpage, search results and record landing page. Below you can see some screenshots of before/after:

Search results

The search results have been tightened up as well. Also, we’ve added support for nested facets, so you can expand broader categories into subcategories (e.g., Publication and Image).

Landing page

The landing page you’ll notice now has Edit and New version buttons (only the Edit button currently works). The right column has been tightened up, and most of the new metadata fields are now properly displayed on the landing page.

Human readable labels

In the facets you’ll notice we now have human readable labels. Instead of 'publication' it will say 'Publication'; instead of 'open' it will say 'Open Access'.

Similarly to search results and landing pages, you’ll now see the correct resource type as well as icons on Open Access.

Localization

Dates, like the publication date which supports Extended Date Time Format is now properly localized as well using the Unicode Common Locale Data Repository. For instance here an English and Turkish localization:

Affiliations

The affiliation display was redone:

Manage section

There’s a new manage section on the record landing page, which allows you to edit a record.

CLI Improvements

We have made it easier to get started with developing InvenioRDM. Partially to make our own developers' life easier, but also to make it easier to customize InvenioRDM.

Shells

You can now easily activate the Python virtualenv shell as well start a Python terminal from your instance using the following commands:

invenio-cli shell
invenio-cli pyshell

Watching assets

We have now simplified how you can change styling via automatic watching of file changes on assets. Previously you had to manually rebuild the assets. Now instead you can simply execute the following commands, and the styling will automatically rebuild once the file changes:

invenio-cli assets --force --development
invenio-cli assets watch

Develop an Invenio module

Developers often need to install the latest development versions of Invenio modules to work on them. This can now easily be done with:

invenio-cli ext module-install ~/src/invenio-app-rdm ~/src/invenio-rdm-records
invenio-cli assets --force --development

Develop a React module

The above works only for Python packages. If instead, you are working on one of our React libraries, you can now easily install and watch the module for changes as well:

invenio-cli assets watch-module --link ~/src/react-invenio-deposit

Install

If you previously installed InvenioRDM, make sure you have the latest Docker image of your choice according to the Python version:

docker pull inveniosoftware/centos7-python:3.6
docker pull inveniosoftware/centos8-python:3.7
docker pull inveniosoftware/centos8-python:3.8

To install:

pip install --upgrade invenio-cli
invenio-cli init rdm
cd my-site
invenio-cli containerize --pre
invenio-cli demo --containers

To stop the instance without destroying the records that were created, you can just run:

cd my-site
invenio-cli stop

To destroy the Python virtualenv, and remove the docker containers, you can now just run:

cd my-site
invenio-cli destroy

Feedback

As always, we welcome your feedback. When you provide feedback on Discourse your message should be pre-populated with the classic template (bugs, what worked well, what didn’t work well, wishes for documentation).

Here is the template to give feedback if it’s not automated:

## Bugs

## What worked well

## What didn't work well

## Wishes for documentation

Take care and stay safe!

We are glad to announce InvenioRDM Alpha 11 (September release)!

What's new?

We now have support for drafts, meaning you can now start a deposit, save it, and come back many days later to publish it.

This simple change marks a very big milestone for the backend of InvenioRDM. During the past months we have been revamping the entire data and control flow of the backend. The flow is what is supporting the core of InvenioRDM - submitting, editing and searching for records - and it determines what is possible and what's not possible in InvenioRDM. These changes enable the customizations needed by partners including customizations to permissions, publishing workflows, metadata fields, indexing and serializations just to name a few.

A high-level overview of the new flow is shown below:

This new flow is implemented in the following Invenio modules:

• Flask-Resources
• Invenio-Records-Resources
• Invenio-Drafts-Resources

Install (TL;DR)

If you previously installed InvenioRDM, make sure you have the latest Docker image of your choice according to the Python version:

docker pull inveniosoftware/centos7-python:3.6
docker pull inveniosoftware/centos8-python:3.7
docker pull inveniosoftware/centos8-python:3.8

To install:

pip install --upgrade invenio-cli
invenio-cli init rdm
cd my-site
invenio-cli containerize --pre
invenio-cli demo --containers

To destroy the Python virtualenv, and remove the docker containers, you can now just run:

cd my-site
invenio-cli destroy

It's the first feature by Rodrigo Almeida, the newest addition to our team!