We are proud to announce the release of Invenio v3.1.0.
Head over to our Getting started to see it in action.
Invenio v3.1 supports Python 2.7 (until 2019-12-31), Python 3.5 and Python 3.6. We expect to add support for Python 3.7 in the near-term future once Celery v4.3 has been released.
What's new in Invenio v3.1?
Webpack build system
The old build system is still available to allow users to upgrade to Invenio v3.1 without first migrating to Webpack. The old build system will be removed in Invenio v3.3
For more information about the new build system, please see:
We have simplified the scaffolding of new Invenio instance by merging the data model template into the main Cookiecutter-Invenio-Instance.
The previous approach of two separate packages -- one for the application and one for the data model -- caused friction and confusion for new users and we, therefore, decided to merge both.
This also fits with our long-term goal, where we want to provide standard data models (such as DataCite, Dublin Core, MARC21) so that users don't have write their own data model.
Docker base image
We have released a new Docker image that can serve as base image for your Invenio instances. The image is based on CentOS 7 and comes with Python 3.6, Node.JS, NPM and some standard libraries often needed by Invenio.
See inveniosoftware/centos7-python on DockerHub.
In order to manage Python dependencies more reliable and securely for your Invenio instance we have moved to use Pipenv which also handles the virtualenv creation. This has all been integrated with the Getting started guide.
New sections where added to the documentation specifically on:
- Build a repository
- Managing access
- Secure your instance
- Infrastructure architecture
Invenio v3.1 has added new features for improved request tracing to allow for better troubleshooting and auditing of problems. The feature allows logging a request id, session id and user id across multiple services such as Nginx and Invenio error logs. This enables e.g. system administrators to identify exactly which Nginx access log line caused a specific error logged by Invenio.
If combined with e.g. centralised log aggregation, this can be used for e.g. viewing requests by a user in real-time, request performance statistics and many other metrics. Please note that in order to be compliant with EU General Data Protection Regulation (GDPR), you must ensure that these logs are automatically deleted after 3 months (the same is the case if you only log an IP address).
- Nginx configuration has been updated to automatically generate a random
request id and add is as
- Nginx log format has been updated to log timing information, request id,
session id and user id if provided by the application server in the
X-User-IDHTTP headers. Nginx will remove both headers prior to sending the response to the client.
- Nginx configuration has been updated to automatically generate a random request id and add is as
- Extracts the
X-Request-IDheader (max 200 chars) if set in the HTTP request and makes it available on the Flask
- Extracts the
- The request id is made available to all log handlers.
- The Sentry log handler will add the request ID as a tag if available.
X-User-IDHTTP headers will be added to the HTTP repsponse if the configuration variable
ACCOUNTS_USERINFO_HEADERSis set to
True. This makes the session and user id available to upstream servers like Nginx.
Minor changes in v3.1
The token expiration was changed from 5 days to 30 minutes for the password reset token and email confirmation token. Using the tokens will as a side-effect login in the user, which means that if the link is leaked (e.g. forwarded by the users themselves), then another person can use the link to access the account. Flask-Security v3.1.0 addresses this issue, but has not yet been released.
Globus.org OAuth Login
Invenio v3.1 now comes with support for login with your Globus.org account. The feature was contributed by University of Chicago.
See Invenio-OAuthClient for details.
/ping view that can be enabled via the
APP_HEALTH_BLUEPRINT_ENABLED configuration variable has been added to support load balancers like HAProxy to check if the application server is responsive.
Backwards incompatible changes
- Pytest-Invenio: The
celery_configfixture has been renamed to
celery_config_extdue to naming conflict with fixture provided by Celery.
Deprecations in v3.1
Following list of features have been deprecated and will be removed in either Invenio v3.2 or Invenio v3.3:
Elasticsearch v2 support
Elasticsearch v2 support will be removed in Invenio v3.2. Elasticsearch v2 has reached end of life and no longer receives any bug or security fixes.
Both the support in Invenio-Search for creating indexes for v2 as well as any v2 Elasticsearch mappings in other Invenio modules will be removed.
Invenio's assets build system based on AMD/RequireJS will be removed in Invenio v3.3.
This involves e.g. the two CLI commands:
$ invenio npm $ invenio assets build
Several Python modules in Invenio-Assets will be removed, including (but not limited to):
Also, bundle definitions in other Invenio modules will be removed. These are
usually located in
bundles.py files, e.g.:
Also, some static files will be removed from bundles, e.g.:
invenio_access.DynamicPermission class will be removed in Invenio v3.2. It has been superseded by the
invenio_access.Permission class. The
Permission class by default deny an action in case no user/role is assigned. The
DynamicPermission instead allowed an action if no user/role was assigned.
The following CLI commands will be removed in Invenio v3.2:
$ invenio records create $ invenio records delete $ invenio records patch
Please use the REST API instead to create, patch and delete records.
AngularJS (reminder from v3.0)
In Invenio v3.0 we deprecated the AngularJS 1.4 application Invenio-Search-JS as AngularJS by that time was already outdated. We have selected React and SemanticUI as the replacement framework for AngularJS.
The new Webpack build system released in Invenio v3.1 is part of the strategy to move from AngularJS to React (note however that you can use Webpack with your favorite framework, including AngularJS).
We have started the rewrite of Invenio-Search-JS and have already released the first version of React-SearckKit which eventually will replace Invenio-Search-JS.
Features removed in v3.1
These following already deprecated features have been removed in Invenio v3.1.
invenio_records.taskswas removed from the Invenio-Records module.
Invenio v3.1 will be supported with bug and security fixes until the release of Invenio v3.3 and minimum until 2020-03-31.
We originally planned to release the Files and Statistics bundle in Invenio v3.1. We however decided it was more urgent to release the new Webpack build system in order to avoid too much code being written against the old build system.
In Invenio v3.2 we are planning to release the Files bundle including:
- Object storage REST API for Invenio with many supported backend storage protocols and file integrity checking.
- International Image Interoperability Framework (IIIF) server for making thumbnails and zooming images.
- Previewer for Markdown, JSON/XML, CSV, PDF, JPEG, PNG, TIFF, GIF and ZIP files.
- Integration layer between object storage and records.