Opened 2 years ago
Last modified 2 years ago
#612 new enhancement
Support for "Task Dependencies" in BibSched
| Reported by: | boltmann | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | BibSched | Version: | |
| Keywords: | Cc: |
Description
For the new module handling videos in Invenio ("BibEncode") we need support for task dependencies in BibSched. There should be a parameter upon task submission that defines an execution dependency on another task that is currently running.
"BibEncode" has a daemon mode that launches multiple other "BibEncode" tasks for transcoding and other stuff defined by a submission.
Let us say that one of these tasks ("A") moves a video file and alters the metadata of the video.
Other "BibEncode" tasks ("B", "C") are going to trancode the video afterwards and extract thumbnails from it.
"B" and "C" cannot be executed before the videos was moved and altered by task "A".
On submission of "B" and "C" a parameter should be settable that means "do not start before task "A" has been executed successfully".
One possible way would be to have the task resubmit itself in case that the task it depends upon was not yet successfully executed.
This is different from the the "postprocess" approach because multiple tasks, possibly in parallel, need to be run.

Additionaly, when the task resubmit itself, it has to choose a time in the future. This could be 60s + max(now, execution time of the tasks it depends on).
In this way the task would be woken up at worst every minute.