Opened 3 years ago

Last modified 3 years ago

#612 new enhancement

Support for "Task Dependencies" in BibSched

Reported by: boltmann Owned by:
Priority: major Milestone:
Component: BibSched Version:
Keywords: Cc:

Description

For the new module handling videos in Invenio ("BibEncode") we need support for task dependencies in BibSched. There should be a parameter upon task submission that defines an execution dependency on another task that is currently running.

"BibEncode" has a daemon mode that launches multiple other "BibEncode" tasks for transcoding and other stuff defined by a submission.
Let us say that one of these tasks ("A") moves a video file and alters the metadata of the video.
Other "BibEncode" tasks ("B", "C") are going to trancode the video afterwards and extract thumbnails from it.
"B" and "C" cannot be executed before the videos was moved and altered by task "A".
On submission of "B" and "C" a parameter should be settable that means "do not start before task "A" has been executed successfully".

One possible way would be to have the task resubmit itself in case that the task it depends upon was not yet successfully executed.

This is different from the the "postprocess" approach because multiple tasks, possibly in parallel, need to be run.

Change History (1)

comment:1 Changed 3 years ago by skaplun

Additionaly, when the task resubmit itself, it has to choose a time in the future. This could be 60s + max(now, execution time of the tasks it depends on).

In this way the task would be woken up at worst every minute.

Last edited 3 years ago by skaplun (previous) (diff)
Note: See TracTickets for help on using tickets.