-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Workflow server #652
Open
bertsky
wants to merge
30
commits into
OCR-D:master
Choose a base branch
from
bertsky:workflow-server
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Workflow server #652
Commits on Dec 1, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 239cf3f - Browse repository at this point
Copy the full SHA 239cf3fView commit details -
add workflow server and API integration…
- add workflow CLI group: - add alias `ocrd workflow process` to `ocrd process` - add new `ocrd workflow server`, running a web server for the given workflow that tries to instantiate all Pythonic processors once (to re-use their API instead of starting CLI each time) - add `run_api` analogue to existing `run_cli` and let `run_processor` delegate to it in `ocrd.processor.helpers`: - `run_processor` only has workspace de/serialization and processor instantiation - `run_api` has core `process()`, but now also enters and leaves the workspace directory, and passes any exceptions - ocrd.task_sequence: differentiate between `parse_tasks` (independent of workspace or fileGrps) and `run_tasks`, generalize `run_tasks` to use either `run_cli` or new `run_api` (where instances are available, avoiding unnecessary METS de/serialisation) - amend `TaskSequence` by `instance` attribute and `instantiate` method: - peek into a CLI to check for Pythonic processors - try to compile and exec, using monkey-patching to disable normal argument passing, execution, and exiting; merely importing and fetching the class of the processor - instantiate processor without workspace or fileGrps - avoid unnecessary CLI call to get ocrd-tool.json
Configuration menu - View commit details
-
Copy full SHA for 0c3d970 - Browse repository at this point
Copy the full SHA 0c3d970View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1cb161c - Browse repository at this point
Copy the full SHA 1cb161cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 63be07d - Browse repository at this point
Copy the full SHA 63be07dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 990857f - Browse repository at this point
Copy the full SHA 990857fView commit details
Commits on Dec 4, 2020
-
run_processor: set fileGrps already during instantiation (as some imp…
…lementations currently expect them in the constructor)
Configuration menu - View commit details
-
Copy full SHA for f4e71a8 - Browse repository at this point
Copy the full SHA f4e71a8View commit details
Commits on Jan 18, 2021
-
Configuration menu - View commit details
-
Copy full SHA for fddb236 - Browse repository at this point
Copy the full SHA fddb236View commit details
Commits on Jan 25, 2021
-
Configuration menu - View commit details
-
Copy full SHA for b4a8bcb - Browse repository at this point
Copy the full SHA b4a8bcbView commit details
Commits on Jan 26, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 6d15084 - Browse repository at this point
Copy the full SHA 6d15084View commit details
Commits on Feb 9, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 6e2e7ff - Browse repository at this point
Copy the full SHA 6e2e7ffView commit details -
Configuration menu - View commit details
-
Copy full SHA for e34b70a - Browse repository at this point
Copy the full SHA e34b70aView commit details
Commits on Mar 9, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 1dd2d54 - Browse repository at this point
Copy the full SHA 1dd2d54View commit details
Commits on May 13, 2021
-
Configuration menu - View commit details
-
Copy full SHA for e637a57 - Browse repository at this point
Copy the full SHA e637a57View commit details
Commits on Jun 9, 2021
-
Configuration menu - View commit details
-
Copy full SHA for e3c992e - Browse repository at this point
Copy the full SHA e3c992eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2949925 - Browse repository at this point
Copy the full SHA 2949925View commit details -
Configuration menu - View commit details
-
Copy full SHA for ccb369a - Browse repository at this point
Copy the full SHA ccb369aView commit details
Commits on Jun 10, 2021
-
workflow server: run multi-processed / queued…
- replace Flask dev server with external uwsgi call - factor out Flask app code into separate Python module which uWSGI can pick up - make uWSGI run given number of workers via multi-processing but not multi-threading, and prefork before loading app (to protect GPU and non-thread-safe processors, and because of GIL) - pass tasks and other settings via CLI options (wrapped in JSON) - set worker Harakiri (reload after timeout) based on number of pages multiplied by given page timeout - add option for number of processes and page timeout
Configuration menu - View commit details
-
Copy full SHA for db14b50 - Browse repository at this point
Copy the full SHA db14b50View commit details
Commits on Jun 11, 2021
-
Configuration menu - View commit details
-
Copy full SHA for e6d61a3 - Browse repository at this point
Copy the full SHA e6d61a3View commit details
Commits on Jun 13, 2021
-
Configuration menu - View commit details
-
Copy full SHA for cac80d6 - Browse repository at this point
Copy the full SHA cac80d6View commit details
Commits on Jun 15, 2021
-
- add `--server` option to CLI decorator - implement via new `ocrd.server.ProcessingServer`: - based on gunicorn (for preforking directly from configured CLI in Python, but instantiating the processor after forking to avoid any shared GPU context) - using multiprocessing.Lock and Manager to lock (synchronize) workspaces among workers - using signal.alarm for worker timeout mechanics - using pre- and post-fork hooks for GPU- vs CPU- worker mechanics - doing Workspace validation within the request
Configuration menu - View commit details
-
Copy full SHA for 6263bb1 - Browse repository at this point
Copy the full SHA 6263bb1View commit details
Commits on Jun 20, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 4b59396 - Browse repository at this point
Copy the full SHA 4b59396View commit details
Commits on Jun 30, 2021
-
Configuration menu - View commit details
-
Copy full SHA for fa1bc37 - Browse repository at this point
Copy the full SHA fa1bc37View commit details
Commits on Sep 29, 2021
-
Configuration menu - View commit details
-
Copy full SHA for fcbcc82 - Browse repository at this point
Copy the full SHA fcbcc82View commit details
Commits on Oct 13, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 8193559 - Browse repository at this point
Copy the full SHA 8193559View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5d48239 - Browse repository at this point
Copy the full SHA 5d48239View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ff1d40 - Browse repository at this point
Copy the full SHA 6ff1d40View commit details
Commits on Nov 10, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 08658f9 - Browse repository at this point
Copy the full SHA 08658f9View commit details
Commits on Jan 17, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 417faf0 - Browse repository at this point
Copy the full SHA 417faf0View commit details
Commits on Feb 8, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 83b10f5 - Browse repository at this point
Copy the full SHA 83b10f5View commit details
Commits on May 4, 2022
-
Configuration menu - View commit details
-
Copy full SHA for d98daa8 - Browse repository at this point
Copy the full SHA d98daa8View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.