evaluations.run() in endpoint-connection mode.
This Service is exposed by the galtea.jobs object.
Remember that we will be using the
galtea object. More information here.What is a Job?
When you callevaluations.run() without an agent, Galtea queues an inference batch and returns a jobId. The platform then calls your endpoint for every test case and evaluates the results asynchronously. The Jobs Service gives you visibility into that background process and lets you cancel it if needed.
A job moves through the following states:
| State | Meaning |
|---|---|
waiting | Queued and waiting for a worker |
delayed | Scheduled to run in the future |
prioritized | Promoted ahead of normal queue items |
waiting-children | Waiting for sub-jobs to complete |
active | Currently being processed by a worker |
completed | Finished successfully |
failed | Finished with an error |
cancelled | Cancelled by a user call to cancel() |