Inference workflows overview

An inference workflow is a small Python program (workflow.py) that Labelty hosts for you. It loads one or more registered project models, runs custom logic on one image per request, and returns predictions as JSON over HTTPS.

It is the product path for “give me an inference API.”

Diagram: Client → Labelty POST /workflows/{id}/run → runner loads workflow + models → JSON predictions (+ optional Raw Data route)

What a workflow is (and is not)

It is	It is not
An orchestrator in front of model inference.py handlers	A replacement for the human labeling pipeline
A synchronous HTTP API on Labelty’s domain	A separate public hostname per workflow (not shipped)
Optional routing of results into project Raw Data	Async job history / webhooks / batch / streaming (not shipped yet)

Auto-labeling vs workflows

	Auto-labeling	Inference workflow
Goal	Annotate assets already in the project	Serve predictions to external systems
Trigger	In-app / internal async job	POST …/workflows/{id}/run
Code	Model package only	Model package plus workflow.py
Response	Annotations written on assets	JSON to the caller

Prerequisites

1. A project with at least one registered ML model that has an inference package
2. Permissions:
   • Deploy / upload / activate: ml.model.deploy
   • Run: ml.predictions.write
3. Auth: session or project API key lty_…
4. Plan features:
   • Active subscription
   • Activating a workflow counts against model deployment endpoints quota
   • External API access must be enabled for your plan when using project keys
5. Wallet balance for PAYG inference billing
6. Platform ops must have configured a runner (LABELTY_INFERENCE_SERVER_URL or workflow Lambda) — if missing, runs return 503

High-level user journey

1. Write workflow.py (WorkflowBase subclass)
2. Create workflow deployment row (deploy_config binds model UUIDs)
3. Upload workflow.py (presign → PUT → confirm)
4. Activate the workflow
5. Call POST /api/v1/workflows/{id}/run with image_base64 or image_s3_key

You can do steps 2–5 in the Labelty UI (Create Workflow) or via REST.

Current limitations (accurate)

Documented as not available on the current product slice:

• Dedicated per-workflow public hostname
• Batch / streaming endpoints
• Webhook or email outputs
• Async run IDs / polling history API
• SageMaker as a user-selectable workflow runner (removed from master; do not rely on it)

Cold starts can take on the order of ~60 seconds; client snippets use a 120s timeout.

Next step

Create and activate a deployment: Create and deploy.