GuidesCreate a response

Structured output

Return typed JSON from Jockey by providing a JSON Schema. Use structured output when downstream code needs predictable fields, when agents or services need typed responses, or when you want to extract structured metadata from a video collection. This guide covers how to define a schema, attach it to a request, and parse the structured response.

Key concepts

  • JSON Schema: A standard for describing the structure of JSON data. You define the expected shape - properties, types, arrays - and Jockey constrains its output to match. Pass your schema in the text.format request field.

Prerequisites

  • You’ve already uploaded your content, and the asset has reached the ready status. See the Upload content page for details.
  • You’ve already created a knowledge store. See the Create a knowledge store page for details.
  • You’ve already added at least one asset to the knowledge store, and the item has reached the ready status. See the Add assets to a knowledge store page for details.
  • You’ve already read the Create a response page and understand the basic request and response format.

Basic usage

Define a JSON Schema describing the output you want, then pass it in the text field. This example extracts themes and entities from a video collection.

1import requests
2import json
3
4HEADERS = {"x-api-key": "YOUR_API_KEY", "Content-Type": "application/json"}
5BASE_URL = "https://api.twelvelabs.io/v1.3"
6
7response = requests.post(
8    f"{BASE_URL}/responses",
9    headers=HEADERS,
10    json={
11        "model": "jockey1.0",
12        "input": [
13            {"type": "message", "role": "user", "content": "List the main themes and key entities in these videos"}
14        ],
15        "knowledge_store_id": "your_store_id",
16        "text": {
17            "format": {
18                "type": "json_schema",
19                "name": "themes_and_entities",
20                "schema": {
21                    "type": "object",
22                    "properties": {
23                        "themes": {
24                            "type": "array",
25                            "items": {"type": "string"}
26                        },
27                        "entities": {
28                            "type": "array",
29                            "items": {
30                                "type": "object",
31                                "properties": {
32                                    "name": {"type": "string"},
33                                    "type": {"type": "string"},
34                                    "frequency": {"type": "string"}
35                                }
36                            }
37                        }
38                    }
39                }
40            }
41        }
42    }
43)
44
45result = response.json()
46for output in result["output"]:
47    if output["type"] == "message":
48        for content in output["content"]:
49            structured = json.loads(content["text"])
50            print(f"Themes: {structured['themes']}")
51            for entity in structured["entities"]:
52                print(f"  {entity['name']} ({entity['type']})")

The response text is a JSON string that matches your schema:

1{
2  "themes": ["outdoor recreation", "team building", "nature conservation"],
3  "entities": [
4    {"name": "John Rivera", "type": "person", "frequency": "3 videos"},
5    {"name": "Yellowstone", "type": "location", "frequency": "2 videos"}
6  ]
7}

Combine with instructions

Use instructions to apply a domain-specific lens and a schema to capture the results in a structured format. This example enriches videos through a brand strategy perspective.

1enrichment_schema = {
2    "type": "object",
3    "properties": {
4        "enrichments": {
5            "type": "array",
6            "items": {
7                "type": "object",
8                "properties": {
9                    "video_reference": {"type": "string"},
10                    "original_summary": {"type": "string"},
11                    "enriched_analysis": {"type": "string"},
12                    "new_insights": {"type": "array", "items": {"type": "string"}},
13                    "tags": {"type": "array", "items": {"type": "string"}}
14                }
15            }
16        }
17    }
18}
19
20response = requests.post(
21    f"{BASE_URL}/responses",
22    headers=HEADERS,
23    json={
24        "model": "jockey1.0",
25        "instructions": "You are a brand strategist. Analyze videos through the lens of brand perception and marketing effectiveness.",
26        "input": [
27            {
28                "type": "message",
29                "role": "user",
30                "content": "For each video, provide enriched analysis focusing on brand messaging effectiveness, audience engagement signals, and production quality."
31            }
32        ],
33        "knowledge_store_id": "your_store_id",
34        "text": {"format": {"type": "json_schema", "name": "enrichment", "schema": enrichment_schema}}
35    }
36)

Swap the instructions to enrich for different contexts - the schema stays the same:

ContextInstructions
Accessibility”Analyze for accessibility: describe visual elements for screen readers, note caption quality, identify audio-only content”
Compliance”Review for regulatory compliance: identify claims, disclaimers, required disclosures”
Education”Analyze pedagogical effectiveness: identify learning objectives, teaching methods, assessment opportunities”
SEO”Extract SEO metadata: keywords, descriptions, suggested titles, topic clusters”

Design a schema for organization discovery

You can use structured output to have Jockey recommend how to categorize your collection. The schema below captures each organization axis with a score, so you can compare approaches programmatically.

1axes_schema = {
2    "type": "object",
3    "properties": {
4        "axes": {
5            "type": "array",
6            "items": {
7                "type": "object",
8                "properties": {
9                    "axis": {"type": "string"},
10                    "description": {"type": "string"},
11                    "expected_groups": {"type": "integer"},
12                    "example_categories": {"type": "array", "items": {"type": "string"}},
13                    "effectiveness_score": {"type": "number"},
14                    "rationale": {"type": "string"}
15                }
16            }
17        },
18        "recommendation": {"type": "string"}
19    }
20}
21
22response = requests.post(
23    f"{BASE_URL}/responses",
24    headers=HEADERS,
25    json={
26        "model": "jockey1.0",
27        "instructions": "You are a content strategist. Analyze the video collection and recommend the most effective ways to organize it. Score each axis by how well it separates content into distinct, useful groups.",
28        "input": [
29            {
30                "type": "message",
31                "role": "user",
32                "content": "What are the top 5 best ways to organize this video collection? Score each from 0-1 based on how cleanly it separates the content."
33            }
34        ],
35        "knowledge_store_id": "your_store_id",
36        "text": {"format": {"type": "json_schema", "name": "organization_axes", "schema": axes_schema}}
37    }
38)

Common pitfalls

  • Response text is a JSON string. You still need to call json.loads() on the text content to get a Python object.
  • Schema must be valid JSON Schema. Invalid schemas cause request errors.
  • Keep schemas simple. Deeply nested schemas may produce less reliable output.

Next steps

  • Streaming - combine streaming with structured output for real-time typed responses
  • Multi-turn sessions - use structured output across a multi-turn conversation
  • Create a response - review the basic request and response format

Jupyter notebook

Download the notebook to run this guide interactively.

API reference