# Merge Studio — complete reference for building an app on it Merge Studio (by FuturizeNow) makes finished videos and interactive lessons on request. Your app sends a short request; Studio writes, pictures, voices and renders; your app collects the result. This document is everything an app builder (or an AI app builder such as Lovable) needs. Always current at: https://studio.futurizenow.co/llms-full.txt · index: https://studio.futurizenow.co/llms.txt ## 1. The rules that make it work - Base URL: https://studio.futurizenow.co - Every call sends the account's secret key: `Authorization: Bearer mk_live_…` - The key is made by the account owner in Studio → Developers (https://studio.futurizenow.co/studio#developers). It is shown once. Store it as a server-side secret (for a Lovable app: a Supabase secret named `STUDIO_API_KEY`). NEVER put it in browser code — every call to Studio is made from a server function (Supabase Edge Function), never from the page. - Making for your own customers: add `X-On-Behalf-Of: ` to every request you make for a customer. Allowed characters: letters, digits, `_ . @ + -`, up to 120. The film is labelled with that customer (`endUser` on the job), you can list one customer's films, and usage is split per customer. Your account is billed for all of them, on one bill. Your customers never need a Studio account. - Send `Idempotency-Key: ` on every POST that makes something. A repeat of the same request (double click, retry after a timeout) returns the SAME job instead of making and charging twice. Remembered for 24 hours. - Everything is asynchronous: a POST returns a job id at once (HTTP 200/202). You then check the job, or ask to be told (webhook). - Finished film files are kept for 24 hours. Download them (with the key) and store your own copy. Lessons (interactive courses) are kept. - JSON in, JSON out. Errors are always `{ "code", "error", "requestId" }`; show `error` to people as written, branch on `code`, log `requestId`. ## 2. A helper for Supabase Edge Functions (Deno) ```ts // supabase/functions/_shared/studio.ts const BASE = "https://studio.futurizenow.co"; const KEY = Deno.env.get("STUDIO_API_KEY")!; export async function studio(path: string, opts: { method?: string; body?: unknown; customer?: string; idempotencyKey?: string } = {}) { const headers: Record = { Authorization: `Bearer ${KEY}` }; if (opts.body !== undefined) headers["Content-Type"] = "application/json"; if (opts.customer) headers["X-On-Behalf-Of"] = opts.customer; if (opts.idempotencyKey) headers["Idempotency-Key"] = opts.idempotencyKey; const res = await fetch(BASE + path, { method: opts.method ?? (opts.body ? "POST" : "GET"), headers, body: opts.body ? JSON.stringify(opts.body) : undefined }); const data = await res.json().catch(() => ({})); if (!res.ok) throw new Error(`${data.error ?? "Studio error"} (code ${data.code ?? res.status}, request ${data.requestId ?? "?"})`); return data; } // Finished files need the key too: download, then save to Supabase Storage. export async function studioFile(url: string): Promise { const res = await fetch(url, { headers: { Authorization: `Bearer ${KEY}` } }); if (!res.ok) throw new Error(`download failed (${res.status})`); return await res.blob(); } ``` ## 3. Make a film from a brief (the main route) `POST /v1/direct` — describe the film; Studio writes the script, makes the pictures, voices it and cuts it. ```json { "brief": "A 30-second trailer for 'The Salt Road', a literary novel about a family of salt traders. For readers on Instagram. Warm, unhurried. End on the title and 'Out now'.", "title": "The Salt Road", "preset": "9:16", "target": { "seconds": 30 }, "generateVisuals": true, "tone": "warm, unhurried", "webhook": { "url": "https://.supabase.co/functions/v1/studio-webhook" } } ``` | field | required | meaning | |---|---|---| | brief | yes | who the film is for, what it says, how it should feel | | title | no | the film's title, used exactly on the title cards | | preset | no | `9:16` vertical (Reels/TikTok/Shorts), `1:1` square, `16:9` wide (default) | | target.seconds | no | length of the finished film | | generateVisuals | no | `true`: Studio makes the pictures. Leave out when you send your own in `assets` | | assets | no | your media: `[{ "id": "cover", "kind": "image", "src": "", "note": "the book cover" }]` — kind is image, video, text or music | | tone, avoid | no | a tone hint; things the film must never show | | narration | no | `false` for no voice-over | | theme, voiceId | no | look and narrator; lists at `GET /v1/themes`, `GET /v1/voices`. Leave theme out (or send `"auto"`) and Studio picks the look that fits the brief — bright for a kids' lesson, dark only when the subject is; the job shows it as `look: { theme, by }` | | score | no | the engine's own synthesised bed: `warm`, `calm`, `bright`, `serious`, `tender`, or `false` for none. Leave it out and one is chosen from the tone and look | | music | no | music is optional — see the table below. Your own `music` always wins over `score` | | webhook.url | no | Studio POSTs the finished job here | Reply: `{ "jobId": "…", "status": "queued", "statusUrl": "…/jobs/" }`. A film usually takes 3–6 minutes. ### Music (optional) **`POST /v1/render` adds no music unless you ask for it** (`music` or `score` in the timeline). Only a brief (`POST /v1/direct`, this section) chooses music for you when you say nothing. Saying nothing lets Studio choose: it writes its own background score for the film (no rights holder, nothing to credit) — and once the music pack holds a track that fits the brief, it picks that track instead. The job's `musicPick` always says what was chosen and why. To choose yourself, send `music` on `/v1/direct` or on a `/v1/render` timeline: | `music` | what the film gets | |---|---| | `false` | no music at all. Always honoured | | `{ "track": "" }` | that track from the music pack (`GET /v1/music` lists them) | | `{ "mood": "calm" }` | a pack track of that mood — `warm`, `calm`, `bright`, `serious` or `tender`. If the pack has none, Studio's own score of the same mood | | `"auto"` | Studio reads the brief, picks the mood, then a track for it. It never picks a track that needs a credit unless you add `"allowCredit": true` | | `{ "src": "" }` | your own music, mixed exactly as before | - **The pack ships empty today.** Until tracks are added, every film still gets the engine's own score — a mood or `"auto"` lands on the score of that mood. Do not promise your users named tracks yet; check `GET /v1/music`. - **Only known words start music:** `"auto"`, the five moods, a pack track, or a `shape`. Anything else ("none", "off", "false", an unknown word like "jazzy", a track that isn't in the pack) gives a film with no music, exactly as before; an unknown word or missing track is named back in an advisory (`music-word-not-understood`, `music-track-not-found`). Out-of-range settings are ignored with an advisory (`music-setting-ignored`), not refused. - **Shape:** `music.shape` is `"bed"` (under the whole film, the default) or `"bookends"` — music only at the start and the end, **nothing in the middle**: the track's opening for `introSeconds` (default 6, 2–20) with a soft 2.5 s tail, and its ending for the last `outroSeconds` (default 8, 3–30), faded out on the last frame. A film under 20 s, or under twice the two windows, gets one quiet bed instead and the job says so. Send no shape and a rule picks it: bookends for narrated films of 120 s or more, otherwise bed. - **Your own file and the score are mixed exactly as before.** Films whose sound is only music now land on -16 LUFS like every other film (they used to be about 1 dB louder). - **Fades:** a pack track comes in over 2 s at the first frame and leaves over 3 s at the last. Change them per film with `music.fadeIn` (0.1–8 s) and `music.fadeOut` (0.1–10 s); shorter on films under 12 s. - **Level:** every track is level-matched when it is added. While someone speaks the music sits about 27 LU under the voice (`music.underVoiceLu`, 18–40); between lines it rises gently, never closer than 15 LU to the voice. With no narration the music carries the film. - **Length:** a long track is cut to the film; a short one repeats without a jump (its ending crossfades into its beginning). - **The receipt:** a film scored from the pack carries `film.music` on the finished job: title, composer, source, licence, `aiGenerated` (true for generated tracks), and — only when the licence needs one — a ready `creditLine` to paste into your video's description. `changed` says what the mix actually did to the track in this film. The job also shows `musicPick`: what was asked, what was chosen, the shape, and who chose them. The same film gets the same track each time it is submitted. - A track's audio is never served on its own; it only reaches you inside a film. ## 4. Follow a job and collect the film `GET /jobs/{jobId}` (with the key): - `status`: `queued` → `directing` (brief films: script being written) → `processing` → `done`, or `failed` / `canceled`. - `progress`: a sentence to show a person. `phase` + `percent`: for a progress bar (percent is per phase). - `storyboard`: appears as soon as the script exists, before pictures are made — show it, and offer "Stop" → `DELETE /jobs/{jobId}` (cancels before the expensive part). - When `done`, a brief or timeline film has: `url` (MP4), `posterUrl` (JPG), `captionsUrl` (SRT), `subtitlesUrl` (WebVTT), `chaptersUrl`, `film { width, height, seconds }`, `notes[]` (anything Studio adjusted, in plain words), `endUser` (the customer you named). - A talking-cast job (section 5) puts its film at `result.videoUrl` instead of `url`. - When `failed`: `error` (a sentence), `retryable` (true/false). - Download finished files with the key (`studioFile` above) — film files under `/files/` require it; talking-cast films under `/v1/assets/` are unguessable links that also work without it. Treat every Studio link as private either way: save files to your storage at once (Studio keeps films 24 hours) and give your customers YOUR link. Webhook: if you sent `webhook.url`, Studio POSTs the same body `GET /jobs/{id}` would return, plus `event` and `deliveryId`, when the job finishes. Deliveries are retried; `X-Merge-Delivery` stays the same across retries (ignore repeats). Treat a delivery as a nudge: fetch `GET /jobs/{id}` with your key and act on that. Keep a fallback: while a customer has the page open, check every 15 seconds. List one customer's films: `GET /v1/jobs?endUser=` (every kind of film). Usage per customer: `GET /v1/usage` — counts films made from briefs, timelines and lists; talking-cast films are listed per customer but not yet counted there, so count those from your own orders table. ## 5. Other things your app can order - Talking cast — `POST /v1/cast`. Animated characters act out your lines, voiced, lip-synced, captioned (under a minute to make). `GET /v1/cast` lists characters, sets, moods and gestures. ```json { "set": "library", "format": "vertical", "cast": [{ "id": "grace", "preset": "grace" }, { "id": "kenji", "preset": "kenji" }], "lines": [ { "who": "kenji", "text": "What's the book about?", "mood": "curious", "action": "raise" }, { "who": "grace", "text": "A family of salt traders, and one long road home.", "mood": "happy", "action": "open" } ] } ``` `format`: `landscape`, `hd`, `square`, `vertical`. Film at `result.videoUrl` when done. Drawing styles: leave `style` out for the house look, `"ink"` for a fine-line comic, or `"stick"` for our simple stick figures — a big round head, dot-oval eyes that blink, a line mouth that opens with the voice, a plain shirt and stick limbs. Every cast member, gesture and walk works in all three. A speaker turned toward a board, a picture or the person they address gestures with the hand on that side. Hands follow the words, on the stressed syllable: "First… Second… Third…" (or "Then", "Finally") counts the points off on 1–3 fingers across a speaker's lines; "everything"/"huge" opens both hands, "a little" brings them together; "I"/"me" goes to the chest; a question ends palms up; "but" hands over to the other hand. Louder emphasis moves more. You can also name these as an `action`: `count`, `big`, `small`, `self`, `ask`, `down`. Up to 6 people speak in one scene. - Crowds (stick look) — add `crowd` to a `POST /v1/cast` scene (or a mix block of type `cast`) for up to 2,000 people who never speak, behind up to 6 who do. A scene with a crowd is drawn in the stick look (leave `style` out or set `"stick"`; another style is refused). `crowd`: `count` 1–2000 (required) · `seed` (a word or number; the same seed makes the same crowd) · `doing`: `stand` | `clap` | `wave` | `cheer` | `jump` | `dance` | `walk` | `march` | `sit` | `queue`, or a mix of up to 4 like `{ "clap": 0.7, "cheer": 0.3 }` (default `stand`; `walk` on a `street`, `sit` in `rows`) · `layout`: `plaza` (default) | `stadium` | `rows` | `street` | `circle` | `scatter` · `mix`: shares of `child`, `teen`, `adult`, `elder` (default 15/15/55/15, honoured to the person) · `palette`: `{ tops, bottoms, hair, skin, headwear }`, each 1–24 `"#rrggbb"` (team colours; otherwise every person gets their own colours) · `density`: `sparse` | `normal` | `packed` or 0.25–4 · `detail`: `auto` | `low` | `high`. The crowd stands behind the speakers in perspective; one speaker before a crowd is filmed wide. ```json { "style": "stick", "set": "park", "cast": [ { "id": "ana", "pronoun": "she", "age": 34, "skin": 0.62, "hair": { "style": "braids", "color": "#1F1B24" }, "top": { "color": "#C2185B" }, "bottom": { "style": "trousers", "color": "#2E3A59" } }, { "id": "ben", "pronoun": "he", "look": "ben-7" } ], "lines": [ { "who": "ana", "text": "A thousand people came out today.", "mood": "excited" }, { "who": "ben", "text": "And every one of them is clapping.", "mood": "happy" } ], "crowd": { "count": 1000, "seed": "plaza-a", "doing": "clap", "layout": "plaza", "mix": { "child": 0.2, "teen": 0.1, "adult": 0.6, "elder": 0.1 } } } ``` - One stick person, any look — no preset needed. Every colour field takes any `"#rrggbb"`; `age` is `child`/`teen`/`adult`/`elder` or years 2–100 (sizes blend: a 4-year-old is smaller than a 9-year-old; years count in the stick look for a person who also uses `look`, `size` or `voiceKind`, or before a crowd — otherwise a number is a grown-up, as always); `skin` is a name, a colour, or a tone 0 (lightest)–1 (deepest); `size` 0.7–1.3 (stick; `height` stays 0.85–1.15 otherwise); `hair.style`: short, buzz, shaved, curly, curls, afro, wavy, long, bob, bun, ponytail, pigtails, braids, locs, mohawk, spiky, sidepart, bald; `headwear.style`: hijab, turban, cap, beanie, hat; `glasses`, `beard`; `bottom: { "style": "trousers", color }` draws coloured trousers (without a style, stick legs). `look: "any word"` makes a random but repeatable person — any field you give wins over it. Up to 6 speakers; a cast over a recording in a mix (`stops` without a `scene`, a corner `host`) keeps 4. What changed for existing stick scenes: side-on faces are turned (two eyes, a nose); hair `braids`, `shaved` and hats `hat`, `beanie` are drawn as themselves (were the default cap); `bottom.style: "trousers"` draws trousers (`jeans` etc. keep stick legs). Everything else is as before. Voices: in a crowd scene, or for a speaker using `voiceKind`, `look` or `size`, a speaker without a preset or `voice` gets one from the house voice library by `pronoun` (`she`/`he`/`they`, which is neutral) and age. It depends only on their own id, so the same id has the same voice in every scene; two speakers landing on one voice are pitched apart. Other scenes keep their voices exactly as before. `voiceKind`: `child` | `teen` | `adult` | `elder` picks the voice's age on its own (children sound higher). `GET /v1/cast` lists all of this under `customise.stick`, `crowd` and `voiceLibrary`. - Your customers' own avatars and voices — `POST /v1/people` `{ name, photo, consent: true, email? }` draws a cartoon avatar from a photo (upload it first with `POST /v1/assets`; use the returned `source`); `POST /v1/people/{id}/voice` `{ sample, consent: true }` copies their voice from 10–30 seconds of speech; `GET /v1/people` lists them. Use one in any cast as `{ "id": "me", "avatar": "av_…" }`. With `X-On-Behalf-Of`, each of your customers has their OWN avatars: one customer can't list, use, change or remove another's. If you send `email`, Studio emails that person once to confirm, with a link that withdraws all permissions at any time. - A topic film — `POST /v1/cast` `{ "topic": "Preparing for a job interview", "avatar": "av_…", "seconds": 60, "audience"?, "notes"?, "set"?: "explainer" | "office" | "newsroom" | "lab" | "cafe" }`. Studio writes a short first-person script ("Hi, I'm Priya…") and the avatar presents it in their copied voice, with slides on a board. No `avatar`: a house presenter. No `set`: the one that fits the topic (office for careers, lab for health, café for food and hobbies…), shown on the job as `look`. `seconds` 20–180. The job shows `script` once written; a refused topic fails with `errorCode: "topic_declined"` and a sentence to show. A ready-made widget and server kit for this: the Avatar Kit (`@futurizenow/avatar-kit`). - One steady camera pan across several pictures — `POST /v1/render` with consecutive image scenes that each carry `"flow": "right"` (or `"left"`, the way the camera travels) in `visual`. The pictures are set edge to edge and one camera crosses them at one speed: no stop, no jump at a join, and joins inside the run are cuts. Repeat the same `src` in consecutive scenes to split one wide panorama across them. Image scenes only, not with `highlights`; anything else is refused. `POST /v1/direct` takes `"motion": "flow"` (or `"flow-left"` / `"flow-right"`) for the same look film-wide. ```json { "visual": { "type": "image", "src": "https://…/street.png", "flow": "right" }, "duration": 4 } ``` - Words laid out as the picture instead of captions (the "block layout" reel look) — `POST /v1/render` with `"visual": { "type": "blocks", "items": [...] }`: `text` (`small` lead-in + `big` phrase), `panel` (picture band with slanted edges, `src` optional, `label`, `from`, `cut`, `height`), `tiles` (`tiles: [{ label, src? }]`, one per spoken word) and `count` (`to`, `small`, `suffix`; counts up). Each piece enters when its own words are spoken (`scene.words`, subtitle cue words, or an estimate from the scene's `subtitle` over its voice), or at a pinned `word` / `at`; everything slides out at the scene's end. Brand it with `timeline.brand`: `color`, `accent` (hex), `logo`, `corner`, `font` (Fontsource name), `endCard: { text }` (closing card with logo + call to action, on any film). Text colour is picked for contrast automatically. `POST /v1/direct` takes `"style": "blocks"` for the same look film-wide. ```json { "visual": { "type": "blocks", "items": [ { "kind": "text", "small": "If you're looking to", "big": "Grow your audience" }, { "kind": "count", "small": "Over the last", "to": 14, "suffix": "years" } ] }, "subtitle": "If you're looking to grow your audience… over the last fourteen years…" } ``` - Small sound effects under the voice — a soft ping when a caption or label pops up, a rising "draw" for a pen circle, and more. `GET /v1/sfx` lists them: `ping`, `ping-low`, `pop`, `draw`, `swoosh`, `tick`, `sparkle`, `thanks-chime`. All are made by Studio itself, so there is no licence and no credit to give. Ask in three places (nothing plays unless you ask): on an overlay, `"sfx": "ping"` or `{ "name": "ping", "at": 0.1, "volume": -26 }` (`at` = seconds after the overlay starts); on a scene, `"sfx": [ { "name": "draw", "at": 1.2 } ]` (seconds from the scene start); on the timeline, `"sfx": [ { "name": "thanks-chime", "at": 41.5 } ]` (seconds from the film start). `volume` is dB under full level, from -48 to -6, default -24, which sits well under speech. `"sfx": "auto"` on the timeline puts one ping under each `popcaption`, `lowerthird`, `lockup` and `title` (never two within 0.6 s). An unknown name is refused with the list of valid ones. Works in every film path. A film with no voice and no music keeps the effects at their own quiet level. ```json { "overlays": [ { "type": "popcaption", "text": "Grow your AUDIENCE", "start": 0.5, "sfx": "ping" } ], "sfx": [ { "name": "draw", "at": 2.0 } ], "duration": 5 } ``` - Band captions and frame balance — for films made from a person's own videos and pictures, so no area of the frame is left dead. (1) `"subtitleStyle": { "look": "band" }` (also `captionLook: "band"` on the caption routes) shows two or three spoken words at a time in bold capitals at the top safe area, ONE word per beat in gold. The gold word is chosen by rule: a word you wrote in ALL CAPS, else a number or rank (40%, $300, #1), else a proper noun, else the longest meaningful word. Beats follow the speech when the cues carry word times (`words: [{ word, start, end }]`, which `/v1/caption` and the transcriber produce); a cue without word times is spread evenly and the result says so. Options in `subtitleStyle`: `wordsPerBeat` ("auto", 1-4), `keyWord` ("auto" | "caps" | "off"), `accent` (hex, default #D9A441), `caseMode` ("upper" | "as-written"). (2) `"frame": { "balance": "auto" }` looks at each video or picture scene, finds the subject, measures the empty zones around it (top, bottom, sides, corners, kept inside the platform safe areas: top 12 %, bottom 20 % of a vertical frame) and puts one small designed block in an empty one: a block caption (small line over a big line, key word in the accent), a counting-up number, a label pill, or a tile. Never over the subject or burned-in text, never in a scene that already carries your own text overlays, at most one per seven seconds of film, none in the first half second, two counters at most. Narrow it with `"fill": ["blockcaption","counter"]` (also per scene: `scene.frame`), turn it off for a scene with `scene.frame.balance: "off"`; `"sound": "soft"` adds a quiet pop or tick. Off unless the film says `auto`. The result's `picked.frame` lists, per scene, the subject box, the empty share, the balance score and what was placed and why; `picked.captions` records the band beats, the gold words and any even-spread fallback. You can also write the blocks yourself as overlays: `{ "type": "counter", "value": 40, "suffix": "%", "label": "faster", "box": { "x": .05, "y": .14, "w": .9, "h": .2 }, "start": 1, "end": 4 }` (`blockcaption`: `small`, `big`, `accent` = index of the gold word in `big`; `pill` / `tile`: `text`). ```json { "frame": { "balance": "auto", "fill": ["blockcaption", "counter"] }, "subtitleStyle": { "look": "band" }, "subtitles": [ { "start": 0, "end": 3, "text": "We grew 40% in one year", "words": [] } ], "scenes": [ { "duration": 6, "visual": { "type": "video", "src": "https://…/me.mp4" } } ] } ``` - Designed frame and smart reframe — `"frame": { "layout": "designed" }` dresses each picture or video scene as a layout instead of a letterbox: a solid band in your brand colour at the top (the caption band's home, so the caption look defaults to `band`), the picture below it, and a lower zone where big bold BLOCKS live: a small line over a big line with one accent word, on full-bleed plates cut on a diagonal that slide in and leave cleanly, a counter that counts up when a number is said, and an optional picture tile with a label. One block at a time, a new one at most every 2 s, always above the app's own buttons (lower 18 %), never over the face. The words come from the subtitle cues, so give them `words` times. SMART REFRAME is the first half of it and works alone: `"frame": { "balance": "auto" }` (or `"layout": "designed"`) crops a landscape picture so the person FILLS the frame, eyes about 38 % down, following them with a smoothed, jitter-free path (needs the cut-out model; without it a busy region or the centre). A picture that must stay whole (a screenshot, a diagram, a cover, a still with no person, a group, or one too small to enlarge past 2.6x) goes on a designed backdrop as a card with rounded corners and a shadow instead. Fields: `frame.layout` ("plain" | "designed"), `frame.reframe` ("auto" | "off": off keeps every picture whole), `frame.blocks` ("auto" | "plates" | "stack": the block look; auto = plates, "stack" = no plates), `scene.frame.fit` ("auto" | "fill" | "card"), `scene.frame.focus` `{ x, y, w, h }` (fractions: zoom the card to the part that proves something), `scene.frame.tile` `{ "src": | "scene": , "label", "start", "end" }` (a picture inset, or a frame of another scene), `scene.frame.phrases` `[{ "big", "small"?, "accent"?, "start"?, "end"? }]` (1-6 blocks you write yourself, in scene seconds; they replace the ones picked from the words; the small line shrinks to fit and a long phrase wraps on whole words). Captions burned into a film (Hindi, Marathi, Bengali and the other Indic scripts included) are shaped properly: the burn uses ffmpeg's `ass` filter, which joins letters and puts vowel signs on the right side. Colours come from `brand` (color, accent, font) else the theme. Every block is measured against the same moment without it: one that makes the balance worse is moved to the other side, and dropped if still worse; one that would sit on the face is dropped. The result's `picked.frame.designed` lists per scene the fit (fill or card) and why, how the crop path moved, and each block with what was said, its side and the balance before and after; anything dropped says why. Off unless the film asks. Scenes with spotlights or highlights are left as they are. ```json { "frame": { "layout": "designed" }, "brand": { "color": "#5A0F14", "accent": "#E2B04A" }, "subtitles": [ { "start": 0.2, "end": 3, "text": "Our book reached number 2", "words": [ { "word": "Our", "start": 0.2, "end": 0.4 } ] } ], "scenes": [ { "duration": 7, "visual": { "type": "video", "src": "https://…/me.mp4" } }, { "duration": 5, "visual": { "type": "image", "src": "https://…/proof.png" }, "frame": { "focus": { "x": 0.05, "y": 0.3, "w": 0.9, "h": 0.3 } } } ] } ``` Under a person who fills the frame the lower zone carries the big two-tier phrase block (small line over big line, one phrase after another, sized to the zone, under the chin and never over the face), kept only if it does not lower the frame's balance by more than 0.16 (moved to the other side, then tried in the smaller plate, then dropped); `"frame": { "faceBlocks": "sparse" }` keeps the older one-plate-at-a-time look. Words in an Indic script get no block (the block text drawer does not shape them) and the record says so. - Pen marks that draw themselves — `scene.marks: [ { "kind", "x", "y", "w", "h", "at", "draw", "hold", "color", "width", "label", "sfx" } ]` puts a hand-drawn mark on a picture or video scene: a red circle forming round a book cover, a box round "#1 New Release", an arrow, an underline (`"double": true` for two), or a yellow highlighter sweep that keeps the text under it readable. `kind` is one of `circle`, `box`, `arrow`, `underline`, `highlighter`. `x`, `y`, `w`, `h` are the target's box as FRACTIONS OF THE SOURCE PICTURE (0 to 1), so the mark lands on the right spot after zoom, reframe or the designed card (`frame.focus` too; a target outside the zoomed part is skipped and said so). `at` = seconds from the scene start (default 0.5), `draw` = seconds the pen takes (default 0.7), `hold` = how long the finished mark stays (default: to the scene's end). `color` is a hex (default red #E5322D, highlighter yellow), `width` the pen as a fraction of the frame width (default 0.0095), `label` a short pill that pops in after the mark closes, `from` where an arrow comes from (`left`, `right`, `top`, `bottom`, `topleft`...), `loops: 2` for a double circle. Same scene id + mark number = the same drawing every time. Sound: marks play the in-house `draw` sweep, timed so its closing ding lands the moment the pen lifts, at -26 dB and never more than one per 1.5 s, plus a quiet `ping` for a label; `"sfx": false` on a mark (or on the scene, or the film) turns it off. At most 6 marks per scene and 12 per film. A marked picture is held still while it is marked (a moving picture would carry the pen off its target), and a scene with `visual.flow` is refused. Marks need `visual.type` `image` or `video`. `find: "the book cover"` in place of x/y/w/h asks the picture-reading service where the thing is; it works only where that service is set up (otherwise the request is refused and says to give x, y, w, h). `"frame": { "marks": "auto" }` turns a scene's A `label` pill goes beside its mark: below, then above, left, right, on the first spot that covers no other mark, label, face or part of the picture's own text; with no such spot it is skipped and the record says `labelSkipped: "no room"` (never laid over something). `"frame": { "marks": "auto" }` also turns a scene's `highlights` regions (circle or box) and its `frame.focus` card (a box just inside it) into marks for you; per scene `frame.marks: "off"`. The result's `picked.marks` lists each mark's box and timing, what was skipped and why. Not built: curl/scribble marks. ```json { "scenes": [ { "duration": 5.5, "visual": { "type": "image", "src": "https://example.com/amazon.png" }, "marks": [ { "kind": "circle", "x": 0.08, "y": 0.2, "w": 0.19, "h": 0.46, "at": 0.6 }, { "kind": "box", "x": 0.33, "y": 0.53, "w": 0.57, "h": 0.06, "at": 2.0, "label": "#2 BEST SELLER" }, { "kind": "highlighter", "x": 0.33, "y": 0.6, "w": 0.5, "h": 0.042, "at": 3.2, "draw": 1 } ] } ] } ``` - Celebration and speech (bursts, bubbles, quote card, sticker, icon pills) — all drawn in code, no emoji font needed, all on `image` or `video` scenes (not with `visual.flow`): `scene.bursts: [ { "kind": "confetti"|"hearts"|"sparkles", "at", "duration", "x", "y", "w", "h", "intensity", "seed", "sfx" } ]` confetti falls from just under the top band in the brand accent, gold, red and white with spin and gravity; hearts rise beside the face (never in front of it) and fade; sparkles twinkle around a box (`x`,`y` centre, `w`,`h` size, fractions of the frame; by default around the subject, e.g. a cover). `at` seconds from the scene start, `duration` 0.8 to 4 (default 2.2), `intensity` 0.3 to 2 (default 1). Same scene id = the same burst every time (`seed` overrides). They are rare on purpose: 1 burst per scene, no two closer than 6 s, 6 per film. Sound: `sparkle` at -28 dB for confetti and sparkles, `thanks-chime` for hearts; `"sfx": false` (on the burst, scene or film) turns it off. Overlays (scene-relative `start`/`end` seconds, like every overlay): `{ "type": "bubble", "text", "speaker?", "side": "left"|"right"|"auto", "icon?", "tail?": { "x", "y" } }` is a rounded white speech bubble with a tail that pops in with a small overshoot; bubbles that are on screen together are placed so they never overlap (side by side or stacked) and never cover the face. `{ "type": "quotecard", "kicker?": "MY CHAPTER", "text" }` is a lower-third card in the brand colour with an underline that draws in under the text (one per scene; whole sentences get their own line). `{ "type": "sticker", "text": "WE DID IT!", "icon?": "party" }` is a tilted pill that bounces in. `{ "type": "pill", "text", "icon?" | "emoji?" }` is a rounded label with a small picture (a pill WITHOUT an icon or emoji, or with a frame-balance `box`, keeps its old look). Icons drawn in code: party, heart, bulb, chart, trophy, speech, sparkle, check, link, star, fire, thumbs. An emoji typed in the text or in `emoji` is mapped onto one of them (🎉 party, ❤ heart, 💡 bulb, 📈 chart, 🏆 trophy, 💬 speech, ✨ sparkle, ✅ check, 🔗 link, ⭐ star, 🔥 fire, 👍 thumbs, and a few cousins); an emoji with no drawing is DROPPED and listed in the result, never a box. Text the film's typeface cannot draw (another script) is left out and said so. Limits: bubble text 110 characters, quote card 90, sticker 24, pill 40; 4 bubbles, 1 card, 2 stickers/pills per scene. Overlay `sfx` is off unless you give it (`"sfx": "pop"`). With a person in the picture the placement search treats the top of the person (head and neck) as a no-go box, keeps inside the platform safe area and the caption band, avoids frame-balance fills, and says in `picked.fx` when there was no room. `"frame": { "fx": "auto" }` adds a burst only where the spoken line earns one (a win such as "number two", "we did it" gets confetti; a "thank you" gets hearts; never in the first 1.5 s; scene `frame.fx: "off"`); without it nothing is added. `"director": "auto"` (top level of the timeline; default off) does this and more from what is said: a win said with energy gets confetti and a sticker made of the speaker's own words ("WE DID IT!", "#1 BEST SELLER"); a thank-you gets hearts; a quote gets a speech bubble (short, with a speaker) or a quote card; evidence ("number 2", "40%", "5,000 copies") over a screenshot scene with a `frame.focus` region gets a pen mark on that region with a label cut from the words, or just a small label when there is no region; a call to action that names its place ("link in the comments", "link in bio") gets that pill with a link icon (nothing is invented when no place is named). Code rules sort each spoken phrase into win, thanks, quote, proof, call-to-action, personal-story or plain with an energy (calm, warm, excited); when the engine has a TypeSafe key it sorts the phrases the rules are unsure about and rates the energy of wins and thank-yous, and code decides what is drawn. Hard caps: one celebration per 8 s (4 per film), 6 pen marks, 3 bubbles, 2 quote cards, 2 stickers, 3 pills, nothing in the first 1.5 s, one thing per scene, never two big things at once; a calm `tone` ("calm", "tribute", "serious"...) gets no confetti and no stickers; a win said quietly gets a small trophy label, not confetti. Scenes you dressed yourself, flowing pictures, non-picture scenes and words in an Indic script are left alone. `picked.director` records every phrase (text, class, energy, `method`: `typesafe` | `code` | `code+typesafe`, the decision and why) and the caps; with no key, or when TypeSafe errors, the code rules alone run and it only ever places fewer marks. A scene that has any of these is drawn into one clip first, so the picture is held while they play (as with marks). The result's `picked.fx` lists each item's box, timing, icon, whether it clears the face, what was skipped and why. ```json { "scenes": [ { "duration": 6, "visual": { "type": "video", "src": "https://…/me.mp4" }, "bursts": [ { "kind": "confetti", "at": 0.6 } ], "overlays": [ { "type": "sticker", "text": "WE DID IT!", "icon": "party", "start": 0.9, "end": 4.6 }, { "type": "pill", "text": "#1 TOP NEW RELEASE 🏆", "start": 2.4, "end": 5.6 }, { "type": "bubble", "speaker": "Boss", "text": "Do this.", "start": 1, "end": 5 }, { "type": "bubble", "speaker": "Team", "text": "Wait... what?", "start": 2.2, "end": 5 } ] }, { "duration": 5, "visual": { "type": "video", "src": "https://…/me.mp4" }, "bursts": [ { "kind": "hearts", "at": 0.5 } ], "overlays": [ { "type": "quotecard", "kicker": "MY CHAPTER", "text": "The Boss Said. The Team Heard.", "start": 1.2, "end": 4.8 } ] }, { "duration": 4, "visual": { "type": "image", "src": "https://…/cover.jpg" }, "bursts": [ { "kind": "sparkles", "at": 0.3, "duration": 3 } ] } ] } ``` - A software walkthrough from screenshots — `POST /v1/render` with one image scene per step. Send each screenshot at 2× the film's size (3840 px wide for 16:9), use `"kenBurns": "hold"` (a moving screen shimmers), and point at what the narration names with `highlights`: ```json { "visual": { "type": "image", "src": "https://…/step-2.png", "kenBurns": "hold" }, "duration": 7, "overlays": [{ "type": "eyebrow", "text": "Step 2" }], "highlights": [ { "x": 0.73, "y": 0.86, "w": 0.065, "h": 0.056, "start": 0.5, "label": "Continue" } ] } ``` `x`/`y`/`w`/`h`: the control's box as fractions of the screenshot (left, top, width, height) — from `getBoundingClientRect()` divided by the screenshot size. `start`/`end`: fractions of the scene, so the ring lights while the control is talked about. Studio draws a glowing ring (theme accent; `color` overrides), holds the picture still, shows `label` as a caption; up to six per scene. A step label (`eyebrow` with no title) sits at the top centre on its own plate. - Pause and explain — in `POST /v1/mix`, a `clip` block's `stops: [{ "at": 4, "cast": [...], "side": "right", "lines": [...], "circles": [{ "find": "the speaker's eyes", "cue": "eyes" }] }]` freezes the recording at `at` seconds while the cast explains. A circle rings what `find` names (or `{ x, y, w?, h? }` as fractions); `cue` is a word in the stop's lines and the circle is drawn as that word is spoken. The ring leaves before the recording resumes. - A show (host + guests in windows) — check `GET /health` lists `"show"` in `features` first (an engine without it refuses the block). In `POST /v1/mix`, a block `{ "type": "show", "sources": { "mei": { "kind": "cast", "preset": "mei" }, "expert": { "kind": "clip", "src": , "name": "Amina", "title": "Medical affairs", "captions": "auto" }, "points": { "kind": "card", "title": "…", "lines": ["…"] } }, "beats": [ { "who": "mei", "text": "…", "layout": "host-with-guest", "guest": "expert", "zoom": "in-out" }, { "who": "expert", "from": 0, "to": 8, "layout": "guest-big" }, { "who": "expert", "from": 8, "to": 15, "layout": "nest", "inset": "points" }, { "who": "mei", "text": "…", "layout": "host", "transition": "hand-back" } ] }`. Layouts: `host`, `host-with-guest`, `guest-big`, `guest-full`, `nest` (a window inside the guest's window), `side-by-side` (`guests: [a, b]`); or `windows: [{ id, source, box: { x, y, w, h }, parent? }]` by hand (fractions of the parent; at most 4 windows, nested at most 2 deep). Transitions (`grow`, `shrink`, `hand-back`, `pop`, `close-nest`, `swap`, `cut`, default automatic) happen in the pause between speakers. The host's captions sit in the band at the bottom; a guest's sit inside their window — unless their video already has words on it: `captions: "auto"` looks (and turns ours off if it finds them), `"burned-in"` / `"ours"` / `"off"` decide it yourself. A clip's `script` captions it when the engine can't transcribe. Landscape, square and vertical (phone) films; `safe: "reels"` (the default for vertical) keeps the phone apps' button areas clear. The finished job's `result.qc[]` is Studio's own check of the show: `report` (plain words: what it checked, what it fixed, what still needs a person), `ok`, `remaining`, `picks` (each judgement and which method made it). - A generated clip (Google Veo; the engine's owner only) — `POST /v1/videogen { "parts": ["scene + what the person says", "what they say next"], "model": "veo-3.1-fast-generate-preview", "aspectRatio": "16:9" }` returns the price and nothing else; the same body with `"confirm": true` makes it (8 s, then 7 s per further part) as a job whose result is a plain mp4 (`result.video`, no captions). Off unless the engine's `VIDEO_GEN=on`; capped per clip and per day. `GET /v1/videogen` shows the budget. - Many films from one list — `POST /v1/merge`: `{ "template": { "preset": "square", "scenes": [ … ] }, "rows": [ { "first": "Priya", … }, … ], "key": "first" }`. Write `{{column}}` in the template; one film per row, very fast. Each row is its own job; `GET /v1/batches/{batchId}` follows the set. - Captions on a video — `POST /v1/caption` with `video` (upload first: `POST /v1/assets` with the raw file as the body and its `Content-Type`; use the returned `source`), optional `captionLang`. - The best moments in a long video, for shorts — **invite-only for now**: only the engine's owner can start it; any other key gets `403` with `code: "invite_only"` until the owner opens it. `POST /v1/moments` with `video` (upload first, as above), optional `count` (1–30, default 10), `minSeconds`/`maxSeconds` (default 20–90), `spokenLang`, `names` (names and unusual words, so the transcript spells them right), `words: false` to skip writing hooks. Poll `GET /jobs/{jobId}`; `result.moments` is ranked: `{ rank, start, end, seconds, at, score (0–100), scoredBy, kind, why[], hook (≤6 words), title, caption, wordsCheck, warning?, text, estimatedCostUsd, shape }`. Every clip is planned vertical: `shape` is `{ aspect: "9:16", width: 1080, height: 1920 }` (centre crop or blurred fill for now; speaker-following later). Nothing is cut or rendered; listing is free. `result.cost` is an estimate in US dollars; `cost.allowance` says the first 3 finished clips a month are free, then 1 credit a clip (not charged yet). Clips start just before a word and never on "and/so/but"; no two overlap; at most two of one kind; weak moments are left out, so fewer than `count` can come back. `{ "from": "" }` instead of `video` finds more in the same video without hearing it again. `GET /v1/moments/{jobId}` is the full record (every candidate and judgement); `GET /v1/moments/{jobId}/transcript?format=json|txt|srt`. Up to 240 minutes a video. - A cover or thumbnail for any film, picture or course — `POST /v1/cover`. Result first and big, name small: the face fills the frame (eyes about 36 % down), the foot fades to near-black, a serif headline in cream, a second line in gold, and a small-caps byline between thin gold rules. Send `from` (the id of one of your finished films) or `source` (a picture or video: URL, data-URL or stored asset), and `headline` (required, up to 90 characters: the result, in a few words), `accent` (the gold second line), `byline` (a name and role), `badge` (a small pill: "New", "Best seller"). `size`: `"9:16"` (1080x1920, default), `"1:1"` or `"16:9"`. `look`: `"testimonial"` (a person fills the frame), `"course"` (the picture on a card over a calm gradient, title and subtitle below) or `"minimal"` (lighter fade, no rules); leave it out and a person gets testimonial, a picture or diagram gets course. `at` (seconds) picks the frame; otherwise the engine scores up to twelve (sharp, steady, well lit, no caption or bubble across the lower third, a person large in the frame) and takes the best. `brand`: `{ color?, accent? }` replaces the near-black ground and the gold (an accent too dark to read is replaced). `format`: `"jpg"` (default) or `"png"`. `vision: false` stops the engine asking its picture-reading helper (when the engine has a vision key) whether the eyes are open and the person is smiling; the choice then rests on code alone, and no picture leaves the engine. Poll `GET /jobs/{jobId}`: `result.coverUrl` is the image (`/files/{jobId}-cover.jpg`), and `result.picks` records every choice (frame time and the scored candidates, look and why, the head found, the crop and where the eyes landed, the type size, the fonts, notes such as "the picture was enlarged 3.2x, so it is soft") so any of them can be overridden. Headlines wrap to at most two lines and the type steps down to fit; a line too long at the smallest size is shortened with "…", never clipped; the words start below the chin. Latin, Cyrillic, Greek, Arabic, Hebrew and Chinese/Japanese/Korean words work, and so do Hindi/Marathi/Nepali (Devanagari), Bengali, Punjabi, Gujarati, Tamil, Telugu, Kannada and Malayalam in the headline, accent and byline: those lines are typeset by libass with the script's own Noto face (fetched once and cached, like the caption fonts) so vowel signs and conjuncts land correctly; `picks.shaped` says which lines. The badge pill cannot carry them yet. A character the face lacks (an emoji) is refused in plain words naming it (a box is worse than no cover). Typefaces: Playfair Display (SIL OFL) for the headline, Inter for the byline, fetched once and cached like the theme fonts. A small source is enlarged and says so; a wide cover from an upright video is a thin slice of it. Not billed as a film. - A voice from text — `POST /v1/voice` `{ "text": "…" }`. Add `"keep": true` to keep the clip in your history (the reply then carries `kept: { id, url }`); without it nothing is stored, as before. - Your history — `GET /v1/history?kind=voice|picture`: the clips and pictures you kept, newest first, each with a `url` (a clip's sound is `GET /v1/history/{id}.mp3`; add `?download=1` to save it as a file). `DELETE /v1/history/{id}` removes one. Kept until you delete it; the newest 100 of each kind are kept. A picture is kept by sending `"keep": true` to `POST /v1/storyboard/image`. Each key sees only its own; a customer named with `X-On-Behalf-Of` on your account key has their own. Films are not here — `GET /v1/jobs` lists them. - Draw your own backgrounds, props and assets in code. No image AI is used, so making and using them is free. A drawing is JSON: `{ "shapes": [ … ], "defs"?: { name: shape } }`. Every shape has a `type`: `circle {x,y,r}`, `ellipse {x,y,rx,ry}`, `rect {x,y,w,h,r?}`, `poly {points:[x0,y0,…]}`, `line {points, width}`, `path {d: "M0 0 C … Z"}` (SVG path data, arcs included), `star {x,y,r,points?,inner?}`, `blob {x,y,r,wobble?,seed?}` (an organic round shape). Groups: `group {children}`, `repeat {count, step:{move,rotate,scale,opacity}, children}`, `mirror {axis:"x"|"y"|"both", at?, children}`, `scatter {count, seed, area:[x0,y0,x1,y1], scales?, rotates?, children}`, `use {ref}`. Any shape can also take `fill` (hex, or `{ "linear": [x0,y0,x1,y1] | "radial": [cx,cy,r], "stops": ["#…","#…"] }`), `outline` (true for the house outline, or a colour), `shade` (a cel shadow, 0–60), `stroke`, `opacity`, `move`, `rotate` (degrees), `scale`, and `animate { spin, bob:[amount,secs], sway, pulse, flicker }`. Up is negative y. A `background` drawing takes `wall`/`floor` colours (one, or [top, bottom]) and `outdoor` at its top level, beside `shapes` (`{ "wall": "#BFE3FF", "floor": "#8CC56B", "outdoor": true, "shapes": [ … ] }`); its y = 0 is where the wall meets the floor, it spans about x −1000…1000, and people stand near y = 95. A still wide shot shows only about x −550…550 and y −430…180; keep what must be seen (a sun, a sign) inside that. For a `prop`, (0,0) is where the hand grips it, and a held object is 30–60 units tall. Try one with `POST /v1/draw` `{ kind, drawing, width?, height?, style?: "ink" }`, which returns a PNG (`?format=json` returns a data URL). Save it with `POST /v1/creations` `{ kind: "background"|"prop"|"asset"|"character"|"part", name, drawing }`. You get back a `previewUrl`, and any film can then use it: `"set": "mine:"`, or `"hold": "mine:"` on a line. Or skip saving and put drawings straight in a cast scene: `"drawings": { "sets": { "meadow": {…} }, "props": { "lantern": {…} } }`, then `"set": "meadow"` and `"hold": "lantern"`. Creations are private to the account, and to each customer with `X-On-Behalf-Of`. `GET /v1/creations` lists them; `PATCH` and `DELETE` work on `/v1/creations/{id}`. `GET /v1/draw` returns the shape list, the limits (3,000 painted shapes, repeats counted) and each kind's frame. - Invented characters — a cast entry `{ "id": "bolt", "drawn": {…} }` is a character drawn in code: a robot, a teapot, a monster. `drawn` is a drawing of the BODY (front view, (0,0) between the feet, about 400 tall), plus `"head": { x, y, shapes }` (drawn around the head's centre, so it turns), `"face": { "eyes": { gap, y, size, color, white }, "mouth": { y, width, color } }`, and optionally `"arms": { "at": [[x,y],[x,y]], length, width, color, hands }`. Don't draw eyes or a mouth: the engine adds eyes that blink and a mouth that talks with the voice. The arms make the cast's gestures and hold props, and the character walks, turns and is framed like anyone else. Give it a `voice` as usual. - Wearables — `"wear": [ { "on": "head"|"face"|"neck"|"chest"|"hips"|"back"|"hands"|"feet", "behind"?: true, "shapes": […] } ]` on any cast entry puts drawn hats, glasses, outfits, capes or wings on that person, and they move with them. For `on: "head"`, (0,0) is the centre of a head about 94 wide and 106 tall; sizes follow the person, so a part drawn for an adult fits a child. Save either one with `POST /v1/creations` (kind `character` or `part`), then use `"drawn": "mine:bolt"` / `"wear": ["mine:top hat"]`. - Pictures in drawings. Upload a picture with `POST /v1/assets`, then place it with `{ "type": "image", "src": , "x", "y", "w", "h" }`, mixed with drawn shapes (a photo backdrop with drawn bunting on top, say). Pictures scale, move and mirror, but don't rotate. A transparent PNG works as a prop. - Let Studio make it, and check it — `POST /v1/draw/create` `{ "kind", "brief": "a grumpy teapot with a bowler hat", "rounds"?: 1-3, "save"?: { "name" } }` draws it from words; `POST /v1/draw/trace` `{ "kind", "picture": , "hint"?, "rounds"?, "save"? }` redraws a picture. Either one makes a first draft, then quality-checks it. It draws a contact sheet of every view and state (talking, blinking, gesturing, turned, from behind, holding something; a wearable on an adult and a child; a set with people in it), runs free checks in code (does the mouth move, does it stand on the floor, are the arms attached, is the floor clear), and has the AI review the sheet and fix what is wrong. That runs **3 rounds at most**, then it stops and reports. Both are jobs: follow `GET /v1/jobs/{id}`; the result is `{ drawing, imageUrl, sheetUrl, qc: { passed, rounds: [{ problems, issues, fixed }], remaining }, saved? }`. Up to 4 AI looks each. `POST /v1/draw/check` `{ kind, drawing }` runs the free checks on your own drawing at once and returns the contact sheet; add `"review": true` for the AI rounds (a job). `POST /v1/draw` with `"sheet": true` returns just the contact sheet. People in pictures become simple generic figures, never a likeness. To put a real person in a film, use `/v1/people` with their consent. ### The creative library — ask for something creative and it comes from here Studio keeps ONE creative library, shared by every app and account: backgrounds (places to stand in), pictures (for a board), the built-in sets and props, and characters. Ask for what you need in plain words. If something in the library fits, you get it; if not, Studio makes it, and keeps it for everyone ONLY if it is generic. Anything that is someone's in particular is made for your film alone and never kept. - Ask — `POST /v1/library` `{ "request": "a tired night-shift nurse", "kind": "character" }`. `kind`: `background` | `picture` | `character`. Answer (HTTP 200): `{ "item": { "id": "lib_…", "kind", "name", "tags", "url"? }, "source": "library" | "created" | "made-for-this-film", "note"? }`. `library` = it was already there (reused, free); `created` = made now and kept for everyone; `made-for-this-film` = made for you, NOT kept (the `note` says why; `item.id` is null). - Browse and search — `GET /v1/library?kind=character&q=nurse` (`kind` and `q` optional; built-in sets and props come back too, marked `builtin`). `GET /v1/cast` also lists the library's characters under `library.characters`. - Use a background or picture — in a cast scene, `"set": "a volcano"` (any place; it comes from the library, or is made) and a board picture written as `"image": "library: a microscope"`. - Use a character — put it in any cast as `{ "id": "nurse", "library": "lib_…" }`. That works in `POST /v1/cast`, a `cast` block of `POST /v1/mix` (and a deep dive's `stops[].scene`), a campaign's cast and an interactive lesson's cast nodes. Any other field on the entry changes it for your film only, part by part: `{ "id": "nurse", "library": "lib_…", "top": { "color": "#E53935" } }` is the same nurse in a red top. An unknown or removed id is a 400 with a sentence to show. Voice: the character library holds no voices — the voice library and the character library are separate and meet only through gender. The voice defaults to a gender-matched voice from the voice library; set `voice` on the entry to choose another; list at GET /v1/voices. (`voices: { "inworld": "…" }` works too.) The default is repeatable for the same cast `id`, and two speakers on one voice are pitched apart. It draws in every style (house, `ink`, `stick`). - What a character is — a small recipe in the cast's own fields: `name` (a role, never a personal name), `age` (child / teen / adult / elder), `pronoun`, `build`, `skin`, `hair { style, color }`, `eyes`, `top { style, color }`, `bottom { style, color }`, `glasses`, `headwear`, `beard`; plus a small preview portrait at `item.url`. NO photo, NO voice sample, nothing about any real person. (To put a REAL person in a film — their face, their own voice, with their consent — use `/v1/people` instead; the two are separate and never mix.) - A made-for-this-film character comes back with its `recipe` but no id: put the recipe's fields straight into your cast entry (`{ "id": "hero", ...item.recipe }`). What is kept (generic) and what isn't: - Kept for everyone: an ordinary thing, place or role — "a cloud", "a science lab", "a nurse", "a grandfather who farms", "a teenage skateboarder", "a firefighter". - Made for your film only, never kept: a real person, a celebrity or public figure, a brand, logo or product, a character from a film, book, game or show ("Elsa", "a Marvel hero", "like Mickey Mouse"), your or your customer's own name, family, colleagues or company, words to print, or distinctive creative detail you described ("a cloud with an orange tinge on the left"). Such a character is drawn as an ORIGINAL stand-in who only fits the general role, never a likeness: it takes none of the character's signature look (for "Elsa": no pale-blonde braid, nothing ice-blue), and the `note` says it is an original stand-in. Links, offensive words and named businesses are never kept either. When Studio is unsure — or can't check just now — it doesn't keep it (it is still made for your film). Limits: a request is up to 200 characters. New characters: 20 an hour per account key (a reused one is never limited; acting for different customers with X-On-Behalf-Of shares your key's allowance) — over that, HTTP 429 `too_many_requests` with a sentence to show. One account keeps up to 100 characters in the shared library, and the library holds up to 2,000; near that, the oldest character nobody ever used again makes room, otherwise new ones are made for your film and not kept. "A happy nurse" and "a calm nurse" are the same nurse: a request with the same role (and no different age or gender) reuses it. Names and tags are plain words (no links, no markup). Who may rename or remove an item (`PATCH` / `DELETE /v1/library/{id}`): only the Studio operator — items are everyone's. Each item says `mayChange: true|false` for the key asking; anyone may use any item in a film. Keeping: library items are kept until the operator removes them (they are not swept); finished film files are kept for 24 hours, as everywhere. Metering: each character made counts once in `GET /v1/usage` as `characters`; each picture made counts in `images`. Worked example — a nurse and a grandfather, the nurse in a red top: ``` POST /v1/library { "request": "a tired night-shift nurse", "kind": "character" } → { "item": { "id": "lib_3f9c1a2b7d4e5f60", "kind": "character", "name": "tired night-shift nurse", "recipe": { "name": "Night nurse", "age": "adult", "pronoun": "she", … }, "url": "https://…/v1/assets/….png" }, "source": "created", "note": "\"tired night-shift nurse\" was made and added to the shared library as lib_3f9c1a2b7d4e5f60 — …" } POST /v1/library { "request": "a retired grandfather who farms", "kind": "character" } → lib_8a1d… POST /v1/cast { "set": "hospital corridor at night", "cast": [ { "id": "nurse", "library": "lib_3f9c1a2b7d4e5f60", "top": { "color": "#E53935" } }, { "id": "grandpa", "library": "lib_8a1d…" } ], "lines": [ { "who": "grandpa", "text": "Long night?", "mood": "curious" }, { "who": "nurse", "text": "The longest. But your results are good news.", "mood": "happy" } ] } ``` The next app to ask for "an exhausted nurse who works nights" gets the same nurse (`source: "library"`). ## 6. Interactive lessons (for learning systems) Build: `POST /v1/interactive` with a node graph. Each node is a short clip (a timeline of scenes) that ends in a question, a `next` node, or nothing (the end). ```json { "title": "Your first bakery shift", "start": "intro", "preset": "16:9", "certificate": { "passingScore": 0.8 }, "nodes": { "intro": { "timeline": { "scenes": [ { "duration": 6, "visual": { "type": "video", "src": "https://…/intro.mp4" }, "overlays": [ { "type": "lockup", "eyebrow": "Lesson", "title": "Your first bakery shift" } ] } ] }, "next": "q1" }, "q1": { "timeline": { "scenes": [ { "duration": 4, "visual": { "type": "color" }, "overlays": [ { "type": "lockup", "title": "Where does it go?" } ] } ] }, "question": { "type": "dragdrop", "prompt": "Drag each item to where it belongs.", "buckets": ["Oven", "Fridge", "Counter"], "items": [ { "text": "Milk", "bucket": "Fridge" }, { "text": "Shaped dough", "bucket": "Oven" } ], "pass": "done", "fail": "why1" } }, "why1": { "timeline": { "scenes": [ { "duration": 5, "visual": { "type": "color" }, "overlays": [ { "type": "lockup", "title": "Cold things stay cold" } ] } ] }, "next": "q1" }, "done": { "timeline": { "scenes": [ { "duration": 4, "visual": { "type": "color" }, "overlays": [ { "type": "lockup", "title": "Well done" } ] } ] } } } } ``` Question types (`question.type`): `single` (default: `choices: [{ label, next, correct? }]`, each choice branches), `multi`, `text`, `dragdrop` (`buckets`, `items[{text,bucket}]`), `match` (`pairs[{left,right}]`), `ordering` (`items` in the correct order; the player shuffles), `fillblanks` (`text` with `{}` blanks + `blanks: [["accepted", "answers"], …]`), `markwords`, `hotspot` (`image`, `imageAlt`, `regions[{x,y,w,h}]` from 0 to 1), `numeric`, plus unmarked `likert`, `poll`, `reflect`, `flashcards`. Every marked type routes with `pass` / `fail` (a node id each). Any question may carry `explain` (up to 600 characters): shown after the learner answers, under "That's right." or "Not quite." (not shown in exam mode, `graded: true`, which only says "Answer recorded." — also in its SCORM package). How the lesson page looks — `look` (optional, on `/v1/interactive` and `/v1/direct-course`): `journal` (Field Journal: science, nature), `clinic` (Clinic Daylight: health, care, safety), `book` (Picture Book: children), `studio` (the Studio house look) or `default` (a light page in the theme's colours). Anything else sent as `look` when creating is set aside (reply: `lookWarning`; a theme name like `"meadow"` becomes the `theme` if none was sent); `PATCH` refuses an unknown name with `400`. A new lesson with no look (or `"auto"`) gets one picked for its topic (TypeSafe, keyword fallback) — always light, unless you chose a dark `theme` yourself. Lessons made before looks keep the neutral default (`by: "none"`) until you choose. A new version (`previous`) keeps the old one's look. `brandColor: "#RRGGBB"` recolours the buttons (darkened automatically until it reads). `celebrate: true` adds a chime and confetti (off by default). The look changes only the page, never the films: change it any time, free, with `PATCH /v1/interactive/{id}` `{ look, brandColor, celebrate }`; preview with `/play/{id}?look=book`. Two different fields: `lessonLook: { name, by: "auto"|"caller"|"none", reason, theme, themeBy }` is the lesson PAGE's look (on both routes' replies, `PATCH`, `GET /v1/interactive/{id}` and `/v1/results` rows); `look` on a `/v1/direct-course` reply is unchanged — the films' theme pick `{ theme, by }`, present only when Studio chose the theme. `PATCH` with `look: "auto"` picks for the topic. Hosted, xAPI, cmi5 and LTI always show the current look; a SCORM ZIP keeps the look (and its typefaces) it was exported with. Reply: `{ "interactiveId", "playerUrl" }`. Check `GET /v1/interactive/{id}` until `status` is `ready` (usually about a minute). - **The story look for courses (covers, trailer, quiz sounds).** Every ready course gets a cover for the course and one per lesson, made from the lessons' own pictures in the `course` look (the picture on a card, the title big, the outcome in gold, the author small between rules). Words: course `title` / `outcome` (else the first sentence of `description`) / `author`; per node `title` and `outcome` (else the Course Director's part title, a reading page's title, the first question, or the node id said like words). A cover with no author shows the course title as its byline. `cover: false` leaves them out. Read them back on `GET /v1/interactive/{id}` under `covers` (`course`, `lessons`, `notMade` with the reason for any that could not be made); files are `GET /play/{id}/covers/course.jpg` and `/covers/{node}.jpg`; the SCORM package carries them under `covers/`; the LTI deep-link reply carries the course cover as the thumbnail. `POST /v1/interactive/{id}/trailer { preset?, seconds? (20-40), label?, webhook? }` renders a 20-40 s vertical film from the lesson titles and outcomes: the cover, the course promise, one scene per lesson, the cover again with a thank-you chime. Each lesson scene carries a block written from the course, never picked from the words: the lesson title in the small tier above its outcome in the big tier (a lesson with no outcome puts the course name small and its title big), wrapped on whole words and shrunk, not cut; the band at the top counts "1 of 5". A scene is held as long as its words take to read (3.6-6.2 s), a lesson with no picture borrows the nearest lesson's (recorded in `picked.notes`), and the band cues are timed to the finished film. The upright (9:16) course cover now fills the frame: a tall picture card and a larger headline, with the whole gold outcome line (up to 60 characters, on two lines) instead of a 30-character cut. The reply and the course's `trailer` say which lessons were used and why (`picked`). `sounds: true` (on the course, or `PATCH` it) turns on a soft ping on a right answer, a quiet tick on a wrong one (the same sounds as films, `sfx.mjs`) and a burst of confetti on a passed lesson (the film burst's own pieces, `fx.mjs`). A muted lesson stays silent, an exam never says right or wrong so it stays silent, reduced-motion devices get no burst, and a "♪" button lets the learner switch the quiz sounds off (remembered on the device). Covers are on by default; the trailer and sounds only when asked. - Learners play it at `https://studio.futurizenow.co/play/{interactiveId}` — public, no sign-in; embed it or link to it. Add `?learner=` to have the certificate made out to them. - Download for a learning system: `GET /v1/interactive/{id}/export/{format}` with the key, format `scorm` (SCORM 1.2), `scorm2004`, `xapi` or `cmi5` → a ZIP to upload to the LMS. SCORM packages hold the whole lesson; xAPI and cmi5 are small launchers for the hosted lesson. - LTI 1.3 (Canvas, Moodle, Blackboard, D2L Brightspace and any LTI 1.3 system): teachers add a Studio lesson inside a course (Deep Linking), learners open it with no second sign-in, and each score goes to the course gradebook (Assignment and Grade Services). Connect a system in Studio under Developers, Learning systems: a one-click registration link (LTI Dynamic Registration, e.g. Moodle), the Canvas configuration URL https://studio.futurizenow.co/lti/canvas.json, or by hand with presets. Tool addresses: login https://studio.futurizenow.co/lti/login, launch and redirect https://studio.futurizenow.co/lti/launch, public key https://studio.futurizenow.co/lti/jwks. From your own server: `GET /v1/lti`, `GET|POST /v1/lti/platforms` ({ issuer, clientId, authUrl, tokenUrl, jwksUrl, deploymentIds? }), `DELETE /v1/lti/platforms/{id}`, `POST /v1/lti/registration-links`. A launched lesson is one of the connecting account's own lessons. - Which package for which learning system (most accept several; this is the suggested one): Moodle, Canvas, 360Learning, Totara: `scorm` (SCORM 1.2). Blackboard, D2L Brightspace, Docebo, Cornerstone, SAP SuccessFactors, Workday Learning, Absorb, LearnUpon, Litmos: `scorm2004`. TalentLMS: `cmi5`. A learning record store: `xapi` or `cmi5`. No learning system: the public player link. Full table: https://studio.futurizenow.co/use/lms-video-api - Results: `GET /v1/interactive/{id}/results` — starts, completions, scores, per question. - A finished sample to try: https://studio.futurizenow.co/play/a685c02d-66dd-4467-9e6f-145860a0180e and its packages at https://studio.futurizenow.co/samples/make-a-course-scorm.zip (also -scorm2004, -xapi, -cmi5). ## 7. Errors | code | HTTP | what to do | |---|---|---| | invalid_request | 400 | fix the request; the sentence says what is wrong | | unauthorized | 401 | key missing, wrong or revoked | | plan_limit | 402 | the account used this month's allowance | | forbidden | 403 | not allowed for this key | | not_found | 404 | gone (films are kept 24 hours) | | conflict | 409 | same Idempotency-Key reused for a different request | | too_many_requests | 429 | wait and retry | | internal_error / provider_failed / unavailable | 500/502/503 | retry later with the same Idempotency-Key | ## 8. What a good app on Studio looks like - One short order form with defaults (vertical, 30 seconds); fill it from what you already know (title, description, cover). Compose the brief in code from the form, not as free text from the customer. - Disable the button after one click and send the order id as `Idempotency-Key`. - Show Studio's own `progress` sentence and the storyboard as soon as it exists, with a "Stop" button. - Webhook first; check every 15 seconds only while the page is open; stop at `done`. - Copy finished files to your storage at once; play from there with the poster and captions on. - Offer "make another version" (another shape or length) instead of starting over. - A staff view: per-customer usage (`GET /v1/usage`) and every failure with its `requestId`. - No dead ends: every finished film offers download, share and "make another". ## 9. Machine-readable extras - `GET /v1` — everything this Studio can do right now, live (with your key). - `GET /v1/themes`, `GET /v1/voices`, `GET /v1/cast` — lists to build pickers from (cache them daily). - Full human guide: https://studio.futurizenow.co/docs ## 10. Automatic quality check Every finished film is checked before the job says `done`. Findings arrive in `advisories` as `{ kind, scene?, message }`; a check never fails a film, so show them as notes ("check this before sending"), not errors. - The job also has `filmcheck` (`ran`, `checks`, `passed`, `flagged`), `deliverySizes` (`bytes`, and `fits` for telegram 50 MB, email 25 MB, whatsapp 16 MB), `contactUrl` (a picture with three labelled frames per scene) and `filmcheckUrl` (the written record). - Advisory kinds: `too-big-for:telegram`, `too-big-for:email`, `too-big-for:whatsapp`, `audio-video-length-mismatch`, `narration-cut-at-end`, `caption-repeats-on-screen-text`, `text-too-small-for-phone`, `text-overlaps`, and from one cheap picture look: `looks-unnatural-face`, `looks-unreadable-text`, `picture-contradicts-words`, `looks-unfinished`. - Size and overlap are only judged for text whose `size` (pixels) or `box` (`{x,y,w,h}`) your timeline states. ## 11. Disagree with a check, or report a film If our automatic check is wrong, or a film has a problem, tell us. It becomes a ticket we look into. - `POST /v1/jobs/{jobId}/report` `{ "message": "what is wrong", "kind"?: "check-wrong" | "film-problem" | "other", "advisory"?: "", "scene"?: 2, "contact"?: "ops@you.example" }` → `201 { ticket, statusUrl }`. Only the key that made the film can report it; with `X-On-Behalf-Of`, the ticket notes which of your customers it was for. Up to 20 reports a key a day. - `GET /v1/tickets` (`?status=open|looking|fixed|wont-fix`, `?jobId=`) lists yours; `GET /v1/tickets/{ticketId}` shows one: `status`, our `reply`, and the `thread`. - Don't agree with our answer? `POST /v1/tickets/{ticketId}/reply` `{ "message": "what is still wrong" }` reopens it. If you still disagree after that, the ticket goes to the owner of FuturizeNow Studio, who answers it there. - A report that reads as urgent (the film is unusable, shows the wrong person, is offensive, or is blocking you) goes to the owner at once. - Re-running a film (`POST /jobs/{jobId}/restart`) whose record shows a technical fault on our side (an engine failure, or a check such as `narration-cut-at-end`) opens a ticket for us by itself, marked `silent: true`, and you can follow it like any other. A re-run because the AI's pictures or words weren't what you meant doesn't. - Show the reply to your user as it is; only we change a ticket's status.