WildflowerJS Logo WildflowerJS

Trillium: three.js in a scene

A Trillium view whose canvas sits inside a scene is drawn once per seek, so a three.js scene renders, checks and plays as the rest of the composition does.

Create the view's renderer with preserveDrawingBuffer: true, or the canvas can read back blank when it is captured. Aim the camera from time in seek(local), so it is in place before any binding reads it. Call renderer.compile(scene, camera) in init(), so the shaders compile before frame 0 and not in the first seek that draws the view.

A view in a scene

A Trillium view whose canvas sits inside a scene is drawn once per seek, once the seek's footage has landed, through view.draw() (Trillium 0.2), whatever its render and maxFps options say. The canvas then holds the frame being captured, and the draw counts toward the seek's time (KN-170).

init() {
    const renderer = new THREE.WebGLRenderer({ canvas, preserveDrawingBuffer: true });
    renderer.compile(world, camera);   // the shaders compile before frame 0
    view = wildflower.three.view('tower', { renderer, scene: world, camera });
}
A three.js scene in a Trillium view inside a scene, with the camera aimed from time in seek(local) and an HTML tag over the point (2, 1, 0) through projectPoint(); click the stage to pause.
  • A view whose canvas is elsewhere on the page draws on the framework's frame loop as on any page, and Kinetic does not draw it.
  • A view inside a hidden clip is not drawn, since it paints nothing there, and is drawn on the first seek that shows the clip; check() draws it on every seek.
  • A view that throws does not stop the other views' draws, and the seek rejects with its error.
  • The promise a seek hands to waitUntil settles after the Trillium draws, so the frame is captured only once the canvas holds it (The composition and the clock).

The renderer

Create the view's renderer with preserveDrawingBuffer: true, so the canvas still holds the frame when it is captured. Development builds warn once per view when it was not (KN-190), since a WebGL context without it can read back blank.

  • KN-190 is one of the KN-19x codes, WebGL in Trillium's views and effects (Diagnostics).
  • A view compiles its shaders in the first seek that draws it, which is one slow seek (KN-170). Calling renderer.compile(scene, camera) in init() moves that cost before frame 0.
  • A post-processing pass that draws through the view's draw option has shaders of its own that renderer.compile() does not build, so one composer.render() in init() compiles them before frame 0 (Depth of field in 3D).

Aiming the camera

Aim the camera from time in seek(local), so it is in place before any binding reads it and the view draws through the same camera. seek(local) runs with the clip-local time after every seek and once at init, before the frame's bindings are written (seek(local)).

seek(local) {
    camera.position.set(Math.cos(local) * 6, 2, Math.sin(local) * 6);
    camera.lookAt(0, 0, 0);
}

An HTML tag over a scene point

view.projectPoint(x, y, z) returns a scene point's place in the view's canvas, in CSS pixels, or null for a point behind the camera. A computed that calls it places an HTML tag over that point.

The view's before() runs when the view draws, after the frame's bindings, so a computed that calls view.projectPoint(), such as an HTML label over a scene point, would read the previous seek's camera.

state: { viewReady: false },
init() {
    view = wildflower.three.view('tower', { renderer, scene: world, camera });
    this.viewReady = true;      // the computeds run again now the view exists
},
seek(local) {
    camera.position.set(Math.cos(local) * 6, 2, Math.sin(local) * 6);
    camera.lookAt(0, 0, 0);
},
computed: {
    // <span class="tag" data-bind-style="tag">, over the point (0, 1, 0).
    // Reading local makes it run again on every seek.
    tag() {
        const p = this.viewReady && this.local >= 0 && view.projectPoint(0, 1, 0);
        return p ? { transform: 'translate(' + p.x + 'px, ' + p.y + 'px)' } : { display: 'none' };
    },
},

The view is made in init(), which runs after the computeds' first evaluation, so viewReady tells them it exists. Without it, a label stays empty on the first frame when the clock does not move between the mount and that frame.

A scene that keeps aiming in before() passes the computed its own camera aimed from the same time, and view.projectPoint(x, y, z, camera) projects through it, in the view's canvas pixels.

Footage as a texture

A footage clip drawn as a texture, a THREE.VideoTexture of the clip's <video>, needs view.video(texture) (Video textures), so the texture is uploaded at the frame the clip was sought to.

const screen = new THREE.VideoTexture(video);
tv.material.map = screen;
view.video(screen);

check() and render()

Each canvas in the composition, a Trillium view among them, reaches a render's file through a 2D copy laid over it and updated after every seek (Canvases in a render). A WebGL canvas needs preserveDrawingBuffer for its pixels to be read.

  • In check(), a WebGL canvas without preserveDrawingBuffer reads back blank in both passes, so it cannot fail there (check(options)).
  • A render with a shutter averages several seeks inside each frame, and blurs a view as it blurs everything that moves (The shutter).

More