Canvas plugins

The canvas plugins are used to create the surface where the video is rendered. The canvas plugins are responsible for creating the HTML element that will be used to render the video. In conjunction with the video plugins, they are responsible for rendering the video stream.

The canvas plugin integrates with the paella player plugins system and is responsible for deciding if the plugin is compatible with a given stream, and for creating the instance of the canvas implementation. As many canvas will be created as many streams are visible in the video, and to determine if the canvas is compatible, the canvas attribute of the stream obtained from the video manifest will be used. This attribute is an array, since there may be more than one applicable canvas type, and the first canvas type that is compatible with the current paella-core configuration will be chosen.

Video manifest:

{
    ...
    "streams": [
        {
            "sources": {...},
            "content": "presenter",
            "canvas":["video360","video"]
        }
    ]
}

Define a CanvasPlugin

The following code corresponds to the most basic canvas plugin: VideoCanvasPlugin, which implements the video via an HTML <video> element.

import { CanvasPlugin, Canvas } from '@asicupv/paella-core';

// Canvas implementation
export class VideoCanvas extends Canvas {
    constructor(player, videoContainer) {
        super('div', player, videoContainer);
    }

    async loadCanvas(player) {
        player.element.style.width = "100%";
        player.element.style.height = "100%";
    }
}

// Canvas plugin definition
export default class VideoCanvasPlugin extends CanvasPlugin {
    get canvasType() { return "video"; }

    isCompatible(stream) {
        if (!Array.isArray(stream.canvas) || stream.canvas.length === 0) {
            // By default, the default canvas is HTML video canvas
            return true;
        }

        return super.isCompatible(stream);
    }

    getCanvasInstance(videoContainer) {
        return new VideoCanvas(this.player, videoContainer);
    }
}

Canvas implementation

When the StreamProvider loads the video streams into the video container, by default a <video> element containing the video player is created. Once the video is loaded, for which a plugin of type VideoPlugin is used, the loadCanvas() function is called. In this function we can make the transformations that we consider relevant, which can be from modifying the style of the <video> element to replacing it with a WebGL <canvas>.

Canvas API

get canvasType(): returns the label that must match with the canvas attribute in the stream, at the video manifest.

isCompatible(stream): returns true or false depending on the stream canvas attribute. You can use this function to determine the compatibility of the stream in a more complex way. The default implementation will use the get canvasType() attribute to determine if the stream in the video manifest is compatible with the canvas plugin:

// Default implementation: check if the stream in the video manifest
// contains the canvas type defined in this canvas plugin
isCompatible(stream) {
    if (Array.isArray(stream?.canvas)) {
        return stream.canvas.indexOf(this.canvasType) !== -1;
    }
    else {
        return stream.canvas === this.canvasType;
    }
}

getCanvasInstance(videoContainer): Factory method that returns the canvas instance. This function will be called when Paella Player determines that this canvas type is appropriate, using the isCompatible() function and the get canvasType() attribute.

User area

Video canvas plugins have an API that allows adding DOM tree elements externally, for example, to create UI elements that can be added on top of each video stream canvas.

Each video canvas has an element that is created on demand called user area. This is a <div> element that is placed right in the video canvas area, and is drawn with the following CSS properties:

.video-container .user-area {
  position: absolute;
    width: 100%;
    height: 100%;
    top: 0px;
    left: 0px;
    pointer-events: none;
}

The following code is an example of using the user area to add a button externally:

const streams = player.videoContainer.streamProvider.streams;
for (const content in streams) {
    const stream = streams[content];
    const { canvas } = stream;
    const button = document.createElement("button");
    button.innerHTML = "Test"
    button.style.position = "absolute";
    button.style.top = "3px";
    button.style.left = "3px";
    button.style.pointerEvents = "all";
    button.click = () => alert("Click");
    canvas.userArea.appendChild(button);
}