ObjectDetector()

Usage of machine learning algorithms to detect objects on an image/video and their location

ObjectDetector uses either CocoSsd or YOLO model to detect objects on the screen.

CocoSsd

CocoSsd (Coco Single Shot MultiBox Detection)

From the creators website:

This model detects objects defined in the COCO dataset, which is a large-scale object detection, segmentation, and captioning dataset. You can find more information here. The model is capable of detecting 90 classes of objects. (SSD stands for Single Shot MultiBox Detection).

YOLO

You only look once (YOLO) is a state-of-the-art, real-time object detection system.

From the creators website:

Prior detection systems repurpose classifiers or localizers to perform detection. They apply the model to an image at multiple locations and scales. High scoring regions of the image are considered detections. We use a totally different approach. We apply a single neural network to the full image. This network divides the image into regions and predicts bounding boxes and probabilities for each region. These bounding boxes are weighted by the predicted probabilities. Source

This implementation is heavily derived from ModelDepot.

Example

const video = document.getElementById("video");

// Create a objectDetector object
const objectDetector = ml5.objectDetector('cocossd', modelLoaded);

// When the model is loaded
function modelLoaded() {
  console.log("Model Loaded!");
}

// Detect objects in the video element
objectDetector.detect(function(err, results) {
  console.log(results); // Will output bounding boxes of detected objects
});

Here is a complete example.

Syntax

ml5.objectDetector(modelNameOrUrl);
ml5.objectDetector(modelNameOrUrl, ?callback);
ml5.objectDetector(modelNameOrUrl, ?options, ?callback);
ml5.objectDetector(modelNameOrUrl, ?suject, ?options, ?callback);

Parameters

  • modelNameOrUrl - Name of the model being used. Current available models are cocossd and yolo.
  • subject - Optional. HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData, p5 image element, or p5 video element.
  • options - Optional. An object describing a model accuracy and performance. For YOLO this are: { filterBoxesThreshold: 0.01, IOUThreshold: 0.4, classProbThreshold: 0.4 }
  • callback - Optional. A function to run once the model has been loaded. If no callback is provided, it will return a promise that will be resolved once the model has loaded.

Properties

.isPredicting

Boolean to check if the model is currently predicting

.modelReady

Boolean to check if the model has loaded

Methods

objectDetector.detect(input, ?callback)
objectDetector.detect(?callback)

Given an image or video, returns an array of objects containing class names, bounding boxes and probabilities.

input - A HHTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData, p5 image element, or p5 video element. If no input is provided, the default is to use the video given in the constructor.

callback - A function to run once the model has made the prediction. If no callback is provided, it will return a promise that will be resolved once the model has made a prediction.

Source

/src/ObjectDetector