poseNet()

PoseNet is a machine learning model that allows for Real-time Human Pose Estimation.

PoseNet is a machine learning model that allows for Real-time Human Pose Estimation.

PoseNet can be used to estimate either a single pose or multiple poses, meaning there is a version of the algorithm that can detect only one person in an image/video and one version that can detect multiple persons in an image/video.

The original PoseNet model was ported to TensorFlow.js by Dan Oved. Check out his blog post.

Example

const video = document.getElementById("video");

// Create a new poseNet method
const poseNet = ml5.poseNet(video, modelLoaded);

// When the model is loaded
function modelLoaded() {
  console.log("Model Loaded!");
}
// Listen to new 'pose' events
poseNet.on("pose", function(results) {
  poses = results;
});

Here is a complete example.

Syntax

ml5.poseNet(?video, ?type, ?callback)
ml5.poseNet(?video, ?options, ?callback)
ml5.poseNet(?callback, ?options)

Parameters

  • video - Optional. A HTML video element or a p5 video element.
  • type - Optional. A String value to run 'single' or 'multiple' estimation.
  • callback - Optional. A function to run once the model has been loaded. If no callback is provided, it will return a promise that will be resolved once the model has loaded.
  • options - Optional. An object describing a model accuracy and performance. The available parameters are:
{
 imageScaleFactor: 0.3,
 outputStride: 16,
 flipHorizontal: false,
 minConfidence: 0.5,
 maxPoseDetections: 5,
 scoreThreshold: 0.5,
 nmsRadius: 20,
 detectionType: 'single',
 multiplier: 0.75,
}

For more information about these options please refer to the original PoseNet code.

Properties

.detectionType

The type of detection. 'single' or 'multiple'

Methods

.singlePose(?input)

Returns and array with a single object:

[
  {
    pose: {
      keypoints: [{position:{x,y}, score, part}, ...],
      leftAngle:{x, y, confidence},
      leftEar:{x, y, confidence},
      leftElbow:{x, y, confidence},
      ...
    }
  }
]



.multiPose(?input)

Returns and array with a multiple objects:

[
  {
    pose: {
      keypoints: [{position:{x,y}, score, part}, ...],
      leftAngle:{x, y, confidence},
      leftEar:{x, y, confidence},
      leftElbow:{x, y, confidence},
      ...
    }
  },
  {
    pose: {
      keypoints: [{position:{x,y}, score, part}, ...],
      leftAngle:{x, y, confidence},
      leftEar:{x, y, confidence},
      leftElbow:{x, y, confidence},
      ...
    }
  }
]

For more details about the poseNet outputs, read more here



Given an image or video, returns an array of objects containing pose estimations using single or multi-pose detection

input - A HTML video or image element or a p5 image or video element. If no input is provided, the default is to use the video given in the constructor.

Event Listeners

.on('pose', function(results){
  console.log(results);
})

Triggers every time there's a new pose detected. If you create a new poseNet method with a video element, this Event Listener will be called continuously over the video frames. Returns an array of objects containing pose estimations using single detection

Source

/src/PoseNet