The first step is to open Google Teachable Machine in order to create a pose recognition model.
Activity 3: Pose Recognition (Python)
Train and Test the Pose Classifier
Next, get ready to record your poses. Make sure that you are centered in the webcam view and that you are far enough away from the webcam to see blue lines appear on your trunk and arms. This means that the Teachable Machine is tracking your skeleton. You will need to remain at about this distance from the camera when using your model, so you may want to mark the position with masking tape.
Add samples for each of your classes. You will want to add 200-300 samples for each class. Because you must be some distance away from the camera, you may need a partner to help you take the samples.
Next, train the model. You may notice that this takes longer than training the image and audio classifiers. Make sure to leave the teachable machine tab open while the model is training, even if your browser pops up a warning that the window is unresponsive.
Keep all of the defaults as they are, and click Upload my model. After your model has uploaded, copy your sharable link. You will need this link to create a Snap! project with your model. Remember to save your model in case you want to reference or change it later.
Using the Pose Classifier inPython
Once you have loaded the model, you can use MachineLearningModel.getPrediction() to get the results of your machine learning model for the current pose. Print these results to the screen and run your program. Remember, it can take up to a minute for the classification to start the first time you run the script. As you use your classifier, make sure to stay centered in the webcam view and at the same distance from the camera that you were when you trained the model.
As always, MachineLearningModel.getPrediction() returns a dictionary. Each class label is a key for the dictionary, and the value that corresponds to that key is the probability that the current pose belongs to that class.
You can use the values returned by MachineLearningModel.getPrediction() to make the Finch perform certain actions when each class is detected. For example, this code changes the color of the Finch beak based on the pose that is detected. This can be a good way to check that your pose model is working properly in Snap!.
Challenge: Write a program to make the Finch respond to each of your poses. As you test your program, investigate what happens as you change your position in the camera view or move closer or further from the camera.