AI with the Finch in Snap!

In today’s world, artificial intelligence and machine learning are constantly in the news, and we are surrounded by devices collecting and using data about us. While the terms artificial intelligence and machine learning are often used interchangeably, they do mean slightly different things. Artificial intelligence refers generally to any system that enables a machine to make a decision, while machine learning refers to systems that use large amounts of data to create models that make predictions. We are focusing on the latter here, so we will use the term machine learning going forward. 

These hands-on activities are designed to enable you to create and test your own machine learning models. You will use your models to write programs with the Finch so that you can get a better idea of the power of these models when used in software. For example, this program recognizes a Star Wars character and makes the Finch perform a song or movements based on which character is detected.

As you complete these activities, think about examples of machine learning that you have heard or read about. What are the advantages of machine learning? What are the potential pitfalls?

Before you get started, get an overview of how machine learning works.

Basics of Machine Learning

The first three activities use Google Teachable Machine to create machine learning models that you can incorporate into your Snap! programs. These models are quick to create online but will enable you to develop some intuition for how machine learning works and how it depends on the data that you use to create it. There are three types of machine learning models that you can create: image recognition, audio recognition, and pose recognition. You will find an activity for each below. In each activity, you will create and test a machine learning model and then use it in a Snap! program. 

In Activity 4, you will create your own machine learning model. The mathematics behind such models can be very complicated, but this activity will use a simplified model to give you an idea of what is happening “behind the scenes” in machine learning.

Activity 1: Image Recognition

Activity 2: Audio Recognition

Activity 3: Pose Recognition

Activity 4: Sketch Recognition

Race after Technology by Ruha Benjamin: Provides a discussion of how technology is not neutral but can be racist and further racist aims, a process Benjamin terms the New Jim Code. Excerpts from this book would be an excellent basis for a discussion of ethics and justice in machine learning.

Coded Bias by Shalini Kantayya: An excellent documentary introducing the ethical issues involved in artificial intelligence and machine learning.

Catfishing on CatNet by Naomi Kritzer: This novel features a machine learning system that has become self-aware and provides an opportunity to discuss the kinds of data collected about us by these systems and how they might be used.

Weapons of Math Destruction by Cathy O’Neil: An engaging and accessible account of how algorithms affect our lives and the bias these algorithms incorporate the biases of society at large. Students may be particularly interested in how algorithms affect the college admissions process.

We would like to acknowledge the work of Ken Kahn and Jens Moenig. Ken’s Snap! project for machine learning was used as a reference when developing Activities 1-3, and Jens’ video tutorial is the basis for Activity 4.