To program using Speech to Text blocks, access the additional blocks available in Machine Learning for Kids, Scratch 3.0 project editor. The standard Scratch 3.0 version does not have speech to text blocks.
Image: Screen capture of Machine Learning for Kids Scratch 3.0 program editor showing Speech to Text blocks
Create this simple script to test the Speech to Text functionality. Add the Speech to Text blocks by selecting the blue Add extension at the very bottom of the code blocks (see it in the screen capture above):
If possible, use a headset with a microphone to limit noise while students program and test by speaking into the microphone. This simple script moves the sprite (cat) left or right depending on the voice command.
Use Machine Learning for kids to incorporate AI into the digital solution. This tool requires a one-time set-up to enable students to train and access an AI model.
The following matrix shows which tools are required for each level in this resource’s Plugged activities.
|Level||Tools required||Artificial Intelligence used|
|1 Easy (binary switch)||Scratch 3.0||No|
|2 Easy (speech to text)||Scratch Editor in Machine Learning for Kids (not Scratch 3.0)||Yes. An inbuilt form of AI in this version of Scratch is used.
No training of this model is required as it has already been trained.
|3 Medium to high||Scratch Editor in Machine Learning for Kids and Machine Learning for Kids (train, test and make a model)||Yes. Training of the AI model is required. This model is then used in the Scratch program environment in Machine Learning for Kids.|
Discuss home automation. Ask, ‘How might voice commands be used in home automation?’ Consider aspects of home automation such as:
Discuss the role of AI in performing the automation. This may prompt students to mention the use of personal assistants.
Ask students to consider how home automation can or might assist those living with disability.
With new technologies there are often risks to consider. Ask what could go wrong. Discuss risks such as privacy breaches from a system being hacked or times when the system might be down due to a power outage (including one caused by a natural disaster such as fire or flood).
Use these suggested activities to explore implementing a digital solution that demonstrates how to control appliances, and to further investigate home automation.
Use Scratch 3.0 or a similar visual programming language (without any form of AI) to create a program that switches appliances on and off using an input: 0 or 1. This task demonstrates the use of binary digits to change state; for example, ‘On to off’ or ‘Off to on’. As this program does not use any form of AI, use it to demonstrate how conventional coding is used to hard-code a particular action.
Depending on students’ familiarity with and understanding of binary numbers, review the use of 0 and 1 and explain that binary means ‘two states’. The two states can be represented in different ways, such as by the numbers 1 and 0, or by text (‘true’ and ‘false’, or ‘on’ and ‘off’). The main point here is that a binary device can be in just one of two possible states. A binary bit is a single on/off value.
In the two-part example below, the sprite (light) changes costume (has a yellow fill) to show an illuminated light.
Image: Screen capture of Scratch 3.0 program giving user instructions for using either 0 or 1 as input
Image: Screen capture of Scratch 3.0 program showing if/then blocks based on user input answer (1), to turn light on
For sample code, see Binary switch light on/off.
This option uses Speech to Text blocks accessed in the additional blocks available in Machine Learning for Kids, Scratch 3.0 project editor.
Note: Speech to Text blocks are not available in the standard version of Scratch 3.0.
Present the challenge of creating a program that recognises the students’ voice commands to perform an action such as turning home appliances on or off.
Discuss the types of voice commands that might typically be used, for example:
Emphasise short commands that are easy to understand.
Initiate each sprite by connecting the when (green flag) clicked block to Listen and wait. Use the when I hear block for each sprite to recognise a command and carry out the programmed action. An action may switch the house light sprite’s costume to the ‘On’ version.
Image: Screen capture of Scratch 3 program, showing that ‘lights on’ is the recognised action in response to that voice command.
Use the cat sprite to say on screen what the AI recognised from the user’s speech. It provides a helpful visual of the recognised instruction for the times when the voice recognition process isn’t accurate.
Image: Screen capture use of Scratch 3 program, showing the use of when I hear blocks to initiate action
View this short video to see the program in action.
Here is a flowchart representation of the steps described above:
Image: Flowchart representation of a sample completed program
Teacher: Visit the Machine Learning for Kids projects page. Select the project Smart classroom. There are several downloads available for it: one for you and several versions for students that include a step-by-step guide, with explanations and colour screenshots for students to follow. Note: although the guide is labelled ‘beginner’ on the site, relative to our activities we rate it at medium to high.
See the following screen shot for the main script students would use. The greatest difference between it and the example of code above is that the AI here is trained by the students and then invoked to determine if a voice command has been recognised. The screenshot below shows the additional black Machine Learning for Kids blocks.
Image: AI model incorporated into Scratch 3.0 using additional Machine Learning for Kids blocks
Ask students to do the project, and to pay attention to ways the AI responds to the different commands. Here are some further questions for them to consider:
Share what you have learned about AI and how ‘smart’ a computer can be. Think about the program you created:
Algorithms and programming are essential to developing machines powered by artificial intelligence (AI). AI is the ability of machines to mimic human capabilities in a way that we would consider 'smart'.
In conventional programming the computer is provided with a set of instructions for a defined set of scenarios. In the binary program, the students hard-coded the program with specific inputs of 0 or 1 to turn the appliance off/on. To include an AI model we used Machine Learning for Kids to use the AI to recognise speech commands as ‘turn on’ or ‘turn off’ for each appliance.
Machine learning (ML) is an application of AI. With machine learning, we give the machine lots of examples of data, demonstrating what we would like it to do so that it can figure out how to achieve a goal on its own. The machine learns and adapts its strategy to achieve this goal.
This lesson focuses on: