Skip to main content
Skip to main content

Home automation programming

Integrating Digital Technologies
Years 5-6

Investigate home automation systems, including those powered by artificial intelligence (AI) with speech recognition capability.

These suggested activities provide a level of differentiation to cater for students’ range of programming skills. Developed in collaboration with Digital Technologies Institute.

This set of lesson ideas is related to Home automation with AI, which explores how AI works within the context of home automation and raises topics including risks and also how these systems might assist people living with disability.



Decorative image

Image credit: Mike MacKenzie/ Flickr, CC BY 2.0


Preliminary notes

To program using Speech to Text blocks, access the additional blocks available in Machine Learning for Kids, Scratch 3.0 project editor. The standard Scratch 3.0 version does not have speech to text blocks.


 

 


Image: Screen capture of Machine Learning for Kids Scratch 3.0 program editor showing Speech to Text blocks

Test Speech to Text set-up

Create this simple script to test the Speech to Text functionality. Add the Speech to Text blocks by selecting the blue Add extension at the very bottom of the code blocks (see it in the screen capture above):

If possible, use a headset with a microphone to limit noise while students program and test by speaking into the microphone. This simple script moves the sprite (cat) left or right depending on the voice command.


Use Machine Learning for kids to incorporate AI into the digital solution. This tool requires a one-time set-up to enable students to train and access an AI model.

The following matrix shows which tools are required for each level in this resource’s Plugged activities.

Suggested steps

Unplugged activity

Discuss home automation. Ask, ‘How might voice commands be used in home automation?’ Consider aspects of home automation such as:

  • security and access
  • climate control
  • lighting
  • entertainment.

Discuss the role of AI in performing the automation. This may prompt students to mention the use of personal assistants.

Ask students to consider how home automation can or might assist those living with disability.

With new technologies there are often risks to consider. Ask what could go wrong. Discuss risks such as privacy breaches from a system being hacked or times when the system might be down due to a power outage (including one caused by a natural disaster such as fire or flood).


Plugged activity

Use these suggested activities to explore implementing a digital solution that demonstrates how to control appliances, and to further investigate home automation.


Use binary numbers 0 and 1 to switch a light on and off (skill level: easy)


Use Scratch 3.0 or a similar visual programming language (without any form of AI) to create a program that switches appliances on and off using an input: 0 or 1. This task demonstrates the use of binary digits to change state; for example, ‘On to off’ or ‘Off to on’. As this program does not use any form of AI, use it to demonstrate how conventional coding is used to hard-code a particular action.

Depending on students’ familiarity with and understanding of binary numbers, review the use of 0 and 1 and explain that binary means ‘two states’. The two states can be represented in different ways, such as by the numbers 1 and 0, or by text (‘true’ and ‘false’, or ‘on’ and ‘off’). The main point here is that a binary device can be in just one of two possible states. A binary bit is a single on/off value.

Suggested steps

  1. Students consider a home appliance they want to include in their program, for example a light or fan, then create the appliance as a sprite. Free versions of icons can be found online, saved to a folder and uploaded, or students can draw their own.
  2. Initiate each sprite using the block ‘when (green flag) clicked’.
  3. Use an ‘ask’ block to enable the user to input a 0 or 1.
  4. Use an if/then block to instruct what happens if ‘0’ is input.
  5. Use a second if/then block to instruct what happens if ‘1’ is input.

In the two-part example below, the sprite (light) changes costume (has a yellow fill) to show an illuminated light.

Image: Screen capture of Scratch 3.0 program giving user instructions for using either 0 or 1 as input

Image: Screen capture of Scratch 3.0 program showing if/then blocks based on user input answer (1), to turn light on

For sample code, see Binary switch light on/off.


Ideas for remix:

Students:

  1. create their own sprites for other appliances
  2. add a block of program code that turns the appliance on and off
  3. add a block of program code that controls multiple appliances
  4. change backgrounds.

Use speech recognition to switch appliances on and off (skill level: medium)

This option uses Speech to Text blocks accessed in the additional blocks available in Machine Learning for Kids, Scratch 3.0 project editor.

Note: Speech to Text blocks are not available in the standard version of Scratch 3.0.

Present the challenge of creating a program that recognises the students’ voice commands to perform an action such as turning home appliances on or off.

Discuss the types of voice commands that might typically be used, for example:

  • Turn on/turn off
  • Lights on/Lights off
  • Turn fan on/Turn fan off.

Emphasise short commands that are easy to understand.


Suggested steps

  1. Students consider a home appliance they want to include in their program, for example a light or a fan, and create it as a sprite. Free versions of icons can be found online, saved to a folder and uploaded, or students can draw their own.
  2. Initiate each sprite by connecting the when (green flag) clicked block to Listen and wait. Use the when I hear block for each sprite to recognise a command and carry out the programmed action. An action may switch the house light sprite’s costume to the ‘On’ version.

    Image: Screen capture of Scratch 3 program, showing that ‘lights on’ is the recognised action in response to that voice command.

  3. Use the cat sprite to say on screen what the AI recognised from the user’s speech. It provides a helpful visual of the recognised instruction for the times when the voice recognition process isn’t accurate.

    Image: Screen capture use of Scratch 3 program, showing the use of when I hear blocks to initiate action

  4. View this short video to see the program in action.

    Here is a flowchart representation of the steps described above:

    Image: Flowchart representation of a sample completed program


Ideas for remix:

Students:

  1. change the value of the answer to a different pin number or string
  2. create their own costumes for the smartphone
  3. add a block of program code that locks the screen after a certain number of failed attempts
  4. create supplementary screens to show after access.

Coding with Scratch 3.0 and incorporating an AI model (skill level: medium to high)

Teacher: Visit the Machine Learning for Kids projects page. Select the project Smart classroom. There are several downloads available for it: one for you and several versions for students that include a step-by-step guide, with explanations and colour screenshots for students to follow. Note: although the guide is labelled ‘beginner’ on the site, relative to our activities we rate it at medium to high.

See the following screen shot for the main script students would use. The greatest difference between it and the example of code above is that the AI here is trained by the students and then invoked to determine if a voice command has been recognised. The screenshot below shows the additional black Machine Learning for Kids blocks.

Image: AI model incorporated into Scratch 3.0 using additional Machine Learning for Kids blocks

Ask students to do the project, and to pay attention to ways the AI responds to the different commands. Here are some further questions for them to consider:

  1. How well did the AI recognise the voice commands?
  2. What did you learn about the training of an AI?
  3. When training the AI to recognise voice commands, background noise was part of the training data. Why was this included in the training data?

Discussion

Share what you have learned about AI and how ‘smart’ a computer can be. Think about the program you created:

  • What were some of the challenges?
  • Where the program used AI, what things affected how well it worked?
  • How might you improve the program if you did the task again?

Why is this relevant

Algorithms and programming are essential to developing machines powered by artificial intelligence (AI). AI is the ability of machines to mimic human capabilities in a way that we would consider 'smart'.

In conventional programming the computer is provided with a set of instructions for a defined set of scenarios. In the binary program, the students hard-coded the program with specific inputs of 0 or 1 to turn the appliance off/on. To include an AI model we used Machine Learning for Kids to use the AI to recognise speech commands as ‘turn on’ or ‘turn off’ for each appliance.

Machine learning (ML) is an application of AI. With machine learning, we give the machine lots of examples of data, demonstrating what we would like it to do so that it can figure out how to achieve a goal on its own. The machine learns and adapts its strategy to achieve this goal.

This lesson focuses on:

  • binary numbers (1, 0)
  • speech recognition
  • training an AI with suitable data.

Resources