A potential future area of research for Sensory Objects is how to make programming easier for people with limited ability to use a computer. Previous iterations of the project have used Arduino microcontrollers, which are programmed in a derivative of C. Though it’s relatively simple for those with programming experience, it does require some knowledge of programming concepts like loops, decisions, and logic. Users also need to have reasonable vision, typing ability, and sufficient dexterity to connect cables and wires to the Arduino boards.
To allow a wider audience to program, we would like to circumvent the computer usage part by providing a ‘tangible interface’ made up of physical objects which users can manipulate to create computer-readable instructions.
Many projects have examined this before us. Here are a selection of them.
Digitally Ready – a project run by Nic and Kate to get Art students using Arduinos.
Mostly focused on LEDs, buttons and pressure sensors. Arduino control appears to have been minimal, main use of them was for powering Squishy Circuits.
Tern is a method of programming Turtles by the use of wooden blocks which have computer-vision readable images printed on them. The blocks represent a single instruction, like “go forwards”, “turn”, or “beep”.
The blocks are not at all electronic, and once a program is ‘written’ it has to be read and interpreted by a computer with a webcam.
No branching, logic, or loops are possible- all programs are straight lines of instructions.
littleBits are small PCBs with magnetic connectors at either end. They are color-coded by category, and a single ‘bit’ is either an input (sensor) or output (effector). The bits signal to each other and provide power through their connectors.
Programming concepts with the default set of bits are limited to IF (IF this switch is pressed, THEN turn on an LED) and AND (IF this switch AND this light sensor are activated, THEN turn on an LED). OR and NOT bits (‘inverter’) are available, as are dedicated AND bits. No loops are possible, but limited timing functions are present.
Tangible Programming Bricks are Lego bricks with embedded microcontrollers. Each brick performs a specified function. The bricks are stacked on top of each other to create a program. They communicate through electrical contacts on their top and bottom sides.
Individual bricks’ functions can be modified by inserting ‘smart cards’ into a socket on one end, altering the time a brick is active for usually.
Some bricks contain sensors or effectors such as IR communications, allowing multiple stacks to interact.
Loops and conditional statements are possible and encouraged, as well as logic.
Paper contains a detailed critique of the concept in the Discussion section.
Electronic Blocks are Duplo bricks with simple embedded electronics, much like LittleBits. Blocks are groups into categories by colour, and images are printed on the sides of the blocks to indicate their functions. Programs are created by stacking bricks on top of each other. Signals flow from the top of the stack to the bottom.
Sensor blocks, effector blocks, and logic blocks are present. Conditional statements and AND, OR, etc are possible, but no looping or branching.
Reactable is a sound synthesis device which consists of a translucent table on which blocks are placed. A camera reads the underside of the blocks to determine their function and position. It also tracks fingertips pressed against the surface.
A video projector provides extra UI elements on the table. These can be used as indicators or controls using the aforementioned fingertip sensing.
Once placed on the table, the blocks can be used to create or adjust various qualities of sound. Moving blocks close to each other causes them to link up and affect each other. Rotating blocks usually affects some aspect of their function.
StoryRooms are specialised rooms containing a variety of toy objects. A user enters the room with a ‘magic wand’ object, which can be used to define interactions between other objects. All objects present are wirelessly controlled by a researcher behind a one-way mirror via a computer. In response to the user performing an action intended to create a relationship between two objects, the researcher would attempt to provide a suitable reaction from the objects.
Programming in this way is done by performing the actions in ‘programming mode’ with the wand, the researcher observes and notes the required interaction, then recreates it whenever the same action is performed out of ‘programming mode’.
Cuboino is a set of wooden cubes with channels cut in them. Most cubes are passive and direct the marble through only the use of the channels. They can be stacked and arranged to create a ‘marble run’. Active cubes can be snapped together to create devices. Some cubes have sensors for light, sound, button presses, or marbles passing over them. Others emit light, drop marbles, redirect marbles, etc. Some cubes simply provide power or transmit signals.
Simple, stylised designs are used to indicate the function of each cube. All of them are made from plywood and are only really differentiable by the components visible on their surfaces.
GameBlocks are large (~300mm) cubes with bright orange symbols on their tops. They are used by placing them onto trays which sense an arrangement of magnets in each cube’s underside. A central microcontroller reads each tray sequentially and performs the action associated with the block.
No sensors are present- they are simply a method of instructing a sequence of movements by a robot.
Dr. Wagon is a method of programming a turtle (a wooden ‘wagon’ with a face on the front in this case), by means of a selection of snap-together wooden blocks. A ‘Start’ block is used to power the blocks, and communicate with the robot. Blocks are connected in a line under the Start block by four copper contacts, and held in place with magnets. Each block contains a PIC on a breadboard which communicates with the Arduino in the start block. When the start button is pressed, the Arduino polls the blocks under it sequentially. Each block sends the Arduino a single line of the program, which the Arduino then collates and transmits to the robot via radio. The robot’s on-board Arduino then compiles and runs the program.
Block functions are indicated by laser-cut writing on the top surface of the blocks. Program flow is also indicated by laser-cut arrows on the edges of the blocks. This does indent the wood slightly, making it readable by touch.
Loops and conditional statements are possible, but logic is not.
Topobo is a selection of plastic parts which connect via magnets, span connectors, and screws. Most parts are ‘passive’ in that they contain no motors or electronics. Some parts are ‘active’ and can move parts attached to them by means of motors.
Active parts can be placed into a ‘learning’ mode during which they record any movements they are subjected to. Once set back to playback mode, they repeat this movement in a loop. This allows users to program a model of their own creation by manipulating it into performing the desired action.