The Ultimate Machine: How would you test it?

As many of you know, we now have our Software Testing Space group in LinkedIn. A lot of exciting discussions are taking place there. If you have not joined it, you should consider doing so.

My friend, Freddy Gustavsson from our STS group posted an interesting challenge on testing the Ultimate Machine. Well, the Ultimate Machine is attributed to Claude Shannon (1916 to 2001). It consists of a simple box with a single switch on its top surface and a closed lid. If you flip the switch on, the lid opens, a mechanical hand arises from it and the hand flips the switch off. Thereafter, the hand goes back into the box and the lid closes. Freddy's challenge was that if he asked me to test this Ultimate Machine, which tests would I perform? Here are the questions that I thought of. I would like to design tests based on these questions.

Positive tests

a. If the tester flips the switch on, does the hand (or an arm in certain implementations) switch it off?

b. If the tester does not touch the switch, does the machine just sit there and do nothing?

Negative tests

a. What happens when the tester flips the switch on and switches it off before the hand can reach it?

b. Is it possible to flip the switch somewhere between the on and off positions? What does the machine do in such a case?

Non functional tests

a. Can the tester operate the machine as it is or does it need to be installed? If installed, what does it take to install it?

b. Once the machine is switched on, how long does it take to switch itself off?

c. How many times can be machine be operated  before it stops working (due to battery discharge, wearing off of mechanical parts and so on)? Does it slow down after a while or show operational problems (make hitherto unknown noise for instance)?

d. Does the machine operate correctly if the ambient conditions are changed?

   i. The machine is vibrated e.g. during travel or a minor tremor.

   ii. The machine is operated outdoors (e.g. during a dust storm or in rain/ sleet/ snow).

   iii. The machine is operated in a strong magnetic field (e.g. certain places on a factory floor or in a hospital).

   iv. The machine is operated in different light conditions (e.g. direct sunlight, in the office daytime with blinds open/ blinds closed, at dusk or in total darkness).

   v. The machine is operated in different temperatures (e.g. at room temperature, in a freezer or near a furnace).

Usability tests

a. Is it easy for the tester to locate the switch?

b. Is the switch easy to flip (works smoothly, clicks to indicate it has flipped)?

c. Is the switch safe for the tester to operate (no rough surfaces or pointed end)?

d. Is the box pleasing to look at? How much space does it occupy on the tester's desk?

White box tests

We could open the box. Then, we can try to understand the mechanism inside and create more tests based on our understanding. For e.g. I would like to know if the time it takes to switch off is configurable and is the machine connected to any external system.

I am sure that you can think of more questions to help test the machine. Did you notice that I used the various types of testing to think about the above questions? What approach would you have used to design tests for the Ultimate Machine?