This Saturday is NEUCS where I’ll be displaying my poster and giving a presentation on WiiPaint.  The same poster will be presented at next weekend’s CCSCNE in Plattsburgh, NY, and WiiPaint will be showcased (in a gallery setting!) at Wellesley for the Boston CyberArts Festival.  The honors thesis paper is due on the Tuesday after that (4/28), immediately followed by another presentation at Wellesley’s Ruhlman conference (4/29). My thesis defense will be on May 15 at 11am.

Whew! I’m very fortunate to be working on something that I’m so excited and proud of I wish I could perfect it some more, but new adventures await me this summer and beyond! I will be staying at Wellesley to do research on supporting scientific discoveries using reality-based interaction (Android! MS Surface!), and this fall I will be joining Carnegie Mellon’s HCII.

Onward!

WiiPaint demonstration 2 from Chloe Fan on Vimeo.

It was clear that if I wanted users to be more natural and fluid with their body movements, I had to do something about the Wiimotes’ affordances. How can we change the look and feel of the Wiimote so that users will approach them a different way? One thought that came up during my weekly thesis meeting was that perhaps the flat front of the Wiimote (where the IR camera is) gave it too much of a point and click remote feel, and that the solution was to create something round that would imply more of a rolling and fluid motion.

We played around with some old Mac speakers, taking out the speakers and wires and putting the clear casing on either end of a Wiimote. The result was a 3-4lb dumbbell-shaped object that was throwable and kickable, but most importantly, made the Wiimote a friendlier device that allowed for more naturally fluid movements and elicited no urge to press the buttons!

I didn’t quite like the weight of the Mac speakers, and they had to be taped together, which made reconnecting the Wiimote/changing its batteries difficult. It was “clunky.” So I went to PetSmart and bought a 7″ hamster ball and crafted a “booster stand” out of foam so that the Wiimote fit snugly inside the ball. I also created a Wiimote coffin — a case for the Wiimote that covered the buttons. It is currently shaped like a brick but I plan to sculpt it wand-shaped.

In addition, I finally put together some parts I ordered from RadioShack.com onto a small breadboard to create an IR LED light array. Each LED can be turned on/off with a switch, and controls a differently-colored dot in a rough IR interface within WiiPaint.

I plan to be finished with WiiPaint by the end of spring break (3/29). After that, I will run some more studies, hopefully in a gallery setting, and start writing the thesis. Next month, I’ll be preparing for the following conferences:
NEUCS
CCSCNE
CHI 2009
ACM Creativity & Cognition
Wellesley Ruhlman
Boston CyberArts

Onward!

At the Cirque, I informally tested WiiPaint with everyone from kids to students to parents, observed their interactions, and marked down some of their suggestions.

- B button to toggle is drawing. Users naturally place fingers on trigger button.
- Confusion as to which dot is whose. Need to find a way to either make it obvious, or remove any distinction to add to the “wholeness” of the application.
- Questions about what the buttons do — should brainstorm ways to make a case for the Wiimotes. Perhaps they can be colored differently to match their respective drawing dots on the screen.
- Unless they are told what to do, users first experiment with xy-plane movements (point and click type movements), but when initially told how to interact, they get it almost right away. Likewise, when told what it does and why, users seem to like it a lot more than just exploring with it.
- Kids love shaking the Wiimote
- It would be cool to add in interaction between Wiimote circles, for example when two streams collide, they will merge colors, or, as one video-game fan suggested, “battle” each other. (Cool idea, but that probably won’t happen ;))

Anyways, stay tuned for some more video clips next week! In the meantime, I’ve created my first Youtube video of a screen capture of WiiPaint

- Wiimote response is slow right now. The bomomo circles are following a tiny dot that rolls around the screen according to the Wiimote’s accelerometer values. This can be easily fixed by increasing the speed of the dot and circles, and will allow users to make more dramatic gestures.

- We want to see support for bigger gestures and features (visual/aural cues) that encourage the creation of these gestures. What will encourage users to “dance” and make other full-body movements? Brainstorm movement qualities we want to capture, such as waving, dancing, throwing, etc.

- Visual effects similar to the iPhone app Spawn Illuminati’s double-tap fireworks effect. Implement similar effects for certain types of gestures. Encourage users to want to try out different things to explore these effects.

- Music is also a powerful tool if we want to influence users’ interaction with WiiPaint, i.e. soothing music will probably encourage slower movements. At the very least, I have found an instrumental music loop that I can use, but it would be nice if I could find one that can be overlapped over it to create another dimension.

- Create a tighter border so that the Wiimote dot pointer will not seem to stray away from the screen. Add interactivity to the dot, such as bouncing off the border, so that it does not stay at the edge of the screen.

- Add some fun physics metaphors, such as ball simulation, to make idle states more interesting. On a similar note, implement animation such as fading when idle so that users will want to continue using it.

These are some great suggestions and I can’t wait to implement them!

There are four basic navigational controls in Google Earth:

(Note: Pitch is tilting the Wiimote up/down. Roll is tilting the Wiimote left/right. IR refers to the Wiimote’s infrared motion sensing capabilities, i.e. pointing on an xy plane.)

Zoom

Slider on right (mouse click)
Scroll wheel up/down (mouse)
Right click up/down (mouse)
Ctrl+Shift+Up/Down (keyboard)
=/- (keyboard)
Pinch (iPod gesture)
Double tap (iPod gesture)
+/- buttons (Wiimote)
B+IR (Wiimote)

Pan

Hand wheel (mouse click)
Drag (mouse)
WASD (keyboard)
Up, Down, Left, Right (keyboard)
Drag (iPod gesture)
Pitch/Roll (Wiimote)
A+IR (Wiimote) 

Tilt

Up/down on eye wheel (mouse click)
Ctrl+Drag up/down (keyboard/mouse)
Shift+Drag up/down [centered] (keyboard/mouse)
Ctrl+Up/down (keyboard)
Shift+Up/down [centered] (keyboard)
Tilt up (iPod gesture)
B+Pitch (Wiimote)

Rotate

Round slider (mouse drag)
Left/right on eye wheel (mouse click)
Right click left/right (mouse)
Ctrl+Drag left/right (keyboard/mouse)
Shift+Drag left/right [centered] (keyboard/mouse)
Ctrl+Left/right (keyboard)
Shift+Left/right [centered] (keyboard)
2-finger rotate (iPod gesture)
B+Roll (Wiimote)
A+B+IR (Wiimote)

It’s quite interesting to note that the Wiimote, as a tangible user interface and game controller, is used similarly to keyboard and mouse gestures. The buttons act as a keyboard replacement, and the Wiimote’s IR capabilities are used for pointing as in a mouse. However, when relying on the Wiimote’s accelerometers (pitch/roll), the functions are more gesture-based, i.e. rotating and tilting the Wiimote to create similar functions on the screen.

If there are buttons or keys involved (keyboard/mouse, Wiimote), they will inevitably be used in interface navigation. The iPod Touch, which features mainly a touch-screen interface, uses intuitive finger gestures for navigation. These devices are designed for a purpose with a specific type of interaction in mind, and the Wiimote’s is gaming-related. For example, in a bowling game, a computer’s interface might involve clicking and dragging to roll the ball, the iPod Touch would require a “rolling” gesture with the device as if one was throwing it as a bowling ball (ex. iBowl), and a Wii game would involve a similar rolling gesture with the Wii remote (Wii Sports Bowling).

One of the research questions that my thesis tackles is, how can we move beyond using the Wiimote as a traditional game controller (similar to function mapping, like on a keyboard), and take advantage of its gesture recognition features? For inspiration, we can look to the kinds of gestures used in Wii games. We can design applications where gesture-based interaction is more intuitive than a point and click interface. Similarly, we can take an existing application (like Google Earth) and re-map its functions to fit a gesture-based user interface.

- Test a GlovePIE script to control Google Earth with a Wiimote.
- Research other ways of using Google Earth with the Wiimote. Google at Macworld had a program that allowed you to “surf” in Google Earth using a Wii Balance Board.
- Brainstorm ways one can use 2 Wiimotes and gestures (without IR) to control Google Earth.
- Short writeup about interaction methods in Google Earth coming soon.
- Read a few recent ACM articles that mention Wiimotes. Nothing new; Wiimotes are still used as a game controller.
- Got WiiFlash 0.4.5 running on Vista using Toshiba’s Bluetooth Stack 6.10.10. This means I no longer need to rely on VMWare to use Windows XP!
- Last WiiPaint 1 update soon, then working on WiiPaint 2.

Some inspirational projects that I found:
- Acura Interactive Oracles – An interactive tabletop interface developed for Acura. The interface allows multiple users to play with it, creating shapes, sounds, and interactions between users.
- reactable – The reactable is a collaborative electronic music instrument with a tabletop tangible multi-touch interface that allows several people to compose music using blocks representing different components of a classic modular synthesizer. This is a really cool device that is intuitive and user-friendly.

The default path was creating WiiPaint 2, which I had been excited about for half this entire time because I thought I could finally create something that would “fix” everything that was off with WiiPaint 1.  This research approach would be “evolutionary” in the sense that it is incremental and makes small steps each time to improve something.  I feel that this is very one track minded and potentially trapping — there is no guarantee that WiiPaint 2 will be the ultimate project that I am hoping it will be.  There is also the danger of striving for perfection, which would have been an issue with my original thesis idea of creating a 3D game — it will not be as good as professionally-made 3D games that already exist.  WiiPaint is bound to come up with new issues that I will continue to have to fix in the search for the “perfect” collaborative art program.

Here is a screenshot of using bomomo, which was a huge inspiration to what WiiPaint 2 would look like created by multiple users.

Inspired by a recent trip to the MIT Media Lab to “think outside the box,” the “revolutionary” approach to research would be to complete scrap the idea and come up with something new and different.  Confused and torn about starting something new, I looked back to my original thesis vision (independent of WiiPaint) for inspiration.

To explore Wiimote capabilities outside of competitive gaming and to better understand and envision Wiimote capabilities in different applications.

Continuing with WiiPaint would have no doubt explored artistic possibilities using the Wiimotes, but it leaves a lot of other areas to be desired.  It has become an issue of whether I want to try to do one thing really well, or try to do many things.  You could argue that either path is the “safer” path, but I’m more interested in which one will better match my vision and passion for this thesis project, and that is the latter.  Overall I’d feel I can accomplish more by not limiting myself to WiiPaint but expanding into other applications as well.

One idea that came up while brainstorming was the GlovePIE script for Google Earth where Google Earth is controlled by one Wiimote.  In conjunction with research done in human-robot interaction where Wiimotes are used for robotic navigation control and robotic posture control, I could create a GlovePIE script that used gestures to control navigation in Google Earth.  The current script only supports one Wiimote and basically maps some of the functions onto the buttons, but I want to figure out how to navigate using gestures with two Wii remotes.  I’ll start with this idea during Wintersession and see where it goes.

It could mean that WiiPaint 2 is among this new series of applications, but in any case, I really look forward to breaking free of the conception that the Wiimote is a game controller and use it as a tangible user interfaces for many interesting research questions, wherever that may lead this thesis project.  To end, here is a sneak peak of the WiiPaint website layout, inspired by Nintendo