User Designer

Today’s idea falls out of a few questions I’ve often wondered about: What is the role of subtle feedback in interaction design and human-computer interaction? How can we recognize design gaps, where subtle feedback would be useful or undesirable?

BTW this post was originally much too long and contained a few different interrelated ideas. Rather than posting a very long post I’m spreading the ideas and questions out over the next few posts.

Mindful Football is today’s fun idea, and it builds on my previously posted ideas around pico projectors. Imagine you’re about to kick a football, and the football knows exactly how you’re going to kick it before you kick it. The Mindful Football knows how hard you’re going to kick it, in what direction you’ll kick it, and it can predict how it’ll behave after you kick it. Of course the surface of the football is also a kickable smashable display, and the ball can project images all around itself (its got inbuilt 360 degree kickable pico projectors!).

Just as you’re about to kick the Mindful Football it predicts and shows you the results of your kick. Close by the football on the ground you see a projected blue arrow, which points in the direction the ball thinks it’ll go. Further in the distance, the ball also projects a big blinking orange circle showing where it’ll land, or what it’ll hit (yes, you want the orange circle to be in the goal or at a teammates feet). Would the real-time visual feedback about your kick enable you to change how you kick the ball, so that it goes where you want at a speed you want? If you used the Mindful Football when playing friendly games, would the real-time visual feedback improve your ball control skills, even when you switch back to using a normal football?

What other forms of visual feedback could be shown to you as you’re kicking the ball? For example, a star could be displayed on the video surface of the football. The location of the star indicates where you should kick the ball to send it in the direction of the goal. You can easily imagine this idea applied to lots of other sports, such as tennis, baseball, rugby, American football, etc.

Another visual feedback possibility includes having the surface of the football show lots of animated red dots, where each red dot indicates where a teammate is relative to the ball. When you have control of the ball and are looking at it, the purpose of the red dots would be to the enhance your awareness of your teammates positions. Would that really increase real-time situational awareness? Would it help you learn how to predict where your teammates normally are relative to you?

Mind you, if you were really dastardly and mischievous, you could hack the opposite team’s Mindful Football so it provides slightly inaccurate feedback. Potentially leading to the players becoming less skilled at kicking the football!

A Mindful Football is an example of providing real-time feedback at the moment of action, i.e. when kicking the ball provide feedback. What about cases where we achieve a goal by performing a lot of interrelated actions that are spread out over time, such as when we cook a meal? I’ll cover that in the next post.

Recently I’ve been debating posting 365 HCI / interaction ideas and inventions on User Designer, one idea a day for the next year. Like lots of other people, I generate and have far far more ideas than I’ll ever use, build or test. Posting an idea everyday for a year would be too time consuming, so what I’m doing instead is posting up a few ideas every week – in one or two blog posts. My aim is to post at least 100+ ideas.

Some of the ideas I’ll post are fun, others are tamer and move obviously useful. I enjoy the fun slightly left-of-field ideas as they’re good for HCI / design conversations, inspiration and follow-on idea creation.

Below I outline three related ideas, which I’ve called FlowOut TV, Pocket CAVE and Half-Real Toys.

FlowOut TV is the first idea and its a bit of fun. The idea is motivated by the question: Can watching TVs and/or playing games on TVs be made more immersive, without having to use 3D displays? Imagine if TVs and computer displays had multiple pico projectors built into them, where each projector points in a different direction. Shown above is a quick sketch of the technology for FlowOut TV. The projectors are mounted on motors so they can point in different directions and project images onto the walls, ceilings and floors around the TV. Included in the pico projectors are video cameras, which can see the area the pico projectors project onto (video cameras enable interactivity).

As you watch FlowOut TV, the projectors are used to enhance the video shown on the main screen. Now imagine you’re sitting at home watching a FlowOut TV, while the screen shows a man dashing along a city street. The street at his feet becomes part of your sitting room because the street is projected onto the floor in front of the FlowOut TV. The sky above the man’s head is projected onto the ceiling, and the crowds walking by the running man are projected onto the walls to the left and right of the FlowOut TV. As the story progresses, the FlowOut TV finds a picture frame in your sitting room, and begins projecting relevant parts of the story into that – such as photographs of the main character’s family. In yet other parts of the story the couch you are sitting on has an image of victorian fabric projected onto it, because the main character is visiting a victorian home. It may even be possible to have the faces of those around you “painted” by the projectors, so they look like characters in the film (give everyone watching Avatar blue skin?!). You can easily imagine lots of ways stories can be woven into your environment.

Now extend the FlowOut TV idea a little further, imagine your laptop screen has the same pico projectors builtin. Fun for computer games? Games could project gloves and body armor onto your arms and wrists…as you move your arms the projected armor would stay projected onto your body, while the walls around you flash with incoming blasts of laser fire! Or more sedately, Facebook status updates could be projected onto family portraits that are near your laptop.

Iterate the idea further, add in front and back facing pico projects and build it into a mobile phone. Suddenly you have a portable CAVE (Cave Automatic Virtual Environment), where any room you walk into can easily be turned into an immersive virtual reality environment. To turn a room into a CAVE, all you need to do is take the FlowOut phone out, place it on a stable surface (optional), let it examine the room so it can figure out the room’s shape and surfaces, then it can project images onto the walls and surfaces around you. Since it has builtin cameras, it can also track how you move around, and could make any surface interactive as long as the cameras can see your physical actions. Prototypes of a portable CAVE could easily be built – its idea number two and I’m calling it a Pocket CAVE (though I prefer the less snappy name CAVE-In-Yer-Pocket).

A final iteration is adding pico projectors with video cameras to kids toys. Imagine building it into a toy car, and as the car is “driven around” a road is projected in front of the car, or grandparents watching from 1000 miles away are projected into the room and they interactively draw the road for the car to follow. There are all kinds of Half-Real Toys (one part projected, one part real), where the act of playing with them in different ways leads to different interactive surfaces getting projected into the surrounding environment. For toys, I’m unsure whether adding projected images is a good or bad idea? It could easily be implemented badly, where the projection replaces the joy of imagining roads and worlds.

Right, enough for now – this post is getting too long. There are lots more ideas which fall out of the above, I’ll follow up with another few in the next blog post.

8pen is a novel alternative to the keyboard. I wonder is it any good and useful? How does it compare to Dasher?

Brain-Computer Interfaces (BCI) are on the rise – a recent Nature paper covers an experiment where measures of thinking are used to separate out images. The BCI research community is gaining traction – keep an eye out for the BCI 2011 conference. BCI products and startup companies are springing up – for example Neurosky have an interesting commercial consumer orientation BCI headset, and a recent toy game from Mattel called MindFlex mixes up a basic BCI sensor with puzzle solving.

Unfamiliar with Human-Computer Interaction quantitative experiment design and statistical tests? Here’s a short introduction to statistics for HCI research. And yes R is great for calcs – though its usability is distinctly lacking. If you want to get serious about HCI stats and experiment design I recommend reading The Whole Art of Deduction. It’ll help you get familiar with Independent & Dependent Variables, Repeat Measures, Between Subjects, Within Subjects, etc, etc. Another handy link is this reference page, which helps you figure out which statistical test to use. Finally, CrossValidated is extremely useful Q&A website for asking and answering questions about specific stats techniques.

Do you think is Divvy a potentially useful user interface tweak for enhancing desktop window management?

Very neat video of a research prototype, which shows off programmable foldable self-reshaping “paper”. How long before we see it in children’s toys? Or your cup is made of it? Or your trousers? Or your display?

Add in a dash of flight and we move in the direction of flying autonomous robots that can self assemble into larger flying structures. Learn more and watch the videos. What would you use them for? How could they be fused with the programmable paper?

Hearing, tasting, smelling, seeing, feeling ain’t enough? Develop a new sense – develop magnetic fingernails and “feel” magnetic fields.

Ice cream, ice cream every kind of flavor you’ve never heard of, or never imagined! I must go along and try some – sounds yum.

Image Swirl is an interesting variant of a graph layout information visualisation from Google Labs. So far I like using it to explore clusters of related images, though I wonder whether the clustering reduces the range of images I see. Do I get exposed to more images that are strongly related to each other and see less images that are unrelated to each other (but may still be strongly relevant to the search term)?

If you like Image Swirl you’ll enjoy playing with Wonder Wheel, also from Google Labs. Wonder Wheel displays a graph layout of text search results. Enjoy.

The Future of Reading Conference could be interesting – especially for the views of authors such as Margaret Atwood. How much do they think the medium matters? Why? Marshall McLuhan has more than a few thoughts in that direction. The conference is in Rochester Institute of Technology and is starting tomorrow.

Very neat – home fabricating “real devices” with Lego. See a video demo of a printer made out of Lego. Yup made from Lego – I especially like how the little Lego people help out! The future of making & user designing is creeping up on us.

Welcome to the nearly-ready nearly-there future! PaperComp 2010 sounds like a very interesting workshop planned for Ubicomp 2010. Their Call For Papers says “This workshop aims at bringing together researchers exploring the future of printed interfaces and associated practices. It will act as a kick-off event for launching a new research community, articulating how different research lines form a coherent emerging field. The proceedings of the workshop ambition to gather the visions of the most active researchers working on augmented reading practices and paper interfaces.”

Quick gimme an electronic paper origami actuator and we’ll add a splash of haptic interactivity….ooooh H Okuzaki et al., are already there doing that in A Biomorphic Origami Actuator Fabricated by Folding a Conducting Paper. Enjoy.

Shown above is the new Coca-Cola drinks dispensing machine. Originally I was delighted because I thought this was a pure co-creation play from a very well known brand – let people mix, match and completely make up new drink flavors – but apparently its not. Boo. Maybe version 2 will…100+ flavors will just have to do for now. Hello, paradox of choice…?

Life v2.0 – Scientists Create Synthetic Organism. Wonderful and impressive.

Get your dancing shoes! A variation on a well known theme, though still a bit of fun…bang bang bits of a jeep and make a good rhythm.

Hi from the United States – in the last month I’ve moved from Ireland and started a postdoc in the Psychology Dept in Stanford University. Woot!

Imagine if every building surface was a display…see this very neat video of a 3D projection onto buildings. I like the way their 3D projection takes advantage of the architecture of the buildings, and doesn’t treat the buildings simply as 2D surfaces. Go on, invent paintable displays – paint that turns into a display once it dries.

Deformable transformable carpet that turns into furniture, tables and seats from Shin Yamashita. Ideas for Version 2: Add in a dash of smarts so the carpet knows its shape, with some idea of its function and enable parts of it to be an interactive surface capable of self-reshaping.

Strictly come robot (competition) dancing – would you vote for Kingrass Hoppers or Nichibu Tokotoko Special?

I’m always a big fan of TED talks…and here’s a fascinating short 8 minute talk, by Pattie Mae’s from MIT Media Lab. She talks about and shows off SixthSense, which is an invention for turning any surface into an interactive gesture controlled video surface. Neat and easy enough to integrate into current mobile devices.

Bosch have enhanced the night vision system system in cars, so that it provides smartly enhanced high contrast images of the road ahead at night. Clever but would you trust it to properly identify which parts of the road are critical for highlighting?

For the last few years I’ve been using and really liked Sony’s eBook Reader, the PRS500. Yup, I was an early adopter and altogether unsure whether I wanted to give up paper books! Recently there’s been an explosion of electronic readers, lead by Amazon’s Kindle. Within the next few years we’re going to see bendable, foldable and colourful electronic paper. For those of you who are design minded and interested in using E-Paper to invent new kinds of interactive visual displays and devices, here’s a handy guide for learning more about E-Paper technology.

Fascinating The Secret History of Silicon Valley – just over an hour long but well worth watching.

How aesthetically beautiful are your photos? Try out Acquine, an Aesthetic Quality Inference Engine. Welcome to the brave new world of computational aesthetics!

Clever – video of evolving a human face using a genetic algorithm. A face detector is used for the fitness function.

Time to start gaming for mental health?

Haunting and beautiful sand art “animation” – very neat and worth the 9 minutes.

Neat video showing off Paper Computing. Author electronic circuits by painting and dropping electronic components on paper! (found via Turbulence.org)

Any ideas on how I can design and build a Charisma Augmentation Device…? Learn about the science of charisma.

Handy handy – a useful new and free visualisation tool called Parallel Sets has just been released.

An arty digital analog clock designed by Humans Since 1982 – Clock Clock: The Analog Digital Clock (thanks Baz).

Me likes MOY – design and adapt the visual design of your car…in real-time! Probably a bit distracting but I suspect it could be useful for making your car visually pop-out in dangerous low-vision driving conditions? MOY is a design concept from Elvis Tomljenovic “The idea behind MOY concept is that everyone can design their own car on their own computer and then apply the design to the vehicle using wireless data transfer or share it with other people through web-site, forum, e-mail etc. To those who lack the necessary skills or time to create their own design, we offer the option of downloading ready made designs. The vehicles are interconnected, so the change is possible in motion. ”

Interesting article in MIT Sloan Management Review on Cracking the Code of Mass Customization. You can register for free to access it for free. The authors identify three required capabilities for mass customization companies. (found via Mass Customization & Open Innovation News)

Read Core77′s writeup Physical pixels: design for the not so near future on the Organic User Interfaces panel at CHI 09, which I previously mentioned. Is it a bit too futuristic? Psst, the answer is no – as long as futuristic innovations feedback into here and now innovations.

User Designer is back…yep, I went very quiet for a few months – but it was for great reasons 1) I became a dad (loving it), and 2) I was writing up my HCI PhD. Fortunately I’m getting to continue focusing on HCI / Interaction Design research, as I’ve just started as a postdoctoral researcher in the CLARITY Centre in University College Dublin, Ireland.

Some day soon I’ll do a proper writeup about the PhD, but in short I was looking at the effect individual differences in low-level vision have on the user experience of HCI designs – a fun fusion of interface / information visualisation design, vision science / optometry, eye physiology and probabilistic modeling.

Anyways lets get started again with a Link Bucket, enjoy!

Did you know that Attractive Things Work Better, written by HCI guru Don Norman.

John M. Carroll, one of the fathers of HCI, writes about the History of HCI (thanks Mads Soegaard @ Interaction-Design.org).

CHI 2009, one of the main HCI conferences has just finished, read about a few neat ideas in The Stranger Side of CHI 2009.

Excellent, HCI and Interaction Design research focused on a malleable and readily adaptable world is really beginning to gain traction. How long before it has its own conference?

There’s a bunch of very interesting workshops at CHI 2009, which will be on in Boston from April 4th to 9th. CFPs (Call for Participation) that caught my eye include:

Super busy at the moment but I’m delighted to announce “BumpList: An Email Community for the Determined” (my project in collaboration with Jonah Brucker-Cohen) is back online after being offline for 4 years! So now is your chance to join the email community that had most people scratching their heads and wondering what happened to email as they knew it.

Join the list here!

We’ve put it back online because it’ll be showing in the San Francisco Museum of Modern Art (SFMOMA) as part of “The Art of Participation: 1950 to Now” from November 8, 2008, through February 8, 2009. Press release for the show is here.

Other artists in the show include “bramović/Ulay; Vito Acconci; Francis Alÿs; Chip Lord, Curtis Schreier and Bruce Tomb (former members of Ant Farm); John Baldessari; Joseph Beuys; Blank & Jeron and Gerrit Gohlke; George Brecht; Jonah Brucker-Cohen and Mike Bennett; John Cage; c a l c and Johannes Gees; Janet Cardiff; Lygia Clark; Minerva Cuevas; Maria Eichhorn; VALIE EXPORT; Harrell Fletcher and Jon Rubin; Fluxus Collective; Jochen Gerz; Kit Galloway and Sherrie Rabinowitz; Matthias Gommel; Felix Gonzalez-Torres; Dan Graham; Hans Haacke; Lynn Hershman Leeson; Nam June Paik; Allan Kaprow; Henning Lohner and Van Carlson; Rafael Lozano-Hemmer; Tom Marioni; MTAA (M.River and T.Whid Art Associates); Antoni Muntadas; Yoko Ono; Dan Phiffer and Mushon Zer-Aviv; Raqs Media Collective; Robert Rauschenberg; Warren Sack; Mieko Shiomi; Torolab; Wolf Vostell; Andy Warhol; Stephen Willats; and Erwin Wurm.”

Enjoy, next week regular blog posting will resume.

All the world is aflame with the iPhone! Have a look at this example of the iPhone used as an Augmented Reality device. (thanks Karl)

See how the world looks to a baby’s eyes.

What is our psychology of time? Read The future is nonlinear on Mind Hacks to learn more.

Drool drool love the visual style in the My Drive Thru music video.

Air Ape art.