BTW this post was originally much too long and contained a few different interrelated ideas. Rather than posting a very long post I’m spreading the ideas and questions out over the next few posts.

Mindful Football is today’s fun idea, and it builds on my previously posted ideas around pico projectors. Imagine you’re about to kick a football, and the football knows exactly how you’re going to kick it before you kick it. The Mindful Football knows how hard you’re going to kick it, in what direction you’ll kick it, and it can predict how it’ll behave after you kick it. Of course the surface of the football is also a kickable smashable display, and the ball can project images all around itself (its got inbuilt 360 degree kickable pico projectors!).

Just as you’re about to kick the Mindful Football it predicts and shows you the results of your kick. Close by the football on the ground you see a projected blue arrow, which points in the direction the ball thinks it’ll go. Further in the distance, the ball also projects a big blinking orange circle showing where it’ll land, or what it’ll hit (yes, you want the orange circle to be in the goal or at a teammates feet). Would the real-time visual feedback about your kick enable you to change how you kick the ball, so that it goes where you want at a speed you want? If you used the Mindful Football when playing friendly games, would the real-time visual feedback improve your ball control skills, even when you switch back to using a normal football?

What other forms of visual feedback could be shown to you as you’re kicking the ball? For example, a star could be displayed on the video surface of the football. The location of the star indicates where you should kick the ball to send it in the direction of the goal. You can easily imagine this idea applied to lots of other sports, such as tennis, baseball, rugby, American football, etc.

Another visual feedback possibility includes having the surface of the football show lots of animated red dots, where each red dot indicates where a teammate is relative to the ball. When you have control of the ball and are looking at it, the purpose of the red dots would be to the enhance your awareness of your teammates positions. Would that really increase real-time situational awareness? Would it help you learn how to predict where your teammates normally are relative to you?

Mind you, if you were really dastardly and mischievous, you could hack the opposite team’s Mindful Football so it provides slightly inaccurate feedback. Potentially leading to the players becoming less skilled at kicking the football!

A Mindful Football is an example of providing real-time feedback at the moment of action, i.e. when kicking the ball provide feedback. What about cases where we achieve a goal by performing a lot of interrelated actions that are spread out over time, such as when we cook a meal? I’ll cover that in the next post.

Add in a dash of flight and we move in the direction of flying autonomous robots that can self assemble into larger flying structures. Learn more and watch the videos. What would you use them for? How could they be fused with the programmable paper?

Hearing, tasting, smelling, seeing, feeling ain’t enough? Develop a new sense – develop magnetic fingernails and “feel” magnetic fields.

Ice cream, ice cream every kind of flavor you’ve never heard of, or never imagined! I must go along and try some – sounds yum.

Bosch have enhanced the night vision system system in cars, so that it provides smartly enhanced high contrast images of the road ahead at night. Clever but would you trust it to properly identify which parts of the road are critical for highlighting?

For the last few years I’ve been using and really liked Sony’s eBook Reader, the PRS500. Yup, I was an early adopter and altogether unsure whether I wanted to give up paper books! Recently there’s been an explosion of electronic readers, lead by Amazon’s Kindle. Within the next few years we’re going to see bendable, foldable and colourful electronic paper. For those of you who are design minded and interested in using E-Paper to invent new kinds of interactive visual displays and devices, here’s a handy guide for learning more about E-Paper technology.

Fascinating The Secret History of Silicon Valley – just over an hour long but well worth watching.

If you only ever subscribe to one podcast then I cannot recommend WNYC’s marvelous Radiolab enough. Over the last few months I’ve been listening to their back catalogue – science radio at its very best. A dash of depth, a dash of humor and lots of interesting diverse topics. It always leaves me delighted and wondering could I do research in that field, and that field, and that field. Boo, there’s never enough time for all the interesting things in the world!

What’s cooking in Research and Development at IBM, Microsoft and HP.

Blog-a-licious 3 Quarks Daily – An Eclectic Digest of Science, Art and Literature.

Have you ever gotten into one of those silly arguments about the colour of something? You know where you’re sure that a t-shirt is red, while your friend is 100% sure its redish yellow. Frustrating isn’t it.

Strange as it is, both of you can be utterly right.

You both “see” a slightly different colour because of individual differences in physiology. The receptors in eyes that help convert light into colour often have slightly different sensitivities between people. For most people the differences are so slight they’re not usually noticed, but people with colour blindness experience a world where colours appear very different. Go here for details about the Ishihara colour plate image, which is used in testing whether people are colour blind.

There are thought to be women who are the opposite of colour blind, they are tetrachromats who are able to see more colours than most people (who are usually trichromats). Damn Interesting has a good introductory article about tetrachromats A Life More Colorful, and the Pittsburgh Post-Gazette has a good article with a little more science background, Some women may see 100 million colors, thanks to their genes.

Previously I’ve touched upon individual differences in genetics for Personalised Medicine and the Psychology of Individual Differences.

There are many other kinds of subtle physiological differences, such as variations in taste receptors and densities on the human tongue. Here’s an introductory article about taste blindness.

Individual differences in physiology can be measured. These measures can be used to shape the design of objects. For example measures of your taste receptors could be used to automatically adapt a collection of cooking recipes to enhance the flavour for your tongue. Or TVs could have inbuilt smarts that adapt football game colours so a person with red-green colour blindness can more easily see their favourite football team. No more struggling to see a team wearing a red outfit running around on a green pitch, or a red snooker ball on a green table.

If the above is to become possible then self-mallable / re-shapable objects that adapt to the individual physiology of users need:
1) measures of user physiology
2) predictive models of the impact due to physiological differences, i.e. if an object is adapting to a user how does it know an adaption has a positive or negative effect?

This builds on implications from When Toothbrushes Mate: Form & Function DNA. Malleable objects and artifacts need to be:
1) self-describing
2) user describing (predicting the impact on user experiences due to physiological differences).

Florian Kräutli has designed and created malleable magnetic curtains that retain their shape. Clunky elegance.

Embodied Cognition “is a growing research program in cognitive science that emphasizes the formative role the environment plays in the development of cognitive processes.” Learn more about this increasingly important area from this website. (via MindHacks)

Gimme gimme one of the Emotiv Brain-Computer Interface headsets. No contact gel required. Of course you could always build your own based on designs from the OpenEEG project.

Neat video of a time fountain optical illusion. Looks like water is running backward defying gravity and hanging in mid-air.

I’m curled up with a bug and head cold so today I’m posting a Link Bucket instead of the normal long Tuesday post. Enjoy.

Looks Tasty – see the world with your tongue.

For the last few years I’ve been keeping my eye on the emerging transdisciplinary field of Augmented Cognition. On the AugCog website it is defined as an emerging field of science that seeks to extend a user’s abilities via computational technologies, which are explicitly designed to address bottlenecks, limitations, and biases in cognition and to improve decision making capabilities. Is it a repackaging of a human information processing approach to HCI, or something more? Either way it has interesting potential.

Takes your breath away – UDF SkyWalker is a massive interactive image from the Hubble Space Telescope. The image shows 10,000 galaxies from when the universe was only 800 million years old. You could also try out GEMS SkyWalker. I wonder how many of those galaxies are now teaming with life?

What would happen if your toothbrush could mate with another toothbrush?

Would you end up with an even better toothbrush – the best of both toothbrushes? How about if your toothbrush mates with 5, 10 or 15 different toothbrushes, with each new toothbrush in turn mating with another new toothbrush?

If you could decide which toothbrushes get to breed would you eventually end up with a toothbrush that’s perfect for you? Its form and function, its colour, feel and shape all bred into the toothbrush children generation by generation. The toothbrushes you dislike don’t breed so don’t pass on their “Form & Function DNA” to the next generation.

How would you tell a toothbrush to mate? Maybe to start the mating process you exchange design DNA by physically rubbing your toothbrush against another toothbrush. After that your smart malleable material toothbrush turns into a lump, which then self-forms into lots of little baby toothbrushes. To decide which mini-toothbrushes breed you crush the ones you don’t like, and rub the ones you do like off each other. Then repeat again and again, till eventually you have a baby toothbrush that you stretch into full size and begin using as your day-to-day toothbrush.

The method I’ve described for breeding toothbrushes is a Genetic Algorithm (GA) search. GAs are very powerful for exploring a large search space. In this case our search space is the potential designs for toothbrushes. Seeing each design generation could be a great way of helping people explore and imagine design possibilities.

Dryad, from Stanford Virtual Worlds Group, is a related example of software for exploring the design space around 3D trees. You can cross breed different kinds of 3D trees. Dryad is freely available for Windows and Macs, go play.

It’s easy enough to speculate about reshaping toothbrushes by cross breeding them, but what about more complex artifacts. Such as doors that slide or TV remote controls. How would you control the cross breeding of what a button does? Functionality is more abstract than form. For example if you cross breed two door handles where one door handle works by turning and the other by pulling…you could end up with a nightmare child door handle that works by turning, then pulling, then turning again.

How can the person doing the cross breeding place limits on what forms and functionality are explored? Maybe by only cross breeding one specific part of an artifact at a time, e.g. only cross breed the handles on the toothbrushes.

For artifact cross breeding to be possible objects and artifacts will need some kind of DNA. At the most abstract level the DNA would encode form, functionality and the relationship between both. Or putting it another way: malleable objects and artifacts need to be self-describing.

Smart Lego (New Scientist article), from CMU’s Computational Design Lab, is an example of a physical / virtual artifact that is able to self-describe. Also for many years various computer languages have been capable of different amounts of self-describing, which in computer science is called Reflection.

Of course now I’m wondering what would happen if you cross breed a door handle with a toothbrush?

Lovely overview about The History Of Visual Communication.

A bunch of interviews with User Experience pioneers.

Can you touch it? Workshop on designing multi-touch interaction techniques for displays (thanks Aaron). Invent a new interaction technique for the likes of Microsoft’s Surface and the DiamondTouch.

Can you sing it? World’s First HCI Rap

Chill out and sketch a personal kaleidoscope. Interactive realtime feedback on how your design changes over time.

Isn’t it strange the way faces look so different? Yet we easily recognise that a face is a face. Imagine waking up tomorrow and everyone’s face has disappeared. Each face is replaced with a blank unexpressive blob. Don’t worry about the eating, seeing, speaking and breathing bits – in this brave new faceless world we can still speak and eat, etc, all without a face.

In Faceless Land would you be more or less easily able to tell when someone is lying to you? Think how many times you’ve chatted with someone close who says one thing yet you know from their eyes, lips, nose and cheeks that they mean the opposite.

For a fascinating article about the science behind our ability to read faces checkout The Naked Face (free download) written by Malcom Gladwell, published in the New Yorker a few years ago.

From the article I learnt that most of us are absolutely terrible at telling whether people are lying. We’re so bad that when it comes to strangers we might as well flip a coin as guess whether they are telling the truth or not. Less than 1 percent of people are extremely good at telling whether others are lying.

How do these the super face readers do it? What do they see in the human face that normal people don’t notice? Psst, Gladwell’s article provides a few answers.

Paul Ekman is one of the pioneering researchers into understanding facial expressions. In the 1960s he helped establish that facial expressions are universal. He also found that in a limited way if you physically arrange your face to mimic an emotion then you begin to feel that emotion! There’s lots more brain food on Ekman’s website via his freely available articles and book chapters.

Ok, that’s it for now with the science – I’ll be back to this topic again as its very relevant, interesting and has lots of potential, e.g. Affective Computing (MIT Media Lab Group), HCI + Emotions (paper discussing applications), etc.

How does face reading relate to User Designer? Computers and other digital tools are currently face blind, to them we are all living in Faceless Land. Cutting edge research has begun to crack the problem of facial recognition but we are still a long way off from having systems that recognise facial expressions with the same accuracy as super face readers.

What are the implications when we can design digital artifacts that read our faces as well as super face readers? Add in a dash of smart materials that can intelligently re-shape themselves, and out pops ideas such as self-reshaping comfort blankets that reassure a child by mimic’ing the movement of a parent’s face. The blanket might be able to “smile” without looking like a face – it creases itself here and it creases itself there.

Or make-up that stimulates your facial muscles to induce you to arrange your face into a smile…smile on the outside so you smile on the inside.

Or a sales technique where the salesperson’s office furniture, cups, chairs and any surface begins to look a little bit like the potential buyer’s face. If it was done subtly enough it might be more reassuring than creepy. Here’s lookin at an office chair lookin like yourself, human.

In today’s Creativity Knowledge post I’m focusing on a series of research projects from Patrick Baudisch, who is affiliated with the VIBE group in Microsoft Research and DUB in the University of Washington. Over the years Patrick and co-researchers have developed a range of innovative interaction techniques focused on enhancing people’s ability to move content around screens.

Interactive displays surfaces are getting larger – for example checkout these exciting demos of Multi-Touch and Surface (thanks to Cormac and co for the link). A usability problem is that content, such as icons, is moved around these large display surfaces using the same techniques as on desktops. We grab an icon with the mouse, we drag the icon by moving the mouse a physical distance proportional to on-screen distance, we then place the icon where we want it to go by letting go of the icon.

The desktop interaction model, which is designed for small displays, has been kludged to work on large surfaces. Imagine moving the icon with a large wall sized display: we grab the icon, we run half way across the room to drag the icon to a desired location, we let go of the icon, we catch our breath then run half way back across the room to where we started.

How would you make it easier to move content around large displays? How about when you can interact with the displays using your whole body?

Check out this interactive online flash demo where you can play with drag-and-pop. Instructions for the demo are here. Drag-and-pop is designed so when a “user starts dragging an icon towards some target icon, drag-and-pop responds by temporarily moving potential target icons towards the user’s current cursor location, thereby allowing the user to interact with these icons using comparably small hand movements”. On the drag-and-pop website you can find videos and papers explaining the interaction technique in detail, along with material explaining push-and-pop, drag-and-pick and Mountaz Hascoët’s push-and-throw. Yep all this pushing, dragging and popping is confusing! Enjoy.

On a related point: I’m regularly drawn to research around innovative interaction techniques but is there too much focus on novel interactions in interaction design research? Are we like moths to the flame of novelty? Does novelty distract from the development of deeper theories? Or does it highlight that HCI can and should also be practiced as an applied field of research?

In the above photo you can see an example where a streak of lightening can be seen on a model’s t-shirt when the t-shirt is viewed via a mobile phone camera. The lightening is printed on her t-shirt but the “colours” (wavelengths of light) used for printing aren’t perceivable by the human eye but are picked up by digital cameras. There’s more details about the technology here.

There’s a number of usage suggestions for the techology on the kameraflage website. You could imagine extending their idea but without digital cameras. Imagine multiple people sitting around using a shared surface computer (such as a DiamondTouch). Each person is wearing glasses with slightly different colour tints, so each person would be able to view a “personal” image on the surface that the other users won’t see. Or you’re going to the cinema and depending on which glasses you wear the film has a different age rating because different parts of the film are “cleaned up” for younger age groups. Alternatively wear these glasses during the film and get in for cheaper because you’ll see inserted advertisements on-screen?

Recently I came across JennyLC’s Intimate Controllers project (via the always interesting we-make-money-not-art). She writes that the project involved “building game controllers into undergarments so that games are played through players physically touching one another. The goal of this project was to research and create objects that challenge the traditional notions and orientation of video game play”. Her demo video is worth watching and if you’ve a bit of time to spare her thesis presentation video is online.

Her work touches on some ideas that have been running around in my head for years. A world where it’s easy to create arbitrary relationships between actions and effects. I wonder could you generalise her controllers so they can be used for creating arbitrary mappings?

For example imagine a product where you buy a box of flexible, durable and reusable controllers that easily attach to clothes, walls, floors, etc. Once the controllers self-network you start creating relationships between controller activations and resulting actions, i.e. press a controller and it turns on a light, or lay out a bunch of the controllers on the floor and walls to create a 3D dance mat for your game console.

Will people move away from buying physical artifacts with pre-build physical interfaces to buying artifacts that can have controls easily attached to them based on their preferences?

Imagine buying a cooker / stove that has heating elements but no buttons, controls or feedback for setting the temperature. When you get the cooker / stove home its up to you to stick a bunch of controllers onto the cooker. If you like you could setup a touch sensitive controller where you adjust the cooking temperature by sliding your hand instead of twisting a knob, or setup controllers so you increase the temperature by dancing fast on a dance mat in front of the cooker :) You could build your physical interfaces for mobiles phones, door handles, etc, in the same way and potentially with the same controllers.

How could you simplify creating a relationship between controller activations and resulting actions? Maybe by fusing Visual Programming Language, e.g. Toontalk, with Programming by Demonstration. That’s a hard but important question.

What kind of easily composable output / feedback / display components would you have? Maybe build souped up versions of LED throwies.

Will we ever have malleable physical interfaces?