[Geeks are Sexy] technology news





Thursday, August 17, 2006

Multi-Touch Revisited

I'm quite the sucker for eye candy. In particular, I absolutely adore eye-candy involving really shiny display technologies. Check out the video below, and join me in the drool-fest.



It's easy to watch all the pretty lights and colors, listen to the high-powered geeks who make them dance explain how their toys work, and think of this stuff as nothing more than that: bright, shiny, complicated toys. But that's not all they are. We need to look beyond that impression.

Awhile ago, I wrote a post on some ongoing research in the areas of human-machine interfaces. Specifically, I highlighted the extremely cool stuff (and the similarly shiny demo video) that the Multi-Touch Interaction Research project has produced. When I originally saw the video, the first thing I thought was how "freaking cool" it looked. The next thought involved how much I would *kill* to get my hands on one of those units and play around with it. This motivated me to post the video and a bit of commentary on this site. But as I was writing, some of the larger ideas that surround these kinds of flashy demos (and the research behind them) started tugging at my brain. It got me thinking about some of my experience and opinions concerning this facet of technology. But those thoughts remained largely in the periphery, and weren't addressed much beyond references to a couple of movies, and a very snazzy video of an awesome artist going nuts with a Wacom.

And then there were comments. Not a huge number, but some of them were surprisingly lengthy and verbose. You'll get a tiny bit of summary of the commentary here, but I strongly encourage you to read them (and check out the link in YeeJen's comment) if you haven't already had the pleasure.

The second comment was particularly interesting. The poster presented an argument that essentially refuted my "Wow! This is really cool stuff that could revolutionize humanity's interaction with technology!" commentary, and postulated how touch-screen technology will "never, and in fact cannot, work in the real world for very simple ergonomic reasons having to do with human anatomy." My personal annoyance at having something I was excited to talk about being called "over-hyped crap" aside, Richard's comment pointed out numerous problems that effect not only currently deployed touchscreen technology, but also the traditional keyboard/mouse implementation. He also discussed various reasons why touchscreens will supposedly never work within a very specific setting. Fortunately, an anonymous poster responded point-for-point in refute of this, using a Powerbook to simulate a touchscreen, and I thankfully don't have to devote any more space responding directly to this very short-sighted scenario.

But something still annoyed me about the comments to my post (with a couple notable exceptions. Ian Mckay, I am delighted that the Smart People of the world have had successful prototypes of this kind of "futuristic" technology as early as 2001! ). Starting with the second comment, discussion largely centered around existing technologies used in a traditional workstation scenario. This is not the space that these alternate interface technologies are trying to occupy. With current technology, we have largely solved the problem of creating a computer workstation that excels at certain desktop jobs. I have set up a number of multi-screen, multi-machine workstations that allow a saavy user to monitor and manipulate enormous amounts of data simultaneously. With a little practice, the only limit to productivity is how fast one can think...but only for *certain tasks*. This is very important to remember. Depending on everything from physical abilities and characteristics, to the task to be performed, to good old personal preference, a 15-inch and keyboard/mouse combo may be perfect. Or it could completely suck.

Consider the task of typing. I think most people can agree that a touchscreen isn't the best input device for most, if not all typing tasks. But what about voice recognition? If you've read my little "Hi! This is me!" post, you'll know that I'm a college student. I've had to write *many* pages of notes, analytical arguments, and various papers. I can tell you with certainty that reliable voice recognition technology would be far superior to the keyboard for this kind of data input. But what of writing code, or some other kind of data-entry that is nothing like spoken language? Imagine how fun it would be to dictate a few lines of PHP or C#. No, I didn't think you'd want to do that either. At least not without a serious IDE to interpret a scheme of special phrases. Our traditional input hardware is obviously not perfect for everything, even within the restrictive workstation setting.

But things, they are a-changin'. Thankfully, people are realizing that the individual workstation isn't the only place for productivity, nor is it the only place for computers. The vast digital world has made its way into our pockets and briefcases, into our boardrooms and meeting places, even into our art studios. If what we are seeing in the touchscreen prototypes noted earlier is any indication, computers are on their way to being in our tables and walls, in any number of settings.

Anyone who has spent considerable time with a professional multimedia creation software suite will quickly understand the amazing benefits of a mutable interface that can be directly interacted with. We currently spend most of our time interacting with these applications with one hand on the mouse, and the other hand trying to feel its way around the keyboard, hunting for various shortcut or selector keys, or stabbing at context modifiers. A lot of these programs are designed to be the best approximations of their old-school physical-world counterparts, but can only manage insofar as the mouse/keyboard model will allow. The large-format, accurate and responsive interactive displays that are likely to evolve from the technology shown in these shiny videos and pretty demonstrations will be incredibly useful for people working with multimedia. And no, I'm not pulling this out of thin air. I have spent many, many hours working with the last few generations of this software. I've done everything from architectural and mechanical CAD drawings, 3D modeling and animation, to audio/video editing, and mounds of Photoshop work. I've seen and felt the limitations of the keyboard/mouse abstraction in these fields. Large work areas that can respond and transform to fit our needs, whether they be on table-tops, walls, or new-age drawing boards, will be be incredibly beneficial to multimedia folks.

Before I wrap up this post, I absolutely have to talk about accessibility. Not everyone has the same abilities to interact with the world around them. There are people who don't have the visual capabilities to see a screen. There are those who can't hear well enough to listen to audio. There are plenty of people in this world who can't physically manipulate a keyboard or mouse. Again, in his comment to my previous post on this, Richard talked condescendingly about gestural input technologies essentially being a form of sign language. But, come to think of it, wouldn't it be great for a deaf person to be able to dictate a letter to their computer instead of having to type it? This is yet another example of how some models work beautifully for certain people in certain scenarios, but don't necessarily make sense for everybody everywhere.

There doesn't seem to be much discussion about current alternative interaction technologies outside of certain usability and accessibility circles because, let's face it: stuff like screen-readers just aren't sexy. Again, I'm speaking from experience. As a web designer who is at least conscious of accessibility concerns, this is part of my world. They may be boring to most, but they are absolutely essential to some. Without them, people with disabilities would be completely cut off from the rapidly exploding society that inhabits digital space. Any new technology that stands a chance of enabling a larger number of people to easily and intuitively shouldn't be dismissed based on first impressions and a couple isolated and limited use-cases.

These technologies aren't being created simply because they are "cool", with the hope that an application will come along to make use of them. They are solutions being engineered for problems that have not yet been solved. Take a second to think beyond the initial reaction of "Hey! That's cool!!" or "That's total crap, and will never work." There's a lot more going on than the cool-factor floating on the surface.

But hey, if there's eye-candy, enjoy the friggin' eye-candy. Oh, and congratulations if you actually read to the end of this post. You're all truly pocket-fulls of awesome!

[Edit] The following showed up in one of my feeds mere hours after this was posted. Not only is it more beautiful eye-candy, but the presenter speaks very well about this technology and its evolving purpose in our world. He briefly touches on some of the same points I brought up in this post.

Inventor Jeff Han demonstrates his intuitive "interface-free" touch-driven computer screen.

Publisher's Page: http://www.ted.com/tedtalks/tedtalksplayer.cfm?key=j_han
Support the Publisher: http://www.ted.com/tedtalks/



2 Comments:

  • Oh I'm seriously drooling!

    By Blogger FreeCyprus, at 9:27 AM  

  • hmm speaking of web design and accessibility...we need to get more people using "1em" instead of "12px" because ems are resizable for those who can't see well, and pixels are...not

    By Blogger Mackenzie, at 4:57 PM  

Post a Comment

Links to this post:

Create a Link

<< Home