This great TED talk introduces the idea that we don’t see what we think we do. Watch the first five minutes, and do what the magician says, and I’m sure, like me, you’ll find yourself fooled.

The idea that someone can be fooled is not new, of course. Magicians and all form of artists (especially the con variety) have been doing it for centuries. But the more I read, the more I realize that we’ve got a pretty weak grip on reality anyway. 

The Study of Attention

If you’ve ever read into a little coginitive science, you would have come across the following experiment: A person is told to watch a video where a number of people are passing basketballs between one another. The experimentee is told to count the number of passes. Midway in the video though, a man in a gorilla suit walks slowly across the middle of the scene. Amazingly, when asked about this later, most people say they didn’t see anything strange in the video. They literally didn’t see the gorilla because their focus was so directed!

This, and other wonderful experiments on attention can be found at the Visual Cognition Lab in the University of Illinois, including, this amazing “Construction worker” experiment:

You can see the original footage on the , construction worker Visual Cognition Lab page but I’ve made a storyboard with Comic Life with some screen grabs to explain what happens. This video is copyright Viscog Productions.
 
Storyboard of construction worker switch.
 
This is an astounding study. The brain is very keen to fill in the gaps, or warp our sensory input to match what we believe to be true. If you want to test yourself, try the ‘Gradual changes to scenes’ examples. Personally I didn’t see many of the changes, and if I did, I only had a rough sense of what changed, and was often wrong.

The Dance Music Display
Before getting into the software world, I was working at a large CD store. There were two levels, with Dance Music on the upstairs level. A massive neon sign pointed along the wall and up besides the escalator to the Dance Music section. But many people would still come to the counter on the lower level and ask if the store stocked Dance Music. “Can’t they read??!” we’d wonder. Then one day, the Dance Music guy from upstairs put a display of a few select titles on a shelf on the lower level. In between the select Dance Music CDs on the display were empty CD cases with the text “More Dance Music Upstairs”.

It was then I realized: People were literally looking for Dance Music, not the indications to it. They were in “find the right CD” mode, not “read signs” mode.

Unread Skitch Tips
In our Skitch application, keeping the interface clean meant many of the advanced features required you to hold down the Shift/Command/Option key. As a helpful reminder, we floated a little tip bezel above the main Skitch window.
Skitch Application screen capture with small window of text tips floating above.

Seeing What You’re Looking For

But you guessed it; nobody reads that. Why? Probably for the same reason. They’re looking for something they expect to be in a certain form (probably an icon). As a result, they literally cannot the text.

I’ve seen some great eye-tracking data from the respected Nielsen Norman Group for this. ‘Heat Maps’, an accumulation of eye-tracking data, show that many people literally don’t see the graphical ads on web sites. Even when the ads provided the answer to the experimentee’s task, eg “Find the lowest price Skiing holiday on the website”.

Fake Ski ad  with caption

How to Make a UI Dissapear

Good UI design is about making the UI mirror the users mental model of the user. But the transmission from the UI to the user is not so clear, linear or instant. And sometimes you don’t want the user to incur the cost of noticing ever-changing states.

Apple Mail icon in the dock without, and with notification badge.

For example; we’re constantly inundated with email, IM, twitter notifications and the like. But the ‘Gradual changes to scenes’ example shows us that changes over a period of 15 seconds can go un-noticed, so it’s possible to change the state of something without triggering our attention.

Instead of popping up a notification, or adding a badge instantly, the UI could slowly fade in a change over 15 seconds. This could be applied across many areas of User Interfaces. And I’d personally love a computer experience which emphasized ‘flow’ and gradual, constant change. No longer would every little change pull your attention away from an important task. Instead, those Mail notifications, system messages and the like could gently change without you noticing, until you decided you wanted to actually look.

SIDENOTE: I actually turn off Mail.app notifications, and simply un-Hide Mail when I want to check for mail. I’m sure Merlin Mann would approve.
 

 

Painting of large letters in a carpark at angles, only clearly visible from a particular angle.

This is a great visual example of using a specific physical perspective in design. German designer, Axel Peemöller has a very cool website portfolio — definitely worth a look. This Melbourne carpark has signs painted with text only readible clearly from the entry/exit it’s aimed for. See the full set of photos here.

app vs web sketch circles on index card with clock in between the apps loop

Developing sites and online applications (for example, Google Docs) is different in many ways to developing applications for desktop machines. Technological differences aside, there is the speed of iteration.

What drives this innovation:

    Usage data: Websites/web applications give developers continuos feedback on the parts of their UI used the most and how 

    Fast incremental updates: Updates can roll out many times a day with their impact measured empirically

    Mutation: Amazon.com randomly switches in slightly different sites to different people; this lets amazon.com practice Darwinian style evolution

    Survival of the Fittest: The hassle of switching to a competitors app is very low

Imagine if desktop apps had these advantages! I would love to know what parts of an application are used more than others, how people are using the application and how multiple UI designs compare. If webapps ever replace the majority of desktop applications, then this closed feedback loop and iterative improvement could be why.

On the flip-side, development tools for desktop applications have been a lot better in the past. But in the last 2 years we’ve seen an explosion of solutions; Microsoft’s Silverlight, Adobe’s Flash and Air platform, Objective-J (really impressive), Sproutcore.

 

In the middle of these two approaches is the iPhone:

    All applications auto-update 

    There is a low cost of entry and exit to applications, not as simple as typing in a new URL, but much easier than installing new software on your PC. Purchase effort is minimal

    iPhone apps are a smaller ‘canvas’ making it easier and faster for developers to improve and experiment

Perhaps iPhone will forecast the relative success of webapps/ ‘cloud computing’ as compared to desktop apps? Or perhaps all applications will grow to fit in between.

I don’t know about you, but I’d be happy to everyday wake up to a slightly improved desktop application for the sake of ongoing improvement. 

T-shirts on a shelf on a store

Once apon a time, I used to have a blog called Songcarver.com with a post titled ‘Search and Save are the same thing’ (or something like that). Some months ago, a spanner fell in the interwebatron works and the domain was rudely turned into an ad. But sometimes a lemon turns into lemonade — as I rewrite some of the better older posts, I get a chance to re-think them, and complete the picture. But first..

A True Story.

The details have been changed to protect the innocent. The computer, it turns out, is not one of them.

A few days ago, my wonderful girlfriend recounted an episode at work. See, she works at a large government organization which regulates governmental quality standards. I would say her workplace approximates the ‘average’ workplace; cubicles, slightly dated PC on every desk and a strict IT department. On this day, a colleague of hers, we’ll call her “Sally”, wanted to achieve a specific task. She wanted to issue a fine to the Bendy Paperclip Company Inc. for not including the regulated minimum number of bends in their paperclips. They were simply putting straight pieces of wire in a box and selling them as paperclips. Clearly an infringement form needed to be sent.

Sally knew there would probably be a standard form for this. She looked everywhere on the shared network drive, poking into all the places she would expect to find it; nothing. She tried some searches for ‘bends’, ‘bent’, ‘bending’ and found nothing relevant. She sighed. And commenced the epic task of creating a new infringement document from scratch.

“Hey, there’s already a form for that on the network” said co-worker Bob, peering over her cubicle.

“Oh, You suprised me!” uttered Sally. “So where is it?”

“It’s in the ‘Quality Infringement Notifications’ Folder”
“No it’s not.. I looked there” retorted Sally.
“Yes it is — look in inside the ‘Minor infringments’ folder inside that one’
“Ahh…”
“I called it ‘Insufficient Stationary Item Curvature Infringement’” he added

So, an hour of Sally’s time was wasted searching for, then commencing a fresh Infringement Document. An hour later Sally recounted the story to my girlfriend.

“And I looked everywhere, but it turned out Bob had called it ‘Insufficient Stationary Item Curvature’”
“.. and he put it here. Wait.. now it’s not in the Quality Infringement Notifications’ Folder…”

When recounting the event out loud, she forgot that the file in question was in a sub-folder of the one she was peering into. With some more frustrated searching, and wasted time, she eventually re-found the file.

This story makes me both mad and embarrassed to be a part of the community that has built the systems which give good, intelligent people such a hard time.

SIDENOTE: I’m not the only one who wondered that there must be a better way. Dr William Jones, of the Keeping Found Things Found research project wrote an in-depth book on the topic. I interviewed Dr Jones a while ago and hope to have that video interview up sometime soon on his research.

How Come Computer File Systems make it so hard to…well… file?

How did we get to this stage? The work done at Xerox Parc around 1973 started the trend towards the real-world metaphor of the desktop and filing system. And the people at Apple built heavily on these ideas to take them to the masses with the original Mac. And these systems in general worked well, as people dealt with dozens of files every day on their virtual desk.

Then as personal computers got bigger, and people got more connected, the number of files exploded. People now work daily with hundreds of files, not dozens. And the response I’ve seen to dealing with all these files has been Search, Scan and Preview. Today, in 2008, a Mac (or most PCs) will let you find a document containing any word in seconds. It will also let you preview almost any document instantly with a keystroke. Or, scroll through 1000’s of images on a single search screen, drawing from image files collated from hundreds of different folders.

It seems reasonable, given all this power that anybody should be able to find anything. But they can’t.

“I’m Looking for Something but I Don’t Know What it Looks Like”

We have only a handful of ways to help make a file findable. One is to put it somewhere where we will find it. Another is to name it so a search will find it, or we will recognize it when we see it. We can tag a file with certain words (this is rarely done) and we can color the file label (also rare).

label
Uploaded with plasq’s Skitch!

So lets look where the problem often starts. The save dialogue box:
Save Dialogue box

No longer are we in our comfortable desktop and multiple window filing view. We’ve been constrained into a tiny window, optimized for a 3-column view. Here we instantly lose the ability to place a file within a folder in a position of our choosing. We cannot tag or color label a file. The default size and view means we need to work harder if we want to see all the other files which will surround our brand new one; this means we lose much of the context of our file.

More importantly, here is where we see the first problem: If I name the file ‘Paperclips’, what happens if I already have a file called ‘Paperclips’? If it’s already in the same folder, I’ll get a warning, sure. But what if it’s not? Remember, our über-search capacity now transcends all folders to find us files everywhere and anywhere. We’ll now have two files which are named similarly, or potentially identically. The job of re-finding the file much more difficult.

How does this differ to the real world? Imagine you own a jewelry box, stuffed full of trinkets.
Jewerly box showing with hand putting hairclip into crowded box
In your hand is a special hair-clip you need for a big night out tomorrow. If you want to find it again quickly, this cluttered jewelery box may not be the place. You would want to put it to the side, or somewhere else. On a computer you simply don’t know how cluttered the box is. You just name a file something and hope it’s unique enough to stand out.

Every Save should Search.

Logically, Search and Save should be part of the same interaction. Whenever you Save, a matching search should run at the same time, to give you a sense of the ’space’. Are there already 100’s of files called “Business Spreadsheet” or similar? Perhaps you should add a few more words to the name. And if Save gave us back the ability to position files within folders, to tag, and to choose the color label, then we need to be made aware of these ’spaces’ too. The information space is defined and understood everytime you do something in it.

As often the case, the inverse is true.

Every Search should Save.

The idea that every search for something actually *creates* information is not new. The Google tech talks include a great video describing the work that Belgian information architect Paul Otlet did prior to WWII. He realized the value in capturing and categorizing the links between sources of information. Google of course has made it’s fortunes from this concept. Yet no such record exists on your desktop or laptop computer. Potentially, these paths can help us find what we seek, faster. These virtual goat-tracks might be created through time, pre-intsalled by the manufacturer or shared with others.

Search and Share.

Del.icio.us has become hugely successful by leveraging people’s experiences through ’social bookmarking’, which means you share and intersect your branches to information with others. The result is a powerful people-driven set of connections and paths between websites. In our real-world story, perhaps Sally could have found the file she was looking for, by simply following the footsteps in the sand? Perhaps she could have even *discovered* that the file existed, when her curiosity was aroused by a cacophony of footsteps leading off into another folder. There’s also a wealth of information in those paths. Who went where and when?

Footprints in the sand

How many of the files we have on our machines actually came from someone else? Perhaps someone emailed you some photos, or you downloaded a song from a your guitar playing friend’s website. Perhaps your sister sent a file transfer of a document of the family tree, or perhaps you have downloaded lots of shareware and it’s sitting in your Applications folder. All of these items posses a secret past, like most characters in an Agatha Christie murder-mystery. If we knew the past and the connections between them, we could better find what we’re looking for and also name and manage our own files in a way that helps us, and others navigate.

It’s said our social nature has been one of the reasons why we have got this far in evolution. In many ways we are cogs in a larger, cultural mind. Sometimes the ‘thought’ process is not within your head, but within the group of people you associate with. Your clan has it’s own thought process, which is independent of, and may be contradictory to, and of the individuals own thoughts. The ability to broadcast our everyday actions not only fulfills some basic need to express, but offers unexpected benefits. Keeping the concept of Share in the Save process means Bob can lean over the cubicle wall and help out pro-actively, if socially acceptable.

Suggest.

Sometimes we don’t know what we’re looking for. Maybe your computer could auto-suggest things to you?Of course, you may be dry-retching as Microsoft Clippy comes to mind. An annoying “cute” animated character which seemed to appear at the most unwelcome moments, with the most banal questions. But ‘Suggest’ doesn’t have to be that. Take two successful examples; the iTunes mini-store that sits at the bottom of the iTunes music player, and Amazon.com’s “Customers Who Bought This Item Also Bought..”
Amazon book store main view with discussed text highlighted.

This idea of a passive view which gently hints new directions has proven very successful. In our real-world example, once Sally had started typing up a new document, perhaps she could see that others had also written similar documents, saving her some effort.

What would a system that showed you your files without *any* cue be like? Well, you may already know, if you’ve ever set your screensaver to show photos. Ever remembered a fond moment when some sweet photo unexpectedly appeared? It’s logical to think the same approach could inspire and motivate in day-to-day work as well. (Hats off to my good friend Cris for this insight)

Combining Search•Save•Share•Suggest

Putting this all together, we can imagine a system where Search and Save are no longer two long-lost cousins.. but part of the one cohesive UI that is Search, Save, Share and Suggest. One would never be without the others:

When Searching: Save where you’ve been, leave trails. Suggest other related items on your computer or elsewhere. Share what you are looking for, other people can benefit or chime in with help. Content creators can see a need and fill it.

When Saving: Auto Search for similiar documents at time. Offer side suggestions with somewhat related content. Share your footsteps between folders and information places with others

When Sharing: Save your footprints and meta-information. Search for anything else relevant. Suggest similar shared ideas or files

When Suggesting: Save the paths you *don’t* take. Add Search to be able narrow your focus. Share the areas where you like hang out.

Any my questions to you are;

What else could this approach achieve? What is beyond Search•Save•Share•Suggest?

Theocacao: Odds and Ends for Aug 17 2008
Uploaded with plasq’s Skitch!

W00t. The wonderful blog by Scott Stevenson, Theocacao, has kindly mentioned UI and us. Scott is not only a respected pomaceeous developer, but also an entertaining blogger. One of these days I’ll get around to interviewing him.