Whole-team User Research

In the 15 or so years I've been designing interactive products, I have always tried to involve the intended audience (users) in the course of product development. A recent article by Jared Spool on UIE.com articulates something different that I've felt but rarely had the data to support: involve the rest of the team in deep exposure to users. I'm in complete agreement with this approach, as long as sensitivity to team structure and dynamics are understood and managed well. Resistance to conducting research (broadly speaking, from field research to usability testing) can come from unexpected quarters, but the usual suspects include Sales (who fear losing control of the relationship they have with their clients), Technology (who look at it as something that will slow down develpment time), and Product (who've invested too much time in their MRD and PRD documentation). The Design team, too, has their share of those who fear user input will impinge on their freedom to create, among other reasons.

I started my career at Morningstar working on Windows-based software, where our team used a waterfall process and testing was difficult. We had to wait until the product was nearly done, which was the only way it could stand up long enough for us to test it with users. Even then the feedback was invaluable, but we rarely had many of the front-line developers on hand to observe. Also, much of the development staff were from mainland China and had a more limited command of English.

Later on at Razorfish, I learned valuable lessons on how not to run user research. One of the failures of the agency model is how often work is done in an assembly-line fashion, with little room for iteration and teams that are separated from each other. My team did excellent work, but too often we weren't tightly coupled with the IA, Design, and Tech teams to provide user feedback in a timely manner. It was also trimmed or eliminated from pitch budgets since it always appeared as a line item.

At Yahoo!, we had a much more tightly integrated UX team, but we weren't physically or organizationally close to Tech. In between us was a firewall of Project Managers (necessary in any large endeavor, I might add). This inhibited our ability to get everyone on board, although we did manage to get some of the front-end developers to some usability tests.

Right now I'm at betaworks, and the situation is different still. Here teams are very small and fast-moving. I don't have a team and everyone is super-busy and its hard to find time to squeeze in anything other than core job responsibilities. There are vastly different outlooks on what roles are critical, and products serve very different markets (although they all have UIs). In a startup environment, there is a lot of pressure to maintain a clear vision and course in the face of all manner of obstacles, users included.

Here lies the challenge: how to integrate user feedback into a team's work so that its not considered unusual or can be sidelined. A few takeaways:

  • Convince teams that something that feels so counter-intuitive is actually better. Spending more time on understanding users, planning and designing actually makes the development cycle shorter (and saves tons of time post-launch).
  • Convince teams that this is as important a part of the process of developing interactive products as continually testing for and fixing bugs.
  • As Spool points out, every discipline needs to be involved to avoid the kind of infighting that comes about when people are basing decisions on assumptions about user goals, not empirical evidence.

Data Overload in the Military

...the screens in jets can be so packed with data that some pilots call them “drool buckets” because, they say, they can get lost staring into them.

It appears the military is bumping up against the limits of peoples ability to absorb and make sense of the streams of data they encounter in their rapidly digitizing field of operations. Myths about people's ability to multitask continue to lead managers astray, I fear. Scientists comment that multitasking actually impedes performance in simulations. If you've ever tried to complete this awareness test, you'll get some idea of these limits.

While well-intentioned, the impulse to add another monitor or another data stream only makes the problem worse for soldiers facing extremely stressful situations. Fundamentals in usability have roots in the military, when aircraft crashes were unexplainably high and researchers determined that standardization of control layout and developing controls with significant tactile differences dramatically reduced error rates. Efforts to "rewire" soldiers to aclimate them to information overload seem foolish to me, as there is only so much change the human brain can accomodate. There is nothing significantly different about the brains of today's 20-year-olds and yesterday's. No amount of video games has made their brains different, but it has acclimated them to this kind of work.

A closer look at integrating data into more holistic dashboards, making data more actionable, and offloading onto the system some tasks so that soldiers can operate better all seem like useful directions, and probably significantly cheaper. This last part is probably of no interest to the contractors supplying expertise and hardware to this effort, however.

Designing Trojan Horses

What do OXO Good Grips kitchenwares and the Apple iPad have in common? It may be hard to think of how chunky-handled potato-peelers and cutting-edge tablet computers may be related, but I think both are really "Trojan horses" of product design. When I read the reviews, look at the UI, and play with it (admittedly, I don't own one yet), I've come to think of the iPad as a device aimed primarily at the Baby Boomer Generation. This is significant when you consider how the iPad is marketed and discussed in the media: like most new technologies, there is a heavy emphasis on early adopters, younger people, and the tech-savvy elites. Apple is too experienced to introduce a product just for seniors or anyone else outside of the 18-34 demographic that content producers trip over themselves to attract. Despite the fact that 'boomers have far more disposable income, the leisure time to use and buy lots of apps and books, and a strong desire to keep up with the "digital natives", it would probably be suicide to market the iPad this way.

To the original question - what do these products have in common? Good Grips were designed using Universal Design principles to the accommodate aging hands of seniors. However, this veneer of utility masked the real market of style-conscious consumers needing a reason to replace their perfectly good utensils with new neoprene-handled models. The iPad is not your grandmother's computer, but with its giant buttons, bright high-resolution screen, and ease of use, it could be. The tenets of Universal Design are centered on the notion that designing to accommodate the requirements of people with disabilities can yield products that by default work well for everyone else. The iPad is a big step forward that takes advantage of the most profound aspect of computers - their chameleon-like ability to adopt forms that belie their endless computational (and networked) abilities - and creates a new category of devices. Even though they have made a clean break with previous computer forms (iPhone notwithstanding, even other tablet devices adhered to the desktop computer metaphor), Apple knows full well to proceed with caution and let the market inform its evolution. Apple promotes it as an empty vessel as much as anything: most of the language from Apple uses word like "magic" and "never before" to describe it, leaving consumers and media to fill in the gaps.  Similar to many other products marketed to younger people, but adopted by older consumers, Apple needs to keep up appearances to through the gates.

Matching real world systems with the iPad

A key usability heuristic termed the "Match between the system and real world", says that software systems should mirror real-world scenarios, nomenclature, and procedures. In the real world, of course, things (especially social) tend to be messy, non-hierarchical, redundant, ambiguous, and often contradictory. A recent post from a designer observed that in his many years of technical support for family members, few could successfully navigate the file system of either PCs or Macs. I, too, have noticed this. People just don't think hierarchically (I have research to back this up, just can't locate it right now). Amazingly, it has taken over 35 years for a software company to completely and successfully shield users from the rigid, hierarchical, and non-intuitive interface known as the "file system" of modern computers. The Xerox PARC Alto and Star systems had, among other numerous innovations (GUI, Ethernet), a user interface that completely hid the file system from users. It introduced the "desktop" metaphor, and that was all users could access - no applications, no file system, just documents. While this approach was nominally copied by Apple and Microsoft, nobody could successfully abandon the file system. Ever since, the user experience of computers has been like an extended version of the ugly, awkward transition of MS-DOS to Windows, where users were sometimes bounced around and forced to open the hood periodically.

Apple's iPhone did this remarkably well on many counts, and seems poised to repeat this success with the iPad. The key is that these aren't explicitly positioned as computers, but computer-enable consumer devices. I'm sure that Apple kept hitting a wall when they tried to bury the file system both technically, and from users - extremely vocal technically adept users shouted down any effort in this area. It used to be Macs were labeled "toy" computers because of the slight layer of abstraction the GUI afforded. They seem to have solved the problem by going full circle, back to the original efforts of the Xerox crew.

Pre-emptive Help

I've been an Amazon user for almost 14 years now, starting with books and moving on to just about everything else they sell. For the last couple of years I've downloaded MP3s (initially just to avoid Apple's DRM, but now because I can find music much more easily). Every time, though I have had to download and install a new copy of their MP3 downloader (nicely done for what it does, by the way). screen-shot-2009-10-28-at-102830-am

The number of times I've bailed on downloading music is much higher than the number of copies of this that are in my Downloads folder (and I've deleted a few copies over time as well).

Today, though, Amazon did something new, which they always do in their subtle, tweakish way. Just below the confirmation message, they offered several answers to what they must know to be common problems. One of them, it immediately occurred to me, was mine.


Even though I'm pretty good with computers and such, for some reason I had never thought to associate the Amazon file type with the appropriate application in my browser. It's something I've done in the past, but not in this case. Anyway, a couple of clicks later, I think I've solved my problem.

The reminder for me is to keep on iterating on everything I put out there. Small adjustments can make a massive change, while massive changes may just introduce more inadequacies to be cleaned up later.

Make it Visual

I saw this on the AirTran site and thought it was a pretty effective way of communicating - in a positive light - what type of seat you can choose for your flight. I can't say that I've ever seen an airline illustrate what seats actually look like, and frankly most seat selection tools seem to be oriented to the experienced traveler. Perhaps that's intentional, and I have no idea what percentage of people stay with their default assignment. The options for "sit together" and "leg room" seem useful ways to prod fliers into thinking ahead and selecting appropriate seats. I imagine this saves the airline a bundle in time savings not having counter personnel playing musical chairs before takeoff. screen-shot-2009-10-11-at-75838-am

AA.com UX pile-on

Not to pile on, but recent personal experiences with the AA.com (American Airlines) site underscore just how dreadful the user experience on the site is, no matter what the excuses are. I've been using - with some success - the aa.com site for some time now, ever since I worked at Yahoo and AA was the preferred airline and I racked up so many miles. I've always found it to be difficult to use, and somewhat stress-inducing given the cramped layout, tiny controls, wonky nomenclature, and tedious performance. Over the last few weeks I've been planning a couple trips, one to Mexico for the family at the end of the year, and one for myself to attend my niece's high school graduation in Madison. I found a good itinerary to Mexico and booked it, then played with several to Madison (putting a couple on hold) before settling on one. The AA site does provide a display of itineraries, which is great, but like most of the site it is poorly rendered.

Here's what it looks like actual size: aa.com itinerary

My nits:

  • The type size, width, and row height of this table makes it feel very cramped.
  • A plethora of uppercase acronyms make it hard to tell which flight is which.
  • The "Reservation Name" isn't user-friendly or even user-generated. It can also be horribly inaccurate: it only shows the first leg in any multi-layover flight. So if I had a flights to Mexico and Madison, both with layovers in ORD, that I ticketed in close succession, I could easily get confused.
  • The Cancel link only takes me to a confirmation page, not one with any more information about the flight, which would be an easy way to help people understand what they are doing. There is no email generated confirming this, no cooling off period, etc.

So what happened to me that was so bad? Well, I blew away the family vacation to Mexico and didn't even know it. I'm sure there are excellent business reason for the way things were designed here, but I also know that they all could be accomodated in a different and much more user-friendly manner.

Don't get me started with the voice-activated phone system I had to use to straighten this out (kudos to the human who did!).

Facial Recognition

Wired Science has a short discussion about how humans recognize and process facial characteristics and why we sometimes stare at people with facial deformities. An evolutionary response causes our brain to momentarily stumble when we see people that don't have symmetrical features:

To decide, your eyes sweep over the person’s face, retrieving only parts, mainly just his nose and eyes. Your brain will then try to assemble those pieces into a configuration that you know something about.

When the pieces you supply match nothing in the gallery of known facial expressions, when you encounter a person whose nose, mouth or eyes are distorted in a way you have never encountered before, you instinctively lock on. Your gaze remains riveted, and your brain stays tuned for further information.

“When a face is distorted, we have no pattern to match that,” Rosenberg said. “All primates show this [staring] at something very different, something they have not evolved to see. They need to investigate further. ‘Are they one of us or not?’ In other species, when an animal looks very different, they get rejected.”

Some of this response might be applicable to interface design, one would think. How do we respond to interfaces that aren't symmetrical or don't fit a recognizable pattern? Are the same processes at work? Some studies show interfaces that are better designed are percieved as more usable than funtionally identical, but poorly designed ones.

New NYC Parking Signs

For the last few weeks I've admired these new parking signs in the Chelsea, New York neighborhood that my office is in. I'm surprised the city used what appear to be high-quality materials and a clean design for both the form and the typography. It looks like something out of Northern Europe. I checked around online and couldn't immediately find any reference to these. Perhaps these are the result of a business improvement district and city partnership, or maybe test models.

Given the uneven track record of city agencies in carrying out revisions to signage, I'm not holding out hope this represents the future of signage in NYC. Paul Shaw recently wrote an article for the AIGA that covers the history of Helvetica as the typface of the MTA subway signage. Beyond the typical organizational cluelessness and inertia that dogged the rollout of a modern wayfinding system, relevant to today is the issue of cost - replacing signage in the subway system, let alone the city streets, must be a huge expenditure. Returning to the design of the signs, it would seem the city has invested in a quality fixture, with a solid base bolted to the street (assume it breaks on impact, sparing the cost of a new one and tearing up the sidewalk to replace), and an interchangeable and adjustable upper portion that is removed with a simple bolt. I'd be curious to know more about these.

MSFT Marketing Masquerading as Usability

I've always been a proponent of leveraging usability test artifacts (video, transcripts, quotes) to help communicate to decision makers the impact of design decisions. I always do this with great care, since I don't want to overstate issues I've uncovered, while at the same time making clear the human effects of software usability. Recently, Microsoft partnered with a branding agency to run what appear to be usability tests of its Vista operating system. The real goal was to counter the the large amount of negative press around Vista. I'm a little concerned that what Bradley and Montgomery has done may damage user experience and usability professionals do by trivializing research and making user look like fools.

  • Turns out that these were just 10-minute demonstrations by experts, not actual usability tests.
  • Apparently none of the participants were current Vista users, so their ratings of Vista at the start were entirely perception-based. Unsurprisingly, their approval ratings were astronomical.
  • This is represented as scientific by calling it an "experiment", and using candid-looking video clips of users in a lab setting. Video clips are not shown in their entirety, nor is a transcript of the interview made available. I didn't expect otherwise, since even if this was real product research, this stuff wouldn't be made public.
  • From a methodology standpoint, they out-and-out lied to the users about what the software is and what their intentions were. To make matters worse, "gotcha" clips are shown of users after it is revealed that they are indeed using Vista.
  • I don't have access to the genesis of this project, but its clear this was not an open-ended inquiry into the usability of Vista, but a marketing exercise from the outset to prove that Vista is fine and there's some unjustified bad press out there.

Is this going to hurt user researchers? I doubt it, given how few people will view the marketing site, but it is a real danger. I wonder how the product folks at Microsoft are viewing this.

*Update: The response at the MS Vista product blog isn't entirely positive either.

Abstraction and Intention

I came across a map of the US Interstate Highway system that got me thinking.  278theinterstatesystem.jpg
I used to teach a class at FIT covering digital product development and UI design. One of the topics covered the representation and abstraction of the physical (real)-world. To make information usable, some level of abstraction is almost always necessary - especially online, where limited screen sizes, the flattening of information, and issues of interaction intrude. All of these issues exist with printed matter, but there have been 450 years to work them out in a relatively stable medium.  To get the class thinking, I showed images of the London Underground map and how the representation of distance and location of stops were abstracted to enhance usability. For example, the incredibly condensed central core of the system is represented as being much larger than it really is, while far-flung stops are brought in closer. Harry Beck pioneered this representation, and it is covered nicely in a book and online.  The class got this and were nominally engaged (and supportive of the concept), but when I showed Massimo Vignelli's maps of the NYC Metro they responded much differently. I think because it hit home, they were much more opposed to losing an accurate geographic mapping of the subway system. This was the opposite of what I anticipated, as I figured they would appreciate being able to accurately and easily find a route or stop and already knew the physical dimensions of the city.Like so many patterns around us, the subway map that New Yorkers have is an evolved, hybrid approach to the problem. There still isn't a completely accurate geographic representation, but it is close enough to be useful in situations where someone might not know the area. Useful detail, however, is often obscured or inconsistent (how express trains work, for example). I'm pretty sure the highway map above is significantly less useful in this form given how people use highway maps. A strong knowledge of  US geography is helpful when looking at this, but even still, a quick glance shows clustering, density, and orientation trends that the loss of noise from a traditional map brings. The abstraction does open up thinking to a system level, which is probably something few people do outside of the DOT. On this level, overlays of population density, usage, capital outlays, infrastructure age, demographics, and many others can take on new meaning.Like any map, the loss of information opens new interpretations and limits others. 

Multitasking makes you dumb

Article in the Atlantic looks at multitasking as a generational and technological phenomenon that may (or should) be peaking . The article is well written and humorous, interspersing personal anecdotes with scientific studies. "Even worse, certain studies find that multitasking boosts the level of stress-related hormones such as cortisol and adrenaline and wears down our systems through biochemical friction, prematurely aging us. In the short term, the confusion, fatigue, and chaos merely hamper our ability to focus and analyze, but in the long term, they may cause it to atrophy."