UX Magazine

Defining and Informing the Complex Field of User Experience (UX)
Article No. 844 July 5, 2012

Overhauling a UI Without Upsetting Current Users

Companies with successful software products often know that their user interfaces could be improved but hesitate to take action because major changes might upset their existing users. “Not only would a redesign cost me a small fortune,” one executive told Macadamian “but look at the backlash against Google and Facebook’s changes. I’m not willing to risk losing customers over design.”

Indeed, Google, Twitter, and Facebook caused quite a stir recently by making dramatic changes to their UIs. In less than 24 hours, hundreds of users formed a coalition against the new Gmail, with a mission to “urge Google to fix the horrid new Gmail interface and take us back to productivity.” And Facebook users began posting “If it Ain’t Broke, Don’t Fix It” banners to express their dissatisfaction.

But for every example of a redesign gone wrong, there is a success story of a company blasting ahead of its competitors by reinventing its product design. Salesforce.com’s dominance in the CRM market, for example, is largely attributable to its innovative design updates.

The risks associated with not updating a UI outweigh those associated with an update. But a successful redesign absolutely requires the right product management and UX techniques to evolve the product carefully and avoid a user revolt.

Consider Form and Function

Ninety percent of the value of an experience comes from how it works, not how it looks. Of course, it’s not that visual design is unimportant; it’s just not nearly enough.

This is the source of one of the major complaints about Gmail’s new interface. “Gmail's new interface is form over function. Strange because the reason everyone went to Gmail was function” says one Twitter user. “Gmail’s new look is uncluttered, which is a good thing until you try to navigate and then, user beware.”

It is a myth among non-designers that the essence of a fantastic experience and a product’s success is its aesthetic value. Although UX pros recognize that visual appeal is just one part of the experience, even they sometimes spend disproportionate amounts of time on visual design, sacrificing other aspects of UX. The importance of good looks may be the initial draw to a product, and visual design can often be a very important factor in influencing purchasing decisions. However, for a product to be endorsed by the market, development needs to be focused on much more than just how the product looks. This is especially true in a world where information and opinions about products travel quickly and unpredictably via social media.

A redesigned UI that looks pretty but fails to deliver new value will not only disappoint new users but will also alienate existing, previously satisfied users, and the news of this failure will spread rapidly. Never make the mistake of thinking that a product’s aesthetic is the same as the product’s actual experience.

Redesign Based on Evidence

In order to make functional (not just aesthetic) improvements, product managers and design teams need to perform research with real-world users. Most people already agree with this in principle, but vary wildly in how well they execute on it.

One of the most common mistakes companies make is to implement UI changes based on what users say they want. Unfortunately, experience shows that what people say they will or won’t like doesn’t always match reality. Moreover, a user’s suggestion on how to solve a UI problem may be unworkable, unreasonable, or both.

While it’s important to solicit and listen to user feedback, users aren’t designers or usability experts so their complaints and suggestions should not be taken at face value. Don’t confuse a design suggestion with a design solution. To identify the real UI problems and solutions, careful UX research must take into account facts, not just opinions.

Two fundamental techniques that should be used in any major UI update are focus groups and usability testing.

Focus groups

Focus groups are a powerful exploratory technique that can provide rich information on the opinions and attitudes of the target audience regarding a new or existing product or idea.

They are best used in the early discovery and assessment stages of a design update project in order to:

  • get a feel for the range of opinions on different aspects of the interface
  • understand the reasons for change requests, new interface ideas, and underlying preferences
  • build a raw list of user requirements

This information can provide direction to a development team in the early stages of their work so they can prepare for, and even get an early start on, development in key areas in parallel with the design effort.

However, focus groups are not a good source of behavioral data. What people say they will do, compared to what they actually do, is notoriously unreliable for several reasons. First, what people say they will do (intent) and what they actually do (action or behavior) often vary because people can't predict contextual factors that may alter their behavior in a given situation. This is why people like test drives and 30-day return policies.

Second, people are not fully aware of all of their behaviors. Some everyday actions are so commonplace that people sometimes forget they even do them. For instance, if you ask people something as simple as, "What features on a telephone do you use and approximately how often?" and compare the results to direct observation data on the same question, the results are usually staggeringly different.

That's where observational techniques like usability testing come in.

Usability Testing

Usability testing is a technique to determine the effectiveness of a design by observing and assessing users as they walk through a predetermined series of tasks in the software. For example, if users consistently stumble during a particular part of the workflow, this is a strong indicator that the design is flawed.

Researchers usually follow-up each usability test session by interviewing the user and asking more qualitative questions about the experience. Usability metrics—such as the time it takes to complete a task, the number of errors, the number of clicks, the success rate, etc.—are collected along with ease of use ratings.

Usability testing is very different from beta testing. In beta testing, users will typically only report usability problems that make it very difficult for them to accomplish a task—in other words, things that are very clearly bugs. They typically won't report that they found something challenging or unintuitive. People don't always like to admit that they failed at something. Also, beta testers (or at least the ones who take time to report issues) are often fans of the product, and are therefore also power users. They may have already learned to work around or ignore usability issues.

Hopefully, by the time a product is in beta, most of the obvious usability issues will already have been solved because the most expensive time to fix a usability issue (or any bug for that matter) is after the product has been built.

Stop the Endless UI Debates

Despite access to quality user research, many projects still get mired in unproductive debate about design and implementation issues. To avoid this, take the development and design team back to the first principles of the product. When arguing over a particular feature or screen, stop to ensure the team is in agreement about the answers to the following questions:

  • Why will a user want to use this feature?
  • What will be their most common tasks?
  • Under what conditions, or in what contexts, will this feature be used?
  • Who are the primary users of this particular feature?

Taking just a bit of time to regroup and make sure everyone on the team knows the answers to these questions will forestall wasted debate and speed project completion. If you don't, the same issues will inevitably come up again and again in the design and development process. Often when a team debates whether to include a certain feature, the true, underlying questions are, "Who are our primary users and what would they value?" If the team has considered these questions already, the answers to feature decisions will be clear, allowing the team to breeze through those discussions.

Integrating Redesign Feedback: Interpret and Prioritize

Even the best products won't be perfect or universally loved; user complaints are a perennial and pervasive part of the design (and redesign) process. So when evaluating user complaints about the UI (there will always be some!) don’t react to them at face value or in a knee-jerk fashion that just closes the trouble ticket.

Experienced product managers and UX designers will take the time to gather qualitative and quantitative data from a variety of users to determine:

  • how frequently each design issue is occurring
  • the impact of an issue on a user
  • the relative importance of one issue over another

This data and analysis allows teams to make sense of all the user reactions so that they can swiftly address issues that will most affect their product (and ultimately their business) success and put aside all the red herrings.

How pervasive is the complaint?

Some complaints may identify a common user problem, but they could also simply be gripes from a very small but vocal group of users.

This is where quantitative data becomes very useful. When Google changed the Gmail UI, it added a pop-up bar in the corner of the screen that asked users for feedback on the new design.

Gmail Feedback Bar

This is an effective mechanism for determining how widespread design problems are, and as a bonus allows users a sense of control as they vent their frustrations in direct communication to the software maker.

However, this method can be costly as it requires staff to sort through the wide variety of comments. The quality of the data provided is also; users will often offer solutions rather than explaining the reason behind their frustration.

For more cost-effective and equally powerful data gathering, consider conducting a random survey of your targeted audience through user interviews or an ethnographic study. This provides the quality and quantity of feedback necessary to ascertain whether a given complaint is common and should be immediately addressed, or is a minor issue that should be lower on the priority list.

Consider the Consequences of not Making a Change

Jumping into a design update can be costly both during the design process and, worse, if the updated product causes user backlash. It’s tempting to take the conservative approach and maintain things the way they are, hoping to avoid hurting customer goodwill with a bad UI decision. But with products such as the iPad and SalesForce.com setting a new bar for software UX, companies are being forced to update their products’ UIs in order to meet rising consumer expectations. The risk of falling behind exceeds the risk of alienating existing users.

Fortunately, you can have your cake and eat it too. Proper product management and UX techniques can reliably produce UI updates that keep a product ahead of the curve without losing loyal users. It’s a discipline based on the fundamentals of good UX design: collecting, interpreting, and prioritizing user research. These are core competencies that companies planning to stay market winners must cultivate within their organizations.

ABOUT THE AUTHOR(S)

User Profile

Scott Plewes is an expert in user experience design, user research, and incorporating the voice of the customer into product design. As Vice President of User Experience Design at Macadamian, Scott has almost 20 years of experience in the field of user interface design, working in both the public and private sector. Scott's experience design skills cover the spectrum from desktop, web, and mobile user interface design through to command line interface and telephony interface design.

Add new comment

Comments

25
28

Good for Hank for calling me out on "Ninety percent of the value of an experience comes from how it works, not how it looks" and asking if the statement has an experimental basis. Its not as if I came up with an operational definition of 'how it works' and then spent a year doing a meta-analysis across a bunch of experiments to come up with a defendable conclusion and that number. The statement was meant in a much more colloquial way. Although I did have in mind a few things. Here's what I was thinking when I wrote the article. One, Steve Jobs well known quote about design along the lines of design is about "how something works not how it looks". Two, my own experience in 20 years or so where most of the experimental evidence I've seen - and even talking to experts - is that much of the value (again not an operational definition) is attributable to things like the matching the users mental model, correct information architecture, workflows and so on. And third, I actually did have in mind a meta-analysis IBM did on usability (which of course is not the same as value) which did have a number like this. So is the "ninety percent" quote defensible? Strictly I'd say not literally or scientifically. And "value", however you may define it, will vary from product to product. Sometimes 'how it looks' may play a much bigger role, and sometimes a much smaller role. However, the gist of the quote I'd still stand by. Typically a large percentage of the value in the user experience of a product goes much deeper then just how it looks. That was really the point. Great question/point though.

31
29

I'm curious how you came to the conclusion that "Ninety percent of the value of an experience comes from how it works, not how it looks." How did you quantify the value of an experience?

26
25

Regarding David's "sometimes you need that power to push a change through so that you can then examine analytics data to validate the changes". I agree that no matter how much you do up front there is still some follow up information that needs to be done. In experimental psychology they call this the difference between an "internally valid" experimental result (ie. in the lab) and an "externally valid" result (i.e. in the real world). You can't learn everything in the lab. But you can learn a lot. Good point.

Regarding the discussion about "releasing slowly" when changing vs. the potential of "fragmentation" of supporting two designs. I'm not trying to be politically correct here, but I really believe they are both good points. And not actually necessarily mutually exclusive. I think there are a bunch of factors at play here. Google can release an old and a new partly because they are Google and can afford to test and iterate the new one while allowing people the convenience of the old one (if they so desire). This allows for a transition to the new where they are learning before completely committing. So, I think, its actually in the spirit of what I said about "testing" with users. I agree though this can't go on forever and then there is eventually a "fragmentation" problem. Its really about timing. You can't support two designs forever. For other companies in different situations they might not be able to afford what Google does, so you have to commit with a more abrupt change; but you still test somehow ahead of time in other ways to try and manage the impact.

25
26

Really interesting article on a topic that I have been wrestling with for a long time. I agree with the focus needing to be on user research. The biggest problem I have faced in re-design situations are summed up in the "How pervasive is the complaint" section. Making stakeholders understand that just because one person doesn't like something doesn't mean you have to change it. Testing with real end users can help, but sometimes you need that power to push a change through so that you can then examine analytics data to validate the changes.

28
32

I really enjoyed this article, very well thought out and logical.

As Steve mentioned in the comments, what would be your opinion on supporting the old design. This understandably would content the unhappy customers, but I personally feel this is the wrong move, progress for progress's sake is bad, but supporting multiple designs just leads to fragmentation and the same displeasure with the audience when it eventually comes to end the support.

Still good article, thanks!

27
29

Thanks for writing the article. Any thoughts on managing the release of the UI so as to cause as little user frustration as possible? I remember reading an example of ebay and how they changed a color by slightly fading it over many releases so it was not as noticeable. I've always liked what yahoo's done and google for that matter. Letting you have the 2 versions for a while and then letting go of the old one after a period of time.

23
27

@Richard, simply put, lack of money. We don't have extra cash for design and engineering work, and we always prioritize content over all else. Hopefully some day we'll have a chance to make our site into a glorious monument to UX, but until then... thanks for reading!

26
30

Great article, but why does a UX magazine not have a responsive site?