Managing Vanguard’s Investor Experience

UX Magazine sat down with Amy Dobra, a principal at Vanguard and head of its Retail Client Experience Group, at the Forrester Customer Experience Forum. This discussion was part of our ongoing effort to profile business UX leaders so other businesspeople can learn from the examples set by other managers and executives, and so practitioners and consultants can hear about UX from the client’s perspective.

UX Magazine sat down with Amy Dobra, a principal at Vanguard and head of its Retail Client Experience Group, at the Forrester Customer Experience Forum. This discussion was part of our ongoing effort to profile business UX leaders so other businesspeople can learn from the examples set by other managers and executives, and so practitioners and consultants can hear about UX from the client’s perspective. You can read more about the Business UX Leaders Series here.

Part 1: How customer experience (CX) and user experience (UX) have taken hold at Vanguard

Jump to the transcript    |    View video on Vimeo

Part 2: The value and role of research; tracking business outcomes; role of vendors

Jump to the transcript    |    View video on Vimeo

Part 1 Transcript

Would you say your focus is on user experience (UX), or on customer experience (CX)?

Dobra: I think that’s an interesting questions because Vanguard is a client-owned company, so everything that we do we feel is related to the client experience. In my particular role, I’m really in charge of all the communications that go to our clients, so whether it’s print collateral, whether it’s a client letter, whether it’s vanguard.com, we oversee the creation of that. So we like to consider it a very important part of the client experience, but it’s certainly not answering the phones or doing the day-to-day servicing.

How did experience-driven design practices first take hold in Vanguard?

Dobra: I’ve been at Vanguard now close to 15 years, and one of the things that they make very clear the first day that you get started at Vanguard is that, again, every decision that you make has to benefit the clients. And you always have to ask yourself, “Is this the right thing to do for the clients?” If by investing in this particular Web project or this communication, is that going to help them reach their retirement goals? Is that going to help us lower expense ratios so that they can have low cost, high quality investments? So from day one, it’s in everyone’s frame of reference for every decision that we make.

When Vanguard first got started [on the Web]—and we were one of the early ones in the financial services industry in the mid-90s—they really treated it like a startup within the firm. So they used a mix of bringing people in from the outside who had some early knowledge of what this Internet thing was all about, as well as people who were steeped in our business, and they built this team. They called it “Web Services” at the time.

That unit stayed together for over ten years. It was a central organization within Vanguard and, over time, they built out the discipline of the user experience within Vanguard. It was, at one point, I wouldn’t say a large organization, but large as compared to other companies that I’ve talked to who tend to have a very small unit and then do a lot of partnerships with agencies. At this time a few years ago, we actually integrated back into the business units, so my user experience group is focused around the retail business line and now I oversee not only the Web, but the print and the social media channels as well. So we’re more integrated from an experience standpoint.

Have people’s skill sets translated well through that transition?

Dobra: It translated fine because they’re still, in essence, doing a lot of the same things that they already were, but now we ask them to broaden their mind and broaden the scope of some of their projects. So if we’re looking at something, let’s say, related to balances on vanguard.com, we’ll also take a look, then—how are we displaying balances on statements or other communications? It provides them with the opportunity to be more holistic, which is what we all want to do when it comes to user experience. So I think it has provided them an opportunity to do a little bit more.

As your UX staff has distributed more evenly across the organization, who’s keeping all their efforts coordinated?

Dobra: We’ve instituted a couple different levels of governance, I’d say. One, we have a brand governance council, that leadership and some of the discipline owners will get together on a monthly basis and will talk about whether it’s design direction or key strategies going on in the individual websites that we oversee. But then we also have a user experience team where all the managers that have user experience professionals working for them, they get together and then they sort of share best practices and make sure that we’re working toward a coordinated user experience.

What do you do to keep everyone’s efforts focused on the customer experience?

Dobra: Specifically to the process that we undertake when we’re, let’s say, focusing on the web design, we’re very focused on doing up-front client research. So whether it’s focus groups or usability studies, we do MVT testing, we do experimentation, and then we share these test results on a regular basis with our business partners and IT partners. So that everyone is knowledgeable about what’s going on. What’s the feedback? How are people reacting to, whether something that’s in production today or something that we’re considering launching. So there’s a really good feedback loop that goes on.

We do a good job sharing how people are using vanguard.com; that’s very much available throughout Vanguard. We do a good job at a project team level sharing the information, the personas, the research. What I think we have an opportunity to do better is making the research more broadly available to those who might be interested in it. So while we have it available through our intranet site, I don’t think we’ve necessarily told a lot of people that it’s out there and that they could learn more about the personas at Vanguard and things that we have learned through research. We have a lot of conversations about that and we’re always looking up ways to better disseminate that information.

We absolutely agree with that analysis that the client experience needs to be something that’s infused through the organization, and it’s something that is a bit egalitarian at Vanguard. Everybody feels, in a way, that they own the experience for clients, and whether that’s on the phones or through operations at vanguard.com, which is great because it means that we’re going to be trying to do the right thing. It can be a challenge at times, too, because when you have ten “owners” in a room trying to get things done, at some point you do need people who are the decision-makers and know how to translate all of these good ideas into the one that’s going to resonate or work best for our clients. So it’s awesome that everyone cares, but I don’t want to say that’s not a challenge because it can be.

Part 2 Transcript

Have you had any difficulty getting support for upfront research, when so often people think it’s an unnecessary delay before work actually begins?

Dobra: We really haven’t, and I think it’s because in years past, when we were just starting out building out our user experience discipline at Vanguard—this has been 15 years ago, now—we made some mistakes, and some pretty public mistakes from a client standpoint on vanguard.com, and those mistakes can be tied back to not listening to the upfront research. So we learned very quickly that research is important and listening to the client and what they want or how they behave is going to be critical to success for a great client experience. And it’s also going to be more efficient in how you spend your technology dollars because nobody wants to have to re-do something or swarm it to fix it. So we have a lot of buy-in around doing research, and we even expanding the amount of research that we do now.

What types of research are you performing?

Dobra: In our arsenal we have all the typical things, from usability tests and accessibility readouts, we have expert reviews that happen. But we started this past year doing multi-variate testing—A/B testing—on the site, as well as experimentation where we can now have a region that’s dedicated to future designs. We invite thousands of clients in at a time and then we observe their behavior, ask them questions about potential designs that we want to put out there.

Research really is running all the time. So at a project level, we certainly want to frontload as much research as we can to make sure that we’re helping in the requirements process. So we use usability tests, we use card sorting, we use focus groups, we’ll look at third party research and trends in user behavior. But then in recent years—in the last year, year and a half—we’ve expanded our use of A/B testing in production, multivariate testing, and also through the introduction of experimentation where we’ve been able to get more volumes of clients through these pages, which has been really instrumental and key to how we’ve been designing the websites. Because in the past, I would say it was driven by best practices in UX development and design, great judgment from people either in the field or great people representing our business. But that was the primary driver, and while I think that was a good strategy back in the day, we have this rich data available to us and we’re now using that data to make better decisions—more informed decisions.

It doesn’t necessarily take the judgment or the years of experience out of the picture, but it’s a better blend. So instead of sitting in a room and maybe evaluating a particular webpage and having a strong personality really disliking a particular color or particular design, we now can say, “We’re going to test this and we’re going to see if it works with our clients or not.” So that’s very exciting to us because it makes us raise the confidence level in what we’re putting out there for our clients and helping to make sure that we’re going to have a better client experience in the long run.

How much user testing do you do during the course of a project?

Dobra: It’s going to depend upon the complexity of the project, but multiple times. It could be, for a small project, it’s two or three times. It could be four times if it’s something a little bit larger. They look to get in front of users at every critical stage. It could be conceptual. They want to get out in front of users. Like, conceptually, are we in the right ballpark? Then we’ll start to get to a more finite level of user testing to make sure that we actually can find things and the details are correct. But there’s not a prescribed number of tests for a project. It really depends on the size of it.

Are there any business outcomes that you track to measure the effects of your UX efforts?

Dobra: There’s plenty. I’ll mention a few. One is completion rate around transactions. It’s a very transactionally focused website, as well as having educational and publicly available information. But we monitor, can people get through the exchange process if they’re trying to move from fund to find. We look at the process for opening accounts on vanguard.com and are there drop-off rates? Is a particular design helping with a particular point of pain? And that’s looked at from our side as well as the business units, who own that process. We look at bounce rates to vanguard.com. Who’s coming in from other sources and coming to Vanguard? Are we engaging them in? Are we helping them find the additional information that they’re looking for? Or they come and, nope, it’s the wrong place or, no, this wasn’t what I was looking for. So those are a few of the measures that we’ve been able to not only have our team care about but the business cares deeply about them. Because they’re very measurable results to their bottom line because it’s going to make a difference to the call center, it’s going to make a difference to how the company’s growing if vanguard.com can’t complete those goals. We’re very much an important part of that because we’re the ones who design the experience.

How much of your UX work is done in partnership with outside vendors or agencies?

Dobra: I’d say probably over 90% is resourced from our team and in-house at Vanguard. That’s not to say that, on occasion, we don’t partner with agencies or other firms, but it tends to be around a couple of instances. Sometimes we’ll bring a partner in and it could be at the very strategic level. Maybe we’re hitting an inflection point with the experience or a new service that we want to roll out and we want to get some external thinking and we want to learn from them and we want to partner with some new ideas. Or it could be that there’s a lot of work going on and we need to work with a firm that can actually help extend our resources. So it can be at the more tactical level. But again, those things tend to happen on the fringe. The bulk of the work is done by our shop.