Top

Conference Review: Enterprise UX 2018, Part 3

May 21, 2019

In Part 1 of my Enterprise UX (EUX) 2018 review, I gave an overview of the conference and reviewed Jorge Arango’s very enjoyable “San Francisco Architecture Walking Tour.” Then, in Part 2, I shared some highlights from Day 1 of the main conference, which convened on Thursday, June 14, at the Mission Bay Conference Center, on the UCSF (University of California, San Francisco) Mission Bay campus. On Day 1, the BUILD theme was the focus in the morning; the COMMUNICATE theme, in the afternoon.

Now, in Part 3, I’ll cover some highlights from Day 2 of the main conference, Friday, June 15, whose sessions focused on the theme INVEST in the morning and the theme SCALE in the afternoon. Issues relating to investment and scalability are key factors in the success of User Experience within enterprise organizations.

Champion Advertisement
Continue Reading…

More Highlights from the Main Conference

In the morning of Day 2, the theme was INVEST; in the afternoon, SCALE.

Morning of Day 2: INVEST

INVEST was the theme for Friday morning’s sessions. The focus of this theme was on investment in strategic gains—balancing long- and short-term priorities—and laying the groundwork for the success of UX teams within enterprises by developing strong, multidisciplinary partnerships.

A Roadmap for Maturing Design in the Enterprise

Presenter: JJ Kercher

JJ Kercher, shown in Figure 1, is Head of Customer Experience, Real Estate, at AppFolio, which develops cloud-based software for vertical-industry SMBs (Small-to-Medium-sized Businesses). She told us their User Experience team was rebranding to become a Customer Experience (CX) team. JJ presented a UX maturity model that could provide a roadmap for maturing any enterprise UX practice.

Figure 1—JJ Kercher
JJ Kercher

The UX Maturity Model

The UX maturity model that JJ presented was from Johan Berndtsson (2014) and, as Figure 2 shows, it takes the form of a ladder, with increasing levels of UX maturity from bottom to top.

Figure 2—Johan Berndtsson’s UX maturity model
Johan Berndtsson’s UX maturity model
  1. Unrecognized—At the first and lowest “rung of the maturity model, UX is unrecognized,” said JJ. “That typically means you don’t have a UX practice.” There is no design team.
  2. Interested—“The second step…is interested. … That means [an organization starts] realizing that UX is important, but it…receives little funding. People aren’t quite sure what to do with it.”
  3. Invested—“The third rung is invested. … [UX] actually becomes very important, and this is when you get some formalized programs around design emerging.”
  4. Committed—“The fourth is committed, so [UX is] critical, and this is where [UX starts piquing] executives’ …interest. … How can [UX] help our business? How can it help our strategy?”
  5. Engaged—“[The] fifth step, which is…engaged, [means UX] is one of the core tenets of the organizational strategy. … UX [has a] seat at the table for strategic discussions.”
  6. Embedded—“[The] ultimate destination, which is the sixth rung, called embedded, [means] UX is embedded in the fabric within the organization and…is included in all the work that you’re doing.”

Climbing the Maturity Ladder

“I’m going to actually walk you through the steps of climbing this maturity ladder,” said JJ, who noted that, as designers, we’re all at least at Stage 2. “As I’m going through this, I encourage you to really think about where…you are in your business. Think about where you want to go. … Depending on how your organization is organized…and the maturity of [your] product…, deliberately focus [your] efforts on where you want to grow your practice to the next level. … It’s impossible to go [directly] from rung one to rung six.”

Stage 2: Interested

Figure 3 describes the general landscape of Stage 2: Interested. “If your business is in Stage 2, this is a point at which [the] design team is disproportionate to engineers—often the design team of one. … UX tends to be positioned as an advisory or consultant role. So the reality is: When the first designer is hired into these businesses, there’s someone else there that probably isn’t a designer saying, ‘This is the way I think it’s going to work.’ A lot of times it’s someone thinking, ‘Hey, the engineers will just come to you when we think we need you or we’ll have you come in.’ I’m sure this sounds familiar to some of you. … At this point, [UX is] definitely tactical, and it’s also nonroutine—and what I mean by that is you haven’t really even established any sort of traditions or regular occurring milestone or deliverables.”

Figure 3—Stage 2: Interested—General Landscape
Stage 2: Interested—General Landscape

“When I started [at AppFolio], there was one designer to three product-facing teams,” JJ told us. “This was an engineering-driven culture whose product had ease of use as the #1 key competitive advantage…, so you can imagine that the engineers were a little skeptical about why there was a designer here. ‘We’re already doing great!’ …

“Every year with my team, at the beginning of the year, we do an offsite, and we try to identify what we think our primary challenge is for the year. Back then, because we were just in Stage 2 and things were kind of messy, we weren’t doing that deliberately, but…I’ve gone through…the last eight years and…wrote down what I think our challenge was at that point.” As Figure 4 shows, “At this point…, it was really simple. It wasn’t to build a design system or to redesign that platform…. It actually was just to integrate a user-centered design practice into an already really well-oiled agile machine.”

Figure 4—Stage 2: Interested—Primary Challenge
Stage 2: Interested—Primary Challenge

“It’s really important to identify what your primary challenge is,” advised JJ. “You can only do so many things well, and this is really about where you invest the time to understand what challenge you’re meeting and knowing that, once I meet this challenge, I can move on to the next one.”

Next, you need to determine where to focus your investment. JJ discussed the three areas in which AppFolio needed to invest at Stage 2, which are shown in Figure 5:

  1. Relationships—“It’s all about your engineers—about really building that rapport, building respect and mutual understanding between yourself and the engineers who are actually building the software. … It means [being] inclusive. In the same way that you would do qualitative research with your customers, do that with your engineers. Understand what motivates them. Understand what drives them. Try to learn their language so…you can weave [in] your stories—your language—in a way that they can…identify with better. What this is really going to do is set the stage for growing internal champions.”
  2. Design Practice—“What kind of investments [do you need to ] make in the practice itself? At this stage, because design is so new, you don’t have routines. I really recommend focusing…on just building empathy. When I started at AppFolio…, the very first thing that I did was set up usability testing because I knew that was going to help me with…relationship building. If I could get those engineers in front of customers, I [could] start to really build the connection of empathy. … Instead of doing mockups right away…, just go to the whiteboard. … You’re going to gain more respect and more collaboration with your Engineering team by doing whiteboarding. … Get to know their visual language…and try to present experiences them to them in the same way.”
  3. The Team—These are the “investments that you’re making in the team for future growth. At this point, there probably isn’t much of a team. There’s one person so the leader really is the individual contributor. What kind of skills [do] you need to learn personally…to take this to the next level? For me, it was all about agile. … I had to really understand: how does design fit into this without disrupting something that’s already working really well? … Research skills because [research] feeds into building empathy, and it also helps you to build relationships.”
Figure 5—Stage 2: Interested—Investment Focus
Stage 2: Interested—Investment Focus
Stage 3: Invested

While JJ didn’t have time to cover every stage in the UX maturity model in depth, she did discuss a some characteristics to consider “as you’re thinking about where you may be” relative to Stage 3. “The landscape…is actually pretty similar to Stage 2. UX is still pretty tactical and probably is still considered an advisory function. … What separates [Stage] 3 from [Stage] 2 is that…your deliverables are becoming more routine. You’ve started doing usability testing. The engineers are bought in and now they’re asking you for more usability testing. … You’re starting to introduce wireframes or user flows, and they’re asking for that. So there’s…more of a cadence. … You’re still experimenting, but I think that’s probably one of the key differentiators there. Usually, this is the point, if you build those relationships and you’re doing those deliverables, [the] best thing you can get is engineers’ asking for more…. That’s what’s going to get you more staff on your team because the demand is higher.”

Stage 4: Committed

“This is critical and when the executives start getting involved.” Figure 6 describes the general landscape of Stage 4.

Figure 6—Stage 4: Committed—General Landscape
Stage 4: Committed—General Landscape

“This is the point where people start realizing [they] need to think in terms of ratios with designers—in the same way that we speak of product managers to engineers. … How many designers to engineers? How many designers to teams? This is…one indication that you’re in [Stage 2]. You’re seeing [that] there [are] gaps, and you’re trying to kind of correct that. Designers move from that advisory role to [being embedded on teams]. … Designers are involved from beginning to end—actually are collaborating in the trenches. They end up on the Scrum teams. Usually, at this point, … specialization [becomes] a need. [At AppFolio], we hired researchers and a visual designer. You just get to a point where…you really need dedicated focus. … You’re still really tactical, but you’re [striving more for end-to-end solutions]. … Through the research and the validation work that you’re doing…, you’re also identifying new problems [you] can potentially solve. That’s setting the stage for that product strategy bit, which is the next phase. … This is…the point where the manager role [becomes] a distinct role. …

“We have…small, cross-functional teams that are autonomous…. As we started growing in teams and…in designers, …people were starting to design…inconsistent experiences. … That’s the trigger for [having] a conversation about design patterns [and making] that investment. … So our primary challenge at this point was delivering consistent and delightful user experiences across all the channels, workflows, and devices, [which] actually requires intentional, cross-functional alignment and shared understanding of the user and business goals,” as Figure 7 shows.

Figure 7—Stage 4: Committed—Primary Challenge
Stage 4: Committed—Primary Challenge

Next, JJ spoke about the focus of the investments they made at Stage 4, which are shown in Figure 8.

  1. Relationships—Everyone needed “to align on what the experience should be, so…we [could] continue to work on these autonomous teams. The relationships, at this point, [comprehend everyone working on] digital experiences, [including] Marketing [and the people creating online Help]. … How do we start forming alliances with the people [working on the] digital customer experience? … You’re building relationships with engineers, but your product managers are also really important. … At this stage, you start to have conversations about why it’s important to prioritize design.”
  2. Design Practice—“[This] is all about evangelism and getting people on board. … We created a Design Patterns Guild—very similar [to] the Spotify model. People from any team can volunteer to be a part of this group. … We talk about design. We talk about design patterns. It’s pretty powerful to get engineers…as part of the conversation. … We picked the battle of our Help content—our knowledge base.” At this stage, JJ’s team created an analog design system, using “Google Drive and Google Docs. … We started introducing tools to share behavioral data, as well to…mix the qualitative data with the quant that our engineers are really asking for.”
  3. The Team—“This is really around shared knowledge. At this point, you’ve got a leader that really should be focusing on building those relationships—coaching, hiring, mentoring, and evangelizing [the] customer end to end. Our [UX] team introduced weekly standups just for our team so…we could all stay in the loop of what was happening across all the cross-functional…teams. We introduced formalized design reviews. … Our team was growing at this point…, [but] we wanted everyone involved in key decision-making so we wanted to keep our team flat. … Even at 23 people, [we wanted] to make sure everybody has a say in…hiring.”
Figure 8—Stage 4: Committed—Investment Focus
Stage 4: Committed—Investment Focus
Stage 5: Engaged

JJ summarized a few characteristics of Stage 5: “This is where you’ve got the executives saying, ‘Yes I’m bought in. Let’s do it.’ You’re invited to the strategy table. The difference is: it’s kind of messy. … What does that look like? … You’re now starting to actually formalize what those programs look like. What are the repeatable processes? How do you continue to hire and onboard really great talent now that you’ve got the investment to grow?”

Stage 6: Embedded

Figure 9 describes the general landscape of Stage 6. “Usually you’re dealing with [a] mature product that’s growing in complexity. That introduces its own challenges. … Your team becomes very structured. You have clear growth paths and career planning. The relationships between designer and product manager [and tech lead] are super solid. … You’re seeing the real three-legged stool —all three working together really closely. You’re seeing design and research—not just within product development but…starting to extend across the [organization]. UX is starting to inform product and business strategy. … Product complexity…really pushed us…to pay down the UX debt. … The business [invested] in an end-to-end customer-experience practice.”

Figure 9—Stage 6: Embedded—General Landscape
Stage 6: Embedded—General Landscape

As Figure 10 shows, “Our challenge was actually to feed the tactical UX insights that we’re learning on the ground back into the business strategy, so…we can reveal opportunities to increase promoters and scale our product…effectively. When you’re at this stage, you have to align your challenge and your team goals to…business goals. … This [was] our key [business] challenge. … [We] go through the business challenges as a team and…pick out the things that we think [UX can] have an impact on. This is…where we place our focus.”

Figure 10—Stage 6: Committed—Primary Challenge
Stage 6: Committed—Primary Challenge

JJ then discussed their investment focus at Stage 6, which are shown in Figure 11.

  1. Relationships—“When you’re embedded, your focus of relationship building [is] nonstop and…cross org. It continues to be product [development], engineering, [and] product management. You’re really starting to ingrain outside-in thinking across all roles.”
  2. Design Practice—“Your practice…becomes really strategic and you’re thinking cross-channel. So you’re still doing the tactical work on the ground on the teams, but you’re also…getting out ahead of understanding some of the business goals you’re trying to solve. This is where it’s…imperative to have your design system. … UX operates as product owner sometimes, depending on…the enhancements that we’re making. We [started] experimenting with things [such as] design sprints; for the end-to-end [customer experience], bringing in people to a workshop to do some journey mapping. … Similar to when we did the cross-functional Design Patterns Guild, we actually created a cross-departmental CX Council—just to talk about things that were related to end-to-end customer experience.”
  3. The Team—“Let’s make sure we have a strong career path so…we can keep, retain, and continue to grow the great talent that we already have. … We introduced the lead and the manager role. We’ve got really clear career paths [so] you can go either individual contributor [or] into a management path, and you can switch…between [these paths]. … We continued to keep decision-making flat, involving people in all the decisions, and it’s really critical…to start to grow business acumen. That means being able to equate the work that you’re doing—the work that you’re advocating—to business impact, and that’s really hard. … When we got a theme to pay down UX debt and there [were] a ton of things that we knew intuitively…we needed to change…, we picked the things that maybe weren’t on the top of our list, but…were the easiest to equate to business value. We get so many cases a month on this one feature that just is a usability issue. Let’s go fix that. Then we can save 15 hours of [Customer Support] time. Multiply that by salary in dollar. … For us to continue to grow and mature as designers and researchers, understanding how we can tie the work that we’re doing back to the business is really critical.
Figure 11—Stage 6: Embedded—Investment Focus
Stage 6: Embedded—Investment Focus

Evolving the Maturity Model

“At AppFolio, …we climbed that UX maturity model and that’s what brought us into CX.… User experience at AppFolio is known as digital product design. … We’ve had to rebrand and evolve our team because we’re getting into customer experience. It’s end to end. It’s every touchpoint. We work horizontally across all the silos, introducing the tenets of service design and applying design as a way to address some of these issues. … Those rungs haven’t been defined. … Our team’s mission statement is really simple: Accelerate customer value through the design and delivery of exceptional end-to-end experiences. … We are driven and passionate about the customer value we deliver. If we can do that successfully, it creates value [for] our company.”

Figure 12 shows “what our team looks like [in this next phase]. We have three distinct disciplines on our team: product design, which…we’ve been doing…since the beginning; user research, which is evolving. We’ve added a couple roles: … insights [and] ops. Then the new…one that we’re really starting to grow: [Customer Experience]. … We have two people on this team because it’s the right investment right now for where we are. … We’re doing service design experiments. … The customer-experience successes are really evident.”

Figure 12—Customer Experience team roles
Customer Experience team roles

“Don’t forget to reflect back and dream about where [you want] to be in the future,” concluded JJ. “Plan the steps that it’s going to take to get there.”


JJ put together a beautiful presentation, was a very effective speaker, and brought life to Johan Berndtsson’s UX Maturity Model.

The Magic Word Is Trust

Presenter: Dorelle Rabinowitz

As Senior Director of Global Design at the very large enterprise PayPal, Dorelle Rabinowitz, shown in Figure 13, leads the product design teams for Consumer, Credit & Identity. She spoke about the important topic of “how to build genuine relationships,” which “improve everything from decision-making [to] collaboration, leading to better and better solutions, but also a happier workplace. The key is, of course, trust.”

Figure 13—Dorelle Rabinowitz
Dorelle Rabinowitz

Dorelle’s organization follows the three-in-a-box approach to partner cross-functionally. “Design, Product, and Engineering work together to create our best solutions,” said Dorelle. “Actually, it’s n-in-a-box” because they involve other functions as necessary. “We genuinely trust each other, …and because we trust each other, we can do everything. Building a trust muscle…takes practice. … We build this trust muscle by building genuine relationships. … A genuine relationship at work…can be the key to really being effective.”

“When I talk about being genuine, what do I really mean? I’m talking about words like authentic, real, sincere, honest, truthful, frank, and candid. It’s really hard to be that way with people at work. But the more you do it, the more you can improve everything about your work experience and your products and how you do things.

“What does trust have to do with it? Imagine this scenario: Your team designs a great experience that you know is going to benefit your customers so much. But your Engineering partners aren’t on board. Maybe there’s an us-and-them mindset and no one is listening to each other. … What ends up happening is that…you may build an experience that’s just not optimal.”

In contrast, Dorelle told us, if you have “a great relationship, a genuine relationship, with your Engineering manager, …you talk it out, you might debate, you discuss the pros and cons. … With a little negotiation, …the team will get it done because you work it out together. You’re able to do that because you trust that you both want the same thing: a great experience.”

9 Tips for Building Trust

At the heart of Dorelle’s talk were her nine tips for building your trust muscle, shown in Figures 14–22.

Figure 14—Tip 1: Assuming good intentions
Assuming good intentions

“It’s easier to assume negative intent,” said Dorelle. “Because, if someone is disagreeing with us, we think, They must be wrong. We have fragile egos. But, actually, just because they have a different opinion that doesn’t mean they’re bad people. Disagreement doesn’t mean a flaw in character. … We can do some things to help us assume good intent like:

  • Be an active listener.
  • Be open to new ideas.
  • Try not to be defensive.
  • Don’t be offended by a contrary point of view.
  • You can’t actually get the right tone in an email, so maybe you need to pick up the phone.
  • If a large meeting is going nowhere, meet one on one.

“The more collaborative you are, the more you understand everybody’s roles, the less likely you are to get caught up in everybody’s individual silos.”

Figure 15—Tip 2: Respecting and valuing different skills
Respecting and valuing different skills

“Success depends not only on the combined skillset of the teams but also on their personalities and different ways of approaching and solving problems. So really you need some empathy. Put yourself in their shoes. … You don’t want to habitually dismiss [someone’s opinions just because they think differently….”

Figure 16—Tip 3: Bringing your whole self
Bringing your whole self

“Letting the people you work with know that you care about them is the best thing you can do. It makes everybody more satisfied, more comfortable, and it leads you to appreciate everyone more as individuals, which helps you to build friendships. … Friendships make the office a safe, happy place to be.”

Figure 17—Tip 4: Bringing people on your journey
Bringing people on your journey

“Focus on the future state and how much better it will be rather than putting down what is, because they might have had something to do with what is. … You may have to repeat your idea over and over…. Sometimes, at first, people are hesitant. The second time, they might start seeing. … The third time, they might think it’s their idea. Let them. … You might find somebody else has a better idea.”

Figure 18—Tip 5: Listening to their ideas
Listening to their ideas

“There are a couple of things you can do to listen better:

  1. Paraphrase back or summarize what they’re saying. …
  2. Probe for more information. Ask how they feel.
  3. Acknowledge and validate their issues and feelings. …
  4. Reflect. …
  5. Use pauses and silence to encourage more conversation.”
Figure 19—Tip 6: Knowing your audience and speaking their language
Knowing your audience and speaking their language

Dorelle described some “different types of communicators:

  • analyticals—These folks prefer data and facts. Detail and accuracy are very important. Many engineers are like this.
  • amiables—These folks focus on feelings and emotions and maintaining relationships. They want everyone to get along, and they don’t want the spotlight.
  • expressives—They get the big picture. Their style is outgoing, flamboyant, spontaneous.
  • drivers—They’re results oriented. They’re very direct and practical and kind of impatient.

“There are different ways to communicate with your audience, but there are also different audiences”—for example, designers, business stakeholders, or engineers. “Speak in a different kind of language for … different kinds of audiences, and it will help bring them along on the journey. And it will help them assume [I have] good intentions.”

Figure 20—Tip 7: Broadening your perspective of roles
Broadening your perspective of roles

“People outside of your Design organization are making design decisions, and those decisions affect your customers is many, many ways. They are also designing your product. … They’re part of your Design team. Now, you’ve got an augmented Design team. It really doesn’t matter where a good design decision comes from. Use their expertise to the advantage of your product and your company. … There’s so much overlap in our roles. We just need to accept it.”

Figure 21—Tip 8: Accepting feedback as a gift
Accepting feedback as a gift

“As designers, we get all kinds of feedback. We get feedback from our users, from our customers, from our teammates, from our stakeholders. Basically, everybody feels that they can comment on a design. … Design feedback isn’t about liking. … That’s not helpful feedback. Helpful feedback needs to be: Is this solving your problem or not? Don’t be defensive.”

Figure 22—Tip 9: Debating
Debating

“If you don’t debate, things can be left unsaid. Disagreements can fester. A vocal few can dominate. Lack of a response can actually indicate approval or agreement—even if you didn’t mean to. Give credit and acknowledge [others’ ideas]. Look for common ground, but note that differences of opinion are the ways you’re going to push the boundaries….”

“You need to adjust for each kind of relationship. For your colleagues, your team, …bring your whole self to the office. Listen to their ideas. … Figure out how to accept feedback from [your bosses] and respect and value their different skills.”

In concluding her talk, Dorelle presented a slide that outlined the benefits of genuine relationships, shown in Figure 23. “It leads to so much goodness,” said Dorelle.

Figure 23—Benefits of genuine relationships
Benefits of genuine relationships

Dorelle’s excellent talk was my favorite of the entire conference because it spoke to the values of teamwork—one of my passions. She delivered her subject matter with great authenticity.

How to Identify and Increase Your Experience Quotient

Presenter: Patanjali Chary

According to Patanjali Chary, who is VP of User Experience at the enterprise FinTech software company Ellie Mae and is shown in Figure 24, “Securing sustainable, significant, long-term investment in User Experience actually requires a different strategy. Part of this strategy is understanding that…great user experiences can be quantified to demonstrate that they increase willingness to pay…for a product or service. To do this, what’s really important is that we learn to speak in the language of the business.”

Figure 24—Patanjali Chary
Patanjali Chary
Some Troubling Questions

Patanjali told us that, as a UX professional and leader, he has struggled to find answers to some troubling questions, as follows:

  • Why do designs often end up sitting on a shelf?—We design so we can actually change the lives of our users. But if [our work] never gets in the hands of our users, what’s the point? Why are we doing this? Even when designers create the right designs in the right way, …is designing the right solutions enough to execute? What is the missing piece here? We all recognize that design quality is very important, but design execution is actually the key thing.
  • Why is UX ROI (Return on Investment) still primarily about reducing failure?—Do design right so your projects don’t fail. Do design right, otherwise, it’s going to cost you lots of money and you’ll have [high] support costs.
  • Whatever happened to UX delight?”—Patanjali shared Aaron Walters’s “Hierarchy of User Needs,” shown in Figure 25, saying, “When you craft a design that’s functional, reliable, and usable, ultimately what users expect is something that’s pleasurable.” He told us that, in talking about UX delight, Kaaren Hansen has pointed out, “Delight accounts for about 75% of the variance in Net Promoter Score (NPS).” “So is that a measure that we could use to…demonstrate…why delight matters?” asked Patanjali. He cited Jared Spool’s recently calling NPS harmful.
  • How can we ultimately quantify that delightful experiences can mean both higher product and higher business value?—If these measures alone are not going to do it, how do we do this?
  • Will people pay more for a better experience?—We know this! If you give users two designs that accomplish exactly the same task, but one of them is simpler and more delightful, you know—even without doing a usability test—that…users will prefer the simpler and delightful one. We…also know that they’ll probably pay a premium for it. …
  • What does it really mean to be in the pilot’s seat?—UX [has] started to…really be a strong presence in an organization. We’re not just something people know that they need, we’re actually starting to lead—to get into the driver’s seat. … Can we keep doing the same things over and over again and expect a better result? The reality is: when you’re in the pilot’s seat, you have a responsibility for your entire aircraft—your passengers. … Ultimately, you need to get the passengers safely from point A to point B. It’s a different perspective being a passenger versus being a pilot.
  • How do we find the synergy between high business value and great user experiences?
Figure 25—Hierarchy of User Needs
Hierarchy of User Needs
Finding Answers in Business School

“My questions remained [unanswered] for years—until…I decided to go to business school,” acknowledged Patanjali. “It completely changed my world because…I got exposed to so many different analysis tools and techniques. … I started to have these epiphanies. … I thought to myself, This is really familiar. I know what this is. This is user experience.” As Figure 26 shows, “[Business] school primarily teaches you to identify who your customers are and understand them really well. It introduces a bunch of techniques—[both] qualitative and quantitative—to measure and to track the operations of a business [and] ways to increase and sustain financial health. … You also learn that, before you make a large investment, make sure you test and pilot it, …reducing the risk.”

Figure 26—What business school teaches…
What business school teaches

“What about UX?” asked Patanjali. As Figure 27 shows, “UX primarily provides ways to identify your users, collect qualitative and quantitative feedback on the user experience, and…increase and sustain the usability and delightfulness of our products. Testing and iterating? … That’s a user-centered design process. That’s what we all do. … I thought, …how can I bring these worlds together? … Could this actually start to answer some of these questions I’ve struggled with?”

Figure 27—What UX primarily provides…
What UX primarily provides
Some Key Learnings

Patanjali shared some of his key learnings from his business-school experience, as follows:

  • Strategy always before tactics. [This is] really critically important.
  • Marketing is actually highly quantitative. I…used to think that marketing was mostly qualitative—opinions and things like that. Turns out it’s really highly quantitative. In fact, a number of my marketing classes were just all math.
  • Customer loyalty is measured in many ways—not just NPS. There [are] actually dozens of…quantitative measures for customer loyalty.” These measures include “Customer Lifetime Value (CLV), …Customer Acquisition Costs (CAC), Customer Retention Costs (CRC), Customer Value, and so on.
  • Investors invest in the dream, not the fix. This is really important. … As UX professionals, what have we always done? We’ve gone in to fix the experience. We continually try to convince management to invest in user experience because this is broken—this needs to be better. … I think we’re missing out on something. The investors actually couldn’t care less about the fix. They want to know: What’s the dream? If I’m going to invest in you, what is going to be my return? What is it? Where is this going to go to and what does that look like?
Searching for the Right Tools

In trying to ascertain “how these things could be put together in some way,” Patanjali began asking these new questions: “Don’t delighted users become more loyal, too? Can we prove it? If we can prove it, is this the real Holy Grail of User Experience?

“So I went in search of the right tools, and I found one: the Service Profit Chain Model, …[which] quantified [the] business concept of increasing a customer’s willingness to pay a premium for a product or service.” The authors of the Service Profit Chain Model, James L. Heskett, W. Earl Sasser, Jr., and Leonard A. Schlesinger, calculated the Customer Value Equation. “They proved that, if you focus on increasing the quality of experience alone, what happens is that the price of that product or service becomes highly elastic. It becomes so elastic that, over time, customers actually start to ignore the price altogether. … What they actually found is: just a 5% increase in the quality of the experience resulted in a 25 to 85% increase in profitability,” as shown in Figure 28. “Better experiences lead to higher profits”—for example, 35% higher profits for software. “So focus on the experience. Focus on investing in the quality of the customer experience. … For us, [this means] creating] better user experiences and making them both usable and delightful.”

Figure 28—Customer Value Equation
Customer Value Equation

Patanjali proposed the User Experience Quotient (UXQ) shown in Figure 29, saying, “Having a high UXQ means that customers become less price sensitive and have an increased willingness to pay for your better user experience. High UXQ is what we’re after, and a high UXQ is what we need to get the business to understand…what we’re after—and what they should be after, too. Now we understand this, we have a way to…quantitatively illustrate why an investment in User Experience can lead to profitability.”

Figure 29—User Experience Quotient
User Experience Quotient
Getting to Design Execution

“How do we convince an organization to invest significant amounts into User Experience, into design quality, so…ultimately, customers buy more and more of the products?” asked Patanjali. “We execute. … It begins with aligning a UX strategy directly with the business strategy, then pitching that as an investment pitch back to the executives.”

Patanjali presented a brief case study on how he did this at EllieMae. First, he conducted a high-level strategic analysis—by doing interviews across all levels of the company; analyzing his detailed, encoded notes; and categorizing the feedback. Next, he applied a strategic model, the Dynamic Capabilities Framework, or Teece Model, which is “a strategy model for constantly shifting business environments. … This model…requires analyzing your firm’s resources and capabilities”—in this case, the capabilities of their UX team. Patanjali modified this model for User Experience, adding the dimension of user-experience quality.

He did a strategy assessment across the twelve dimensions shown in Figure 30, subjectively rating their perceived importance and perceived relative strength on a scale of 1 to 10. The first four dimensions are resource categories; the next four, capabilities; and the last four, quality. Patanjali then plotted the data on a Teece graph with four quadrants, shown in Figure 31. Figures 32 and 33 show the actual analysis and Teece graph, respectively, for the EllieMae UX assessment. The red circles show the team’s weaknesses—“the things we may not be doing as well at and we need to invest in.” The yellow circles that, in the graph, are on the midline indicate “the things that we’re actually doing a pretty good job of, but [aren’t] really…a strength yet.” The green circles indicate strengths.

Figure 30—Strategy assessment
Strategy assessment
Figure 31—Teece graph
Teece graph
Figure 32—Initial UX assessment
Initial UX assessment
Figure 33—UX assessment plotted on a graph
UX assessment plotted on a graph

The key observations from this initial UX assessment included the following:

  • The UX team had “many key weaknesses across resources, capabilities, and quality.
  • “Brand and product experiences [needed] better alignment.
  • The team had a “strong work ethic, but [lacked] core strategic design thinking.”

Patanjali used the same dimensions-of-analysis table to map the team’s long-term goals—again, rating their importance and strengths. He told us, “You’ve got to do that thoughtful analysis to understand what’s important to [your] company and users at the current time. Of course, we ultimately want everything to be a strength.” By plotting the current state and the goals together, as shown in Figure 34, you end up with vectors that show what you need to invest in to turn a weakness into a strength.

Figure 34—Vectors of improvement
Vectors of improvement

“I took all this analysis and created an investment pitch [that] I presented…up the chain all the way up to the board,” said Patanjali. “We secured a five-year commitment [for] a very significant investment in User Experience [and] over a 500% increase in the UX budget. … [At] the end of the presentation, my last section was an investment request [with] very large numbers.” Leadership’s response was: “If that’s what we need to do to get the quality of experience to get to that level of o growth and willingness to pay, yeah sure, that makes total sense.”

Patanjali wrapped up his outstanding talk by saying, “In summary, …we have to learn to understand the business—speak in the language of the business. We have to…align our UX strategy—using some of these tools—with the business strategy, …communicating in the language of the business. … You have to pitch the dream. … We often get ourselves into the weeds—we almost become our own enemies—because we start to talk about all the fixes. We lose people. We lose the business. We need to focus on the dream. You need to actually pitch the dream. That’s what the investors want to hear and that’s what they care about. … We have to deliver delight.”

Patanjali presented a comprehensive and very useful approach for convincing an organization’s leaders to make significant, long-term investment in User Experience.

INVEST Discussion

The morning’s speakers participated in this discussion, which was moderated Amy Marquez, who took questions from the audience. Here are highlights from this discussion.

Question: The first question was for Patanjali Chary: “Thinking about enterprise user experience, where the purchasers of our products are not necessarily the same people as the users of our products, what particular challenges or nuances do we need to address with your model to make sure that we get that same delight and willingness to pay more?”

Patanjali Chary: “You have to understand a couple of things about the business you’re in. How does the business actually make money? Enterprise software companies have a couple of traditional models. Today, if you’re a cloud-based company you have a SaaS (Software as a Service) model, usually called PEPM (Per Employee Per Month)…. We use a metric called RPL (Revenue Per Loan) because the way we make money is when lenders use our platform to originate and close loans. That’s when we get paid. As we add more services and capabilities [such as] compliance stuff and they go through us to do it, then our RPL just increases. So first thing is to understand how your company makes money.

“Then, the next thing to understand is who makes that decision? As you pointed out, unlike in consumer spaces, the buyers of these experiences are often very different [from] the users themselves. But there’s something very interesting that I’ve observed [recently], which is a pretty huge convergence…. I believe the reason for that convergence is because everybody’s got a mobile device now and everybody expects a certain kind of experience. … I think the expectations have changed so surprisingly. When I presented this to the business in the way that I did, they [realized] our lenders want the best experience for our borrowers. We need to give them the tools to do that [they] need to do. So I didn’t have to do as much translation…. That’s good news for all of us.”

Question: “All of your presentations touched on the idea of quality and driving quality about design. As you scale your organizations, how do you continue to conduct quality design reviews?”

Dorelle Rabinowitz: “It’s really the one of the biggest issues that we have to scale design reviews—especially because, when I was talking about three in a box that means the three-in-a-box functions need to make decisions about the products we’re building and design is part of that. So, not only do we have design reviews in individual teams, then for the design team overall, we also have design reviews for our PayPal UI team, which is our design system team. Then, we also have product reviews at the product-team level. Then, at the three-in-a-box level, …we have what we call power hours, where folks bring in their products and we talk about design in the context of the whole product rather than just a design review. So teams are really caught up in preparing their work—even though, say at my critique, it’s all about just show me your stuff. … Mostly at the team level is where folks do a lot of things like studios, where they’re sharing their work and really getting feedback, and they’re all sketching together.

“So it depends on the level of feedback that you’re looking for in the quality. … If I have 75 designers and each of those designers [is] working on five projects, I can’t see everything. But I never, ever want to walk into a review with my stakeholders at the SVP level and see something for the first time. That’s happened.”

Amy Marquez: “Amazon’s a little scaly. What we do at Amazon is: every division of Amazon is almost like its own business unit. It’s almost like its own business—its own little corporation. So there’s something called the CXBR. which is the Customer Experience Bar Raiser. Anytime you’re going to market with something new, anytime you’re significantly changing, you go through the CXBR. So you’ve gone through your internal reviews with your teams, then you go up to a broader, more horizontal review with senior executives in Product and in Design and in Legal and in PR, and it can be a little intimidating. But those people all want you to succeed, so they’re all going to give you genuine feedback on what they feel is missing or what they feel is working.”

JJ Kercher: “I think quality, because you’re working with all different functions, is going to be sort of subjective. So how do you align on what that looks like? [For] design reviews…, we involve everybody in those conversations. One thing that’s really worked well for us to improve quality is showing instead of telling. So we actually have regular meetings where someone from our Customer Success department will come in and they’ll demo something that could be improved. That really sparks people’s interest because they see it, they experience it. We often have engineers go home…and fix it over the weekend, so that’s been really beneficial as well.”

Dorelle Rabinowitz: “One thing that’s helping us with the quality—and also with the validation or getting the best design done even if there’s a tight deadline—is aligning on what we’re calling product principles, not design principles. We created them in a three-in-a-box way and now I can use that as, ‘Well, we all agreed that we need to be findable’ or ‘we need to be able to have the same experience across channels’ or whatever the principles are—and they’re pretty minimal—that helps us say, ‘Oh, this is meeting that or not meeting that.”

Patanjali Chary: “One of the ways that we’re trying to address this and…we’re still kind of getting up to speed…, but the way we’ve organized the team…is sort of like all these high-intensity design teams. So designers are almost always paired with another designer, and…they review things off…each other. The usability engineer is actually part of that exercise as well. So what we try to do is reduce the risk, before we go in front of a stakeholder, that we haven’t thought this through. … I love what [Dorelle] just said. … I have a saying, ‘Show them, don’t tell them. So, if you go into a stakeholder, you show them something and, even better, you whiteboard with them—you bring them in. What happens is, when these people start to have skin in the game, all of a sudden they feel like their thoughts are being taken care of. Then you can start to control the quality.”

Question: “Knowing, as a company scales, it can often go into different groups and business units, how would you help one side of the company uplift in UX when one side is doing better than the other? So, if you have a B2C side and a B2B side, and [on] the B2C side, [UX] is really well respected, and on the B2B side, [UX] is often [not] seen as…important, how could the B2C side aid the B2B side?”

JJ Kercher: “One thing that I think is really important in any organization is: make a point to identify your strengths and build from them. So, in that example, where you’ve got a B2C side and they seem to be…a little bit more mature, pinpointing what it is that’s working and then going in and trying to experiment with whether or not that will work in the other organization…. Depending on the situation, …is the B2B side open to hearing from the B2C [side] about how to be better or not, because…there [are] other relationships…to build. … It’s really pinpointing what those strengths [are]. Even if it doesn’t seem like the most important thing that you could fix right away, just start implementing something that’s already working because you’re going to get a lot more success that way.”

Amy Marquez: “[Create] a kind of a cross-company support team for the design teams, …so you’re all communicating together. If one side of the house is really solid with a product team, really solid with their engineering team, and the other side isn’t, teach them what you know, talk to them about it, meet with them, help them understand that whiteboarding with your product team, whiteboarding with your architects, whiteboarding with them, listening to their ideas, showing them your ideas in real time, builds a lot of trust—it builds a lot of affinity between you. And always bring food.”

Dorelle Rabinowitz: “I’m not going to disparage the other side of PayPal—the merchants’ experience. These are the experiences for people who process payments using PayPal, and that experience is very different [from] the consumer experience. What we’ve been doing, because we have customers who are on both sides, we’ve been inviting the merchant folks to our design jams to our sprints to all of our stuff to model the behavior that we hope them to have. Then we’ve also started thinking about, if you use our component libraries, it’ll reduce your complexity and your time to market, so maybe it makes more sense to use these more well-thought-out pieces. That’s gone pretty far.”

Patanjali Chary: “I’m going to take a slightly different tack to answer your question. I’m going to start by asking a couple of questions. … Is the investment in the B2C side by the business greater than that on the B2B side? It is. So the first thing that I would say…is each of those sides contributes revenue to the bottom line in some way. You need to first understand how…the company gets paid from a B2C model. How does the company get paid from the B2B model. Once you understand that, if you can start to then connect the dots on how…improving the user experience on the B2B side will increase the willingness to pay of that customer like it has on the consumer side, you will have their attention. Once you get their attention, you start to formulate a strategy and go through the steps…to do that. Then you’ll get the investment and then you’ll get the attention. Investment is attention. If the business says, ‘We’re going to fund you, we believe in this, you’ve got their attention. …

“That is something we often just miss because we don’t think about that as designers. … It’s not natural. It wasn’t natural to me…until…I kind of realized this is what I have to do. So that’s how I would go about trying to address what you’re talking about—in addition to what they both said. … When it comes to the execution, you really have to do everything. But to get the mandate to execute requires going above that and understanding those things. When you go back to them, …they’ll be impressed when a designer knows the business.”

Amy Marquez: “One more quick tactic: find your senior-executive ally. If there’s somebody at the senior-executive level who understands design, who supports design, who knows what it means, who knows the significance of it, go skip all the levels…to talk to that person. Then, tell your boss afterward, ‘Hey, I did this thing so it doesn’t sideswipe him or her. That has worked for me in the past and so far I’ve not been fired.”

Question: “Have you encountered any ethical problems in the course of your work, and how have you begun to discuss them with your team members or with your management reports.”

Dorelle Rabinowitz: “I don’t really know if this is an ethical problem, but we are doing some work for the financially underserved. It’s one of our biggest missions, in fact, for PayPal. There’s this ethical question of: do we want to make money over this financially underbanked population? We talk about that a lot. Can we offer this to them for free and then have our merchants pay more? Those are the discussions we have in terms of the offering that we have. In terms of behaviors and how people operate, PayPal has values that we really try to tie everyone back to. That helps me when I see more of the us-and-them mindset and people arguing over small things. Always remember that we’re one team, we’re doing the right thing for our customers, and be the…right kind of person.”

Patanjali Chary: “I’m going to give a slightly odd kind of twist on this, which is: who would ever think making your Chief Legal Officer your best friend as a designer makes any sense. … From a business perspective, …the person in the company who’s most concerned about ethical stuff…actually can be the lawyer in the house, [who] basically is worried about getting sued by doing something that’s unethical or whatever. In our case, since we are a platform for mortgage lenders, …we’re regulated by FINRA and CFPB…, but we have to follow certain very strict compliance requirements.

“One of those is the Fair Lending Act. What’s interesting is: we’re about to release a product…to customers internally soon, \ and it’s a whole brand-new interactive analytics experience. We’ve taken about 20 years of historical mortgage data, and we’re allowing lenders and eventually the public to be able to go and actually see the trends of mortgages and different things. From a user-experience perspective and from a product perspective, we want to provide the ability for people to do this very easily and to get the information they want. It turns out some of the information that they want could potentially break the Fair Lending Act, which says, if you allow people to figure out certain information that could allow them to discriminate who they lend to, that’s a problem. Of course, the Product people just wanted to have little popups that say non-compliant…, but that’s not going to work. So we actually turned it into a design challenge, then in talking to our chief legal person, …who’s very concerned about this, he all of a sudden has been brought into the design process, and he’s very pleased. He’s actually our biggest advocate now. …

“You can actually do good if you just understand the mechanics of how. I think in every business it’s different. That’s just in our case.… I’m sure at PayPal, you’re dealing with money and merchants and privacy. We have that, too.”

JJ Kercher: “There [are] always ethical considerations, [but] I don’t feel like it’s been a big deal at AppFolio. … One of our core values is: We do the right thing that’s right for business, and we…live it. … We have some ethical things that come up with different demographics of different users, or we have products for tenants—low income and high income—but we just pretty much embrace that. I think the most challenging one has been in some of the push and drive for improving the accessibility of our products, which is just another thing that’s just not going to happen overnight. But we are able to have really meaningful conversations with people across the organization about…the roadmap for how we might pay that down. I guess …I feel like we’re not getting much pushback for doing what’s right….”

Amy Marquez: “One of the things we we do at Amazon is every team has tenets, and a tenet is different in that it’s a belief that you stick to even when it’s uncomfortable. One of the core tenets of the Alexa Personification team is that we don’t advertise. Other areas…of Amazon will come to us and say, ‘Hey, can you have her personality say this?’ ‘No we can’t, and here’s our legal counsel telling you why we can’t. You always have to assume, in a household, there may be a child in the room, and that’s wrong. You don’t advertise to kids. … So keeping that in mind, we use the personification of this AI (Artificial Intelligence) for good, for fun, for delight, for deepening a relationship, and not for advertising. … I basically have to put on Wonder Woman bracelets every day and just kind of bounce back the requests for advertising.”

Question: “It already can be an uphill battle to kind of sell the value when you’re pitching a project of delight and experience—when it’s going to be a consumer-facing product and that’s really important. … What advice do you guys have on pitching the importance of delight and experience when the users are not going to be people choosing to spend money—they’re people working internally in a company and being forced to use the product. Therefore, do we really need the delight—the experience side of it? That pushback can be difficult to deal with.”

JJ Kercher: “I have a story I can tell. Very similar. In enterprise, you’ve got buyers; you’ve got users. It’s really easy to tell [UX] to do what the buyers are asking for. … Find out…where they are sticking points and see if you can learn more about the two personas to bring back to the table. In our example, we were really super interested in…feature adoption, and there [are] certain features that you can use in our software that actually cost more money, and then…that brings in additional monthly recurring revenue for us.

“We were really kind of scratching our heads at some points about adoption and it’s relationship with churn. Why are some people adopting? Why aren’t some people adopting? We actually did a research study, and we took buckets of companies that were high adopters [and] low adopters. One of the insights that we learned, which was really valuable, is that the culture of the company—the relationship between the buyer and the user—actually had an impact on whether these companies were high adopters or not. So we could then go back to these buyers, who a lot of times when they’re not adopting are less happy with the software, and [try] to get them to adopt more features. But then we would find that they weren’t empowering their people.

“Now we’re putting programs in place to create these champions. Champions of our product tend to be end-users, not buyers, so our Marketing department has been working on…identifying and growing internal champions within these organizations that can speak up and say, ‘Hey, we should be using this. We should be using that.’ It’s kind of new. I don’t have any results from that, but I think bringing data back about why both matter and the connection to that and then to things that the business is thinking about in terms of adoption and churn and recurring revenue, that’s one success story….”

Afternoon of Day 2: SCALE

SCALE, the biggest differentiator of enterprise UX, was the theme for Friday afternoon’s sessions, which explored various challenges of remaining customer centric and achieving scalability within very complex environments.

Standardizing Product Metrics for Leaders, Designers, and Everyone

Presenter: Jennifer Cardello

Jennifer Cardello, shown in Figure 35, is Head of Design Ops and Executive Director of Design Research at athenahealth. She spoke about approaching “scale from a meaningful impact perspective. So scale isn’t just about quantity. It’s not just about embedding yourself throughout the organization. It’s about the quality that you are delivering and the impact you have on the outcomes of the organization.”

Figure 35—Jennifer Cardello
Jennifer Cardello

“When I think about where I’m going to work or what client I’m going to work with, I…think about whether they are design forward to make sure it’s going to be a good experience for both of us,” said Jennifer. “So I created a litmus test [for evaluating] these organizations,” as follows:

  1. They’re seeking human-centered innovation. What I mean by that is: … they really mean it, and they see humans as the center point of…how to innovate.
  2. [They recognize] Design as a strategic lever. So it’s not just a support function. It’s not: go make it pretty, go add some design. It’s saying we’ve got this really messed-up…system…and experience design is going to save us from that. That’s really meaningful. That’s an impactful statement.
  3. They’re ready, willing, and able to invest in Design.” They’ve invested in hiring designers and researchers who are “partnered with Product and Engineering. They [are] embedded in the organization, working on projects.”

Jennifer thought: athenahealth “is a mission-driven company that meets my criteria. So I was ready to work there and be radically simple, refreshingly candid, powerfully relevant, and pushing boundaries. That’s our new branding. But…what I found when I got there:

  • Complicated experiences [were] driven by incrementalism. …
  • User frustration [was] super high. …
  • Designers [were] super frustrated. …
  • Product and Engineering [were] not bought in.”

“This company passed my litmus test!” Jennifer exclaimed. “What’s going on here? Why isn’t design having an impact at scale. It’s a large department. It’s got its tentacles throughout the organization. [The] CEO is out talking about innovation, human centeredness, experience design. So what what gives?”

The Enterprise Software Delusion

Jennifer told us that the enterprise software delusion occurs “when an organization isn’t honest with itself about success drivers and underinvests in experience design. It [was] not honest about how it got where it got. … [A] government-subsidized market demand [was] making it possible for [athenahealth] to actually ship pretty poor software, but still do well. Then [they underinvested] in Design because [they] don’t see it as valuable to get those results. [They] don’t see it as a lever to get those results.”

Figure 36 shows the mechanics of the enterprise software delusion—weak versus strong competition, high versus low switching costs, and whether the purchaser is an agent of a company, or a customer, versus an actual user.

Figure 36—The mechanics of the enterprise software delusion
The mechanics of the enterprise software delusion

“When competition is strong, …switching costs are low, and your purchaser is your user, design is your differentiator—the experience is your differentiator—so you want to invest in Design,” said Jennifer. “Many [B2C (Business to Consumer)] organizations…spend a lot of time and energy making sure that Design is at the forefront and is driving their innovation.”

For a company that is suffering from the enterprise software delusion, “the truth is: your users aren’t happy because they’re trapped,” acknowledged Jennifer. “They have no sense of agency about whether they use [the software], and no one in the organization is thinking that [market] conditions are going to change, so there’s no urgency for them to do anything about that and invest in Design…. But we all know that the one thing we can count on is that things are going to change,” as Figure 37 shows.

Figure 37—Change
Change
Increasing Design Influence Across the Organization

At athenahealth, Design devised a strategy to address these issues and “increase design influence across the entire organization,” which Jennifer detailed, as follows:

  1. Capture all the attention of the organization.” Not just Product and [Engineering], but everyone—Marketing, Sales, anyone who will listen. … Get their attention
  2. Establish a sense of urgency. Make [everyone] understand that this is an urgent situation. The market is changing. We can rescue you from that. Let us help you!
  3. Then, deliver the goods. Start making things that make a difference and show the proof.”

Jennifer recommended using three tactics:

  1. Prove that Design impacts outcomes.” Build an ROI model. “We were inspired by Len Schlesinger’s Service Profit Chain. He’s one of our advisors.” To prove that Design impacts outcomes, you must know “what metrics are important. Most organizations have a corporate scorecard, …[but] over time, …they [can] become unwieldy and unfocused…. Figure out the metrics that matter right now. They can make a difference. We found two: …retention, or…attrition, and…referrals. … We spent four sprints on a regression and mediation analysis. We found the data that we needed in the organization. We spent [three] sprints…cleaning the data, so pre-processing was a big deal. Then we ran the analysis. Happily what we discovered is that users’ perception does predict attrition and paid referrals,” as Figure 38 shows.
  2. Create a shared language for design. If we’re the only people speaking that language and no one else has the nomenclature, we can’t talk to each other—we can’t communicate. … How might we help our coworkers or colleagues identify, articulate, and quantify poor experiences?” Their goal was to reduce users’ workload by eliminating tasks, removing unnecessary functionality, and redesigning “the remaining functionality to improve efficiency, effectiveness, and satisfaction.” So they did a workflow analysis of “the tasks that our users are conducting in athenaNet. [They] worked with Product Management to identify 55 critical workflows, …looking for things that have high usage, high friction, or could be differentiating [capabilities] that we offer, [but] our competitors don’t. Heuristics provide a shared language. We used Jakob Nielsen’s heuristics largely [to lend] credibility to the effort.” Step 1 of the Workflow Improvement Initiative comprised scoring sessions, as shown in Figure 39. “We didn’t do this alone. … [During three-hour] scorecarding sessions…, [which included a facilitator from design research, designers, product managers, and subject-matter experts], we would watch a video together, …[go] through the heuristics, [noting] everything that we see, then [discuss and calibrate our scores].” Step 2, which is shown in Figure 40, consisted of “in-depth audits…, going through the whole workflow and…highlighting all the things that could be changed in the system so it would work better. We did this…on a two-week sprint cycle. … We didn’t want to have a big-bang release of these results. We wanted to be putting them into the organization to inform backlogs as we were learning things. So we would record for the first two days…, do those…collaborative scoring sessions. We’d spend five days doing the deep-dive audit. We’d do a readout every other Thursday and…a [retrospective] every other Friday [to] see how we could improve the process. We got through this in nine…sprints, so 18 weeks,” as shown in Figure 41, then followed up with a meta-analysis. “We broke out [our] findings by user role so we could see which users were suffering the most…; by the heuristics so we could know [whether] there were patterns…; and [by] distribution…to see where the low-hanging fruit [were] and the most severity and urgency…. Then we wrote very detailed reports for all of our product groups so…they understood [where to focus and what user stories they should add to their backlog].” They employed John Kotter’s “model for change management and behavior change in an organization,” which is shown in Figure 42, throughout this process and use it for all of their DesignOps initiatives.
  3. Redesign product decision-making. … The big thing here is that we want the [product] teams to get better at [determining] when to pivot, shelve, and proceed. We want them to have the autonomy and the power to make those decisions. They should be making those decisions and making those recommendations to leadership instead of the inverse. So, in DesignOps, we’ve created a variety of systems” and deployed them across the product-development lifecycle—through Discovery, Experiment, Alpha / Beta, and Generally Available—as shown in Figure 43. These tools include [an] Experience Definition Framework; an Experience Measurement Framework, shown in Figures 44 and 45, which provides “rich quantitative and qualitative information from existing and prospective customers” that helps teams to “identify the right things to build…and prioritize improvement and growth opportunities, …standardized methods to rapidly [test and] iterate concept prototypes [through] qualitative feedback and quantitative validation, expedite best-solution identification, [measure internal alignment], [and test and monitor live features], and standardized methods to measure design and usability quality, user perception, and usage to ensure we build…things right;” a Research Council—which is their user panel—and the Forge Design System. “All these things are about building the right things and building the things right. This is empowering our Scrum teams to work as efficiently and productively as they can.”
Figure 38—Results of the metrics analysis
Results of the metrics analysis
Figure 39—Workflow Improvement Initiative: Step 1: Score
Workflow Improvement Initiative: Step 1: Score
Figure 40—Workflow Improvement Initiative: Step 2: Examine
Workflow Improvement Initiative: Step 2: Examine
Figure 41—Scoring and Examination Sprints
Scoring and Examination Sprints
Figure 42—John Kotter’s change-management model
John Kotter’s change-management model
Figure 43—DesignOps systems across the development lifecycle
DesignOps systems across the development lifecycle
Figure 44—Experience Measurement Framework
Experience Measurement Framework
Figure 45—Tools and instruments
Tools and instruments

Jennifer has now improved upon her litmus test for evaluating organizations by adding two more points. The company is

  • honest with itself about success drivers
  • uses evidence to make decisions

Jennifer delivered a valuable talk about delivering quality, having impact on your organization’s outcomes, and measuring that impact.

We’ll Figure That Out in the Next Launch: Enterprise Tech’s Nobility Complex

Presenter: Nancy Douyan

Nancy Douyan, International User Experience Researcher at Uber, Haitian child prodigy, and world traveler, who is shown in Figure 46, discussed “how Uber uses informed, international insights to spread their products across the world.”

Figure 46—Nancy Douyan
Nancy Douyan

“In enterprise, we all want to design for scale—meaning that we all want our solutions to reach more customers,” said Nancy. She spoke about the need for “empathy—understanding perspectives from another’s point of view. Companies today are adopting empathy as a way to build their products to reach more customers, but empathy has become a bit of a buzzword. It’s a fad. You’ll hear empathy maps and empathy technologies and empathy product design and empathy economy and empathy engine—just to name a few. All in the name of designing solutions that have global, scalable impact…. Is empathy enough to build for scale? That’s the question we’re going to try to answer today.

“At Uber, here’s an example, we’ve been trying to incorporate empathy more in our design, in our work. In many ways, human-centered design has established empathy as a baseline, and it gives…credence to having enough humility that we may not have all the answers, but we can try. Being empathetic can also sometimes be a shortcoming, I would say, as you think more and more about scale.”

The Nobility Complex

“What is the cost of our quick fixes to our relationships with our customers?” asked Nancy. “Can we be agile / Lean—maybe even at the beginning of a startup—and build for scale with just an empathy add-on? Fostering empathy should be a baseline for any job that you have when you’re thinking about designing for reach. But here’s the gap that I noticed”—for which Nancy has coined the term nobility complex—“when people in positions of privilege [in Westernized societies] unintentionally create solutions without accounting for their own explicit and implicit personal biases.

“Your nobility complex can lay the groundwork for misunderstanding and lead to colossal disasters and unseen dimensions such as

  • limiting your revenue potential—Due to product confusion, users don’t convert if they don’t understand what [a] product does. If you don’t consider local appeal, [you may lose to your] competition because [you] don’t know how to strongly favor…these cultural practices.
  • spending inefficiently—People, in some…countries, are three times more likely to buy a product if it’s localized in their language and interpreted correctly in their language. At Uber, for example, we translated a ride at your fingertips, and that ultimately translated to walking around on your tiptoes, which isn’t very good for business, is it?
  • [limiting your] future growth—We’re not creating more opportunities for people to use our products because we don’t have enough context. What that means is that we’re seeing five…to twenty times losses…because we didn’t take into account some of this context before we launched the product.

“We’re not scaling, essentially. We’re here in San Francisco—where we think we know what we’re doing because we’re so privileged and so knowledgeable and so empathetic. But let’s consciously address the unconscious bias in our tech today.”

For example, “Tay was a chatbot that Microsoft created that basically shows the hidden dangers of AI (Artificial Intelligence). It was well-intentioned. The experiment was to have this artificial intelligence and machine learning that would learn from Millennials from Twitter and other messaging apps. …
Within less than eight hours, Tay became a racist, sexist monster. What’s even more interesting about this is, when you speak to this team, they tell you, ‘But we did test this. We tested this for a couple years before we even launched it.’ … If you put garbage in, garbage comes out. So if people are racists and making racist comments and making sexist comments, …Tay [is going to] spit out the same garbage. …

“We’re privileged, …Westernized people designing for ourselves without realizing it. We’re feeding back into this loop of a luxury. I think about these problems at Uber every day. Our products should also in work in places like [a marketplace in India]. … How do we design pickups for this? How do you design…for the fact that…the estimated time of arrival [might be delayed] if a cow crosses the road…. Who’s to blame [for] all these things—these limitations in our own perspectives. This is where we start to use our own solutions, but our solutions sometimes turn into…something that’s hurtful.

“We need to stop designing for a Western market. This is the next billion users type economy right now. We’ve got to make our products more scalable, and the only way to do that is to…get these voices that are not just our own into our product process.

Creating Global Scalable Research

“I started at Uber a year ago,” said Nancy. “I wanted to work with people from all over the world, and…they tasked me with scaling their research platforms around the globe. At the time, they had some researchers. They had in-product researchers, in-region researchers who conducted research under something called product growth. We conducted that research based on Uber’s headquarters priorities, and we did explore some regional opportunities. However, there were a lot of gaps. We had issues with efficiency, visibility. There was inconsistent collaboration with stakeholders—inconsistent communication with stakeholders and with the regions—all due to having limited bodies, limited resources.

“So we developed the global research team, and we came together to understand what worked and what didn’t work and decided to make some changes so we can improve impact around the world. I decided to create a subchapter in that group called Global Scalable Research Platforms. In this program, I want to explore global design opportunities for more than 77 countries that Uber operates in around the world—in Uber Eats, Uber Ridesharing, Uber Elevate, Uber Freight. … It’s intended to influence product teams at headquarters and around the world to design and test in global regions.

“When I built this program up, I was very fortunate to have a manager [who] helped me to execute and refine this program. … It was a monumental group effort to bring us all together to help the folks in [headquarters to] understand its importance and try to involve our customers to build something truly special, truly global. Not just US focused, but global focused. Something that would power our drivers, our riders, people who are interested in Uber Eats, our customers everywhere, and so many other experiences that we’re planning to bring forth.” Figure 47 shows some of the things that Global Scalable Research does.”

Figure 47—Global Scalable Research
Global Scalable Research

“What we do, as a particular approach in this program, is we manage a bi-weekly research cadence program,” said Nancy. “Like other UX programs, you have stakeholder asks, you design and conduct studies, and you synthesize that feedback. But the difference here is that we run parallel, agile, fast, regional research, with at least two plus country checks per request or feature. We provide cross-regional analysis and repeat engagements as a design improves.

“Not only that, [we’ve] made it systemic,” as shown in Figure 48. “[During] our product requirements process, everyone now has a chance to…test their designs earlier through a global checkpoint. … That…means we’re not just taking features from the US, then launching them…. We’re saying, ‘We’re going to test this in Brazil, in India…, or in Mexico and Japan. Bi-weekly, constant, for the last six months, we have just been conducting vigorous research.”

Figure 48—Doing global, scalable research
Doing global, scalable research

“This is our process,” said Nancy. “We have folks submit an ask. We localize the prototypes. We have our teams who actually understand and interpret actual local experiences, so we can catch problems earlier on. Then we contribute to the research plan. We have an interview guide. We conduct the research. Then we have preliminary findings about 24 hours after the study, and then the following week, we do a final report out where we don’t just invite the stakeholders who requested the ask, but we involve people across organizations, across the company, [who] might be impacted by this process and results.

“What we found out in just six months is that this was a game changer to accessibility around the world,” as shown in Figure 49. “This program was designed to check and acknowledge our own power dynamics as designed frequently from the [headquarters] in San Francisco to the rest of the world or to the rest of the United States.”

Figure 49—A game changer for accessibility

“We very quickly saw that, when we launched this program, there was a huge difference in our ROI because of the improved designs and innovations,” said Nancy. “We had a wider radius of influence. Not only that, we were picking up innovations from other places and testing them in the United States to give us opportunities to see things differently as well. This directly informed the trajectory of our projects. We had global and regional guidelines created so people could…start focusing on other problems…. We were able to scale this out to a multi-thousand organization. In six months, we’ve completed over thirty projects. We’ve done over ten research methods. We’ve done over 15 Uber-wide global collaborations. We’ve had over a 150 actionable insights and millions in projected savings—just from testing a product in another country.

Key Learnings

“What we’ve learned is that, because we didn’t understand our own power dynamics, we saw ourselves as the newest, most empathetic, knowledgeable innovators in design. We didn’t understand our own nobility complex and how that limited us with scaling our products globally. There’s still an unarticulated sense that innovation is being bestowed on needy communities by the largess of the Western product teams. But we can change that. …

“If you want a scale, you’ve got to incorporate UX research for a global customer base, your global customers, from day one—or in the country from day one. You don’t want to design for your immediate target market. You want to start to empathize with marginalized user groups—long before the product launches into [the] market. If you want to scale, use UX research to help you understand the context of your customers from their angle of [view], not just from your own understanding. Once you’ve been able to internalize that perspective, you can champion…their needs and create opportunities for voices that normally don’t make it into our products.

“By leveraging our privilege, which is the ability to influence an outcome, by looking beyond our own solutions, we open up doors to these marginalized communities. When you think about products that we’ve worked [on] for just accessibility—[for example], Google autocomplete [and] Google Voice [were] accessibility products before they became products that all of you could enjoy.

“I want to leave you guys with three things that I feel are actionable—that you guys can do within your own organizations: …

  1. Design a platform that provides more than two country checkpoints or two underrepresented groups. …
  2. Turn your assumptions into questions. [Not] just with the work that you do day to day, do this with everyone. …
  3. Test your learnings from these marginalized communities in your immediate and primary markets. What you’ll start to see when you start testing these things in reverse is that you’re opening up opportunities for other customers, and you start to see how your product might be seen as a luxury to certain groups, which is limiting your profitability.

“You need to slow down to ensure that your solutions build change throughout the tech ecosystems. … Design is not art. Design is problem solving.” Nancy concluded by saying, “If you can understand how you can incorporate some of these other perspectives [in what you’re] building, you’re going to make the world a better place.” Nancy exudes positive energy and I enjoyed her talk.

Making People the X-Factor in the Enterprise

Presenter: John Taschek

John Taschek, who is shown in Figure 50, is Senior Vice President of Strategy at Salesforce and began his talk with some self-deprecating humor. He spoke about his work at Salesforce and how he wants to make the company a little more human.

Figure 50—John Taschek
John Taschek

Figure 51 shows what John actually does. “It’s a different way of looking at the world. In enterprise software, you get a few things: You get people with analyst-relations programs. These are the people who run industry analysts. You have competitive intelligence, which is either classified as your corporate spy or just looking at the product strategy and making sure you have the best product. You have…pricing and packaging—usually in different groups run off an operations team—where you look at how your products are priced and simplified and packaged. Then you have the voice of the customer, which [is] a survey of the customer and the insights. … [Finally, people in] strategic-influence relations are evangelists.”

Figure 51—Customer focused
Customer focused

“We center everything around the customer,” said John. “So analyst relations means that’s the customer. Our analyst relations are the industry analysts, the Gartners, the Forresters, the IDCs of the world. They are a customer. We have to listen to them. We want to put the customer first. [You can look at] competitive intelligence like deal-desk support, or winning deals all the time without lowering prices. You can look at pricing and packaging [as] just how to make more money. You can look at the voice of the customer [as] just generating positive NPS. But what we do is bring this together and put everyone at the center.

Humanizing Technology Companies

“Here’s…the way we do that. I have two [simple] goals at the company. I’ve always had the same two goals for bringing more humanity to technology:

  1. Start with a person first. That person’s not necessarily a customer. It’s your internal customer [or] external customer. It’s a person.
  2. Make empathy our top value. … The reason I do that is because of my experience with the computer industry. … I saw how much people struggled with their computers. … I [wanted] to make it better. … How can I do that, but actually have the same job that I just described, which tends to be kind of operational?
Overcoming the Obstacles to Humanizing an Organization

“There are obstacles [to humanizing an organization],” which John outlined, as follows:

  • Obstacle 1: Scale is hard. We are trying to simplify our solution [and] make it as easy to use—make enterprise software as easy to use—as Amazon. … It was not hard at the beginning because it’s very simple to understand. It was hard to compete, but it wasn’t hard to understand this. These days, …Salesforce [is] competing on many fronts. …It’s a very fragmented, very niche oriented world where hundreds of companies may have [only] $200 million in revenue combined, but…it is something that we can benefit from or it’s something that we should pay attention to. So that’s where we have to come in and listen and that would be the competitive-intelligence part.” As shown in Figure 52, “Growth is great, but scaling is hard. … We have to adjust. So how can you do that?
  • Obstacle 2: I have personal issues. I hate meetings. I actually hate presenting, too. … I hate meetings because…I’m kind of an introvert. … I just hate meetings and don’t be don’t like being there. I like to listen to everything, but what I what happens is: people just pontificate or take over the meeting. I don’t like it. I resist it. … I think all people should resist all meetings at all times. But when you resist all meetings at all times, …you get that feeling that you might be a little bit isolated from the rest of the company. … I am not shy about saying I hate meetings. …
  • Obstacle 3: Scale introduces systematic dehumanization. Part of it is in your own company. I think Salesforce has been really good at protecting against this, but it still happens, and it’s going to happen in all the companies who are our customers as well.” As Figure 53 shows, “There are processes inherent in scale” that result in dehumanization. “How can we achieve success without dehumanizing? You can’t. It’s part of the nature of the growth of a business. There [are] contractual agreements. You have legal issues. You have just the number of people that are brought into your company as [it] grows. … A lot of them are no longer customer facing. … We really want to try to change that—make sure everyone is talking to customers, everyone’s talking to detractors, everyone’s talking to promoters. We need that voice of the [customer]…to achieve success without dehumanizing.”
Figure 52—Salesforce headcount growth
Salesforce headcount growth
Figure 53—Scale causes dehumanization

“Before talking about humanization, we have to figure out…what [it is],” acknowledged John. Figure 54 shows the principles underlying Gartner’s “Digital Humanism Manifesto:

  1. Start and end with people.
  2. Embrace serendipity.
  3. Give people space.”
Figure 54—Digital Humanism Manifesto
Digital Humanism Manifesto

John told us, “The Pope said, ‘We need to bring tenderness into technology.’ … When you do humanize, you need to identify goals for rehumanizing. … There’s a hierarchy of needs,” as shown in Figure 55. “But when you scale, that hierarchy is flipped. The more technology you bring in, the more it’s flipped,” as shown in Figure 56. “So your self-actualization becomes your agency, and your empowerment is removed. Your esteem then is taken away because there [are] so many processes you have to do. … And this is your customers. Everyone is going through this, not just inside the company. Interaction becomes more difficult because there [are] so many specializations in your group. There [are] specializations and compliance and legal, different product areas, [and] so on. Then, of course, your job security…becomes at risk. and you eventually lose your job to systems. We need to protect against that at all costs because we need to flip this hierarchy back.”

Figure 55—Maslow’s Hierarchy of Needs
Maslow’s Hierarchy of Needs
Figure 56—Hierarchy of Needs flipped
Hierarchy of Needs flipped

“People have the understanding that, if you work hard, there’s kind of some kind of career ladder, but…really it’s a heterarchy. … To achieve a path, there might not be a straight line to get there, and that comes with scale.

Overcoming the Dehumanizing Effects of Scale

“With scale, company culture must adapt humanism,” said John. “I’d rather be known more for bringing humanity into the tech world than anything I’ve actually done. So the solution. What do we do? …

  • Solution 1: Resisting systematic dehumanization. Adapt to career ambitions. The first thing is: I wanted to…separate fact from emotion. … You have to have emotion and you have to have fact. It doesn’t mean you need to conflate the two. … The other thing I wanted to do was deprioritize the hierarchy.… I have direct reports who have direct reports who have direct reports, but a lot of times that means the communication channels between or among those groups are fractured. I wanted to network them. The way that would play out is: if somebody…needed to tell our CEO something, …they should be able to, and we should encourage it. But but it doesn’t have to go through me. … I don’t like that hierarchy. … The other thing is using communications to include people, instead of exclude. … Email is the most exclusionary technology that’s out there. It is meant to send a message to somebody, but not somebody else. As you scale up, you see these processes put in place where you can’t send wide distributions of emails out anymore. It’s meant to fracture your communications, and I wanted to change that. There’s technology to do that now [such as] Slack. … Things that can open up transparency. … You want to establish trust through transparency and what’s real. … Be real about it—authentic. … I’ve [fostered transparency around] competitive intelligence [and] analyst-relations reports—[both negative and positive]—directional statements [and] feature checklists [that would encourage commoditization], and the [organizational] structure itself. … There is an org structure, I just don’t like it. So the more I don’t like it, the more I actually act like there is no org structure. … A lot of this comes back to Frederick Taylor,” as shown in Figure 58. “There’s such a focus on efficiency or ROI that you’re actually losing the bigger picture. … I think it dehumanizes people. … So what’s the the answer? … Taylorism is about efficiency, scientific methods, and…I think that’s also applicable today. But…there’s a future that may expand beyond that. … Elevate intuition to a peer level of the efficiency people—your operations people. This is your core management. … Encourage new thinking. … Think [not just] about how you can do something better, but how to [do] something new altogether and actually resist efficiency. … Monitor the outcomes of your customers, whoever the customer is—the analyst…, the people who are dealing with pricing [and] packaging, the actual customers, detractors, and promoters. … [Having] a big middle-management layer [is] also dehumanizing. … Why don’t we make all managers individual contributors.
  • Solution 2: [Making] sure humanism is part of your alignment.” As Figure 59 shows, this encompasses “vision, values, methods, obstacles, and measures”—what Salesforce calls V2MOM. “You put your vision forward and…your values. … Your vision [has] got to be…about what you want the company to do…to increase [its] relevance…in the world. The values are…what you feel—empathy [or] storytelling. … The methods are how you’re going to do it, the obstacles are what gets in the way, and the measurements [are] how you’re going to [know when you’ve succeeded]. The problem with what I do, in general, is that you can’t measure some of the things that we do. It’s very soft, and you have to be able to resist…when people start going after it.
  • Solution 3: [Changing] the metrics for [the] team’s success. The new metrics…would be: There’s efficiency, but why don’t you just make it [customer] success. Instead of looking at the number of interactions you have, make it about relationships. …
    There’s a focus on…reduction in cost, but you also have to maximize opportunities. Finally, …don’t think about career growth so much. Think about the impact you’re going to have on your company, and your career will grow. Just a slightly different way of looking at it.”
Figure 57—Solution 1: Resisting systematic dehumanization
Solution 1: Resisting systematic dehumanization
Figure 58—Taylorism versus the future
Taylorism verus the future
Figure 59—Solution 2: Making sure you align on humanism
Solution 2: Making sure you align on humanism
Figure 60—Solution 3: Changing the metrics for team success
Changing the metrics for team success
What Worked at Salesforce?

“What did I do?,” asked John. Figure 61 shows some of his tactics for humanizing the enterprise. “I try to put in a intuition above lots of things. I tried to hire actual misfits people who don’t ordinarily fit into the meeting culture that kind of happens with scale. I wanted people to think…differently about different approaches. I also wanted to elevate the opinions of every single individual as an equal—[which is hard to do]—even if they weren’t articulate…, even if they were negative…, even if they didn’t even know what you were doing. … Probably one of the most important things that we overlook is [finding] people who don’t know you and [learning] what they think. Then, figure out those internal and customer issues. … Listen. … I listen quite a bit….”

Figure 61—Tactics for humanizing the enterprise
Tactics for humanizing the enterprise

According to John, “A lot of people tier things” such as firms, roles, ideas, or channels of communication, according to their importance. As Figure 62 shows, “I don’t like tiers. … So I try to avoid [them]. I don’t think that anyone’s better than anyone else. I don’t think that there should be tiers. There [are] priorities. … If you tier at all, you should do it based on an individual, not by any organization, but I think that it’s a mistake to tier.”

Figure 62—Flipping the hierarchy
Flipping the hierarchy

“How can you measure this?” asked John. “[You can’t.] There’s no true measure. There [are] no measurements of a lot of those things I’m talking about. There’s no true measure of the cost of doing nothing—like keeping everything status quo. You can always get more efficient. You’ll benefit your bottom line. … Do it anyway because it’s going to benefit your top line and actually your future.

“Did this stuff work? [Yes.]” Figure 63 shows Salesforce’s core values:

  • Trust—You can’t do anything unless you’re trusted. You don’t have to be a trusted advisor [or] trusted salesperson. You have to be trusted across the board. A service [such as] Salesforce…has to be trusted. [For a cloud service that] means it has high availability. You have to trust that we have a vision.
  • Customer Success—Drucker said something like: a business only has two purposes. One is to create a customer and the other is to retain a customer. That’s customer success.
  • Innovation—You have to innovate—in the technology world especially, but everywhere—because every company is going to be changing. There [are] lots of kinds of digital transformation happening.
  • Equality—You have to innovate and to do that [requires] a diversity of people. That doesn’t have to be by races or genders or anything. It could be in addition to that. It’s got to be equality of thoughts from across a variety of individuals with different opinions.”
Figure 63—Salesforce’s core values
Salesforce’s core values
5 Key Takeaways

John wrapped up his talk by presenting five key takeaways, as follows:

  1. Identify your customers—internal [and] external—promoters, and detractors. Those customers may not just be promoters. They might not be customers. They might not be users of your system. They could be people [that are] just not part of it.
  2. Elevate the counter-intuitive nature of misfits. Embrace the misfits. Have maybe somebody who could articulate the misfits point of view. But bring that seat to the management table so you’re not just having this kind of microcosm of a hermetically sealed environment where everyone’s telling you everything’s okay. You have to have the other the other thoughts.
  3. Don’t worry about your next meeting; worry about the two meetings that will follow. I hate [meetings]. That’s why I go into the attitude where I don’t really care about this meeting right now. Just keep talking. Let’s get it over with. Why do you have to make it thirty minutes when it could be twelve? … I don’t worry about that meeting. I worry about [the] two or three from now. How can I be a participant and add value or be of service in the meetings—two or three meetings from now?
  4. There often isn’t good or bad in an organization. There is just [what] is. Separate the emotion. … If something goes wrong…, you know, we’re in the tech industry. We can fix things. We can just prioritize and fix things. But you have to be aware of them. So there’s no good or bad…. … An organization just is. If you identify the problem, fix it.
  5. Humanization is ironic and necessary. Create your own narrative. [Ironic] because, as you scale, it’s going to become dehumanized—your company, your customers, and so on. So, do it, then tell your own story. Make your own story about why you’re in the technology world.”

I really enjoyed John’s refreshing candor. His primary theme of humanism really resonated with me.

SCALE Discussion

The speakers who spoke on the SCALE theme participated in this discussion. Tricia Wang was the moderator and posed some questions to the panel. Here are some highlights from the discussion.

Tricia Wang: “You guys are all clearly contrarians and really able to speak up in your organization about very difficult topics, but you’re also at the point in your career where you know how to be more collaborative and make sure it’s received well. What advice do you have for people who…know that something isn’t right, but…may be junior in their career or just don’t have the right buy-in?”

Nancy Douyon: “Very early in my career, I learned that you need to build credibility. It’s not enough to just state that you see a problem. They need to know that you will do the work behind it and push toward it. … Try to find someone on your team that you’re willing to work with to do maybe a small project together that you will present out. … The first thing I do is try to find a way to build credibility as quickly as possible.”

John Taschek: “It’s iterative. The biggest thing you have to have is patience and the ability to have people say your idea is wrong—even if you know it’s right. Because, at some point, they’re going to realize you were right and then that credibility develops over time. So I’ve thrived on being wrong and making mistakes—although they weren’t mistakes and I wasn’t wrong—but I thrived on the perception.”

Nancy Douyon: “One of my things that I’ve gotten really good at is being uncomfortable. … I’ve learned to kind of embrace that discomfort. So, if someone calls me out or tells me I’m wrong or any of this stuff, I’m okay with being wrong. It doesn’t mean anything about who I identify as or anything like that. So I would also suggest being comfortable with discomfort is a great way.”

Jennifer Cardello: “If there are people issues, if you see people behaving in strange ways, we have a thing at athena about positive mutual regard. It’s very important for us—we’re empathetic, … lot of us understand psychology—to sit down with people and get at the key drivers, the root causes for why they behave the way they do. Often, those conversations are incredibly powerful in building a bond, so those people see you as a collaborator and someone who can help them out of a situation, instead of that UX friction and combat. We’re not in combat. I’m trying to help.”

Tricia Wang: “It’s like what JJ was saying—oftentimes just doing this over a drink or food just neutralizes everything.

“What kind of what questions do we have from the audience?”

Question: “I have a question about self-correction in an enterprise organization. When you’ve managed to have enough buy-in so you get the quick wins, …but things just seem to be going off the rails, …share your experience on how you’ve dealt with trying to be that source of change—where you see something needs to be corrected, but management may not be listening to you, and you’re getting frustrated as a designer, as a problem solver. What are some things that you’ve learned from that?”

Nancy Douyon: “I have tons of stories. I have a lot of different experiences with that. One thing that I’ll suggest that has been really effective for me is, when some of my ideas don’t make it into a product or something doesn’t happen and the product launches, I try to do a postmortem and just basically share with the team opportunities that were missed because certain things weren’t considered earlier in the process. So I bring people together and say, ‘What can we do better next time in this process…? I like making things a little bit more systematic, and I like making sure that…, if I’m in a meeting, there are going to be actionable things that come out of it. So one of the things that I would probably get out of that, now that we know there are some things we could do better, let’s make sure we incorporate that into our process going forward so certain things are considered, so we can really scale or do better across the organization.”

John Taschek: “I can give an example of something that I would do for problem solving. One part of the job I do is the industry analyst—so that’s analyst relations, that people call your babies ugly and stuff. … [The] Gartner Group calls this the Magic Quadrant. … The philosophy of a larger company is you have to win or move up on the Magic Quadrant, so they focus all the attention on that Magic Quadrant. But I don’t care about that, and I run industry-analyst relations. So I care about three Magic Quadrants from now. I want the analysts to be chasing me. I don’t want to be chasing some document somewhere. So you’ve always got to be looking ahead of what you want to do, instead of looking at what’s right in front of you. It’s called…target fixation. Wherever you’re looking is where you’re going to drive to. Look farther out. … There’s still a focus though. It’s not like I don’t care about the next thing. It is a focus. It’s just looking out longer term.”

Question: “I have a question for Nancy. Right now, we are working our internationalization of our design system. I just wonder about your approach and your story in Uber and [what are] the impediments and [what are] the exciting things for you.”

Nancy Douyon: “I found mentors and sponsors within the organization that were willing to leverage their privilege to make sure that I was going to get as far as possible. … I have to find those people that will help me get to that next level, so I try to find people that trust me. What’s great about Uber is my manager and director have become my mentors and sponsors. So when I have an idea and I want to do something, they say, ‘Try it. We’re not afraid of you failing at this. Give us some reasonable information. Give us some data behind it.’ I am in research. I went from human-factors engineering to research because you can plan to bring some data behind what you’re presenting ahead of time. So I was able to show a gap and show a way to fix it, then basically get their buy-in to start a small pilot program. … Everyone needs to find mentors to get their projects through. No matter what level, you still need that.”

Question: “[Do you] have any thoughts on balancing the incremental change that’s often necessary in addressing UX debt with bigger more innovative work, which is often necessary to keep things growing.”

Jennifer Cardello: “We’ve talked about it as a portfolio strategy at the company. It doesn’t feel good for it to be mandated to Product Management or to Product Owners—like 30% of your work needs to be UX debt work and 40% needs to be innovation work. So you have to encourage a culture of going after those things. It’s like establishing that sense of urgency. …

Companies [operate] on these S-curves—you go off and then you plateau—and you’ve got to have started your next S-curve right below that. So you can’t just be working on table stakes and the incremental fixes. You’ve got to have this urgency around getting to that next level. so it’s embedding it in the organization, in the culture. So it’s more conversation, not so much I’m going to insist upon your portfolio allocations. Let’s talk about where this could be. Oftentimes, what we’ll do is accelerators to help a team do a Northstar vision—something that’s like a year, three years, five years out—that they can backcast into a place where they can start working on a piece of that at that moment. One of the best things about that is: those types of initiatives are incredibly engaging for employees. [They’re] the types of things that they say, ‘I wish I got to do this all the time.’ Making leadership understand that doing that Northstar visioning is important to get people to stick around—it’s a good selling point. Then they will encourage the cultural behavior.”

Nancy Douyon: “With scale, I’ve noticed that my efforts are always to try to make things systematic. So, in this particular case, I’m actually working on this problem where I’ve built a matrix that takes a look at importance as well as difficulty to really understand what are quick wins, what are luxuries, what are nice-to-haves, and we can pinpoint them. So I’ve created a set of questions [to ask] people. Do you have the resources? Do you have the buy-in? Do you have all these things so you’re…working on products that are actually meaningful. I’ve been testing this process…, but I definitely think looking at importance as well as difficulty is a great way to measure what you should be working on versus maybe what you shouldn’t be paying attention to.”

Question: “In the 80s and 90s, people talked a lot about globalization and how multinationals were moving over to other countries and changing the way of life. I think it’s pretty relevant to the conversation today. … As companies necessarily want to go global and profit, [what are] other ways that UX is standardizing and creating these experiences globally [that] preserve the way of life without imposing a certain new standard or Western ideology? How, as UX designers, can we not just be ethical, but look for ways to preserve local heritages and foster the delight of going to new places and seeing how people live elsewhere.”

Nancy Douyon: “Look for things that are great about [other cultures] and bring [them] back [to the US] to make us better. There [are] tons of innovations that are going on around the world that are really creating opportunities—not just for people globally, but for people here. If we can just try to learn to get of our own way and understand that we have to respect certain differences…, we can start to think about…things we can learn from these folks that will help improve the American lifestyle. It’s [about] shifting [your] mindset and…making sure you’re asking people what they need. … Systemization does require standardization, [but if a company wants a product] to really meet the needs of the people, they’re going to have to customize it or make it work for that that culture. … There’s a lot…we have to learn about the trade-offs with systematization and standardization.”

Closing Keynote: Creativity and Principles in the Flourishing Enterprise

Presenter: Richard Buchanan

Day 2 concluded with a keynote address by Richard Buchanan, Professor of Design, Management, & Innovation at Case Western Reserve University, who is shown in Figure 64. He talked about three things:

  1. A new kind of design practice”—He said, “I want to tell you about that practice because I’ve seen signs of it here—not completely, but signs of it.
  2. The nature of principles—I’ve heard the word used a lot in the conference and usually not very well informed. …
  3. Creativity—I used to hate that word. Whenever it came up, I thought, Oh, this is just terrible, dreadful stuff. That’s because I’d been reading cognitive psychology on creativity, which is such an amount of bullshit, I can’t believe it. … I have a different take on creativity today, and I want to explain it to you….”
Figure 64—Richard Buchanan
Richard Buchanan

Buchanan kind of lost me from the very beginning of his talk. To some extent, I disengaged before he had even finished telling us what he was going to talk about. As Dorelle Rabinowitz said in her talk, “There’s really nothing more insulting than to present basic information to a highly knowledgeable audience.”

Buchanan began by talking about his history at Carnegie Mellon University: “In the early 90s at Carnegie Mellon, we created not one, not two, but three interaction-design programs. One was shaped around the rhetoric of experience, and that’s an interaction matter deep and meaningful. The second was a poetics of experience. Poetics being about the making of a fulfilling, unified whole. The third was a new product–development course. These were all important and meaningful courses. No one, at the time, understood why they were different, but I knew why they were different. …

“Curiously enough, it’s out of the industrial-design side of new product development that the concept of design thinking emerged for us at Carnegie Mellon and continues. I want to explain briefly because there’s such misunderstanding about that term. Someone said, ‘Maybe it’s not useful any longer.’ Well, I don’t know. Maybe it isn’t. But I want to explain that term for us at the time. We realized that new-product development had to incorporate the engineering and other technical disciplines to make effective products. … We also believed that marketing and strategy had to be part of design. Think of what I just said, marketing and strategy are part of design. That’s the nature of design thinking. That meant that we had to change the concept of marketing.

“The old marketing is about selling products, finding the right markets and the segments of markets, and pressing ahead with products. The new marketing is not about that at all. The new marketing is about preparing users or customers for the new kinds of products that will emerge as time moves forward. It’s a forward-oriented kind of marketing. You will not find it a lot in this world that we live in, but you will find it in some places. It’s a form of marketing that is gathering strength. I regard that as a part of design thinking.

“With regard to strategy, the old strategy involved competitive positioning of products with your competing companies. This is the Michael Porter material. … But the new strategy is about understanding what that future world will be—whether it’s three years from now or five years from now. What kind of world will we be living in, with some technology, but maybe other things as well. That sense of bringing customers and users to prepare them for the new products and then to understand where we’re going and then build the products to fit that. So it’s this combination of three features that makes effective design thinking….

“I don’t reduce it to cognitive psychology as Herb Simon does and some others, too, but that’s the focus that I have. Our programs were quite unusual in this respect. I think, as I look at the group and the presentations I’ve seen now, I’ve seen evidence of what I call design thinking. Although I’m not sure that your work practices bring you as closely in connection with the new marketing as perhaps it should and, certainly, perhaps not with the strategy as it might be. Some companies do and some don’t is the way it is, I think.”

The Four Orders of Design

Buchanan presented his four orders of design, shown in Figure 65. “This is part of the the work we’re doing. … I want to explain to you where I see your work emerging now and be real clear about this because this is pointing toward a new kind of practice. … That’s my core point. … The four orders of design are concerned with problems of communication and problems of construction—making things. These are the two great practices of design at the beginning of the 20th century—graphic design [and] industrial design. Engineering comes [in] there, too, and I suppose architecture as well. … I try hard not [to] talk about architecture, but sometimes I have to. It indicates the first and second orders are straight ahead. We have programs and have [since the] early…20th century.”

Figure 65—Four orders of design
Four orders of design

“The curious moment comes with what I call third-order design—the move toward activities, services, designing actions,” said Buchanan. “Activities—those are interaction areas of work. I used the word interaction in a very broad sense. … For me, it includes services and policy formations and lots of other things that are not necessarily digital—and maybe not even fundamentally digital. But it’s about how human beings relate to other human beings. That we can design those relationships. I think that’s what the focus of this group is fundamentally about. You are dealing with individual human interactions.

“[Third-order design] is the formation of our interaction programs…for sure. But, at the same time, I began to think, There is going to be another practice that will emerge. … This move into environments, organizations, and systems. I call that fourth-order design, and I want to explain very carefully about that…. Fourth-order design is about understanding the system structure and understanding how we intervene in that structure to create new pathways of experience. … A few of the presentations [at this conference] have touched on what I call fourth-order issues. … There are a few organizations around the world that are doing this. … Over the past 10–15 years, there are now probably 15 or 20 design firms in the world that are doing what I call forth-order design. … There are a few design firms that work at this high level, and it’s hard to characterize what they do.”

As shown in Figure 65, “I had names for the first, second, and third orders: graphic design, industrial design in some sense, and interaction design—this broad term that I’m using, encompassing as I said service and some other individual relationships. I never had a term for fourth-order design because the practices seemed to be so different and so diverse… [but] within the last two years I’ve realized that probably the best word for it is dialectical design. … The common practice in fourth-order design is a discussion—a conversation. We call them strategic conversations—typically at a very high level in an organization, but sometimes…in product work. It’s where conversations take place in an effort to find the core ideas and values that animate the work we do. At the highest level, these are very serious deliberations about the future and a good strategic conversation can take months.

“A dialectic is when there are two people talking together and, curiously, in fourth-order, the role of the designer changes. Instead of being the controlling influence, the designer more and more steps back and becomes the facilitator of discussions by others. … One of the features of fourth-order design that I want to be clear about [is] what we mean by experience. … This is an idea that’s still molten and taking shape.”

Principles

“We’ve used the word principle frequently—more frequently than I expected—in this conference,” noted Buchanan. “But every time we’ve talked about a value or a principle, they’ve not been seriously exploited or explored. We tend to mean principles of practice [or] principles of method—not really principles…, [but] guidelines for how we practice, and it’s fine to have those. But principle, for me, is something more than just a guideline. …

“A principle is an idea or a value that gives unity to an organization, an enterprise, a system, a product, a service, or a human experience.”

“We have a big difficulty talking about that kind of principle in design, and that’s actually the second point that I want to bring. Not only are we moving into fourth-order design—some of us and some very powerful work being done in this—but down deep, we’re looking for principles and don’t know how to talk about them. That’s a characteristic I’ve found in design generally. But I also, at the same time, find that there are more and more designers asking the question: Why do we design? Why do we do this? It’s not [an] easy [question] to raise when you’re in the middle of an enterprise driven by profit and competitive positioning. That’s true, but that’s the way it is. There are principles that guide our actions and…they need to be discovered and shared.

“I can’t put it into a phrase like doing good to people. … I have no idea what good means until I have a chance to talk to people and see what good might mean to them and what it might mean to me. That’s why the conversation takes place in fourth-order design. We talk with people about what is good and what is right.

“I was at a conference…on the relation of systems thinking and design thinking, and I’ve got real hesitations about systems thinking. … Systems thinking never led to a design. Systems thinking is a great way to analyze a situation, but it will tell you nothing about how to act—what to do. For that you need design. [Through] systems thinking…, I can understand the complex situation, but if I want to change the situation, I need design. …

“There are first principles and then there are all the other principles. First principles are the deep ones that we value and discuss and don’t fully understand maybe, but by conversation we gradually understand it better. The trick is this: Once we’ve gained that understanding, how does that design principle, in general, …work its way down into the products and the relationships of people and what we do. That’s the concern in fourth-order design. … I think we’re…coming to a period [when] the design community is going to begin discussing principles in serious ways. …

“I think the design community is going to have to learn how to talk about principles in more than a trivial way—as more than just rules of thumb. That’s a daunting task, but I think it’s going to happen. I think it is happening, in fact. I’ve been startled by this conference—that it’s surfaced as often as it has, for good or bad, well used or poorly used. … People are not entirely happy with what’s going on. The values aren’t quite right. My concern, of course, is that, in the big tech firms and big organizations generally, they’ve forgotten about principles. … I want to know what [a principle] actually means, and I want to see it embedded in the products and services that are provided. No small task, but that’s the role of designers today: to facilitate the discussions—to discover those principles. …

“We’ve got to avoid being caught up too much in the processes and the techniques and remember that there’s something else beyond it. …

“I think though that we are on the verge of new discussions in design about what constitute the principles of our work—why we design. I think it’s time that we have those discussions. … What I see as emerging fourth-order design around the world is surprising, even stunning today. … There are projects taking shape that are engaging social-innovation and transformation changes, and designers are facilitating those and, in many cases, stepping back and letting the people carry it forward. That’s meaningful and significant to my mind.”

Marketing

Buchanan told us about a paper by Philip Kotler called “Humanistic Marketing: Beyond the Marketing Concept,” written in 1987, in which “he identifies four kinds of customers,” shown in Figure 66. “Each one could be a principle or be the focus of a principle. In one case, providers look [at] users [as] creatures of desire and whim, and we design for those whims and desires. … The stranger is a person who has a want or a need, and they are to be served by us with the product we deliver. We provide some way to fulfill that want or that need. … Neighbors are people who have wants and needs, but the wants and needs evolve and change over time, and you treat the customer in a different way. You don’t treat them as creatures of whim and desire. You don’t treat them as strangers merely fulfilling their want or their need and helping them with information about a product type and specs. You actually try to engage what they are thinking and what they’re doing and what their needs and wants may be as they evolve. A very unusual kind of interaction. … Friends are people whose lives we want to enrich—and not necessarily limited to commercial enrichment. But the notion of enriching people’s lives… becomes a very powerful driver and a significant matter.”

Figure 66—Marketing perspective
Marketing perspective

“Just as the provider has a notion of the user, users have notions about the providers,” said Buchanan. “In that first case, the creature of desire and whim [sees] the company or the provider as a foe. They know they’re going to be tricked. There are efforts to manipulate them. That’s the characteristic of these creatures and the way companies treat people. … The providers are also strangers. People who come to our services, our Web sites, our companies…want that product with certain characteristics and specifications, and you need to provide it. There’s no close relationship—just can you give me what I need, what I want. Not just a whim or a desire, but a specific want and need. … The neighbors…see the company as a helper, and to be a helper is a different…kind of interactive relationship…, and the way we design our services and our offerings to be a helper and not just to rip people off. … Friends… Is there a company or a product that you like so much, when the product fucks up, you want to help the company straighten it out? … There are a few cases where the company has made such an impact on us that, even if they mess it up, we want to help them do it right. … [There] is…a bond between [the user] and the provider that’s unusual. … In serving the friends, the goal is to enrich their lives—not necessarily entirely commercial. You may all have run into situations of service where the providers have been gracious and do a little extra, and sometimes you can sense that’s manipulation, but sometimes it’s genuine.”

In Figure 67, Buchanan mapped his four orders of design to the marketing perspectives shown in Figure 66. He asked, “Are you moving to third- and fourth-order relationships …neighbors and friends? … I think this is where you are. You’re dealing with third- and fourth-order issues.”

Figure 67—Mapping marketing perspectives to four orders of design
Mapping marketing perspectives to four orders of design
Creativity

“I want to talk about creativity,” said Buchanan. “This is the word that I hate so much because the cognitive psychologists took it over. They think of creativity [as] a matter of information processing. It occurs in the brain. Creativity is a process in the brain and, typically, it involves a search of earlier experiences that have been successful [to] find those patterns and bring them into the present situation. … This notion of creativity…is not not adequate for what we know in design. For me, creativity occurs in the mind, not in the brain. The mind is a different matter. … I’ve actually looked more seriously into creativity. It has a history in Western culture and…in Asian culture. There are different words that we’ve used over time for creativity. Our current word is innovation.

“The things we know we give names to, and the names are categories. The category fixes the meaning. … But the trick of creativity is how to break the…categorical meanings to discover new meanings. Creativity means a change in perception. We change from what we see that is familiar to what is not familiar—to what may [even] seem to be ridiculous initially. … When you’re able to merge two categorical terms…in your imagination and see a possibility of a new meaning, you change…the meaning…of that category. … We invent a new possibility. …

“If you take that invention that occurs in the mind, you say there’s a possibility here. You then turn to the world that we know—the experiences we have around us. We use that invention to…discover new facts about the world—new facts that support that invention, that carry it forward and change our ways of thinking. … That’s a discovery and lots of other discoveries that go with that because it’s a body of new facts and new possibilities. Things ahead we hadn’t thought about before—unfamiliar.”

Figure 68 depicts Buchanan’s creative matrix, which shows the progression from invention to discovery to innovation to intuition.

Figure 68—The Creative Matrix
The Creative Matrix

Buchanan said, “The third step for us is innovation. … The innovation comes when two stories come together and there’s a spark that crosses the stories. We see suddenly a possible connection that we hadn’t thought of before. … Our ability to see stories comes in the mind. We see a connection and say, ‘My God! That could work. We look for those a lot. Those are powerful moments.

“There is a fourth word [that] is not used very much with regard to creativity: …intuition. … Intuition has had a history in Western culture of profound significance. Spinoza thought it was a higher level of knowledge than logical argumentation. Intuition ranked higher. What is intuition? Intuition is the ability to see a connection and realize [it] is so important that you could build a system out of it. You grasp a principle. Remember, we started talking about principles that organize. If you see a connection and realize how important that connection is, that’s an intuition. People…have these abilities to make connections. The connections don’t get made by the computers, they get made in the mind—not by searching old information bases, although that can happen. …

“Creativity [is] an activity in the mind. It’s a change of perception, and we can change perception in a variety of ways. We can change it by merging words together and saying, ‘What happens if…? Just suppose…. …

“Once [an] invention has taken place, …it’s…established fact. It’s back to categories. …

“None of us have the ability to talk about principles these days. All of us seem to have lost that ability over the last 150 years. We’ve become obsessed with facts and data, and our problems lie in the ability to make connections…that are significant. I guess that’s the story that I’m telling about creativity: how we move from that obsession with facts and data to finding significance in our lives and our practices. … I think, down deep, you’re looking at what the experience of human beings ought to be, what it can be, what it could be, what it should be, how we can flourish as human beings.”

While Buchanan made some interesting points, I didn’t really enjoy much of his talk. However, the part of his talk that focused on creativity resonated with me. Particularly at the beginning of his talk, his tone was pedantic, condescending, and off-putting. His subject matter was rather academic for a context whose focus was all about industry, and I was not inspired by this talk. While that might seem to be a high bar to attain, it’s what experience has taught me to expect from closing keynotes. The many great speakers at this conference were tough acts to follow. Only a truly inspiring talk could have closed the conference on a strong high note.

Conclusion

This excellent conference covered a wealth topics on enterprise User Experience, an essential area of focus for today’s UX professionals. For large enterprises in diverse domains to deliver superior user experiences, compete successfully in today’s marketplace, and achieve optimal business results, they must invest in User Experience. Only enterprises that are committed to investing in User Experience and scaling their UX teams can deliver optimal results. Within enterprises that choose to make the necessary level of commitment to User Experience, UX professionals who can solve complex problems can make a huge difference to their companies’ business outcomes. Thus, working in enterprise domains today presents great opportunities for UX professionals.

Rosenfeld Media’s newly renamed Enterprise Experience conference will take place June 3–5, 2019, in San Francisco. This year, the organizers have decided to switch things up and start off with the two-day main conference, then follow up with a full day of workshops. I hope to attend the next iteration of this conference! 

Discount for UXmatters Readers—To attend Rosenfeld Media’s Enterprise Experience 2019 conference and workshops, register using the discount code UXMATTERS, and you’ll get 5% off the price of your conference ticket.

Principal Consultant at Strategic UX

Founder, Publisher, and Editor in Chief of UXmatters

Silicon Valley, California, USA

Pabini Gabriel-PetitWith more than 20 years working in User Experience at companies such as Google, Cisco, WebEx, Apple, and many startups, Pabini now provides UX strategy and design consulting services through her Silicon Valley company, Strategic UX. Her past UX leadership roles include Head of UX for Sales & Marketing IT at Intel, Senior Director of UX and Design at Apttus, Principal UX Architect at BMC Software, VP of User Experience at scanR, and Manager of User Experience at WebEx. Pabini has led UX strategy, design, and user research for Web, mobile, and desktop applications for consumers, small businesses, and enterprises, in diverse product domains. Working collaboratively with business executives, multidisciplinary product teams, and UX teams, she has envisioned and realized holistic UX design solutions for innovative, award-winning products that delighted users, achieved success in the marketplace, and delivered business value. As a UX leader, she has facilitated conceptual modeling and ideation sessions; written user stories; prioritized product and usability requirements; established corporate design frameworks, standards, and guidelines; and integrated lean UX activities into agile development processes. Pabini is a strategic thinker, and the diversity of her experience enables her to synthesize innovative solutions for challenging strategy and design problems. She is passionate about creating great user experiences that meet users’ needs and get business results. A thought leader in the UX community, Pabini was a Founding Director of the Interaction Design Association (IxDA).  Read More

Other Columns by Pabini Gabriel-Petit

Other Articles on Conference Reviews

New on UXmatters