Goals Beyond Gold: How to Prioritize Projects with Dr. Maraki Kebede

Gabrielle: Thank you, Maraki, for joining us today! You helped to design a five-step process to decide which projects receive support from the UK Sports Institute. Can you explain how this plan helped to guarantee that decisions are fair and include everyone on the team’s input?

Maraki: Yeah, sure. I think maybe a quick backdrop on how we approached the project would help. The UKSI team, more broadly, supports Olympic and Paralympic sports in various ways. In particular, their data team supports about 50 or so customers in ensuring that they can incorporate data into how they support athletes. 

So, the team’s goal is not just to win medals for the UK, but also to win them while protecting the health of the athletes, making sure they're “winning well.” This is why TDL was particularly interested in working with them.

Our approach going in was first to understand what their goals were. The data team asked us to help them with their prioritization process. Essentially, if they have 50 customers that they're working with, we needed to figure out how they should prioritize all the different projects that are coming to their table when they only have so many team members to work on them.

Even though the team wants to work on every project that comes to their desk, they have to make sure that they prioritize ones that are more aligned with their strategic goals, and more impactful. And so they needed an evidence-based process to do just that.

What we learned was that the team wasn’t necessarily all aligned on what the goals were, even though they were all following the same strategy documents. So first we had to get everybody on the same page as to what they were working toward.

Then, based on that, we were able to refine the process by which projects are introduced. This can’t just be dependent on inbound projects, because then you're just kind of going with whatever comes to your door instead of seeking out projects that are aligned with what you want. This process also included steps that allowed the team to make sure that every member was involved in the decision-making process. So not just the team members that will work on the project, but also the broader data team that can support the project in various ways.

Initially, the team would hold a single meeting where everybody sits at the table, one person comes with the project that they're suggesting, and everybody decides whether or not to go with the project. What we suggested is to give everybody time to reflect on the project proposals in advance of the meeting, which allowed them to make sure that no voices were lost in the meeting—that everybody had a chance to reflect and come to the meeting having already voted on the project. The meeting, in this case, is just kind of a final discussion point.

Gabrielle: Bouncing off that, when you're introducing new methods like this, they can sometimes face pushback or unexpected issues. What were the biggest obstacles that you encountered when introducing this new decision-making process at UKSI? And how did you deal with them?

Maraki: That's a great question. I think perhaps it's not just an issue that we faced, but it was an issue that the team was grappling with even before they came to TDL. With every project that the data team at UKSI took on, the person who proposes the project to the team is the person who ends up leading the project. And oftentimes, that person is particularly tied to the project. And so they were worried that in the prioritization process, the individuals leading a project would take the deprioritization of their project personally.

With this in mind, we suggested that they do a blind review. So the person who's leading the project would put together a project proposal, not just leaving out their name, but not even including the customer's name. This is because usually, each data team member is associated with certain customers. So this allowed us to remove that tie between the data team member and the project. This way, everybody else was able to vote on it before they knew who was actually responsible for putting the project together. 

When they come together for a discussion, they're able to talk about the project on its merits alone.

The other challenge I think that is worth noting is with the blind review, if the data team member putting together the project proposal is particularly good at selling the project, people might vote it higher just because it was pitched better. The team was concerned that the decision on the project would be based on how well the person sold it instead of the merits of the project itself.

So what we did to alleviate this is the proposal document that they would have to fill out—we called it a “viability form”—would have to be something that required short responses, that didn't rely on the ability of the data team member to sell it. This also protected the bandwidth of the team from finding ways to sell a project. So we kept the questions short, prompting them to answer in very specific ways so that the human element was addressed, not just focusing on the insights alone.

Gabrielle: When coming up with these initiatives, you used some interesting techniques, like implicit association tests that revealed hidden preferences, and qualitative discrete choice surveys that asked people to choose between different options. What surprising insights did you gain from these methods and how did these help to shape your project?

Maraki: I'm kind of proud of how we approached the project, to be honest. What we did is we went in first with very broad, semi-structured interviews. Here’s some important context: the data team is comprised of 11 people. So that kind of puts you in a position where there's only certain things that you can do because it's such a small team. And so we first did semi-structured interviews to be able to understand what their process entails.

Once we have a solid understanding of their process, we wanted to simulate how they weigh out the different options when they're deciding between projects A and B. So we did that in two ways.

The implicit association tests help us understand whether the key biases that we suspected were present based on the interviews were actually there. That allowed us to see that there were moderate levels of bias, particularly emotional bias and confirmation bias. If they're fond of a particular customer or if they've already spent a lot of time working with them, they probably want to rationalize working with them again and working with them more.

But the discrete choice component was something a little bit new that we did. Typically with discrete choice surveys, you're testing different “attributes” or levels, as we call them. So testing different features of a project to be able to test what part of that project is held as more important by the team.

But in order to do that in a quantitative way, for a team as small as 11 people, they would be required to answer too many questions to get at the statistical significance that we are trying to get at. So instead, we came up with five key choice sets for them to compare it to. So five projects, A and B projects to be able to compare. And with each one, instead of trying to get a quantitative analysis, we actually asked them to qualitatively describe why they picked one or the other.

And the awesome thing that came out of that a very large portion—roughly 90%—were prioritizing strategic alignments, or making sure that they're picking projects that are most aligned with their strategy. But when we asked them how they were making these decisions and whether or not the rest of their team would likely make the same choice, we found out that their understanding of that alignment is actually highly conditional.

For example, they would pick a project that's strategically aligned if they have the resources, if everyone else was in agreement, or if they had the bandwidth. So we were able to qualitatively understand the project attibutes that we may not have thought of if we had run it quantitatively.

Gabrielle: Wow, that's very impressive! So with these methods in mind, what advice would you give to other leaders who want to use similar methods in their organization to make better decisions? What kind of environment or mindset is necessary for these methods to work well?

Maraki: Well, a lot of things came out of this project, but I think the really important takeaway is goal alignment. So when our point of contact with the data team approached us, they had a very specific ask. And part of, I think with TDL, when we're working with clients, we also help the clients think through whether or not they're asking the right questions so that they get the impact that they're looking for.

I think their first ask was about how we “make sure we're prioritizing projects that are aligned with our strategy and that are most impactful,” but through doing workshops with the team and interviewing individual members to understand their goals, we were able to see that not everybody was being included in the decision-making process. And so we were able to point this out to our point of contact who then said, “Oh, this is an extremely important thing to us.”  Even before they asked TDL to support them, they actually brought the team meetings together to kind of foster more cohesion.

But what we were learning from our interviews was that the meetings were happening together, but the decisions were still siloed. And so she was under the impression that they had addressed that already, but people were still feeling like they weren't being included. So we were able to incorporate in the prioritization process a way to make sure all the voices are being heard.

I think the big thing is always to start with making sure you're clear on your goals. And some of those goals are external impact, but some of those goals are also internal. And sometimes your internal goals help you meet your external ones as well.

Gabrielle: Okay, so what you’re getting at is that we can apply this strategy not only across different kinds of teams but also different industries?

Maraki: Oh, for sure. Whenever you're thinking about impact, in any context, you need to first understand the impact for what and to whom. And I think those questions can only be answered if you're clear on your goals. But regardless of what industry you're in, thinking about your goals both internally for your team and also externally for your organization is important to always balance things.

Gabrielle: Very SPICE-y!