Why card gaming helps teams model security and privacy threats [transc

One of our challenges at Agile Stationery is explaining why simple cheap paper products, like Elevation of Privilege - can help solve a complex urgent and highly valued problem like cyber security. There is a long trail of reasoning involved and the people that blazed that trail are rarely available to explain themselves.

We were therefore delighted when two of the inventors in our community delivered an insightful speech at AppSecCali 2019 - a leading Application Security conference in California under the purview of the cyber trade body OWASP.

Noticing that AppSecCali is back on soon we decided it was about time to bring you these insights, so we have produced an edited transcript of the session in three parts. I will let the speakers introduce themselves:

[MARK] Hi everyone thanks for joining our talk. This is “Game on - Adding privacy to Threat Modelling”.

I'm Mark Vinkovits. I work at LogMeIn as manager for application security. I come from the Budapest office so if you're interested in what our clan on the other side of the planet is doing, I'm also happy to talk about that. Apart from that I used to work a lot in user centered design and usable security. I've also been running the security champion program at LogMeIn and I'm building that up if that's something you're interested in.

Also for a couple of years now privacy has become one of my passions. Not just the legal side of it, but actually I think it's a very fun and interesting technical challenge. That's how I ended up adding privacy to the STRIDE wall and the Elevation of Privilege cards.

[ADAM] Hi I'm Adam Shostack. I helped create the CVA. I'm on the review board of Blackhat. I've written a couple of books and most relevant to this talk I created the Elevation of Privilege game to teach people how to threat model and help them do so. And so we'll be talking about how that's evolved over time.

[MARK] …. So we're just going to give a quick overview of the Elevation of Privilege card game. Then Adam is going to go a little bit deeper into why games are good not just for threat modelling but for security in general and other areas, and then I'm going to talk about how I came up with the idea or what the motivation was to add privacy to this and the methodology used to do this so we ended up with STRIPED which is adding Privacy to it. We'll have a small game but we come to that later.

So let's look at threat modelling.

You don't have to read this. This is the threat model for SSL but just from the picture, if you look at it, this is something that we appreciate. This is a well-made threat model. It covers a lot of areas. Compared to TLS covering a quite small use case it is a very comprehensive and nicely made threat model.

I think we also agree that doing threat modeling as part of your application security program or general security program is a really good thing to do and we motivate people to do this. When we are not that good at it, it is when someone starts fresh as a security engineer…The usual advice as to how he should start this is just do brainstorming. Academia has quite a good definition of what modeling is and they weren't thinking about brainstorming when they came up with that definition.

Just saying do brainstorming is not really going to help them get practice in this. So there is a second piece of advice which is a bit more methodological and that is to look at all your assets and think of all the ways something could go wrong. Now again that's exactly what the problem is - they have no clue what all the ways are in which something could go wrong.

So they could start looking at threat libraries or threat catalogs but those are huge. Like doing those as a regular exercise really works only if you have that amount of time to do threat modelling.

And some go to the extreme saying that security professionals just look at a system and they know all the problems. They don't have any kind of methodology. They know it...

[ADAM] I just want to add that I'm proud of the threat modeling book that I wrote and it's 600 pages. It's not something you give to a beginner and say “read this and then get started.” It's way too much.

So when I think about threat modeling and when I think about modeling with experts in the room, we're engaged. We're having a good time. We’re playful in our exploration of ideas. And when someone is new to something, they find it challenging. There's a learning curve that they go through.

And so when I started thinking about the gap between how I threat modelled and how people using the Microsoft Threat Modelling Tool threat modelled, there was a real difference and I was responsible for that tool so I'm casting shade on myself not anyone else and there's a model from positive psychology which is when there's a balance between the skill and the challenge that it's applied to. People develop this feeling of flow, that they're engaged, that they're enjoying, that they're productive.

And when we go from point A to point B, we go from a low challenge to a higher challenge without a corresponding increase in skill, people develop anxiety. On the other hand if you go from point A to point C your skill increases and challenge doesn't increase at the same time, you get bored and so we need to build this balance between skill and challenge.

And you'll see this in video games. When you start a new video game there's a training level in which they're explicitly and intentionally taking you through this challenge:

walk to the door
open the door
pick up the kit
do this thing with the kit.

You're developing these skills in a low challenge environment. Then a monster might show up. And so this approach can be used as we engineer and design systems of teaching, design systems to bring people new skills.

When I think about threat modeling I think about four key questions:

what are we working on
what can go wrong
what are we going to do about it, and
did we do a good job

There's lots of ways to answer each of these questions ranging from “what are we working on” - this might be a whiteboard diagram or it might be something very fancy. “What can go wrong?” Different ways to answer the questions.

Elevation of Privilege was created to be the easy way to learn how to threat model. We start with something super simple like a whiteboard diagram and in software we draw diagrams like this all the time. So that's what you do to get started with Elevation of Privilege: you draw a picture.