How to play the Elevation of Privilege threat modeling card game? [tra

In the second part of our transcript, Mark and Adam demonstrate how to play the game with worked examples. We loved how this part of their talk drew on their years of experience of putting games to work. They showed how games can resolve problems in scenarios where other kinds of collaboration don't work as effectively.

If you've missed the first transcript of the Adam and Mark's session, you can find it here:

Why card gaming helps teams model security and privacy threats [transcript part 1]

Who plays?

[MARK] So who do you play the Elevation of Privilege Card game with? Our experience, at least the way that we apply this in LogMeIn, is that having 5 – 6 engineers in the room together with someone from the security team is the best way to have a good pace at which you are working. Obviously involve quality engineers. Very rarely, we get the chance to have a product manager in there, which is always a fun experience.

So you get these people into the room and you hand out the cards and you explain that this is actually just a regular card game, only that the suits are replaced with elements from the STRIDE model and the basic rule is that there is a calling card, After that everyone should stay in suit. We're going to show that whenever you're playing a card you should read out what's written on the card and work with that and you play until you have time or you run out of cards.

Very importantly you also should appoint someone to take notes. Hopefully someone who is familiar enough with the system that the notes they are going to take are going to be understandable for the people in the room later on.

How to play the EOP card game?

[ADAM] So we have here a simple little diagram, what I created for this deck ten years ago.

This was part of one of our training exercises. It's an integrity monitoring system like Tripwire. So we've got some software out on hosts. We've got some software at a console which keeps track of what's going, what files are being changed, what our expectations are. So we can have a model like this and use it as we play Elevation of Privilege.

For example Alice might play a card the three of tampering and the card says an attacker can take advantage of your custom key exchange or integrity control which you built instead of using standard crypto.

Now as security people we know that's something that you don't want to do but if Alice is a software engineer or a network engineer, she might not have that knowledge but the hint is on the card in front of her and she can say maybe that applies to this network connection right here (see red arrows).

[MARK] So next one up is Bob. He has a really strong hand and he plays the tenth of tampering.

It says that an attacker can alter information in your information store….He says well there's a file store on the integrity checker console site, so if we have weak ACS there, then someone might alter your configurations or alter the way the software works.

[ADAM] So Charlie might play the 5 of tampering.

An attacker can replay data because your code doesn't have timestamps or sequence numbers and looking at the diagram he might say that applies to this integrity data flow that comes back (see the red arrow).

[MARK] Finally Lisa plays the 8 of tampering saying an attacker can manipulate data because there's no integrity protection for data on the network.

Not very often the answer to this is, we're using TLS but just like people don't know the difference between having encryption and doing integrity checking, what Lisa says is that there's a connection there (see the red arrow) which might have this problem.

The basic rules of the game

[ADAM] The card game derives from spades if you play a lot of card games so you have to play in suit. The high card wins the hand unless someone plays a trump card and a trump is the suit that always wins so elevation of privilege is the trump suit. There's a mechanism for the game which is there are aces and an ace is you've invented a new threat that's not in the deck and to support that there are cards in the deck that say here's a list of all the threats that you might see on all the cards and then you give out some points. You give one point for each threat. You give one for winning the hand. Some people play this competitively and do a better job of tracking points than threats. That's a mistake. But some people like to know who's winning and some people play very collaboratively.

The key idea is that you're actually engaging with the threats and the possibilities and having a conversation about the second question at the heart of threat modeling which is - what can go wrong.

How the EOP game can identify threats in a system they claim is secure?

[MARK] As I said, you play until you have cards or you run out of time...Very importantly after the session make sure that you triage the items, you talk to your project managers about how important the items are from his business perspective and then make sure that those which should be handled, actually get into the ticketing system.

Now after listening to us and if you have never tried the game out, I really motivate everyone to try it. And this may sound strange, like threat modelling - it’s a beautiful art that experienced security engineers are doing so why are we doing a game out of this. Surely it cannot work. Like people are going to focus on very bad aspects of the session, they are going to focus on playing the high cards and so on and you lose the whole idea.

Now those arguments might be true but the experience is that it works. So we have made regular threat modeling a practice for more than two years now and we always use the elevation of privilege card game and there's always two three high findings in every threat modeling session which comes out by playing those cards which have not been discovered before...

Now one of my favourite findings which I would like to share with you. There was a component written a couple of years ago.

It's job was quite simple. The idea was that an admin can upload a file or a patch he wants to distribute in the network. He uploads the file, it provides us the URL where this file can be found and then our component picks up the file and distributes it in their system.

The engineers thought there's not really a high risk here because the admin who wants to distribute the file is already the admin of the network. He’s a local admin on all the machines so there's not much that can go wrong here.

Also the software has been developed by some of the demigods of the company like the ones who've been working for 20 years... and I'm still quite fresh. How am I supposed to tell them what could go wrong. They’d certainly thought of all the things.

So we went through the session. As I said …. the admin provides us the URL of the file we should be distributing. When we were talking about the validation of that URL, we thought there's not much we can validate because it's a file in the customers’ network. He might be providing an IP address which is only visible to him or an s3 bucket,or uploading something to their own domain. So what do you validate against..?

Now as it turned out there was one important piece of validation which was missing which was that it shouldn't point to our internal network. This was because the component was running an OR firewall segment so if he provided an internal IP address, we would just be picking up the file from our internal network and sending it out to his computers. So we discovered quite a nice way to use the component to extricate data.

Why I really like this one is this whole story around it. Like there were principal fellow level engineers. They were really security sensitive. Two of them were security champions for a long time and before that they'd been writing some of the very sensitive code in the company. But still there was like one quite critical part of validation that they forgot and that came out with the Elevation of Privilege card game.

Why are games good for cyber security?

[ADAM] So more generally than elevation of privilege I've come to the understanding that games are really good for security…Games enable help us solve important problems which we face and there's a bunch of reasons that games work as a tool.

So if I bring out a card deck, this is a very modern sort of game.

They are attractive. They're intriguing. If I start with some cards and I show these to you it's like oh that's interesting. Right? What is that? Let me learn about this and that's powerful as we're engaging with developers who might think that they have other things to do. It's powerful as we engage with operations people.

I talked about flow. Flow is important. You saw me get into a little bit of a flow State as I was talking about those things and I forgot to hand off to mark.

The other thing that it does is if I'm going to hand each of us a hand of cards, as play progresses around the table, I cannot be a Wallflower. I've got this card in my hand and I've got to say huh, how does this connect to the system. Not sure? I'll go to my next card, then I'll go to my next card and so it requires participation without being aggressive or demanding. If people are feeling a little frozen and I see this happen, they'll say I think this card might have something to do with what we're working on but I'm not exactly sure and they'll get help from the people around them or they'll just use the hint on the card and be able to act and so it creates this very fast feedback.

More importantly perhaps even most importantly is that when we're playing a game, the act of playing gives us permission to behave differently than we might otherwise be in a meeting. I can explore an idea if I'm sitting with the most senior developers in the company who have founded it, built the code, they might say this is safe. I might freeze. I might not feel like it's okay for me to tell these folks to explore. But if I'm playing a game with them I could say yeah it probably is but what about this right. I just have to play my card in this hand, so let's see what happens and that permission also extends to disagreement... [basically] the game gives me a different context for a conversation and as security people that can be incredibly powerful because we don't have a lot of playful conversations with the people around us.

The other thing to mention about Elevation of Privilege is that it produces real threat models. It’s not simply a training game that you play once and then you're done and you know how to threat model. So that's a valuable property of the game.

Now the game sits in a line of games. It was directly inspired by Protection Poker by Laurie Williams at North Carolina State. I heard a podcast that Laurie was in talking about it and I said that's fun let me see if I can build a game to help people and then I learned about this whole 'serious games' movement and I thought, well if threat modeling should be simple, fun and have flow, I can build games for this.

I did this while I was at Microsoft. If you go to github Adam Shostak EOP you can download copies and we'll give you more links at the end of the presentation but I do want to say thank you to Microsoft for enabling this open-source sort of engagement where Mark can take the game, and I heard from him after he had built the privacy extension, so if you want to do similar things, it's worth thinking about serious games because this is a big field. It’s got its own conferences and a serious game is a game that has an explicit and defined educational purpose. The goal is not to play for pleasure, the guilt - you can, but the goal is present.

And there's all sorts of things - table top exercises, persuasive games, games for health. We see a lot of use of gamification - points, badges, leader boards to help motivate people and there's ups and downs to these things which I'm not going go into but I do want to give a shout out to a colleague at Microsoft who used gamification to help deliver Windows 7 so Windows is chipped in. I don't even know, I think it's now a hundred and thirty different languages. And QA in the translation from English into these other languages is a big expensive project. And so they gamified it. They said what we're going to do is present you the English text and the Hungarian text and then they reached out to our office to their office now in Hungary and said hey all of your people should look at these screens and tell us where the translations are good and where they're bad and it turns out that that work resulted in lower cost and better translation because of the use of a gamified structure.

So you can really think about games solving problems that you have - around engagement, around how do I get lots of people to do an activity that they might not understand really well. You know we don't want to just present them walls of text and so I think serious games are important to security.

In the final part of the transcript, Mark Vinkovits will talk about how and why he created the Privacy Extension to the EOP card game and his key motivations behind doing so.