Today’s episode, recorded live at RSA 2022, features a great conversation with Tomás Maldonado, NFL Chief Information Security Officer, and Brad Garnett, Director of Cisco Talos Incident Response. The dialogue is led by none other than the brilliant Tazin Khan.
Taz talks to Tomás about his early life and career (“I didn’t want to be another statistic”) and how he approaches new opportunities. He talks about how he communicates his vision for a cybersecurity strategy, as well as how he ‘blends the tracks’ between a technical and managerial style of leadership.
Tomás then goes into details about how he builds steering committees across the NFL so that people have a voting stake in technology and security decisions. Together with Brad, they discuss how Cisco and the NFL worked together to secure the most recent Super Bowl (“This sounds like a movie script”), and how they created a playbook based on learned threat intelligence, to proactively secure future major events.
For more details about the work Cisco has been doing to help secure the NFL, visit this blog https://blogs.cisco.com/security/nfl-teams-up-with-cisco-to-secure-super-bowl-lvi
So we are live from RSA 2022. And I am sitting in our meet and pod lounge with Cisco secure, alongside really two incredible people. I am your host Taz today normally co host Hazel We miss you always. But today I have the privilege and the honour to speak to a Yeah, privilege and honour. You hear that? A really great friend and also somebody that is very respected in the security industry and especially here at Cisco secure. Tomas Maldonado. How're you doing?
I'm doing all right. No pressure at all. Haha. Way to build that up. Thank you.
Yes. I didn't mean to put pressure and Brad, how are you doing today?
I am awesome.
Amazing. Amazing. So I'm, I'm super excited to be interviewing you, Tomas. I feel like you've been mentoring me anyway, for the past, like, seven years that I've known you. But we're gonna have the opportunity to talk about leadership, we're gonna have the opportunity to learn about your story, because that's what the security stories podcast is about. But tell us a little bit about yourself. Before we dive into the very deep existential questions, I'm gonna ask you,
man, it sounds like a loaded question. So Tomas Maldonado, obviously, as everybody knows, see, so for the NFL. I spent, I've been at the NFL for I'll say close to three years now. December 2019. will be three years. Okay. Little three Super Bowls under my belt.
Which fuse? Do you get the rings to? Of course, I've got a good old No. Should we we should make a security Super Bowl ring for you. Yeah,
I was actually joking with the old see. So that sat in the seat. I said, Hey, we should have like a C. So Hall of Fame. because there'll be two people in there that we know for sure. Yeah. No, but look, I've been I've been at the NFL for nearly close to three years now. It's been an awesome experience learning about a completely different industry that I've sort of grew up in, right in comparison to financial services. So 17 years in financial services, four and a half years chemical manufacturing, and now at the NFL protecting, you know, not only America's game, right, but the world's game, because it's one of the most viewed shows, or, or events, if you will, is speaking about the Superbowl is one of the most viewed events around the world. And so you know, I take that very seriously. And we obviously at the NFL take that very, very seriously. So I'm honoured to be sitting here with you both. And I'm honoured to have an opportunity and a platform to share my story. So that's, that's a very quick intro.
Yeah, you're gonna get it. This seems like you're doing a lot of press lately. No. And all of the things that you've highlighted, right, I think, brings me to my first question. So tell tell us about the moment I guess that you knew you wanted to pursue a career in security? Was there a moment or was it just something that kind of happened?
Yeah, it must have been around 1977. The year the blackout. I was one years old. And I knew always get, like I always say, I always say that I stumbled into security. And I really mean that right? I was a computer scientist by trade. I graduated from Fordham University in the Bronx, New York, with a computer science degree, but I never programmed outside of college, I actually did my thought process. And please don't hold this against me people. But you know, think about that time period, we rented a nerds movie was out my thought process around people that did sort of programming word are guys that have pocket protectors, you know? And so I didn't have a lot of role models. Let's just put it that way. Now that I've been in the field in history for so long, but my thought process was, I'm more of a people person. And I didn't want to be sort of in a in a room Gemini where a keyboard just pushing out code everyday. I thought that was what programmers did at the time. Right? Hindsight. They don't only do that there's some that do right, but most don't. And so the my last two years in the university, the Internet was starting to evolve. You know, I created the Internet. The Internet was starting to evolve. And I decided to take more networking classes. And I remember, which is, it's ironic that I'm speaking with Cisco, I remember there being at the time less than 100 CCI ease in the world. And I worked with I was, I was interning at a consulting company, there were two. And I remember thinking, wow, I want to do what that person does, because they're travelling the world and going to different countries to set up the internet for companies. Ecommerce was starting to evolve, right? Think about web one dot o if you will. And so I'll fast forward a little bit I got out of school took a first job at Bloomberg. I didn't know what I wanted to do. I was getting interviewed. They're asking me questions, what do you want to do, I knew I didn't want to do programming, I didn't really know how to explain or articulate that I wanted to do networking. And so I did what any other IT person at the time would do. I worked at Bloomberg and I did the help desk. And I was the first line Bloomberg tech support. And I learned very, very quickly, I want to say within like maybe a minute that I did not want to do to help this for too long. So I given that I was somewhat educated and networking, they promoted me to second level support, I did a lot of second level support. And I was on my way to being a network engineer, which is what I thought I decided to leave that row and go to another company. And remember it as if it was yesterday recruiter. So they told us I know you want to do networking. But I've got this network security officer role open up at this small investment banking firm, why don't you go in and see if you'd like to see what they say. So I went in, I interviewed loved it met with the CTO, they hired me, I joined they sent me on to do cybersecurity training on Iran, learn about TCP IP security, I learned a lot about firewalls, I got firewall certified, not Cisco firewall. But to get firewall certified, I learned a lot about intrusion detection systems. And I learned how to hack and what, you know, this kid from the Bronx, where my family, you know, my dad was a cop, my family are cops, my goal in life was not to become a statistic and end up in prison, I found that being able to take control over a computer was very interesting and empowering. And so I stumbled into security, because I really did that set off my career, you know, I spent a large majority of my career in financial services, just exercising that skill of being able to take over a computer. And then I started to grow past that, and being able to assess different types of technologies, assess companies as we acquire them, and then start to build programmes. And, you know, somebody decided to hire me as a CFO. And, and, and I've been able to run programmes for very big, multinational global companies. And so, you know, that's sort of my my story in a nutshell,
that I remember you telling me about the Bloomberg experience, or at least your beginnings of security, but hearing you talk about your family, and the way that you were empowered, right to take over a computer and not be a statistic. I mean, all of that is very much vision, it was a visionary moment for you, right to be able to see that and then implement it into your life, which is really awesome.
At the time, it wasn't sort of visionary, it was more of like, hey, I want to get out of my mom's house. Yeah, you know, I want to make a few more dollars in my pocket, because I had to pay for student loans, you know, so at the time, it really wasn't visionary. But when I do look back, you know, there have been those pivotal moments in my career where I've made decisions to leave a company because it wasn't the right opportunity. Or I've made decisions to pursue one track because I saw that as being an opportunity. And maybe I didn't think about it so fully, like, you know, like, you'll say, Well, what's your 10 year plan? Yeah. And I saw that the internet was starting to evolve in a way that I used to do a lot of web design as well, right? Computer science background. And so I saw that very early on, you could do things like, you know, modifying a field in a text box and have a company send you money versus you actually pay for a product. So what that would, what that allowed me to do was figure out, this is not going anywhere. And a lot of people around that time period were like, what's this email thing? And why are you emailing me send me a fax, I said you need to catch on to this technology. But because this is not going anywhere, this is where we will be. And so I stumbled into probably the best future, which is cybersecurity because we do cybersecurity professionals are engaged in every aspect of the business. And if you're not engage in a cyber secure, professional aspect of your business, chances are your business is not going to do very well because digitalization and the digital aspects of what we do is here to stay.
I couldn't agree more. And so I guess with that, how would you say that you come up with a your strategic vision for security, right, and what are some of the key elements for that vision? And does it have anything to do with your personal narrative and your journey coming into the industry and do you see Are you that bleeding into your work today?
I do. So I'm very curious about aspects of a business that, you know, I'm very curious about, about how we actually do business. Right. And what I mean by that is, I grew up right, in a and I educated in a very technical way. And, you know, one of the things that that lacks in academia is that is those courses around business, and how does the business opportunity operate? How do they run and things like that? So I'm very curious, you know, wherever I work, how do we do? How do we make money? Right? And the reason why I'm very curious about that, because if I can figure out how does the company actually become profitable or gain revenue? I can also start to figure out what what am I actually protecting? And so when I think about a vision for a security programme, it all centres for me at least all around, where's the west of business objectives? And where are they where's the business trying to actually get to, you know, and grow, whether it's different country where the different product lines, whether it's different sort of revenue targets, and I try to apply my security programme, to helping and enabling the business, make better decisions, take more risk, right? quantifiable risk, but under an understandable risk, but trying to allow them to fully appreciate where they're trying to get to and the cybersecurity impact of that, so that they can then try to up or down or make more investments. So you know, we can do reverse management. So that's, that's usually how I try to evolve my vision around around a security programme, and I tried to adapt it to whatever company I'm working for. So a lot of conversations, a lot of listening to very smart and senior people, trying to get a perspective from them around what a cybersecurity means to them, and what keeps them up at night. And then just wrapping a programme around trying to hit those marks, and then also trying to look at everything else that we have to worry about, which is you know, what next regulation is out there, what latest adversaries trying to attack us? Or what could potentially impact us. Because it's easy to solve security problems, I have a very simple solution. Disconnect the internet. Problem solved. But yeah, companies won't make money. So it's all risk management. So that's how I do it.
Okay. And I, you touched on this a little bit already. But I do want to ask, Was it a natural transition for you to go for from being somebody very technical, somebody that is breaking, you know, empowering themselves to take over computer using email being an early adopter of technology? And like you mentioned before, that you didn't have a lot of role models for programmers, right? Like back in the day, nerds weren't as cool as they are today. No, nerds are definitely awesome. And so did the alignment to business and the way that you actually are able to speak to C level executives and highlight the importance in simple language enough that right people outside of the scope of our industry can understand. What was that journey like for you? Was there a large? Was there a latency period for you? Is that something that came naturally? I know, you said you were doing a lot of listening and learning. But was there a mentor in your life that kind of influenced the way that you approach security from a business perspective? Yeah,
no, no, there were there were definitely mentors, you know, whether they knew it or not. For more formal mentors, it was not an easy journey. Right? It took me it took me a while to really figure out that, you know, yeah, it's fun being able to have this technical skill set to be able to do things right. You know, the time period that I was sort of more hands on was Wi Fi was still was starting to evolve. Voice over IP was starting to come around and people were starting to use it, you know, and I was the I was the security person that had to go and assess those technologies and figure out well, how do we actually do this securely? How do we transfer information to these mediums, and make sure that we're that nobody else can break into it. And I enjoyed that I enjoyed hacking those things. I had a laptop, that dual booted right into Linux and Windows, I enjoyed that. But while I was at Goldman Sachs, you know, I enjoyed that for I'll say for like, maybe my first three or four years of my career there. And then I realised that I didn't want to be the only go to person for very specific, I'll call it niche or very specific projects. I wanted to be responsible for initiatives, I wanted to be able to drive programmes and drive initiatives. And then, you know, as I was, I was I spent 11 years at Goldman Sachs. So as I was progressing through my career at Goldman, I started to realise I started to realise those specific aspects, and it allowed me to change my approach to the initiatives that we were working on. And that approach was, you don't always have to be that smart person is probably good. So try to learn more. about what the business is actually trying to do, and try to translate to the business. How does technology impact. And so, you know, I spent a lot of time at Goldman growing up in that in that environment and growing my call my leadership style, my leadership ability, but it wasn't really until I went to JPMorgan Chase that actually had an impact on an individual's life. Right, that's when you switch from being an individual contributor, contributor, driving very big initiatives, right, to actually having to do performance reviews for people, you know, and being, you know, somebody's psychologists, if you will, right, when they have a bad day, or being a, a role model to them when they want to be you in the future. And I realised that, you know, it was great doing the technical stuff, but it's so much more fulfilling to be able to teach and educate and help people along on their journey. And it was so much more gratifying for me. And so, you know, that opportunity while I was at Chase allowed me and afforded me to exercise and grow in that capacity. And it wasn't honestly, it wasn't until I sort of left Chase. And I left the sort of it construct of work that within JPMorgan Chase that I went to international flavours and fragrances, we actually had to work for General Counsel to my boss was the GC at IFF. And she's amazing. And, you know, I'm not gonna say everybody do this journey, but it really helped me fine tune how I would articulate cybersecurity to a non cybersecurity person. And that really helped me be able to explain in regular practical terms, what is risk, what is risk management, I would use things like, you know, you have a house, you've got cameras, you've got, you know, doors, locks, things like that you're walking across the street, you're doing risk management every day. So I would try to keep it very simple and relatable. And that really helped me as I was growing, and my career presented to the Board of Directors. And, you know, it's helped me and my current role. So that journey, and that trajectory was really points in time, and spas, very specific initiatives, throughout that sort of path, that I haven't really gone into too much, but very specific initiatives that help pave it helped me pivot my style. And, you know, the fact is, you, the people always say you have to have two tracks, you want to manage the track, or you want to do the technical track, I don't think there's really two tracks, I think you can still blend the tracks. But at the time there was that was the sort of deciding factor. And I said, I thought to myself, well, it's great being a technology track, but I actually want to be able to manage people, so I gotta go to manage them out. And I found and I started to seek opportunities to help me fine tune those skills. And obviously, I took training, yeah, I took a lot of, you know, personal type of education and training, to be able to help me progress and grow in that area that I didn't learn while I was in, in academics.
I'm so happy that you just shared that you took training to become a better manager, because I think that that was something that we talked about this morning. about leadership. Yeah, it's okay about leadership around. A lot of times, especially in the security industry, something I say often is that right, technical people are incredible. And they're, they're really great at what they do. And, however, not always great at people management or innate innately. You know, sometimes, there are many skills that we all have to learn, as opposed to have an intangible quality that exists within us. And I think that at times, and often when we're kind of naturally moving into leadership positions, and something that I personally experienced with some leaders that I've had in the past is that just because there was a natural progression in your career does not mean that you don't need to train or learn to become a better leader. And I find that fast, like, just really, like humbling and fascinating that you did take the training, and I'm so glad you're sharing. So I hope that people that are listening, if you're a manager, if you're looking to be some sort of a people leader, training is not bad. It does not mean that you are not capable, it means that there's always room to be better.
Absolutely, yeah, absolutely. And you have to listen to people, right. And mentors are very, very important. I've had bad managers throughout my career. I'm not going to name them when I go because they're still in the industry. Yeah. But I've had horrible managers, like, they were really smart. But they were bad people, managers. And so, you know, I you start to see those types of characters that you work for characteristics and people that you work for. And you try to model the things that work for you that you see work for you and you try to leave all the other stuff to the wayside, right. Yeah. But yeah, absolutely. Only you have to be a continuous learner.
So then my next question, we've talked about leader, your journey leadership, we spoke a little bit about your strategic visioning. But I guess how, how would you because I feel like I, you said it, you translate things into simple language. But is there any other way that you think you personally inspire innovation related to risk based thinking? Because I know that compliance can be very rigid, right? So how are you bringing the creative sauce into risk based thinking?
Wow, that's a that's a, that's a really interesting question.
You know, I'm full of, I gotta put a sheet right
on spot there. But look, I will say, having, and I don't want people to listen to this and take this the wrong way. But you know, 17 years of financial services, very regulated, very sort of rigid, and very, you know, structured, if you will, leaving financial services. So maybe not even leaving, just having the opportunity to work in a different industry has allowed me to open up my creativity based upon the industry that I've been able to work in, right. So, again, highly regulated industries, there's maybe not a lot of room for innovation from a cyber standpoint. But as I've moved out into different opportunities, it's allowed me to then bring a little bit more of the creative aspect of information security, to the table to be able to meet the business demands of where the business is trying to actually head to. So I'll say, you know, things like, you know, working in a chemical manufacturing company, it allowed me to allow not only me, but even the company to start to go down the path of leveraging blockchain for the supply chain. So being able to leverage a cybersec I'll call it another cybersecurity solution, but a more technical, security based type of a solution to help innovate within the manufacturing environment to be able to be able to, to be able to handle things like recalling a product and figuring out where that product was made, right where it down to the farmer of where it was collected from, right. So, you know, I like the I like the fact that I'm forcing myself to learn more about the business and where they're trying to get to. And then the, obviously, a lot of this comes with years of experience of being able to adapt those years of experience in information security, and then look at the different industries that I've worked in, I can almost cherry pick different solutions that might work in a different area. So NFL very creative company. We, we do things that impact people's lives every day, the prior company I worked at it was that was their motto, right? You want to get some emotion out of somebody. So I tried to adapt information security in that business context, and tried to try to just take that creativity to a different level. Think outside of that box, if you want to use that cliche.
Yeah, I love that. And I definitely we're gonna get into all of the technical stuff with Brad, because that's why my man, Brad is here today. And I have questions for the both of you. I do have a few more leadership and human related questions for you. And specifically, I guess, how would you describe your leadership style? In both how you aim to get the best out of your people on your team, but also leading folks within the NFL to adopt perhaps security within entertainment retail industry that perhaps didn't have security front of mind?
Yeah, so I like to say that my leadership style is very sort of, I'll call it consensus driven, if you will, right. It's it's an it's a way of approaching a problem, not only as it being my problem, but our problem, and how are we going to get that together? Right. I usually say, you know, security is not only my job is the job of everybody in the company, everybody in the room, you know, for lack of a better example. So with that mindset, the one of the first few things that I do as I start to develop a security programme is I try to meet with senior leaders within the organisation to do a lot of listening, understand where they're going from a business standpoint, but also start to put out a campaign along the lines of like, we would like to create a steering committee made up of senior people, like you'd be able to nominate somebody from your for to represent you and have a voting stake at the table for things that are going to impact you. What is policy, what is new technology solutions or security solutions? What are its budget that we're going to ask for, you know, and the reason for that is because, again, it's not only my problem, it's everybody's problem. We get if we get people involved in that journey, we build that consensus upfront, then we're able to lead effectively and we're able to continue to evolve in our practice of assets. As security practitioners, so my style from from a programme standpoint, and from a people standpoint is that's very similar model, I have an open door policy with my staff, I meet with, with my directs every week, individually for an hour, and I meet with them as a group for an hour as well. And I meet with my broader staff every two weeks, and we try to have open dialogue around, you know, big ticket items, things that are impacting us things that are impacting the company. I'm Stern, you know, so I'm not gonna, I'm not gonna say that I'm not Stern, this things that need to get done a certain are very specific way. Because, you know, we deal with crisis all the time, you know, at specific times. And when you have an incident, there's a specific sort of flow to that that needs to occur. But other areas, I try to allow my team to be creative, and come up with ideas so that I'm not the only one, you know, leading them, they're leading me,
I love I love that so much. You're not the one leading them, they're leading you. And it sounds like at least from what I'm hearing your leadership style, a part of it consensus getting everybody on board with you. And you're like inspiring people, you're inspiring people to take initiative, you're inspiring people to be more involved. And I want to say that there's a way to teach people that but I feel like that's kind of an intangible quality, that people kind of some people are inspiring and know how to be. And I think other people probably takes a little bit more work to get there. But it's, I see it translating. It's translating to me right now, I feel very inspired. I will say that I was
just trying to trying to get people to that next level in their career and ultimately have them, allow them or for them the opportunity to take my job at one point in time because I love to continue to grow in my career. I'm still growing, I'm not ready to retire yet. But you know, there might be something else that I can do. That's not only security related, it might be business related, right? Yes.
Okay. Tell me a little bit about that. What what might Tomas Maldonado outside of the security industry look like? And I'll end it with that that will be the last leadership oriented question or human oriented question. And then we're going to jump in with with Brad after that. Yeah, who are you outside of security? I guess as far as aspirations are concerned,
well, outside of security, I'm obviously I'm a husband, I'm a father, I'm a I'm a brother, I'm a son. So first and foremost, that's, that's the family that core sort of, you know, the core to my, to my being, if you will. But I'm also you know, I like to talk as you can tell from this recording, I haven't, there's no shortage of words, you know, that come out of my mouth. And sometimes I don't even know what I'm gonna say at times. But, you know, I actually I like to talk with very smart and intelligent people. What are those people who are new entrepreneurs? What are those people are people starting out their careers in security, what those individuals are, you know, future CISOs I like to have those that type of conversation with them. And you know, I don't think of it as as inspiring and I appreciate those kind words, I just thinking of as just sharing what I've been able to learn from others, with with people that I see that, that have an interest in it, right? People that are closed off, you know, there's probably no point in having a conversation. But if you're open to having a conversation, I'm opening the show to sharing my knowledge. So outside of outside of doing my day to day, which I feel like that's very few and far between, because I'm always doing my day to day. Yeah, I'm trying to meet with up and coming entrepreneurs in the security space, trying to shape the community. You know, I host a weekly sort of pocket podcast of sorts, you know, as my way of sort of giving back to the community, you know, so those are the types of things that I like that, that keep me motivated. And keep me still engaged in this in this security community.
Yeah. Giving back staying creative and innovating. Right. In the radio. Yeah, anything for sure. Okay. So that also, I just want to say this before we move in is that it makes me appreciate this dialogue so much more to and just this podcast, the security stories podcast, because while we are at Cisco, a security organisation, I think this podcast allows us to dig deep with the people we work with on a human level, which I think allows us to as an organisation, make better business decisions, because we're learning about people's narratives. And speaking of business decisions, and I want to talk about the details and all of the bits around Cisco secure, and the Superbowl and the work that we're doing. And Brad, I want to, I want everyone to know about the work that you do who you are. So why don't you take a second tell the world who you are, and a little bit about yourself, the team you're on and then we'll jump into some more questions.
Awesome. Well, happy to be here and I love Tomas, his story. So Brad Garnett, I lead the Cisco Talos incident response team and a lot like Tomas, my journey and security. You know, I spent over a decade in law enforcement became the computer guy in house, moved into high tech crime investigations, spent time on a mini Task Force back in the day and kind of journey made my journey into forensics network intrusion. From there. So outside of my work day to day leading our global incident response team. I'm a dad, I'm a husband, and I'm also a football coach. And that's one of the things that I love. And like I say, My motto is incident response really, is that ultimate team sport kind of bringing all aspects all capabilities of an organisation together, focused on the mission at hand. And so yeah, happy to be here.
Yeah, this is exciting. Also, while you were talking about forensic, your work, like the forensic intelligence, and that I had the Law and Order SVU theme song playing in the back of my head. Yeah, that was really great. I watched a lot a lot in order, guys. So anyway, so tell us about I guess, and the both of you can answer this. But you your first experience when it came to collaborating with Cisco secure, right, like, what were you looking for in a security partner? And then Brad, I'd love to hear how you were able to complement that. And work with one another?
Yeah, so for me, I want to say after after, or just say Tampa Superbowl, Superbowl 50. I can't remember the actual number. I don't know why I can remember now. But how many rings 55. Thank you, after Superbowl 55 in Tampa, I remember as if it was yesterday, I was I was talking to after the Super Bowl. There's always a Super Bowl wrap party for staff. And they could actually play back the Super Bowl. Because since we're working on securing the Super Bowl, it's hard for us to actually watch the game and a halftime show and all that stuff. So we're there we're having a rap party, and we're sort of going into that, to those sort of motions and eating is a lot of us don't actually get to eat. And we don't we can't drink anything adult beverage like, you know, I was talking to my to my boss, Cathy Lanier and I said, you know, for next year's Super Bowl, I have this idea of potentially bringing in a solution that we can sort of put in a call it a wrapping around the venue to get more visibility and telemetry data around what's actually occurring, who's connecting in what are types of attacks look like in our tax surface. And we had something already within Super Bowl 55. But we're always trying to innovate and or iterate and innovate on solutions. And so that was my idea. I said, I need to find a partner. You know what I said, I was making a pitch to say, hey, I need some budget for to do this. But I was also making a pitch of like, I need to find a partner that can help us accomplish these specific goals that I have. And she's you know, she said, sounds great. Go out and do it. Right. So then after Superbowl 55, you know, you might say, well, Tomas, when do you actually start planning for Superbowl 56? But we actually start like maybe a month later, right? You go and I say we've been the security department, the US department store like years in advance, right? Cuz you've already selected the venue. So we started to go down the path of explore really exploring that idea that I had. And I chatted with my team. And I said, What do you guys think if we did this, who can help us do this, and we put out an RFP to a few vendors. I'm not going to say we just automatically selected Cisco, but we did put out an RFP, and Cisco were, I'll say they were the ones that listen to us the most. I love that they were the ones that that actually heard the problem that we were trying to solve, and created and came with a very creative solution that will not only meet our expectations, but also meet our budgetary requirements that we have, right, because we can't spend millions of dollars to try to secure the actual event. I mean, from a cyber standpoint, because we need to iterate in a at a very measured and calculated pace, right, again, is risk management. So, you know, after we after we were able to identify identify Cisco to help us with that solution, we then started to iterate on that I want to say took us months, I'm not gonna say we actually got it down within the first year time, but it took us months. And it was those months, months of having conversation and Cisco coming back to us with Hey, how about this? What if we did this? What if we added this? What would you like to sort of do we took the approach of and I should have probably said this we took the approach of for Super before SoFi for Super Bowl in LA we want to not be too disruptive in the solution that we put out from a cyber standpoint. And the reason for that was because it was gonna be the first time that we did this okay. And you don't want cyber to impact the game right? I mean, it's you know, call it that career suicide.
Hello, tell what cyber to impacting the game. Right. Right. Right.
So, so we worked you know, we we worked very effectively with Cisco to come up with a solution that made sense, not only from a technology standpoint, but process wise people, how many people we actually needed on site? How does that fold into my team and Cisco was essentially an extension of my security operations team on game day. And then we also worked through what were their sort of boundaries, what we were going to do and what we were not going to do, which was probably much more of a very detailed conversation. Because of the nature of an approach that we took, try not to be too disruptive in this first, sort of go live, if you will. Since then, we've secured, we secured Super Bowl, which was again, very, very awesome. Did you see that that cyber incident that happened during the game? No, it did not happen. That's why you didn't see Yeah, no cyber,
I did not we secured the heck out of that exactly, we did.
And then we secure we managed to also work on on this very similar design, and a slightly scaled back, I'll say scale back in terms of we didn't actually have people on site. But we did have a very similar design, very same processes, same service offerings, with Cisco secure, and tallis and secure x. And the extra equipment to help us security NFL draft, which was, you know, arguably, in my opinion, is actually one of the harder events to secure than Super Bowl. And the reason for that the reason why I say that is because the draft is three days. So an adversary has three days to try to iterate their attacks and try to get in to disrupt the draft versus Superbowl, you're really confined to that set of hours that the game is being played. Once the game is over, we wrap up, we close up shop it, stuff is done, the broadcast still happens and people are sort of you know, confetti and everything. But from a cyber standpoint, we shut it all off. And we move on to the next next venue.
Wow, that so many gems that were just laid out. But Brad on your end. So that was Tomas his perspective, right, of how it went down. I'd love to learn from your perspective, perhaps what that process looked like and how we leveraged our team to get together and aligned to what Tomas and the team were looking for.
It was like a swan. Right. So So from my perspective, it's like you're watching
recording this right? We gotta make sure we get these quotes on the last one that is Cisco secure. Your swan.
It was a swan. Right? It was very smooth. That was what we were able to portray. And below the scenes, there was like hundreds of people but go ahead. Right.
Yeah, no, I mean, Tomas is point the really the months ahead, really leading into it, right. I mean, just like everything, you know, we under, you know, initial planning, understanding scale, understanding scope, things kind of changing, you know, parties kind of, you know, involved in it. First thing, how many different sites are we securing? Right, what are we looking at? What is our security instrumentation going to look like, from an incident response perspective? What, what's the escalation procedures working directly with the, with the sock, you know, building an incident response plan. I mean, we had a very solid plan of action that we put in place kind of ahead of time, right? I mean, train how you fight, because you'll find how you try and building that playbook ahead of time, understanding the escalations, right, that's one of the things that, you know, from an incident response standpoint, that I, that we often see is sometimes people fail to escalate. But again, table topic, all those different scenarios. When I'm at the scoreboard goes down, what happens, you know, if the power goes out, you know, all those different aspects having a plan. So we've built that trust, ahead of time was key, I think, kind of the lightbulb moment for me, Tomas is was there's no, I think eight minutes left to go in the game, fourth quarter, and Tomas gives our kind of the green flag to kind of beat the traffic and I'm thinking, here's myself sitting on WebEx from only thinking, Oh, my goodness, you know, we've got eight minutes to go in the game. And we're already kind of letting our folks get out. But that really, I think, speaks to the trust and then understanding the capabilities, whether it was inside of SoFi or remotely, Tomas knew that he had, you know, Cisco behind me, but also, all the other, you know, the law enforcement partners as being a tier one event, the other intelligence partners that were involved. So, I mean, I, you know, leading up to the big game, I know, we had a ransomware scare kind of on that Friday kind of leading up, not directly, you know, at so far, but we were on a heightened state of alert kind of going into going into game day on Sunday on that Sunday, Super Sunday. Yeah,
I feel like I'm in a movie, an action film, or a heist film of some sort. And I'm just imagining all of you at the Superbowl like, Come on, guys. We got to make you know, like underground.
Stand up like that every
quarter. It was it was actually like that. It was very intense like that. Yeah.
Did you all do security as well?
Well, I'll just kind of echo that. That's one of the things that I really enjoyed how I'll Tomas and really the league and his team approach security. You know, just like in it, we talked about that convergence between OT and it. How, you know, Tomas and his team handles security, the physical security folks know what's going on from a cyber standpoint. So threats are people the other day, it doesn't matter if it's cyber, you know, fans, you know, possibly in the stadium, right, I mean, really treating security and threats as a singular, you know, approach. And I think that's, I mean, that's, that's, that was key, I think on game day. Yeah. And
from a from an NFL perspective, does a control room at the at the stadium where it has very senior folks that are essentially putting on the show. So you broadcast your events, you're planning, local law enforcement, and I was my boss is there from a physical security standpoint. So we have all little senior people within that control room so that if something does happen, everything goes up to control and it fell control manages the incident, and then it goes back down to the folks on the ground that have to make to have to execute on the decisions that were made from NFL control. So my job in NFL control was to not only manage what's happening, and from a cyber standpoint, my team's job that are working with Cisco, and trying to say defuse different situations that may pop up, but they'll escalate big ticket items for me to then have a conversation around what do we want to do. And if there is something that will impact will has the possibility of impacting physical security or any other aspect of the of the user experience as you're going and coming into the stadium? My job is to communicate that effectively to the other leaders and so that we can make a decision as to how we can handle that going forward. So very, very, very, very well coordinated. Yeah, it sounds like it really does. Like we like ties, we actually have like earpieces, we cool. See,
I'm literally envisioning all of this right now. I wish I was there. I wish I was a part of it. I am also obsessed with red teaming and offensive security strategies. Yeah. So I do want to ask this and set the lay of the land because we have a very diverse audience that listens to us. And something that we didn't talk about, though, is like, what would a worst case scenario look like? Like, let's think about Super Bowl bad actor dystopia? What would be some of the worst things both from your perspective, right? As a team managing, and then you as the CISO? Like, what does a true bad actor look like? And how would that really affect game day? I know how bad this can get. But tell me I want to see how creative you can get. So
I'm not gonna get too creative, because there are people that will listen to this. Yeah, you're right, and get overly creative. And I still have a few more Super Bowls to secure during my career. Yes, but But I will, I will say, you know, a truly bad day is that a fan? Does not have their team win the Superbowl, you know, maybe maybe that's a bad day. No, look, diplomatic answer. A bad day would be something that starts and originates as a cyber incident and may potentially impact health and safety. Which could ultimately impact obviously, the game being played, or have the referee stop the whistle. So our goal from from security in general is to ensure that nothing from a security standpoint, whether it's physical or cyber stops the whistle being blown, so that it stops the game of play, but also that it impacts your experience as you're interacting in the in and around the stadium. You know, you're going about your day, right? So those will be bad days, I don't want to I really I honestly don't want to give too.
I really I realised that I can appreciate that very much. So we'll keep our secrets to ourselves and our imaginations to ourselves, however, I guess as we as we look forward, because now we have a Super Bowl under our belt, right? But how how are you both still working together as organisations? How are we still working together to build repeatable success, right, a repeatable network security playbook that we can just regularly deploy? And is this something that is reaching beyond just the headquarters at the NFL? Are you working with? Because from what I remember, the NFL is siloed. Right? Like it's franchise? So does that impact security? I just asked you two questions at once.
Yeah, so I'll say I'll let you answer in a second. But one of the outputs of our first engagement with Cisco was to document how we actually set up the design so that we could actually be make it repeatable for other other locations, right. So so that output we've used that effectively, we've secured the draft and we're looking at leveraging the technology potentially for our international series games. It really depends on on what the footprint of it looks like NFL it and not nfo sort of corporate headquarters it but the IT network that's established for our staff for our vendors and our partners to connect into. In other words, what am I actually protecting when we actually put put onto the when we actually go to the venue? So it depends heavily on on that footprint around what we would actually implement from from the Cisco standpoint, but we have a playbook I already have, we have the playbook sort of documented. Now what we're doing is we're actually being more proactive, with our call blocking and tackling of incident. So, again, first Superbowl, we wanted to see prove out the technology throughout the design, do we have the right visibility, can we react to things we did block a certain amount of stuff before actual game day, but we try to do, we try to institute a change free so that way, we can normalise and baseline what traffic looks like, right? As we go to Arizona, which is the next Super Bowl next year, or this coming season will be more proactive and actually start to block a lot more things that we see before they actually make their way into the environment. So the you know, we're versus us what we were doing it doing so far, we were doing more risk management is this. Alright, we see that? Is that really an issue that we need to worry about? In other words, is it impacting something, some parts of the game, if it's not, alright, leave that alone until it becomes an issue that we need to really worry about. But keep an eye on it, versus our next period is going to be? Let's Let's block all of these things that we know should not be occurring, again to reduce the footprint or vector of attack in our space.
Okay. Yeah, Brad?
Yeah, I know what I was just gonna say, just to kind of echo what Tomas said, you know, really taking the lessons learned. I mean, that's I'm a huge fan of, you know, taking lessons learn those after action reviews, what did we learn from, you know, this event, our front, you know, incidents are going to happen? What did we learn? So, from this perspective, right, taking some of the things like Tomas said, what we learned baking that into the playbook going forward. So those business decisions that you have to make, regardless of the event, is it the Superbowl? Is it the draft, you know, some type of event that you know what we've seen that before? Let this is what we're going to do based upon what we're seeing, and having that instrumentation and that visibility. And again, understanding I mean, that's that's key.
And we carried that forward, right? We did. We did a we call it so we do it after action report, where we also did a hot wash with the incumbent stadium, what's hot, what's a hot hot hot wash. So it's a hot wash is a it's basically a lessons learned with the new state with the upcoming stadium that's going to host the Super Bowl, we had a meeting with Cisco and a lot of our partners that helped us secure. So far, we had that sort of exchange of information with the with Arizona, and a lot of the folks that are going to help us secure Arizona. So we that's baked into our process to continue to iterate as, as Brad said, right to work through the lessons learned. Yeah,
yeah, absolutely. Yeah. hot wash, sharing successes, opportunities, you know, bringing all parties together and kind of laying everything out there. For me a perspective, everyone's different perspective, right? Because we all look at things we all experience things in the world, the paradigm a little differently. So being able to share collectively how we experience things again, to make to make the situation and prepare for the next of it.
Yeah. And how does perhaps the Talos, intel on demand service kind of tie into all of this?
Yeah, absolutely. Right. So the telephone tell us on demand is one of our more popular proactive services that we have in our Cisco TELUS Incident Response retainer. So, you know, I mean, today, right? I mean, you're just like leading up to the big Super Sunday. organisations don't need another threat feed they need that really, that's trying to understand really, what's what are the threats? Were these specific actors? Doing? So being able to answer a very specific question with our Threat Intelligence. That's, that's key. And our intel on demand service allows us to be able to take those specific questions or things right regarding threats, malware research topics, to be able to help customers on their risk management journey on their resiliency journey to be able to make informed business decisions, you know, at the end of the day.
So we're coming close to the end of our time together. Oh, I know, one five, so quickly. I do have another question. And this one is for you, Tomas. I'd say how and for you, Brad, I feel like this is important to to identify, especially from a vendor perspective, right of like, how would you both summarise the key areas of success from this relationship? Right? And well, both in terms of the relationship with Cisco, but also what's truly making it work, right to be able to share with our audience that this is best practice and engagement like this. And are Are there any key things that you think has allowed now from that relationship that now has allowed the NFL to find more resilience in their security posture.
So I'll say, you know, defining success to me is not only being successful on game day and having a successful outcome, yes, that's probably the clear objective. But it's also that ongoing relationship where we're constantly having dialogue and continue to iterate on what we've actually implemented. And continue to use opportunities to, to leverage either more technology or more service offerings from that will help us enrich the the the controls and the information that we're getting, we're able to get out of the solutions that we put in place, but also ensure that we are being very effective and efficient with, you know, managing our budget, because we still have to manage a budget to implement security controls. So as we continue to grow our with our partnership and our relationship, it's, you know, it's really Success for me is, you know, what, you know, Cisco knows our roadmap for the next three years, right, in terms of locations that we'll be playing in, you know, from from a state and venue standpoint, you know, with that in mind, we should be thinking, Yes, we should be thinking about and focus on Arizona, but we should also be thinking about, you know, the International Series games and the next location after Arizona, right, wherever that might be. I can't remember if that's public or not. So I won't say. But it is a funny put out in case it is a fun place. I'm sure everybody's gonna have fun. But it's really looking at that roadmap for us, and helping us continue to grow based upon where Cisco is going with their service offering, what can we potentially take advantage of? Maybe not for Arizona, but for, you know, for the next location? I almost said it.
Yeah. Brad, from a success perspective, and yeah,
no one, you know, I'm going back to thinking back to game day in our for the Superbowl Tomas in our briefing. And you know, when you were kind of kicking things off talking about, well, it's kind of an annual performance review for all of us. And, you know, like, for me, okay, the flyover is occurs in an hour. And it's like, Alright, I know, from a Cisco perspective, I know what are holding statements are going to look like Monday morning after the game, you know, but then the back of my mind, but what happens, you know, if there's an attack or something like that, so I, you know, the transparency, I think the collaboration that we've built from Cisco, and the NFL has been key, you know, the, the candid feedback that we get, like improving things, I mean, that's really been key, being able to innovate kind of on the fly, we were on like a WebEx bot during the Superbowl, and shout out to our technical Alliance team. So we could feed specific alerts into WebEx that so I think being able to innovate and meet the demand and have a very changing environment, you know, is key at the end of the day, being able to map the requirements of the organisation for the NFL, and then how we deliver upon that and be able to support them, you know, on the on the biggest stage, right? I mean, it's, you know, yeah, the Superbowl, you know, the world's biggest stage, but then also globally, as well as these other events and venues. So I'm, you know, happy to be here and the relationship that we've been able to build and, you know, really grow, and also the same thing, it's really helped us as well. I mean, the feedback from my team, in particular has been awesome. So
yeah, this is, this was one of the funnest podcast interviews I have done. And I'm super thankful. I feel like this is a full circle moment in our worlds right up past life to a now life. And just hearing some of the key elements that I personally as a non technical individual, I'm going to leave this conversation with is that it's really about innovation. It's about collaboration, and it's about proactiveness and how you're approaching your tabletop exercises or any kind of risk assessment that you're doing. And in order to make that happen, you have to inspire people in your dialogue through simple language. So those are my takeaways. And I hope that everybody listening in this room, our live audience, enjoyed this dialogue and conversation. I am once again your host of today but co host normally Taz and this is another episode of the security stories podcast live here at RSA. Thank you, everybody.