Every breach is but a surprise. No CISO expects to be breached next week, and that expectation is precisely the problem, because a breach is almost always the result of something the organization got wrong about the adversary or about itself.
Jason Rivera began as a US Army intelligence officer, moving into offensive cyber operations at US Cyber Command before taking that experience into the private sector. At Deloitte he built threat intelligence and security operations programmes for Fortune 500 companies and government agencies. At CrowdStrike he ran intelligence solution engineering alongside the threat hunting and vulnerability management products. He is now Field CISO at SimSpace, where he works with governments and large enterprises on whether their teams and their tools actually hold up when tested under realistic conditions.
This is the full conversation behind an issue of InfoSec Leadership, the newsletter from InfoSec Relations. Hosted by Samarpita Pattnaik, a volunteer host at InfoSec Relations.
Watch the full conversation below or read the interview.
SimSpace did not sponsor this content
The transcript below has been lightly edited for clarity.
Take us through how you got from the Army to where you are now.
Rivera: When I started my career in the Army, I was actually doing traditional intelligence. So think of things like counterterrorism, or understanding foreign adversarial systems. Towards the latter half of my Army career, I ended up being stationed within US Cyber Command, where I did intelligence support for various cyber operations, and that's how I got into the offensive side. I was in something called a combat mission team, where we support different combatant commands and create cyber effects in order to support combatant command objectives. Things that might control adversarial maritime systems, air defence, planes. We want to be able to affect the systems, and so that's how I got my start, very offensive-minded. I then went to Deloitte, where I took all of that offensive experience and applied it from a defensive standpoint. I had the opportunity to build threat intelligence programmes and SOC programmes for several amazing Fortune 500 organisations as well as government agencies. I got to build out threat intel programmes for some of the largest government agencies in the world, particularly within the law enforcement space, but also worked in banking, telecom, and medical. I got a very wide variety of the different problems that different sectors experience. From there I went to CrowdStrike, where I ran their intelligence solution engineering team and was also responsible for the threat hunting and vulnerability management products. Since then I've been focused on what I'm doing at SimSpace, where I'm trying to figure out how do we simulate things to work out what might happen in the future. Because we're seeing the adversary very effectively using AI to increase their speed, to increase the scale and volume of attacks, and to increase the sophistication of attacks. And they're doing so not at a linear rate, but at an exponential rate. So one of the concepts we're exploring is how do we create training, testing, and validation scenarios for these different AI agents on the defensive standpoint. How do we accelerate our own defensive capabilities in the same way the adversary is accelerating their offensive capabilities.
You spent years running offensive operations against nation-state targets before you moved to the defensive side. What was the first thing you noticed about how organisations protect themselves that looked completely different from how you had been trained to attack them?
Rivera: Everybody's different. The way that you protect yourself depends not only on your skill and your budget, how many people can I have, what can I afford, how good are these people. Of course those are factors. But it also depends a lot on the critical assets you are defending. This idea of critical assets, or what some people call crown jewels, really is the foundation of your defence. If you're a hospital, then the number one thing that must happen is MRI machines, dialysis machines, X-rays. The things keeping people alive, those systems must not be compromised. Private healthcare information must not be compromised. So for a hospital, they're focused on those things. Whereas an oil and gas company or an electrical power company has a very different focus. Now we're talking about ICS and SCADA systems, programmable logic controllers, the things that provide power, the things that provide water. That is a different protection strategy. Whereas a government entity might be focused on national security information, classified information, and that in itself implies a different protection strategy. So from an offensive mindset, what I realised is that no one target is the same, and therefore no one attack strategy must be the same. The attacks that work are the ones that are thoughtful and well-planned, and the ones that are highly targeted and highly bespoke to the target. I think that's one of the reasons why the United States excels, not only within cyberspace but in all domains of warfare. They're very focused on precision. They're very focused on achieving the objective. So if I think about what the offensive experience did for me, it made me very objective focused. And what I realised in the process was that my ability to apply my objectives was very much a function of how the defender defended themselves.
What is the one thing security teams consistently do that makes an attacker's job easier, something most of them do not realise they are doing?
Rivera: Security professionals often fall into the trap of tunnel vision. They have a little checklist. They think if they do the checklist and they do all the right things, oh, I bought my EDR, I bought my firewall, I'm doing alerts and triage, I'm doing threat hunting, I'm doing incident response. But then the question is not just are you doing it, but are you doing it well? Are you doing it correctly? Are you doing it in a manner that's actually going to defend yourself, or are you doing it in a manner that's going to maybe achieve compliance and maybe make your boss happy, but you're not really affecting what you're trying to accomplish, and you're not really defending the assets that you want to defend? Every single breach that happens is always a surprise. No CISO actually believes they're going to get breached next week. A breach is always a strategic surprise. So I would argue that the main thing anybody does incorrectly when they get breached is they've miscalculated. They miscalculate on two fronts. The first is the adversary themselves. They don't understand the adversary, they don't understand what the adversary is capable of. But that misunderstanding is then further amplified when not only do they not understand the adversary, they don't know what they should do about it. You're going through the motions, and the motions feel good, but the motions aren't actually what's going to protect you in real life. So miscalculation, if I were to put it all in one word, miscalculation is always the reason for a breach, and it's the one thing we always try to focus on as defenders. How do we miscalculate less?
Was there an assumption from your NSA days that turned out to be wrong once you saw it from the other side?
Rivera: There are many assumptions, I think, that not only I realised to be wrong at a personal level, but assumptions in general from an entity as vast as the United States and the Department of Defense. They operate on assumptions, but those assumptions change. The battlefield changes, not only in terms of technological change, but also in the change of dynamics. If you look at the use of unmanned aerial systems or drones within both the Ukrainian theatre of conflict as well as the GCC and Iranian theatre of conflict, one major assumption is that even with drone warfare, we thought the bar was a little bit higher than it was in real life. We miscalculated these low-cost drones, hooking up a drone to a fibre optic cable and not having the drone emit a signature. So I think miscalculation often occurs not only in the physical sense, but of course the cyber sense, and those miscalculations are further amplified by technological change. When AI first came out, I remember ChatGPT coming out in 2023. I couldn't imagine we'd be where we are today, that agents would be doing the things that we're making. So when I think about assumptions that I made that were incorrect, sure, I made tons of assumptions that were incorrect, but I think we all do. That's the nature of assumptions. You make them, you have to operate on them because there are lots of unknowns. But then battlefield dynamics and technological change force us to adapt our assumptions. So for me it's less about was I right or wrong. The question is how quickly can we see it, and how quickly can we adapt so we don't get beaten on the battlefield, whether that battlefield is kinetic and physical or within the cyber domain.
You now watch organisations get tested under realistic attack conditions. Walk us through a case where a team looked strong on paper, mature tooling and documented playbooks, and still fell apart.
Rivera: There are many cases, and to some degree you read about these all the time. Think about all the breaches that are publicly announced. The telecom sector early last year, done by an adversary referred to as Salt Typhoon, which is a Chinese nation-state threat focused on the telecom sector, that was one major miscalculation. When you think about the Fortune 500 telecom entities in the United States, these are very well-funded, very mature, very strong organisations. But even at the highest tiers, miscalculation still happens. Sometimes it's one of those things where it's almost inevitable, if that makes sense. Because when you think about the nature of offence versus defence, the defensive side has to get it right every single time. The offensive side only has to get it right one time. And not only does the offensive side have that advantage, they also have the first mover advantage. One of the interesting things about the cyber domain relative to the physical domain is that in the physical, you see things coming. We can observe. We have satellite imagery, we have signals intelligence. We're able to see things as they're moving. In the cyber domain, it is very difficult to see movement, and because it's difficult to see movement, you see that strategic surprise happening. This indicates a fundamental challenge in how we approach the problem, because cybersecurity is inherently a responsive discipline. You get the alert, you respond to the alert. It is called incident response. Threat hunting implies that you're hunting for something that has already happened. Every single defensive discipline implies response. But the rate of attack continues to increase. There's a thing called breakout time, which is the amount of time it takes the adversary once they gain access to move laterally, because once the adversary moves laterally they have expanded access. Now they can go into multiple machines, they have multiple points of presence. I remember when I first joined CrowdStrike in 2018, the fastest breakout time was like eighteen minutes and fifty-one seconds, and it was done by the Russian GRU, which is their military intelligence group. Last year, according to their global threat report, the fastest breakout time was twenty-seven seconds. So think about that. In a seven-year time span, it went from eighteen minutes and fifty-one seconds to twenty-seven seconds. And the very obvious question I'll throw to the audience, where does that breakout time go in the next five years? Right above zero. So let's think about that for a second. What does the world look like when the breakout time is right above zero? There is no more alert triage. There is no more incident response. Or the way that we think of these things ceases to function in their current state. That's the urgency. It's not that we have to do it because it's better or faster. I think we have to do it because of the inevitability, that if we don't do it, we're going to be fundamentally behind our most capable adversaries.
In those moments, what tends to break first, the people, the process, or the tooling?
Rivera: It depends, and the adversary is very opportunistic. I once said in a podcast that the adversary is like water. Think about the nature of water. You ever watch water go down a stream, throw a rock in the stream. What does the water do? Does the water get angry? Does it get upset? No. It just goes a different direction. You poke a hole in the dam, now the water's coming through that place. So when I say the adversary is like water, their ability to move and adapt is very fluid and very rapid as well. Think about some of the changes in polymorphic malware. Polymorphic malware is basically malware that can change its appearance and change the way it operates. Polymorphic malware is not new, it was happening five, six, seven, eight years ago. But now polymorphic malware can not only change its appearance when it's in the environment, it dynamically adapts and makes judgment calls and changes its capability on its own. So going back directly to your question, what fails first, the people, process or the technology? The answer is that it sort of depends on the situation. But to put it even more bluntly, and to take this at a leadership level, the failure is always people, is it not? It's always a poor decision that we made. Our technologies and our processes do not exist in and of themselves. They exist because a leader made a decision based off their understanding of the situation. So part of this is our own ability to have accountability on how we think. Even in a future when the agents are on the front line, it's still going to be people who decide which agents, how those agents operate, what their decision thresholds are. At the end of the day, it truly is a people thing.
What does a live-fire exercise expose that a tabletop simply cannot?
Rivera: My answer again would be it depends. In cybersecurity, unfortunately, and especially in less mature organisations, we often think of things as binary. Do I have the EDR or not have the EDR? Did I do the live-fire exercise or not do the live-fire exercise? Am I compliant or not compliant? We have these binary ways of thinking, and it's very comforting to be able to do the thing and check the box and pat yourself on the back and say, yes, I did it. So more directly to answer your question, with the live-fire exercise, sure, it helps you to prepare. But then my question back to anyone, okay, but what kind of live-fire exercise? How realistic was it? Did it actually incorporate your environment? Did it incorporate your tools? Did you practise against adversaries and threats that you are likely to face in real life? Or did you just check the box and play a video game thing, and there was like fireworks and confetti at the end with a gold medal? Here you go, here's your gold star, you did a good job. That happens a lot. Because again, this compliance mindset, this idea that we can check boxes and be okay. So live-fire exercises certainly help, because there are really two ways to learn. You either learn by doing an exercise or education, or you learn by real life punching you in the face. So sure, let's do some education. But then of course it begs the question, what is the extent of that education, and what is the extent of the live-fire exercise? Applying more urgency and more effort towards the thing will yield more results.
Why do so many organisations still stop at tabletops?
Rivera: Time, money, effort, stress. It's the same things that stop us in anything. Why do we not finish that master's degree that we always wanted to finish? How come we never run that marathon we said we'd run? How come that project we said we'd do, and everyone was like, yay, it's a good project, and then we kind of all... Why does anything die on the vine? We're all limited. We're limited with our time. We're limited with our capability. We have lives. And especially when you go to smaller organisations, those limitations become more and more severe. Everyone wants to do a good job. Everyone wants to do the right thing, but part of it is just those real-world limitations. So it's one of those things where it's not only what should I do, what is the right way to do it, but then what is the best way to do it given my constraints, given my objectives? Because the truth is you're not always going to have all the money in the world. The truth is you might not have all the time, and you might not have all the capability. So given those constraints, what is the best way we can do this? The more that we honestly ask ourselves the question, the better off we are, whereas the more that we just say let's check the box, that's where the danger comes from. So it's a mentality thing. How do we get into this mentality of doing the best we can given the constraints and the limitations that we have?
When you test the products organisations have already bought, what is the gap you see most often between what a tool claims to stop and what it actually stops in practice?
Rivera: Let's take any tool, whether it's an EDR, a SIEM, a firewall, identity, whatever. It's one thing for something to work in theory, it is another thing for something to work in real life. Let's take a weapon first. An aircraft, a drone, artillery. Part of understanding whether that weapon works is using that weapon in realistic conditions. The military, whether it be the US military, the NATO alliance, or any advanced military in the world, is constantly training, testing, and validating under realistic conditions. When you think about this within the cybersecurity tool space, that's sort of a difficult thing to do. How do you validate under real world conditions? Sure, we have pen tests and we have breach and attack simulations, but here's what I can tell you with certainty. There is no pen tester, and there is no breach and attack simulation capability, that is going to deploy ransomware on your environment. There is no pen tester or BAS tool that's going to blow something up in real life. There's no pen tester or BAS tool that's going to take your data and put it on the dark web. There's only a certain extent that these capabilities will go within the cyber domain, because what they don't want to do is cause real damage. Therein lies the solution of a simulation. Because for me to know the truth, if I'm going to make a good decision, I must know the truth. Not the truth I want to hear, but the full extent of the truth. And for me to understand that full extent of the truth, I have to be able to test against the worst of all possible circumstances. So for me, testing is about not just looking at the claim and not just doing it in a nice pretty place where everything works, but doing it in a messy place where things don't work, where life is falling apart, where attacks are constantly happening, where weird stuff is happening on the network. For example, I have a printer behind me. Maybe in my simulation, the printer should break. Maybe the printer should be the point of entry, or maybe it should be some type of network flaw. Real life is messy, real life is dangerous, and we get that in the physical sense, but we don't get it in the cyber sense. So when I think about tool testing, the evolution comes when we realise that we can't just test in these safe, easy environments. We have to test in these dirty, dangerous environments. That to me is the next stage, because we don't have time anymore. We don't have the luxury of time and adaptation. Things are getting faster. Things are getting more severe.
For a security leader who has never stress-tested their stack, what is the realistic first step to find out whether the tools actually work?
Rivera: The realistic first step for me is first understanding your own expectations, and whether this is the right tool. What am I trying to accomplish? What is the problem I'm trying to solve? And then the tool. We often have this reverse understanding. I'm going to get the tool, I'm going to do the thing, whatever my boss says, whatever compliance says, but we don't have a problem-focused mindset. So to me the first step is, do you even understand what you're doing? Do we understand why we're buying this thing? Do we understand what we hope to achieve? Because once you understand what you think you're doing and what you hope to achieve, then you can run a test. What is a test? A test to some degree is kind of like the scientific method, and we all go through this as children. What is the very first step of the scientific method? It's the hypothesis. It's the expectation of what might happen. So the second step is that once you have that understanding of the situation, you develop a hypothesis, and the hypothesis itself is what allows you to construct the test. Because it's not only about the tool. The tool itself is operating within context. It's operating within the context of the environment, the critical assets, the subnets, how things are configured, the people, but then of course and equally important, the adversary itself. And if you are not testing all of these things together in the same place, when you only do it in isolation, you get an isolated answer that doesn't represent the real world. So that's the sequence. Do I understand my objective? Do I have a hypothesis? And can I set up a complete realistic test environment that allows me to test my hypothesis against the ability to achieve the objective?
You have seen threat intelligence done well and done badly. What separates intelligence that actually changes a decision from the reports that get produced and never read?
Rivera: An old mantra of intelligence is that intelligence is timely, accurate, relevant, and predictive. Good intelligence possesses those four traits, because if you are timely, you get the information in time. If you are accurate, you have the right understanding. If it's relevant, it matters to you. And if it's predictive, most importantly, it helps you understand what might happen next. Those four things combined allow for action. I cannot take action on something that does not happen on time. I cannot take action on something that's inaccurate, and so on. So good intelligence, to sum it up, is actionable. Bad intelligence is noise and distracting. Intelligence can actually send you backwards. When intelligence is noisy, when it's misleading, when it's irrelevant, not only does it create more and more noise, which distracts our attention, but it lowers our understanding of the real-life situation. Intelligence itself is an evolving discipline, but the thing that seems to be changing a lot really is two features. It's the timeliness piece and the predictability piece. Those pieces appear to be strained. Think about intelligence back in the World War One and World War Two days. In World War One, we were sending hot air balloons up to see over adversary lines. The hot air balloon would go up, and they would observe a thing, and then they'd write a note, send a telegram back. This process maybe takes days, a week. Speed up to when we have modern radio communications and satellite communications. Now it's happening faster. But now with AI, now it must happen even faster. And one of my concerns about intelligence is that the nature of it is that it lives in the past. Every indicator of compromise happened in the past. Every threat report happened in the past. Every adversary profile we wrote is about who the adversary was in a time, not now, in a place, not here. So how do we become predictive, particularly when it's getting faster? For me, the evolution that's coming is that intelligence is going to go from proactive or predictive to preemptive. Intelligence must think of what the adversary is going to do before they do it, and that's why AI plays a critical role. AI is what allows us to understand what the adversary will do before they do it.
If a security leader has a limited budget and wants to build an intel function that matters, what is the first capability they should invest in?
Rivera: For me, intelligence is in many ways a function of the information that you possess. Good intelligence provides good information around actionable things that are relevant to your ability to make decisions. So for an intelligence leader, my very first question is, do you actually understand yourself? Before you go and get threat feeds and do whatever it is you think you're going to do, are you aware of your own stuff? Do you have access to your alerts? Do you have the right tools in place? Because if you're not collecting on yourself, then you might have a great understanding of the adversary, but you can't apply that to who you are. That's one thing an intelligence leader needs to think through. But beyond having good information on yourself, the question is why am I doing this? Many intelligence leaders fail to ask themselves that question. Because if you don't know the reason you're going to do something, then you're just going to do stuff, but it's not really going to get the results you want. You can think of intelligence as a four-step life cycle. There's planning and direction, collection and processing, analysis and production, and then dissemination and feedback. Four steps. There are different versions, some are five steps, some are six, some are eight, but I'm a simple person. Four steps. And intelligence leaders often skip the first step. They go straight into collection. They go straight into analysis. They go into dissemination and releasing reports. But they don't understand the priority intelligence requirements. They don't understand why they're doing it. So for me that very first step is, do you understand yourself? Do you understand what you want to do? Do you have information on yourself? And then from there, now you can go collect, now you can analyse, now you can disseminate.
You have worked inside several security vendors. When a CISO is being pitched AI-driven defence today, what should they be most skeptical about?
Rivera: The skepticism that I often see is around the area of trust. The AI capability claims it can do a thing. It looks good in the demo. But how do I validate trust? And in what location do I validate this trust? Not only trust, but also efficacy. I need to know that if I allow this thing to have control over aspects of my environment, it's not going to behave destructively. For me that's step one. Because it can be effective. Say it's great at finding alerts and triaging and doing whatever, but in the process it's destroying business-critical assets, then it doesn't matter how good it is, because I don't trust it. So the biggest thing a lot of leaders are thinking through right now is the idea of trust. How do I know I can trust this? But then beyond trust, the next step is, is it effective? It's one thing for something to work in theory. It's one thing for something to work during a demo or a presentation where there's cool slides and flashy stuff going on. It's another thing for something to be effective in real life, which gets back to this idea of realism. Whether it be trust or efficacy, I cannot know either of those things unless the situation in which I'm validating or testing is a realistic situation. So that's what I'm seeing CISOs and security leaders think through these days, because they all know it's inevitable. It's happening right now. AI will be on the front lines everywhere within five years. I say that in almost every conversation that I'm in. Some people believe me, some people don't. Some people think it'll be faster. But I'm almost certain every single large organisation will have AI on the front lines within five years. So the question is, how do you get there? How do you do it in a manner where you know efficacy, where you can believe in the trust?
For a security leader who knows their organisation is not ready and does not know where to start, what would you tell them to do first?
Rivera: Always the basics first. It's hard to do anything advanced or special or amazing and interesting when you don't have the basics covered. So basic things like, do you have visibility in your environment? You've probably heard this a million times, but if you don't have EDR, if you don't have visibility, if you can't see what's going on, then nothing else really matters. Please don't get AI. Please don't do other things until you have the basics. So there's the idea of visibility. Can I see what's going on? There's the idea of the capacity to respond, the capacity to contain, the capacity to isolate, to prevent. When you think about these core defensive functions, for me the first step is always that. Can I even do these? Because if life is on fire, if the building's falling down, if things are in a mode of crisis, then there is no special project to go after. You're in a crisis. You need to solve the crisis. Most intelligent security leaders are always knocking that out. But then once you realise you're doing the right thing, the question is, am I doing it the right way? I have my EDR, I have my firewall, I have my tools, but am I even using them correctly? Am I doing so in a manner that's effective, or in a manner that's wasteful? And then once you know you're doing something the right way, now are you doing it the best way? You might be doing the right thing, but are you doing the right thing in the best way possible? Could you do it more effectively? Could you optimise better? Could you do it faster? Could you have spent less and achieved the same or better outcomes? The best leaders are good at continuously challenging themselves and continuously iterating through that process. Poor leaders go through the process once and believe so much in their own decision-making and their own understanding of the situation that they fail to challenge themselves, which gets back to how we started this conversation. Why does every breach happen? Every single breach happens because of a miscalculation or misunderstanding. So for me it's the ability to methodically go through that process. Methodically, but also humbly. Can you humbly accept when you are wrong? Can you not be offended by being wrong? Can you accept being wrong with grace and rapidly make a change? That's a big one too.
How has the job of security changed in the last few years, and what should someone entering the field now be learning?
Rivera: I think the obvious answer is AI. But what does that even mean? AI is as significant as any of the most important inventions in human history. It is as significant as the vehicle, as significant as the internet, as significant as agriculture and energy. And given its level of significance, I do not think there is a future where any security professional is in this industry who does not understand not only how AI operates, but also how to use it. There's something called the four S's in AI, and whenever anybody asks me about how to use AI, I almost always quote the four S's. The four S's are basically the things AI is always better at than humans. Those are speed, scale, sophistication, stamina. It is faster, it does more, it's more sophisticated, it finds more linkages and does more interesting analytic things, and stamina, it never sleeps. So I think the cybersecurity practitioners of the future don't ever go away, but they learn new skills that make them more effective. An analogy I often use is, you ever seen those cartoons where somebody gets inside a giant robot and they fight people inside other giant robots? That's kind of what's about to happen. We're all about to put on our robot suits, and those robot suits are going to be powered by AI, and they're going to make us faster, scale more, make us more sophisticated, and make us able to do things our human capabilities cannot do. But then the challenge is that everybody's going to do that. It's not just the defenders, it's the offence that's going to do that as well. So I don't think AI is going to massively replace cybersecurity professionals. I think it might in some areas, but at the same time I think it might also increase the need for a cybersecurity professional, because I think more things are going to happen. It might make defence more effective, but if there are ten, twenty, thirty, a hundred times more bad things happening, then I guess you need more defenders. Either you need more defenders, or those defenders have AI helping them. So I don't think the security profession goes away, but I think it does adapt to AI, and I think it does it at a much quicker rate than a lot of us previously thought might happen. It's just being honest with yourself. Do we see the problem? Do we see what's happening? Can we accept it? Can we humbly accept it? And then most importantly, can we respond and evolve in ways that are necessary in order to not only protect the organisations, but to empower ourselves to continue to be relevant in this industry.
What is something you believed early in your career about nation-state threats that you no longer believe?
Rivera: I once believed that nation states had exceptional, unmatched capabilities. I once believed that if you had all the money and all the technology and all the power in the world, that you could have your way on the battlefield. And beyond what I believe or don't believe anymore, just look at what's happening. It might not matter how much money or how much power or how much capability you might have. There are certain circumstances and certain things that might happen that change these assumptions. So for me, the battlefield is sort of evening. I once wrote on this concept in 2014 about the idea of unconventional and asymmetric warfare. Asymmetric warfare is basically this idea that you have a more powerful adversary against a less powerful adversary. Those lines are blurring. Things like AI, things like drones, things like decentralised cryptocurrency wallets, all of these things are creating capabilities that allow those who have less capability and less funding to sort of even the playing field. That's been, for me, one of the main things that's changed. Nation states are not as powerful as they used to be, and power is beginning to become more and more distributed because of technological advancement.