«02:14» The Night Nothing Happened

This story begins with a server and a decision that many IT teams have made before. Monday, 4:44 p.m.: Somewhere on the network, an automated scanner is scanning the externally accessible systems of a regional bank and finds what it’s looking for. A VPN gateway whose software is one version too old. A patch for the vulnerability has been available for weeks, along with a support ticket and a scheduled maintenance window. It was postponed twice, deliberately and for good reason: Another project took priority, and no one wanted to touch a central system during live operations of all times. This wasn’t a failure, but a perfectly normal trade-off in an organization where a hundred things are priorities at the same time. Every IT team makes exactly these kinds of trade-offs every day, and that’s precisely why a second line of defense is needed in case one of them happens to be wrong.

The attacker exploits the vulnerability and gains a foothold in the system. Up to this point, there is no difference, absolutely none, from a typical attack exploiting a known, unpatched vulnerability. Same type of attacker, same method, same quality. The difference begins with a decision this bank made two years ago. Not out of foresight, as the IT director candidly admits today, but under pressure: Regulators (FINMA) demanded proof, the board of directors asked questions, and a risk analysis brought it to light: no one was monitoring the systems at night, and no one knew who to call in an emergency. Since then, the bank’s alerts have been routed to a Security Operations Center, there is an incident response retainer with contractually guaranteed response times, and once a year, senior management runs through an emergency scenario in a drill. During the first drill, the IT director recalls, no one knew who was leading the crisis team. It was an uncomfortable afternoon. It was the most valuable afternoon in years.

Monday, 9:03 p.m.: From the compromised gateway, someone is making their way deeper into the internal network. The activity is inconspicuous enough not to trigger an alarm, since systems communicate with one another, that’s their purpose. The SOC monitors them nonetheless. The attacker remains quiet, looks around, waits. At 2:13 a.m., he takes the step every attacker must eventually take: He attempts to access an internal file service from this server, where the credit files are stored. And at that very moment, the correlation engine in the SOC combines several alerts that, taken individually, seem harmless: that a server accessible from the outside suddenly launches a command line, that a connection to a foreign country is established from this exact server, and that it is attempting to access an internal system it has never touched in its entire operational history.

2:14 a.m.: An incident appears on the screen of the analyst on duty. 2:15 a.m.: The automation system has already isolated the affected server from the network and terminated the active sessions; the attacker loses access before a human has even intervened. What follows is not a heroic feat, but routine, and that is precisely the point: The analyst realizes that the attacker has already attempted to access the internal systems. Since the scope and duration of the attack are still unknown at this point, he escalates the incident.

At 2:31 a.m., the IT director’s phone rings, not as a panic call, but as step three of a playbook familiar to both parties. He asks two questions specified in the playbook, confirms that the system has been isolated, and calls the hotline of his incident response service provider, which has a contractually guaranteed response time. Within minutes, an incident responder is on the line. During the scoping call, they discuss the incident so far, define the course of action, and exchange information on findings and activities to date. The IT manager is relieved to be able to follow a prepared incident response plan and remain capable of taking action despite the late hour. At 3:45 a.m., the CSIRT begins the actual forensic work. The CSIRT collaborates with the SOC to efficiently review the gateway logs and thus identify the initial attack vector via the exploit. Combined with the network logs, it becomes clear that the attacker has not yet been able to move laterally within the network. The CSIRT maintains regular communication with the IT director and the SOC, reporting on new findings, any logs that may be needed, or actions being taken. The entire environment is scanned: no additional compromised systems, no second trail, no backdoor left behind. The attack was stopped early, before the attacker could move laterally within the network.

After an intense night shift, the incident had been clarified enough by 8:30 a.m. that everyone involved could catch up on their lost sleep before the follow-up work began the next day. The IT director would later say that this was the moment when two years of preparation had truly paid off for the first time: not the technology, but the calm. Responsibilities were clear, everyone was able to work in an organized manner, and it was known exactly who would answer the phone at 3:00 a.m.

A few days later, management receives a one-page report: what happened, what occurred automatically, what the CSIRT verified, and what still needs to be done. The gateway is taken offline, cleanly reinstalled, and finally patched, closing the vulnerability. A new patching process is defined to update critical systems in a timely manner, and the necessary resources are approved. No one is held responsible for the postponed maintenance window, because the decision was the right one at the time, and this is communicated internally as such. After all, anyone who is singled out for a delayed patch will be less likely to report the next outstanding issue at all. The problem wasn’t the prioritization, but the assumption that prevention alone is enough. The lesson that remains is a different one: prevention had overlooked this one vulnerability. Monitoring caught it, and the prepared response put an end to it. That is exactly what both are for. According to the established process, a report is sent to the appropriate departments, not out of fear, but because the procedure for doing so has long been defined. And two weeks later, what turns a thwarted attack into a win for many happens: The insights from the incident flow into the SOC as refined adjustments and into the retainer as updated playbooks. The next attack will not only be detected sooner but also responded to more quickly.

The most remarkable thing about this story is what’s missing from it. No crisis team meeting lasting for days, no anxious questions about a ransom, no five-day shutdown. For the SOC and the CSIRT, a case like this is business as usual; attacks like this are regularly stopped in exactly this way. On Tuesday morning, senior management learned that a system that had been waiting for an update for weeks had nearly been the start of something major. Almost. It was the often-feared scenario that had always seemed to happen only to others; now they were affected themselves, but without any damage and without any major business impact. At the bank, people now talk about Monday night as the story where nothing happened, and the IT director corrects them every time: “It’s the story where everything happened. We just prepared for it two years earlier and were able to respond accordingly.”