We have now finished a first round of tests to evaluate how our users work with the CS-AWARE system in their own contexts. The users were tested with simulations: for various reasons it was better to ‘inject’ cyberthreats than to wait for them to happen in real time. As a consequence, our users were subjected to many threats following each other: one threat resolved, the next one appeared almost instantly. Much of the rest was real: users could view their own system network, and threats came with actual and real-time information from reliable internet sources. Users could see where in their network a treat was showing up, a diagnosis of the threat and suggestions for its resolution (if possible: self-healing), what other parts of their network were in danger to be compromised and pertinent information about the seriousness of the threat (e.g. the number of attacks, the urgency or severity level).
However, while the CS-AWARE system is very good in diagnosis, and all users agreed is does this very well and very clear, the system cannot handle what happens after the diagnosis, except suggesting possible remedial action. Enter the human factor, and here we found many differences between our users. Let’s discuss some striking ones, and then draw some general conclusions about training for cybersecurity awareness.
In most cases (of a simulated threat) the system administrator testing person was used to consulting with others about what action to undertake. This was especially relevant when the system component at which a threat was directed was managed by another department, or when a service was outsourced to an external agency. Consultation was also the normal practice when the possible remediation involved (temporarily) shutting down some network service or network component. In some cases, however, the solution proposed by CS-AWARE was simply accepted without further reflection. In general, we can say that users have their established procedural strategies for handling cybersecurity threats, and that they apply these strategies also with the new CS-AWARE tool.
Another possible issue is the handover of a threat. Monitoring of cybersecurity, and consequently of the CS-AWARE system, is not in the hands of a single individual. Therefore, the handling of threats by more than one individual must be dealt with, to some extent. We are working on the implementation of a ticketing system for assigned users of the CS-AWARE system.
We also discovered specialised knowledge about network threats, and the absence of it, when our users were handling our simulated threats. Some users were database specialists, others specialised in particular software threats. In other words, relevant knowledge about cybersecurity is distributed within any organisation, even within a specialised department. In addition, user roles and responsibilities may differ. This is not an exceptional situation: system administrators did not all participate in the same curriculum, and their job descriptions may vary widely between contexts.
This means that training in cybersecurity awareness with the CS-AWARE tool does not stop at the interface. New users and new uses will be part of regular practice. Therefore, training cannot be of the type ‘one size fits all’. In the next phases of our project, we will look at cybersecurity awareness of our users, and on how that evolves with the practice of using the CS-AWARE system.
Jerry Andriessen
Wise & Munro
