...
Similar, and preferably aligned, tabletop tests may be executed at the SOC level, to build robustness and resilience at the SOC level (see Potentials for Enhanced Operation ).
Functional Exercises
Functional exercises are more extensive than tabletops, requiring an event to be really played. Functional exercises include simulations and “war-gaming”. Often, scripts are written out for role players pretending to be external organisation contacts, or there may be actual inter-agency and vendor/service provider participation.
...
This type of exercises will be ideal contexts also for building robustness and resilience at the SOC level, as recommended in Potentials for Enhanced Operation of the SOC.
Building Scenarios for the City as a whole
Although IMPETUS is primarily focused on cybersecurity, the scenarios for verifying and demonstrating a city's preparedness should be extended to other areas. The goal is to test the city's resilience in a manner as close as possible to a potential real-world situation. Therefore, situations such as blackout, flood, earthquake, epidemic or similar should also be practiced. This is also due to the fact that often there is a synergy of crisis phenomena (looting and social disorder during a flood, protests during a blackout and non-functioning of supply ...).
More complex scenarios will allow to better assess the resilience of the city as a whole and provide incentives for improving the robustness of security measures. Details of resilience and robustness you find here.
Modelling and simulation
...
It is very difficult to assess the real preparedness of the city for crisis situations. An exact measurement of anything is almost impossible and most of the time the result depends on subjective assessment by qualified persons. In order to compare the results of different exercises over time, or to compare different cities with each other, a common metric needs to be established and a team of calibrated assessors needs to be created.
The metric can usually be simple - a multi-stage assessment of the execution of scenario steps or the quality of decisions.
In order to apply the metric, it is necessary to train evaluators who have sufficient knowledge and experience in the field and, in addition, are "calibrated" so that the variance of their ratings is not too large. This can be achieved through years of practice.
This process works quite well, for example, in emergency services exercises, trauma plan reviews or fire response. For complex scenarios in smart cities, there is the potential of creating such a team in the future, for example within COSSEC.
Exercise results - follow-up measures – Robustness vs Resilience
The distinction between robustness and resilience (see Robustness vs Resilience ) applies for the city as well for the SOC (as described in Potentials for Enhanced Operation for the SOC).
The scenarios for city level exercises should as far as possible be commensurable with the Regular, Anticipated and Surprise scenario types (see scenario building) applied in SOC training and exercises
...