Much has been written in the last few digital years about advantage that data metrics now offer businesses on the marketing front, but it appears that a growing number of organisations are using metrics to measure – and improve – their users’ security.
According to Scott Greaux, vice president of corporate phishing training specialist PhishMe, most security awareness programs fail to gather metrics – despite the advantages they offer.
Those that do, he says, typically measure inputs instead of outputs.
What this means, he adds, is that many teams are measuring items such as the number of users who have completed a CBT course or attended a lunch instead of the number of incidents related to a specific IT risk area.
This is similar, the PhishMe vice president says, to looking at the number of times someone visits a dentist each year instead of the number of actual dental incidents (e.g. cavities, root canals, etc.) and using that data as an indicator of good dental health.
By measuring your company’s susceptibility after each security awareness exercise you complete, he adds, gives the IT department a perspective of which concepts are working – and which ones are not – and allow your business to refine the techniques to improve the program.
Greaux goes on to say that collecting metrics will help reduce the percentage of susceptible users in a given business.
The challenge, he says, is that traditional security awareness involves implementing a variety of security awareness initiatives and hoping for the best.
Whether they are posters, knick-knacks encouraging employees to change their passwords, or annual classroom training, most of these initiatives fail due to a lack of measurable effectiveness – amongst other things – and offer no idea whether they are really improving employee behaviour.
Just to make life interesting, he says, it comes as no surprise that security awareness budgets are often limited, given the absence of proof that we are actually accomplishing our goals, since there is effectively no justification for more resources.
Collecting metrics, he adds, offers not only information about the past effectiveness of our programs, but also a path toward continuous improvement and better security posture.
Against this backdrop, Greaux argues that every security awareness initiative you implement as a business is an opportunity to collect information.
“Metrics measuring overall vulnerability to phishing emails are useful as a baseline to assess your readiness for a phishing attack, but offer much more insight. By measuring your susceptibility after each security awareness exercise you conduct, it gives you perspective of which concepts are working and which ones are not, allowing you to refine your techniques to improve the program,” he explains.
This is, he says, a great example of a directionally correct output metric.
Metrics, the PhishMe vice president says, should not end with something like the behavioural metric detailed above – they should extend into the real treasure-trove which is your internal review process.
“Using phishing as an example (probably one of the best to use, not because we’re PhishMe, but because it’s the most common initial entry point) teams can measure outputs such as the change in the number of phishing related incidents, the time from incident to detection, and the number of user reported phishing incidents,” he says.
“All these metrics can show that behavioural change is happening (or not) and provide you with cost benefit data to support your initiatives,” he adds.
Collecting these metrics, says Greaux, helps organisations to keep their security awareness training fresh by mixing up the topics and methods they use.
Simulated exercises using various tactics and at various times throughout the working week, he argues, will not only make things more interesting for your users, you will receive valuable feedback about the areas where your users are most susceptible as well as when they are most vulnerable.
“Your security awareness program should allow for collecting information about individuals, departments, etc. in your organisation, and discerning which users are susceptible and which are more security-savvy,” he says.
This knowledge, he adds, enables your IT department to tailor programs to users based on their level of knowledge.
An at-risk user, says Greaux, can be given more remedial training, whilst advanced users can be trained on more advanced topics such as conversational phishing.
Greaux concludes that metrics that tell you which users are most knowledgeable about security can aid in incident response – providing you encourage users to report potential security incidents or suspicious activity.
And, he says, if the internal review team can prioritise reports from users who are known to be savvy – security-wise – helps to increase the efficiency of response and remediation processes.
Acknowledging users for successful reports, he says, provides positive reinforcement, and makes them feel like they are contributing to the overall security of the organization.