Category Archives: General

Root Cause Analysis in Vulnerability Management

Posted on by
Reply

 Root Cause

If you are not familiar with the subject of “Root Cause” or Root Cause Analysis I would encourage you to read about it on Wikipedia before reading the rest of this post. In short, Root Cause Analysis is trying to determine the earliest factors of the Cause of an event or issue and not simply treating the Symptoms of an event or issue.

Nothing New Here..

I worked in IT Infrastructure and studied business for years before I started working in IT Security. I’ve found that most operational management principles apply to Information technology and Information security processes in nearly the same way as they apply to manufacturing or other business processes.

Root Cause Analysis is yet another operations management topic that directly applies to information security vulnerability management.

Some use cases for Root Cause Analysis in Information Security Vulnerability Management.

(Root cause analysis for these 4 cases below will be broken out into another post and linked here when complete)

  • Why are system vulnerabilities there?
  • Why do system vulnerabilities continue to show up?
  • Why are coding weaknesses in my code?
  • Why do coding vulnerabilities continue to show up in our code?

 

Isn’t this Common Sense?

No.  See below for why…

Treating or Correcting Root Cause is Harder than Treating Symptoms

Treating symptoms is nearly always quicker and easier than resolving root cause.

– Treating symptoms gives an immediate short term relief. This short term & quick fix creates a very direct emotional response. If you are in pain, and a doctor gives you a shot to numb the pain, that feels great right? But what if they never fix the reason you are having the pain in the first place? You could keep needing those shots every day. After a while you will probably decide that putting in the effort of fixing the “root cause” of your pain is a better option.

Resolving the root cause to an issue typically doesn’t have that immediate emotional feeling of relief because it takes longer. It takes planning, discipline, and often a strategic patience to influence others to help resolve the root cause of an issue.

I think that treating symptoms is the more “natural” reactive response to a problem. The more proactive and mature response to an issue is to take the time to determine and analyze root cause.

Examples…

– Reboot it? A great example of this issue is very common in IT Infrastructure or Development operations. An application area or systems team has an application or system that starts behaving strangely or stops working. The common response is to just reboot the system or restart the process. This may resolve the problem that time, but it is likely that the problem may re-occur, or just be an earlier indicator of a larger problem. If you take the time to gather documentation (memory dumps, logs, etc..) for root cause analysis before rebooting the system or restarting the process you will be able to research root cause. This is more difficult initially because the process of getting memory dumps and logs may take a while longer than simply restarting something. If you never address the root cause, these symptoms will keep stacking up and drive up your support and maintenance costs as well as impact availability.

– Patches – Is it easier to install a bunch of patches on some systems, or to implement a solid process and resources to ensure that patches are always correctly installed and validated on systems? Installing a patch is treating a symptom. Implementing an effective patch management process is treating root cause.

Some may argue that root cause of patching starts all the way back at the processes of the operating system development. That is true, however you always have to realize that there are some root causes that are out of your control. In this case, you can effectively treat the root cause reason of why the patches are missing, but not why they are needed in the first place.

–Social Issues – Social and political issues most often have symptoms treated because resolving root cause is typically assumed to require behavior changes or other changes that are considered too difficult or unpopular to implement.

Should my focus always be on fixing Root Cause?

Now we are getting into opinion, but I think that root cause should be identified and how to resolve the issue should be analyzed. Choosing to address root cause or not is a business prioritization issue just like any other project that should be evaluated. However, the choice to Not address root cause needs to be documented and known. Why? Because the symptoms of that root cause will continue, and they should be expected to continue.

 Hybrid Approach

I think that taking the parallel approach of treating some symptoms while working on remediating root cause is a reasonable approach. Unfortunately, since the temptation to focus on treating symptoms is so strong, it often takes a very determined person to ensure that root cause is addressed.

Bird Flu and Security Bugs – Research Gone Awry?

Posted on by
Reply

Several times in the news lately I’ve heard about the Bird Flu Research controversy. Each time I hear about this controversy I want to compare it to the recent controversy around SCADA IT Security research being published directly to security testing tools companies. I don’t think it is a stretch to compare these 2 topics. While there are some obvious differences, many of the arguments are similar..

 

To Publish or Not to Publish?

One of the main concerns around the bird flu research is whether the results and methodology of the research should be fully published.

The premise used to justify publishing vulnerabilities in the IT security industry is that exposing IT security vulnerabilities and making them easier to exploit forces companies to patch these vulnerabilities and create more secure software and systems over the long run. I believe this premise is true. Most companies would not enhance the security measures in their code or systems unless necessary. The cost for developing software is mainly in the initial development, anything afterwards is maintenance cost that is typically a cost center and not a revenue generator. So any company will attempt to keep their maintenance (creating code patches) cost as low as possible.

This same premise has been used to defend publishing the research on the bird flu. A nation that wanted to do its own malicious bird flu research could do that if they wanted, so we should understand and be prepared for that scenario. So just like we should improve software or systems, we should improve our ability to identify and respond to a malicious strain of bird flu.

 

Does the research help or hurt??

The concern around the bird flu research information is that malicious actors could use the information to create some kind of biological weapon. The very same type of concern exists around IT security research. If you show a bad guy how to exploit a vulnerability on a system, they are more likely to use it, or it makes greatly reduces the time and effort needed to create their own exploit.

Whenever you reduce the effort (time, cost, risk) of exploiting a vulnerability or showing how to develop a stronger virus, you initially will increase the risk of people using that information for their own purposes.

So does it help or hurt? Initially it hurts. The publishing of the vulnerability forces companies to dedicate resources to the analysis, development, and deployment of patches. After the initial pain, it helps the companies by ensuring their code and systems are more secure. The long term effect that *should* happen is that development companies should change their processes to ensure they develop code security, consumer companies should ensure they have enough resources to keep systems patched, and the whole cycle should gradually become a less hectic normal maintenance routine.

For Bird Flu research it can help to ensure that public administrations prepare and have plans in place for dangerous virus outbreaks.

 

What is the real question that needs to be answered?

Is the initial increase in risk caused by releasing research information worth the mid-term and long-term reward of improving the products or being more prepared for a lethal virus outbreak?

Unfortunately I wasn’t able to find any real data to support whether security research disclosure truly helps improve security over the long run.  I think that it does, and I think the IT Security industry believes it does, however this seems to largely be “common wisdom” and not based on any hard facts. (please correct me if I am wrong)

 

Conflict of Interest?

There are some obvious conflicts of interest that create “grey” areas in IT Security research. When a security researcher works closely with vulnerability testing companies to incorporate working exploits for vulnerabilities they have found instead of working with the companies that publish the software with those exploits, it makes me question their motives.  Also, if my company sells security testing software, then having a check for a vulnerability that no other company has and/or has no patch is a competitive advantage. Is the security company truly first concerned with the security of their customers or with their sales?

For virus or disease research I don’t see anything like this happening that I know of. I suppose a researcher could work directly with a pharmaceutical company, but the whole concept doesn’t apply very well to disease research and pharma development.

How to Resolve *Some* of these Questions

Security testers and security companies that deal with exploits and vulnerabilities should be very clear about what responsible disclosure guidelines, code of ethics, or methodology they follow when disclosing vulnerabilities.(if any)  Customers of security services or security testing software should ensure that they purchase from companies or researchers that align with their own code of ethics.

So what about Bird Flu research? Why can’t the National Academy of Science or World Health Organization provide guidance around research regarding increasing the virulence of a virus or disease? Research labs or Universities should clearly define what guidance or methodology they follow around this type of research and it should be a condition of disclosure when applying for funding and grants.