Efficient Vulnerability Management with Qualys

Posted by on August 12, 2014

Vulnerability management (VM) is the process of discovering and remediating system vulnerabilities with the purpose of keeping information security risks at acceptable level. All IT systems have vulnerabilities. A vulnerability is a weakness which allows an attacker to reduce the confidentiality, integrity and/or availability of information. We still remember the major vulnerability discovered this year in OpenSSL, Heartbleed, or another big one from a few years ago – DNS cache poisoning affecting many DNS implementations.

Vulnerabilities exist in operating systems, applications, databases and network devices. Each month a number of new vulnerabilities are being published by vendors and independent researchers, so it is critical to discover and remediate them according to the risks they introduce. In this blog post I am sharing the details of the ThousandEyes VM process.

VM Process

At ThousandEyes we automate a VM process using a service called QualysGuard by Qualys. The process includes the following sub-processes (or phases of VM):

  1. Network mapping
  2. Asset classification
  3. Network scanning
  4. Vulnerability prioritization
  5. Vulnerability remediation
  6. Closure of vulnerability
Figure 1: Vulnerability management process
Figure 1: Vulnerability management process

Network Mapping

Network mapping is an essential step in discovering vulnerabilities and consists of enumeration of all IP addresses in registered networks in an attempt to find live hosts. Network mapping is implemented using QualysGuard Vulnerability Management Scans – Maps. Mapping is implemented as a weekly task, one day prior to scanning.

Figure 2: Network mapping process
Figure 2: Network mapping process

All discovered hosts are entered into the QualysGuard asset inventory. Refer to the next section to understand how we manage assets.

Asset Classification

All technology assets have a criticality level assigned to them. At ThousandEyes we use a very simple and effective approach – asset classification by location within the network. QualysGuard uses Asset Groups to manage assets. We have one Asset Group created for each subnet. Refer to the table below for an example of asset management (IP addresses are provided here as an example).

No. Asset group Criticality Comments
1 208.185.7.x 5 – most critical Permiter network
2 10.10.10.x 4 Web applications
3 10.10.20.x 4 Database systems
4 10.10.30.x 3 Backend systems
5 10.20.10.x 2 Supporting infrastructure
6 192.168.20.x 1 – least critical Internal lab
Table 1: Asset management

You can apply different strategies to classify the Asset Groups by their criticality; here I am using an example of an exposure-driven approach. You can also select data-driven approach or a combination of these two. Based on the table above, a system located in 10.20.10.x subnet will be assigned criticality level 2, a system located in 10.10.10.x subnet will be assigned criticality level 4, etc.

Based on the results of mapping, Information Security operations personnel add newly-discovered systems into appropriate Asset Groups for upcoming scans.

Figure 3: Asset management
Figure 3: Asset management

Network Scanning

We have implemented two types of vulnerability scans on a weekly basis – external (using the Qualys cloud scanner) and internal (using an internal QualysGuard appliance). The external scan shows us our exposure to the bad guys on the internet and are being used for general reporting only. To maximize results and improve their quality, internal scans run as an anonymous process and as an authenticated process, against all defined Asset Groups. Results of the scans feed into the Qualys remediation module for prioritization and ticket creation.

Prioritization

Not all Qualys users are aware of this, but Qualys comes with a powerful policy-based ticketing module called Remediation (exists in Express Suite and above) at no additional cost. Figure 4: Qualys remediation module

Figure 4: Qualys remediation module

To take full advantage of the Qualys Remediation module, you have to document your risk management policies related to vulnerabilities and set your remediation targets (operating level agreement, or OLA) with those who will fix the vulnerability. Feel free to reuse what we have put in place at ThousandEyes.

Each discovered vulnerability has a risk level associated with it. These risk levels are calculated using a combination of Criticality of an asset (1 through 5, see table 1) and a severity assigned by QualysGuard (1 through 5, according to Qualys Severity Levels). See the table below for detailed information about the risks associated with vulnerabilities:

Table 2: Calculation of the risk for confirmed vulnerabilities
Table 2: Calculation of the risk for confirmed vulnerabilities

Our Information Security standards set the following remediation targets:

No. Risk Level Risk Rating Remediation Target
1 HighRisk vulnerabilities 25 – 100 30 days
2 MediumRisk vulnerabilities 10 – 25 60 days
3 LowRisk vulnerabilities 5 – 10 90 days
4 SlightRisk vulnerabilities 1 – 5 120 days
Table 3: Remediation targets

Tables 2 and 3 are documented in the Policies section of the Qualys Remediation module. It is important to keep in mind that if you add new Asset Group, you need to update the policies.

Figure 5: Remediation policies
Figure 5: Remediation policies

I have a small trick for ease of reporting: each vulnerability gets a risk level assigned as the user name. As you can see from Figure 5, I have created 4 users and called them 4 High Risk, 3 Medium Risk, 2 Low Risk, and 1 Slight Risk. Let’s look at what’s inside those policies.

Figure 6: Policy conditions
Figure 6: Policy conditions

This is the policy “High Risk – Asset 5 Severity 3 4 5”. It has the Asset Group with criticality level 5 from Table 1 (208,185.7.x). According to Table 2, vulnerabilities with severity levels 3, 4, and 5 in the “Criticality 5” asset group will result in HIGH RISK vulnerability. The remediation target for those is 30 days. Qualys also assigns this vulnerability to the user 4 High Risk (see Figure 5), so we can clearly identify this vulnerability as high risk.

Essentially, to document your policies in Qualys you have to go through your risk calculation table line by line and enter all data in the remediation policies. For example, to fully document table 2 you’ll need to create 17 policies in Qualys.

Remediation

What we have done so far was only the preparation, and now the real work starts. After the first scan you’ll have the list of prioritized vulnerabilities with due dates defined and risks associated. Go under “Tickets” in the Remediation module of VM to see the list. All resource administrators work off this prioritized list to investigate and remediate vulnerabilities.

I will not be sharing any screenshots of our vulnerabilities; however, I will share the metrics I use for reporting. As part of the Security operations metrics, each month our CEO is presented the following data:

  • Number of vulnerabilities remediated last month
  • New vulnerabilities discovered last month
  • Currently open vulnerabilities
  • Vulnerabilities in violation of OLA (expected to be none)

Figure 7 shows our reporting screenshot related to vulnerability management. Feel free to reuse it. The same metrics are provided for the past 12 months.

Figure 7: Reporting screenshot
  • * Weighted is calculated based on risk ratings as:
    (100 x High) + (25 x Medium) + (10 x Low) + (5 x Slight)
  • ** Absolute is calculated as:
    High + Medium + Low + Slight
Figure 7: Reporting screenshot

Closure

Vulnerability tickets will be closed by QualysGuard automatically when it no longer detects them as open. If a vulnerability ticket was opened by an authenticated scan, it can only be confirmed as closed by an authenticated scan, and the same applies to anonymous scans. We do not close tickets manually (unless there is an exception); everything is done by QualysGuard automatically.

Suggestions to Qualys Team

Below are the items which would improve my experience with the Qualys service and save even more of my time:

  • Elevate Remediation module in the solution architecture, so it can cover both VM and Web application scanning vulnerabilities.
  • In addition to severity of vulnerability, add a risk to the ticket (though could be back calculated as a combination of severity and timeframe between due date and created date, ticket OLA). IN our standard they are High, Medium, Low, Slight.
  • Add reporting in time as described in Remediation section.

Conclusion

An effective vulnerability management process is a must in today’s fast-changing environment. The same vulnerability in a production database server and in the IT lab most probably poses different levels of risk and must be treated with different priority. There are tools available to track service exposure and criticality of assets. If you already use QualysGuard for scanning, you can implement an efficient process by maximizing its use.

** Qualys, the Qualys logo and QualysGuard are registered trademarks of Qualys, Inc.
All other trademarks are the property of their respective owners.
Processing...
  • Sai Ramanan

    Have you performed Qualys scanning as a service on the Amazon AWS environments? How do you think this approach can scale over there (except that the servers reside in Amazon)?

    • Alexander Anoufriev

      Thank you Sai for responding to this blog post. AWS (and generally IaaS/PaaS) and security scanning is certainly a very interesting topic.
      ThousandEyes has a very limited footprint on AWS. In our VM setup, systems running on Amazon Web Services comprise their own QualysGuard asset group which we scan from the Qualys cloud. Configuration of those instances is quite simple and we have similar installations inside of our network with full visibility during the scan.
      For companies with extensive use of AWS, the implementation of Qualys appliances (or other scanning solutions) inside of AWS seems like the right approach. To address your second question, I think it will scale as you scale the rest of your AWS infrastructure. For example Qualys allows you to spread the tests among the scanners for a single asset group.

  • Aditya

    Thanks for sharing such a well defined process.
    I have few questions:

    1.How do you manage vulnerabilities which does not have a patch?
    2.If there are 100 tickets open for a particular vulnerability and lets say we have to ignore or close them all.
    Do we have to do it manually or is there a any other way

    • Alexander Anoufriev

      Aditya, thank you for your comment. Both are very good questions indeed.

      1. Right, sometimes there is no patch or the patch may break expected functionality or affect the component’s availability. In such cases an asset owner needs to request a security exception and suggest alternative controls designed to lower the risk of not remediating the vulnerability. All exception requests are assessed from the risk management perspective and can be either approved (for a specific time period, for example, 6 months) or rejected. In case of approval, additional mitigating controls might be requested.

      2. We perform bulk operations on Qualys tickets by applying specific filters to the list, for example, by host IP, by QID, etc. When filters are applied, actions can be performed in bulk, like “Close/Ignore” with a comment on all 100 tickets.

  • Hemanath Baskaran

    Does this process work in the cloud environment ? Cloud is a very robust environment. Can share your insights on AWS or in any other cloud setup.

    • Alexander Anoufriev

      Hemanath,
      Yes, the process works in cloud environments as long as the scanner can connect to the computing asset via the asset’s IP address. The scanner can be external (hosted in the Qualys Cloud) or internal (usually an appliance under your control). If using an appliance, you would preferably host it within the same cloud as your asset. For example, there is a Qualys virtual appliance for AWS. Alternatively you can scan your assets from the OS level of each asset (scanning agent), which is a new feature announced by Qualys and supported by other vendors.

  • Victor.C

    Does the Qualys WAS integrate with the Remediation Module? I find no way to manage the vulnerabilities in web applications

    • Alexander Anoufriev

      Victor,
      Unfortunately not, WAS has its own “Detection” page which is not as comprehensive as Remediation (of VM). I know feature request to decouple Remediation from VM was registered in Qualys development queue, but unsure about the priority and timing. Feel free to request that and hopefully increase its priority.

  • Tom Schollmeyer

    Great job ThousandEye in building and documenting. I had my team implement exactly as described. Can you tell little bit more about reporting you use? Thanks

    • Alexander Anoufriev

      Tom, we use three different reports for three different audiences:
      1. For technical teams: detailed vulnerability listing (list)
      2. For management: Monthly – remediated last month, new vulnerabilities last month, open vulnerabilities currently, in OLA violation currently, all by risk level (table)
      3. For executives: Monthly trend – overall risk level calculated as described in Remediation section (graph)