Do WAFs dream of static analyzers?

Positive Research Center - 24 Říjen, 2017 - 13:18

Virtual patching (VP) has been one of the most popular trends in application protection in recent years. Implemented at the level of a web application firewall, VP allows protecting web applications against exploitation of previously defined vulnerabilities. (For our purposes, a web application firewall, or WAF, will refer to a dedicated solution operating on a separate node between an external gateway and web server.)

In short, VP works by taking the results of static application security testing (SAST) and using them to create rules for filtering HTTP requests on the WAF. The problem, though, is that SAST and WAFs rely on different application presentation models and different decision-making methods. As a result, none of the currently available solutions do an adequate job of integrating SAST with WAFs. SAST is based on the white-box model, which applies formal approaches to detect vulnerabilities in code. Meanwhile, a WAF perceives an application as a black box, so it uses heuristics for attack detection. This state of affairs makes VP sub-optimal for preventing attacks when the exploitation conditions for a vulnerability go beyond the trivial http_parameter=plain_text_attack_vector.

But what if we could make SAST and a WAF "play nice" with each other? Perhaps we could obtain information about an application's internal structure via SAST but then make this information available to the WAF. That way we could detect attacks on vulnerabilities in a provable way, instead of by mere guessing.

Splendors and miseries of traditional VP
The traditional approach to automated virtual patching for web applications involves providing the WAF with information about each vulnerability that has been detected with SAST. This information includes:

  • Vulnerability class
  • Vulnerable entry point to the web application (full or partial URL)
  • Values of additional HTTP request parameters necessary for the attack
  • Values of the vulnerable parameter constituting the attack vector
  • Set of characters or words (tokens) whose presence in a vulnerable parameter will lead to exploitation of the vulnerability.

The set of HTTP request parameters and dangerous elements of a vulnerable parameter can be defined both by bruteforcing and by using a generic function (typically based on regular expressions). Let us look at a fragment of code from an ASP.NET page that is vulnerable to XSS attacks:

By analyzing this attack vector code, we can generate a symbolic formula for the set of attack vector values:

{condition = "secret" ⇒ param ∈ { XSShtml-text }}, where XSShtml-textis the set of possible vectors of an XSS attack in the context of TEXT, as described in the HTML grammar.

This formula may yield both an exploit and a virtual patch. The descriptor of the WAF virtual patch can be used to generate filtering rules to block all HTTP requests capable of exploiting the relevant vulnerability.

Although this approach surely heads off certain attacks, it has some substantial drawbacks:

  • To demonstrate any given vulnerability, SAST needs to discover just one of the possible attack vectors. But to ensure true elimination of a vulnerability, it is necessary to address all possible attack vectors. Passing such information to the WAF is difficult, because the set of vectors is not only infinite but cannot even be expressed in regular expressions due to the irregularity of attack vector grammars. 
  • The same is true for values of all additional request parameters that are necessary for vulnerability exploitation.
  • Information regarding dangerous elements of a vulnerable parameter becomes useless if an attack vector, between the entry point and vulnerable execution point, undergoes intermediate transformations that change the context of its grammar or even its entire grammar (such as with Base64, URL, or HTML encoding, or string transformations). 

Due to these flaws, VP technology—which is designed for piecemeal protection—is incapable of offering protection against all possible attacks on SAST-detected vulnerabilities. Attempts to create such "all-encompassing" traffic filtering rules often lead to blocking of legitimate HTTP requests and disrupt operation of the web application. Let us slightly modify the vulnerable code:

The only difference from the previous example is that both request parameters now undergo a transformation and the condition for the "secret" parameter is weakened until the sub-string is included back in. The attack vector formula, based on analysis of this new code, is as follows:

(CustomDecode condition) ⊃ "secret" ⇒ param ∈ (CustomDecode { XSShtml-text }).

The analyzer will derive a formula at the relevant computation flow graph (CompFG) node for the CustomDecode function to describe the Base64—URL—Base64 transformation chain:

(FromBase64Str (UrlDecodeStr (FromBase64Str argument))). 

It is still possible to build an exploit on the basis of such formulas (we have considered this issue in a previous article), but the classical approach to generating virtual patches cannot be applied here for the following reasons:

  • The vulnerability may be exploited only if the decoded "condition" parameter of the request contains the "secret" substring (String 12). However, this parameter's set of values is quite large and expressing this set via regular expressions is infeasible due to the irregularity of decoding functions. 
  • A request parameter that is, in fact, an attack vector also is decoded (String 14). Therefore, SAST cannot describe that set of dangerous elements to the WAF. 

Since all the problems of traditional VP stem from the inability to interact with an application at the WAF level based on the white-box approach, the obvious solution is to implement this capability and make further improvements so that:

  • SAST provides the WAF with full information about all transformations to which a vulnerable parameter and variables of attack conditions are subjected, from entry point to vulnerable execution point. This enables the WAF to compute argument values based on the values of the parameters of a given HTTP request. 
  • For attack detection, heuristics are replaced with formal methods that are based on rigorous proof of all statements and describe the exploitation conditions for any particular vulnerability in the most general case, instead of haphazardly describing a limited number of cases. 
  • Thus was born runtime virtual patching.

Runtime virtual patchingRuntime virtual patching (RVP) is based on the computation flow graph model used in PT Application Inspector (PT AI). The model is built using abstract interpretation of an application's code, expressed in semantics similar to conventional symbolic computations. Nodes of this graph contain generating formulas in the target language. The formulas yield the set of all allowable values associated with all data flows at the relevant execution points:

These flows are called execution point arguments. CompFG is evaluable, and thus able to compute sets of specific values for all arguments at any execution point, based on the values that have been set for input parameters.

RVP occurs in two stages, which correspond to the application lifecycle: Deployment (D) and Run (R).

Deployment stage
Before a new version of an application is deployed, the application is analyzed by PT AI. Three formulas are computed for each CompFG node that describes a vulnerable execution point:

  • Conditions for reaching the vulnerable execution point
  • Conditions for reaching values of all its arguments
  • Sets of values of all arguments and corresponding grammars 

All formula sets are grouped by the application entry point to whose control flow the vulnerability relates. The very notion of entry point is specific to each web framework supported by PT AI and is defined in the analyzer's database.

Then a report containing the list of vulnerabilities and related formulas is extracted in the form of code written in a special language based on S-expression syntax. This language describes CompFG formulas in a form that does not depend on the target language. For instance, the formula describing the value of an argument for the vulnerable point in the above code sample is as follows:

(+ (+ "Parameter value is `" (FromBase64Str (UrlDecodeStr (FromBase64Str (GetParameterData (param)))))) "`")

The formula for reaching the vulnerable point is:

(Contains (FromBase64Str (UrlDecodeStr (FromBase64Str (GetParameterData (condition))))) "secret").

The report is then uploaded to PT Application Firewall (PT AF). On the basis of the report, a binary module is generated, which can compute all the formulas contained in the report. For example, the decompiled code for computing the condition for reaching the above-mentioned vulnerable point is as follows:

To make formula computation possible, PT AF must have one of the following:

  • Pre-compute database of all functions that may occur in the report
  • An isolated sandbox with runtime environment for the language or platform on which the web application runs (such as CLR, JVM, or PHP, Python, or Ruby interpreter), and libraries used in the application 

The first method ensures maximum speed but requires a huge volume of manual work by the WAF developers to describe the pre-compute database even if the developers restrict the scope to standard library functions). The second method allows computing all the functions that may occur in the report, but increases the time needed to process each HTTP request, because the WAF needs to access the runtime environment to compute each function. The most appropriate solution here would be to use the first approach for the most common functions while using the second approach for the rest.

It is quite possible for a formula to contain a function that the analyzer cannot process (for instance, calling a method that involves a missing project dependency or native code) and/or a function that PT AF is unable to compute (for instance, a function for reading data from external sources or the server environment). Such functions are flagged "unknown" in formulas and processed in a special way as described below.

Run stage
At the run stage, the WAF delegates processing of each HTTP request to the binary module. The module analyzes a request and detects the relevant entry point in the web application. For this point, formulas of all detected vulnerabilities are selected and then computed in a specific way.

First, formulas are computed for both conditions: 1) reaching the vulnerable point and 2) reaching values of all its arguments. In each formula, variables are substituted with values of the relevant request parameters, after which the formula value is computed. If a formula contains expressions that are flagged "unknown", it is processed as follows:

  • Each "unknown" flag spreads bottom-up through the formula expression tree until a Boolean expression is found. 
  • In the formula, such expressions ("unknown" regions) are substituted with Boolean variables, so the Boolean satisfiability problem is solved. 
  • The assumption formula generates n conditions by substituting possible values of unknown regions from all the solutions found in the previous step. 
  • The value of each formula is computed. If at least one formula is satisfiable, the assumption is deemed satisfiable as well. 

If computations show that the assumption is false, then the HTTP request in question cannot lead the application to a vulnerable point even with dangerous values of all request arguments. In this case, RVP simply returns request processing to the WAF's core module.

If attack conditions are satisfiable, the value of the argument of the vulnerable point is then computed. Algorithms used depend on the vulnerability class to which the analyzed point belongs. Their only similarity is the logic used to process formulas that contain unknown nodes: unlike assumption formulas, argument formulas cannot possibly be computed, which is immediately communicated to the WAF. Then the next vulnerable point is computed. To better flesh this out, we shall now review the most complicated algorithm, which is used for detecting injection attacks.

Detecting injections
Injections include any attacks that target the integrity of text written in a formal language (including HTML, XML, JavaScript, SQL, URLs, and file paths) on the basis of data controlled by the attacker. The attack is carried out by passing specifically formed input data to the application. When this data is "plugged in" to the target text, the boundaries of the token are exceeded and the text now includes syntactic constructions not intended by the application logic.

If a vulnerable point belongs to this attack class, its argument value is determined using incremental computation with abstract interpretation using taint analysis semantics. The idea behind this method is that each expression is computed separately, from bottom to top, while the computation results obtained at each step are additionally marked with taint intervals, given the semantics of each function and rules of traditional taint checking. This makes it possible to pinpoint all fragments that are the result of transformation of input data (tainted fragments).

For instance, for the code above and the following HTTP request parameter ?condition=YzJWamNtVjA%3d&param=UEhOamNtbHdkRDVoYkdWeWRDZ3hLVHd2YzJOeWFYQjBQZyUzRCUzRA%3d%3d, the result of applying this algorithm to the formula of a vulnerable point argument is as follows (tainted arguments are marked in red):

The value is then tokenized in accordance with the grammar of the vulnerable point argument. If any tainted fragment matches more than one token, this is a formal sign of an injection attack (based on the definition of injection given at the beginning of this section).

Once formulas have been computed for all vulnerabilities pertaining to the current entry point, request processing is passed on to the WAF's core module together with detection results.

RVP advantages and specific features
This approach to application protection based on code analysis has a range of substantial advantages as compared to traditional VP:

  • The shortcomings of traditional VP are addressed, thanks to the formal approach described above and the ability to take into account any and all intermediate transformations.
  • The formal approach also completely rules out the possibility of false positives, so long as the formulas do not contain unknown nodes.
  • There is no adverse impact on web application functionality, because protection is built on the functions of the application, as opposed to simply trying to work around them. 

For testing the technology and confirming its effectiveness, we have developed a prototype of an integration module for PT Application Inspector and PT Application Firewall, in the form of a .NET HTTP module for IIS web server. A video of the prototype handling the code example above is on YouTube. Performance tests on around fifteen open-source content management systems (CMSs) have shown great results: the time required for processing HTTP requests with RVP is comparable to the time that it takes to process such requests with traditional (heuristic) WAF methods. The average performance hit for web applications was as follows:

  • 0% for requests that do not lead to a vulnerable point
  • 6–10% for requests that lead to a vulnerable point and are not an attack (depending on complexity of the grammar of the vulnerable point)
  • 4–7% for requests that lead to a vulnerable point and are an attack

Despite obvious advantages over traditional VP, RVP still has several conceptual shortcomings:

  • It is not possible to compute formulas that contain data from external sources absent on the WAF (including file resources, databases, and server environment).
  • The quality of formulas directly depends on the quality of approximation of code fragments during analysis (including loops, recursion, and calls to external library methods).
  • To describe semantics of transformation functions for the pre-compute database, some engineering work from the developers is required. The description process is difficult to automate and is prone to human error. 

However, we have managed to mitigate these weaknesses by offloading some RVP functions to the application and by applying the technologies that underlie runtime application self-protection (RASP).

To be continued in Part 2. 

Author: Vladimir Kochetkov

Announcing Bochspwn Reloaded and my REcon Montreal 2017 slides

j00ru//vx tech blog - 20 Červen, 2017 - 17:14

A few days ago at the REcon conference in Montreal, I gave a talk titled Bochspwn Reloaded: Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking. During the presentation, I introduced and thoroughly explained the core concept, inner workings and results of my latest research project: a custom full-system instrumentation based on the Bochs x86 emulator, designed to detect instances of uninitialized kernel memory disclosure to user-mode applications. This work was largely based on the original Bochspwn research, conducted by Gynvael and me in 2013, whose goal was to identify so-called double fetch conditions in the kernels of various popular operating systems (see SyScan slides and whitepaper, Black Hat slides and source code on GitHub). Bochspwn Reloaded repeated the success of its predecessor, so far having found nearly 30 infoleak vulnerabilities in Windows, and more than a dozen lesser issues in Linux.

The most relevant part of the abstract is as follows:

This presentation will introduce another subtle class of kernel vulnerabilities – disclosure of uninitialized stack and heap memory to user-mode applications. Since information leaks of this kind leave hardly any footprint, they are rarely noticed and reported to system vendors. However, we have found that it is still a prevalent problem in current kernels (especially Windows), and can be abused to defeat certain exploit mitigations or steal sensitive data residing in ring-0. In order to address this matter, we have developed a new Bochspwn-style instrumentation based on rudimentary kernel memory taint tracking, which we then used to discover 30 memory disclosure issues in Windows alone. In this talk, we will discuss the kernel design problems behind the bugs, the design of our tool, and the exploitation process of some of the most interesting findings.

Without further ado, the full slide deck presented at REcon can be downloaded below:

Bochspwn Reloaded: Detecting Kernel Memory Disclosure with x86 Emulation and Taint Tracking (6.45 MB, PDF)

The trophy case of bugs indicated by Bochspwn Reloaded thus far is as follows:

  • Details and proof-of-concept programs for all 30 Windows kernel memory disclosure vulnerabilities fixed by Microsoft in April – July 2017, hosted in the Project Zero bug tracker.
  • Bugs detected in Linux kernel 4.8:
    • Kernel stack memory disclosure, llcp_sock_connect in net/nfc/llcp_sock.c (report, fix)
    • Kernel stack memory disclosure (root-only), ctl_ioctl at drivers/md/dm-ioctl.c (external fix)
    • Use of uninitialized stack memory, bind() and connect() handlers in multiple sockets:
    • Use of uninitialized stack memory, deprecated_sysctl_warning in kernel/sysctl_binary.c (report, fix)
    • Use of uninitialized stack memory, SYSC_epoll_ctl in fs/eventpoll.c (external fix)
    • Use of uninitialized heap memory, devkmsg_read in kernel/printk/printk.c (refactored out on 4.10+ kernels)
    • Use of uninitialized heap memory, dnrmg_receive_user_skb in net/decnet/netfilter/dn_rtmsg.c (report, fix)
    • Use of uninitialized heap memory, nfnetlink_rcv in net/netfilter/nfnetlink.c (report, fix)
    • Use of uninitialized stack memory, ext4_update_bh_state in fs/ext4/inode.c (external report, external fix)
    • Use of uninitialized heap memory, nl_fib_lookup in net/ipv4/fib_frontend.c (external fix)
    • Use of uninitialized heap memory, fuse_release_common in fs/fuse/file.c (report, fix)
    • Use of uninitialized stack memory, apply_alternatives in arch/x86/kernel/alternative.c (report, fix)

During the presentation, I also showed animated visualizations of tainted memory layouts of Windows 7, Windows 10 and Ubuntu 16.04 (slides 67, 68 and 117). Since they ended up exported as static images in the PDF, I’m including the original GIFs below. These are 1024×512 (or 1024×256 in case of Linux) views of the entire kernel address space, with lower addresses at the top and higher ones at the bottom. Each pixel represents one 4 kB memory page, and is colored green for stack taint, or red for heap/pool taint. Other characteristics such as the total visualized run time, intervals between subsequent frames (memory state snapshots), and actions performed on the systems are listed next to each specific animation.


Windows 7, 40 minutes of run time, 20s. interval, boot + initial ReactOS tests


Windows 10, 120 minutes of run time, 60s. interval, boot + initial ReactOS tests


Ubuntu 16.04, 60 minutes of run time, 20s. interval, boot + trinity fuzzer + linux test project

Chystá se konec a stěhování. - 28 Květen, 2017 - 01:00
Takže, tohle je poslední příspěvek na starém A také jeden z prvních , co se objeví i na novém, provozovaném na
Kategorie: Blogy

Aktuální vypsané termíny školení - sociální sítě, bezpečnost, internetový marketing, public relations - 17 Květen, 2017 - 01:00
Více informací o všech školeních i dalších možnostech školení na míru najdete na
Kategorie: Blogy
Syndikovat obsah