Automating ZAP through Gauntlt — A DevOps Solution

Rugged DevOps, or DevSecOps, is a method for developing software that is gaining much traction in recent years. However, the security tools and practices may not merge well with automation.

This produces bottlenecking or delays security processes until time-consuming manual tests at the end of a release cycle. Such delays in security testing greatly increase the risk in identifying a security vulnerability which delays or halts a release schedule. That is not ideal when working to design a DevOps pipeline.

I recommend using Gauntlt to help resolve these issues and incorporate automated security into DevOps. Gauntlt is a free and open source tool which bundles together various common security tools and can add more to further customize a test suite.

These security tools include network testers, such as NMap or SSlyze, and SQLmap or Arachni for testing of injection and XSS vulnerabilities, respectively.

Individual test attacks may be applied through Gauntlt or a group of attacks may be initiated. This is similar to a list of tasks used to test an application within the DevOps process.

These attacks are written in the Gherkin language, which can be used to describe automated tests without defining exactly how those tests are implemented. For those familiar with Cucumber, Gherkin uses Cucumber syntax to define the tests in a series of 3 basic statements:

Given a scenario,

When an action is taken,

Then it will result in a specific outcome.

Gauntlt’s test suite can be combined with the OWASP Zed Attack Proxy (ZAP), a great web application vulnerability scanner, to be more robust. Specifically, ZAP was implemented via zap-cli, developed by Grunny, to start the zap daemon and initiate a scan. Below is the attack file, myAttack.attack, which may be used by Gauntlt to automatically

@final

Feature: OWASP ZAP vulnerability scan

Scenario: The ZAP attack is designed to run a vulnerability scan to identify a range of issues within the target application. Note that this takes a highly conservative approach of finding 0 issues. This attack will fail in most applications.

Given The default aruba timeout is 120 seconds

When I launch a "generic" attack with:

"""

zap-cli --verbose --port 8099 --api-key 12345 quick-scan --self-contained -o '-config api.key=12345' --spider -r http://localhost/

"""

Then the output should contain:

"""

Issues found: 0

"""

 

Speaking Gherkin

Tags are used by Gauntlt to group various attacks and convey the meaning of the attacks between developer, security, and operation resources. The @final tag indicates the attack is ready to be used. Alternatively, a @draft tag may be applied for attacks being developed and tested. The recommended practice is to explicitly define the tags for various attacks. Some tags to include are @slow for Gauntlt tests that take a long time to complete and @production for tests that are run in live environments.

The Feature and Scenario statements make the scripts more user-friendly by explaining the intentions of the attack. These are not executed by Gauntlt when the attack is executed. The Feature statement defines the high-level attack or attacks, while the Scenario defines the specifics of its Given, When, Then. Please note that multiple Scenarios and Given, When, Then attacks may be defined for a single Feature within one .attack file.

For the Given statement, I do not like including magic numbers but needed to set one for ZAP to run correctly. Without defining the Aruba timeout, default Gauntlt commands are allowed 3 seconds to complete. Those familiar with ZAP, or vulnerability scanners in general, know these take significantly longer to complete. Hence, I needed to define a longer timeout variable. The Given statement can be optional when initial requirements are not needed. In most cases, the Given statement is used to check whether the security tool is installed. This only works for security tools which are natively bundled with Gauntlt

The When statement defines the exact command line statement that will be executed by Gauntlt. I recommend running all attack commands via the command line to ensure they work as expected prior to running them through Gauntlt. That will save time when debugging issues. In this particular case, zap-cli must be installed and added to the system’s PATH as well as the zap.sh script. Otherwise, the zap-cli tool will not work properly. Gauntlt does have a small set of security tools it can use natively. Anything outside of that list, such as ZAP, will need to be defined as a “generic” attack.

The Then statement defines the exact regular expression expected to be found at some point in the output of the previous command line’s execution.

To learn more about Gauntlt’s finite list of command statements, visit this site or type the following:

gauntlt --allsteps

Scenario outlines may be applied to reduce the clutter within an attack file. These create placeholder values within a generic scenario which may be applied through a table of values when executed. The placeholder is defined within <> braces, similar to <placeholderValue>. This can be used to run multiple instances of the same scenario using different input. The sample Gauntlt curl attack, verbs.attack, illustrates a scenario outline:

@slow
Feature: Evaluate responses to various HTTP methods.

Background:

Given "curl" is installed

And the following profile:

| name | value |

| hostname | google.com |

Scenario Outline: Verify server responds correctly to various HTTP methods

When I launch a "curl" attack with:

"""

curl -i -X <method> <hostname>

"""

Then the output should contain "<response>"

Examples:

| method  | response                          |
| delete  | Error 405 (Method Not Allowed)    |
| patch   | Error 405 (Method Not Allowed)    |
| trace   | Error 405 (Method Not Allowed)    |
| track   | Error 405 (Method Not Allowed)    |
| bogus   | Error 405 (Method Not Allowed)    |

In this case, the curl request will be made to a single host, google.com, multiple times according to the responses defined. The target hostname is set in the profile and applied uniformly, while the examples set the individual methods and expected responses of each iteration. The output of this attack is shown in a later section. To use a different hostname for each individual test, a third column may be applied to the examples.

Running Gauntlt

To have Gauntlt execute every attack in the current and sub-directories, simply run:

gauntlt

This is the equivalent of:

gauntlt ./**/*.attack

Gauntlt can be further tailored to run a specific attack pattern:

gauntlt ./myAttack.attack

To run Gauntlt and display the output in an easily readable format, such as html, use the following command:

gauntlt --format html > /tmp/report.html

Understanding Gauntlt’s Output

The html report gathers the direct output from Gauntlt and color codes it in terms of passing tests, green, and failing tests, red. These include the Given, When, Then statements which Gauntlt ran.

When attacks fail, the command’s output is shown in red. The line beginning with the -, in this case -Error 405 (Method Not Allowed), shows the specific reason this attack failed. This was intended to check that the HTTP Status returned for a DELETE method request was 501 to indicate that the method is not implemented. However, the response included a 405 error to indicate the method is not allowed. This attack should be rewritten as the DELETE method vulnerability is not present in the environment.

Troubleshooting

Under certain conditions, the zap output, and by association, the Gauntlt output, will fail and not provide good error reporting for troubleshooting. When configuring and testing the attack above, I kept running into an error alert which culminated in the following message:

File “/home/osboxes/.local/lib/python2.7/site-packages/requests/sessions.py”, line 609, in send
r = adapter.send(request, **kwargs)
File “/home/osboxes/.local/lib/python2.7/site-packages/requests/adapters.py”, line 473, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: (‘Connection aborted.’, BadStatusLine(“””,))

Annoyingly, searching for help on this issue led me to believe a bug in a python2.7 dependency was preventing the connection. However, that is not correct. As of ZAP 2.5.0, an API key is needed to prevent ZAP from running against its host machine or network. This security feature makes automating or command line calls more difficult.

In the above Given, Then, When, I used zap-cli to configure the API key value, with -o ‘-config api.key=12345’, and applied it to the current instance with, –api-key 12345. Please refer to the github location above for usage of zap-cli.

The Gauntlt tool can be used to incorporate numerous scanners to identify low hanging fruit. Most importantly, it can move security processes earlier in the SDLC.