On offensive engagements, we frequently encounter centralized internal password managers that are used by various departments to store incredibly sensitive account information, such as Domain Admin accounts, API keys, credit card data, the works. It used to be that these systems were implemented without multi-factor authentication. “Hacking” them was as simple as finding somebody that had access, then logging in with their domain credentials. It is now commonplace to see these systems instrumented with multi-factor authentication, making it much more difficult to compromise via the standard user interface.

One such system is Thycotic Secret Server (TSS), a comprehensive solution for internal password management that does what it does really well. To be clear, this article does not drop any exploits against the app itself. It is simply a walkthrough for exposing the protected, encrypted secrets offline and decrypting into a plaintext, readable format.

If you are on an internal test, here is the access and components you’ll need to pull this off:

1. An account that has read access to the Secret Server database instance. I do not know if TSS supports various database platforms. In this article, we are assuming Microsoft SQL.
2. The TSS database instance name. I used Metasploit’s auxiliary/admin/mssql/mssql_enum module for enumeration.
3. An account that has read access to the TSS web site files. This might be the web server itself, or a back up server. You specifically need these files from the webroot directory:
   1. \encryption.config
   2. \bin\Thycotic.AppCore.dll
   3. \bin\Thycotic.ihawu.Business.dll
4. Clone this brilliant piece of work onto your Windows machine. I don’t know who made this, but you have my gratitude! https://github.com/denandz/SecretServerSecretStealer
5. Important note: This methodology works at least up through TSS v 10.4. Not sure about 10.5-10.7. Newer versions of TSS (e.g. v10.8+) use a different database format for storing the encrypted data. You know you are working with the newer format if you see “FFFFFF…” for the IV/IVMek. All the data you need to perform an offline decrypt of this data is in the extract you receive, and I may explore it in a different blog post, but as of right now, I’ll leave that as an exercise for the reader. ????

If you have all those pieces, then you are ready to fire the query that will retrieve all the encrypted information. Use your favorite way of executing MSSQL queries from your attack machine. I used Metasploit’s auxiliary/admin/mssql/mssql_sql.

Before executing the query, turn on spooling (assuming you’re using MSF) with spool /root/secretserverdata.raw.

The query to use is:

select s.SecretName, f.SecretFieldName, s.[Key], s.IvMEK, i.ItemValue, i.IV from <strong><DATABASENAME></strong>.dbo.tbSecretItem as i JOIN <strong><DATABASENAME></strong>.dbo.tbSecret as s ON (s.SecretID = i.SecretID) JOIN <strong><DATABASENAME></strong>.dbo.tbSecretField as f on (i.SecretFieldID = f.SecretFieldID)

(Again, huge credit to the author or SecretServerStealer for the tool and the query!)

Note that you need to substitute the actual database name in the query above. This query might take a while depending on how much the secret server has been used. It will also dump out a truckload of data to the spooled file. In my case, the query took about 5 minutes and the resulting file was 56MB, most of it whitespace.

Once that’s done, bring the file to your machine for processing along with the webserver files mentioned in step #3 above. I must stop and remind you that you are handling some of the most sensitive information in the company, please use proper precautions when handling this data. I used shred to securely wipe all temporary instances as it moved through infrastructure, and gocryptfs to ensure the file was securely stored at rest.

If you used MSF, you are going to need to do a bit of processing on the file in order to clear up the whitespace and convert it to a comma delimited file:

1. Remove any MSF spooling comments at the top and bottom of the file
2. I used sed to replace any 10 or more spaces with a comma. This worked reasonably well, but your mileage may vary.
   1. sed -i -e 's/ \{10,\}/,/g' /path/to/secretserverdata.raw

Now it’s time for the fun part! SecretServerStealer was meant to be run on the Secret Server itself. It makes straight database calls to pull the information back and parses it out. We are doing this offline, so we need to add some code to wrap each entry in the decryption routine. Before we get there, do a test to make sure you can actually decrypt something with the data you have.

Look at the Invoke-SecretDecrypt method in the PowerShell script. It requires 5 parameters:

• EncryptionConfig - The path to the encryption.config file.
• Key - The encryption key used for encryption. Corresponds to the Key value in the database extract.
• IvMEK - Corresponds to the IvMEK database field.
• Item -The item to decrypt. Corresponds to the ItemValue database field.
• ItemIV - Corresponds to the IV database field.
• NewFormat (Optional) - Tries to decrypt the encryption.config file of a newer installation of TSS. Try with and without if you don’t know the version. I had to use it as I was on a newer version of TSS.

Source the SecretStealer.ps1 file, then load up a set of values from your database extract. If all goes well, you’ll see the decrypted value!

(Side Note: I saw several entries in my data set that were missing pieces of data like Keys and IVMeks. I’m not sure why this is the case, but I’m assuming it’s due a database upgrade process or these are simply archive entries that are no longer valid. In any case, I removed them from my data set.)

Now we need to simply wrap the Invoke-SecretDecrypt function with some PowerShell code that iterates across our data set. Grab the Invoke-SecretWrap function from here and paste at the bottom of SecretStealer.ps1. Source the file, then run the Invoke-SecretWrap function.

While the glorious creds flow, take a minute and pour your favorite beverage. xDD

Defensive Measures

This entire attack hinges on you having the right pieces of data: the TSS database encrypted values, and the encryption.config file. Without one, the other is useless.

Protecting the TSS Application Server

1. The TSS Application server is the most important piece of the TSS infrastructure, far more important than the database itself. If an attacker gains access to the Application server, then a full offline decrypt can be performed even in the most recent versions of TSS and without the attacker having credentialed access to the database server! If your organization is saving critical secrets/credentials/data in TSS, then you should consider the TSS Application server on par with a domain controller regarding security:
   1. Multi-factor authentication for remote access wherever possible, especially RDP.
   2. Implement firewall rules to restrict remote administrative functions to trusted administrator machines, preferably using a Privileged Access Workstation model.
   3. If possible, set tripwires for the access of the TSS application DLL files that are being touched outside the IIS process. Giving you pearls here, Charlie…
   4. Log critical IIS changes, such as the adding of new sites to the App Pool.
2. At a minimum, TSS supports DPAPI encryption of the database.config and encryption.config files. This should be implemented. It’s not perfect, because DPAPI keys can be extracted easily, but again, this forces more work and risk onto the attacker.
3. The Cadillac solution: TSS supports protecting the encryption.config file with a Hardware Security Module (HSM). HSM’s are pricey, but simply outstanding as a security control and can be used to protect much more than a simple string. They can encrypt private keys, whole file blobs, just about anything you can throw at it. The keys never leave the HSM, so it is absolutely a worthwhile investment for the encryption of your crown jewel data.

Protecting the Database Server:

1. Use the local firewall to restrict incoming connections on the TSS database instance port to just the TSS application servers.
2. Monitor for certain “select *” type queries wherever possible.
3. Ideally, follow similar principals for protecting the database server as you would the application server.

Until next time. Happy Hacking!

@curi0usJack