Encrypting MySQL Backups
Encryption is important component of secure environments. While being intangible property security doesn’t get enough attention when it comes to describing various systems. “Encryption support” is often the most of details what you can get asking how secure the system is. Other important details are often omitted, but the devil in details as we know. In this post I will describe how we secure backup copies in TwinDB.
See the picture. This is what happens when encryption is used incorrectly. The encryption algorithm can be perfect, but poor choice of the mode results in a quite readable encrypted image. This mode is called “Electronic Code Book”, avoid it at all means.
Another bright example of improper encryption use was illustrated in Venona project.
There is encryption algorithm that is mathematically proven to be unbreakable. That means you cannot decrypt a cipher text even with brute force attack and having unlimited computing power and time. It is One-Time Pad. It’s very simple and fast – it’s a XOR of a plain-text message and a key. The only problem with the algorithm is – it requires a key to be as large as the plain text message. But if you can securely transfer the key why would you need the encryption at all? That’s a reason why One-time Pad is not so popular. But it’s still usable. For example, you can generate a really large key, much larger than a typical plain text message. Then use some secure channel to transfer the key and then use parts of the key to send encrypted messages. Basically, this is what Soviets did to spy in USA. They generated a bunch of keys and used them to encrypt messages to a Moscow recipient. Because Soviets were broken by design they either ran of keys or some lazy-bone didn’t destroy used keys and at some point the keys were reused to encrypt more than one message. That broke their encryption and the USA managed to decipher many messages.
Thus there is a fundamental rule. Never use or design home backed encryption algorithms.
The public key of the dispatcher is hard-coded in the agent (I’ll refer to it as firstname.lastname@example.org). The agent uses the public key email@example.com to send encrypted messages to the dispatcher. When a user registers new MySQL server in TwinDB the agent generates its own RSA keys pair, it’s referred as <UUID>@twindb.com, for example firstname.lastname@example.org .
sec 2048R/17B2D796 2014-12-08
uid Backup Server id e456b18e-11eb-49ff-93f9-5341656799fe (No passphrase) <email@example.com>
ssb 2048R/604969DA 2014-12-08
Here’s how you can generate a pair of GPG keys in Python non-interactively:
server_id = "e456b18e-11eb-49ff-93f9-5341656799fe"
email = "firstname.lastname@example.org"
gpg_cmd = ["gpg", "--batch", "--gen-key"]
gpg_script = """
%%echo Generating a standard key
Name-Real: Backup Server id %s
Name-Comment: No passphrase
""" % (server_id, email)
p = subprocess.Popen(gpg_cmd, stdin=subprocess.PIPE)
The agent sends its public key to the dispatcher and the dispatcher since then can communicate securely with the agent. It’s nice that gpg can sign messages, so the dispatcher can authenticate the agents, too.
Encrypting MySQL Backups
When the agent takes a backup it streams the backup copy to gpg process and it encrypts the stream with its own public key.
innobackupex --stream xbstream ./ | gpg --encrypt --yes --batch --no-permission-warning --quiet --recipient email@example.com
Hence one of the biggest features of TwinDB I’m proud of. The agent encrypts the backup copy with its own key. So, we, at TwinDB, can not decrypt user backups! We did it because of two reasons. First, we want our users to trust TwinDB. “You can get much farther with a kind word and encryption than you can with a kind word alone” (c) almost Al Capone. Second, it’s a kind of protection for us. If TwinDB is broken (I have no illusions, there are smarter hackers out there than we are) the burglars won’t gain access to user data.
The private key is generated locally and never leaves the server where it was generated.
But how can we restore the database if the server is destroyed with the private key?
We ask a user to generate their own keys pair and give us the public key. The user should keep the private key in a secure place, print it out and put in a bank deposit box.
If the public key is available the dispatcher schedules a job for the agent and ask it to encrypt its private key with the user’s public key and send it back to the dispatcher. In case if the server is completely destroyed the user can get the encrypted server’s private key, decrypt it and decrypt the backup copy to restore the database.