In message-driven systems, messages flow in a lot of directions. This behavior helps to decouple applications to make these systems better scalable. When it comes to sensitive data, we need to know who has access to this information. To prevent unauthorized people from reading sensitive data, you can apply encryption to these fields. A key should be generated for every resource. This key is stored and only given to the consumers entitled to process this sensitive data according to the security regulations (like the General Data Protection Regulation). Encryption of sensitive data has more advantages. For example, when logging exceptions, no sensitive information will be present in the application logs when this data is encrypted and production data where personal information is encrypted can more easily be used for testing.

When working in an event-sourced application, the events are the single source of truth. This makes the event store the place to encrypt sensitive data attributes. In such an application the events are very important because they will provide an audit trail, and are used to rebuild the state. Your application may not function properly anymore when you delete events, and that’s why events are immutable; they cannot be updated or deleted. 

The right to data erasure is a requirement that is common in security laws. At first glance, the right to erasure seems to be contradictory to event sourcing. A solution to overcome this problem is crypto shredding. Crypto shredding is the concept where one deletes sensitive data by deleting the encryption key that was used to encrypt it. If the encryption is sufficiently strong, and the encryption key is lost, you cannot decrypt the data anymore.  

The Data Protection module 

AxonIQ’s Data Protection Module takes a declarative approach to encrypting sensitive data. It can be used to restrict access to information by certain components, as well as for crypto shredding. This module is available for different versions of Axon but can also be used without Axon. 

The Data Protection Module uses Advanced Encryption Standard (AES) as an encryption algorithm. The default key size is 256 bits, which is highly recommended, but you can also use 128 or 192 bits.

The CryptoEngine is the driving part of the data protection module. You can:

  • Retrieve an already existing key 
  • Given an id, retrieve the existing key or generate a new one
  • Delete a key
  • Override the default key length for new keys
  • Obtain the Java cryptography Cipher object for performing actual encryption

Except for the deletion of a key, these methods are used by the FieldEncrypter class and not in your application code.

Six implementations of the CryptoEngine can be used:

  1. VaultCryptoEngine to store the keys in HashiCorp Vault
  2. InMemoryCryptoEngine for unit testing
  3. JPACryptoEngine to store the keys with JPA
  4. JdbcCryptoEngine for plain JDBC instead of JPA
  5. JavaKeyStoreCryptoEngine which uses the Java Keystore to store the keys
  6. PKCS11CryptoEngine that supports Hardware Security Modules (HSM) using PKCS11

Getting started

To give you an idea of how it works, I am going to implement the Data Protection Module in an example Springboot Axon application using Java and Kotlin. 

First, you need to add the provided jar to your project. In the download package, there are four different versions available: axon4, axon3, axon2, and core. The latter does not depend on Axon at all.

In this blog, we use the axon4 version of the module:

The next two chapters explain how to configure an application with the JPACryptoEngine and the VaultCryptoEngine.

Setup using the JPACryptoEngine

The JPACryptoEngine stores the secret keys in a table in a relational database. To use it, add these beans to the Spring configuration:

The event serializer needs to be configured to use the FieldEncryptingSerializer. This is a wrapper class for the Serializer and will encrypt the sensitive data before applying the serialization. 

The next step is to create the table for the secret keys:

Setup using the VaultCryptoEngine

The VaultCrytoEngine uses HashiCorp Vault to save the keys. In general, Vault provides a more secure location to store the keys, which themselves should be treated as sensitive data too. The following configuration enables the VaultCryptoEngine:

 The prefix configured here is the name of the KV secrets engine used in the Vault Server. The version of that engine should be 1:

Identify the sensitive data

Now that the CryptoEngine is set up for managing the keys, it is time to identify our sensitive data. In our example, when an Account is registered, the AccountRegisteredEvent is published. We will add  the PersonalData annotation on the password of the account to mark it as sensitive data and use the accountId as the identifier of the data subject:

The prefix is optional and will be used in the Keystore to be able to distinguish between the different groups of data subjects. The group, which is an optional attribute, tells the Data Protection module which category of sensitive data this field belongs to. Fields in different groups will use different encryption keys, which allows you to control access to each group separately.

Running the application

Before you start the application, you need to provide it with a valid license file. You can store the license on your local machine and point to it by adding the VM option: 

-Daxoniq.dataprotection.license=path-to-your-licence/axoniq.license

You can run AxonServer locally and start the application and apply the AccountRegisteredEvent with a password Welcome1. This event will be stored in the event store, and the payload looks like this:

You can see that the password was saved in an encrypted format. If you have configured the VaultCryptoEngine you can see these entries in your Vault server:

The Data Protection Module in action

Until now the examples used in this blog were tied to Axon but the Data Protection Module can also be used without it. The unit tests below, written in Kotlin, showcase the capabilities of the data protection module.

Example 1: 

Encrypt and decrypt the sensitive data inside the AccountRegisteredEvent:

The output of the logger statement:

Encryptedpassword [CAEVlUhITxoQLm3mZGxmGtnTG1NwEQfwPSFMJZjivdkLQinruTXtUGHa1g==]

This encrypted value of the password is now saved in the password field.

Example 2: 

Encrypt and decrypt data in custom objects annotate them with DeepPersonalData: 

In this code sample, some new annotations are being used. The DeepPersonalData annotation tells the DP module that it needs to look into the Address object to find more annotated fields.

The house number in the Address object is annotated as SerializedPersonalData, and an extra field houseNumberEncrypted was introduced, defined as a byteArray. This additional field is needed because the result of the encrypted house number can not be stored in a field that is defined as a Number.  

When running this test, the output of the log statement is:

Encryptedaddress [Address(street=CAEVanORchoQ7mXP0GTuig8piCE7PGc+tSHMhjm+YxulXik+L0TdBLQu5A==, houseNumber=0, houseNumberEncrypted=[8, 1, 18, 3, 105, 110, 116, 34, 43, 8, 1, 21, -91, 93, 39, 86, 26, 16, -13, -45, -102, 96, 78, 120, 96, 72, 63, -104, -125, -102, 87, -27, -17, 7, 33, -30, 98, -73, 63, 100, 106, -31, 72, 41, -32, 49, -36, -109, -57, -77, -47, -52], zip=zip, city=CAEVdVSjAhoQeBE8kG4862AIZXOYilp9giGu11H0CmGTOCn3PVgEI55Mag==)]

Example 3: Crypto shredding

In this test, the event is encrypted, and then the key is deleted. After decrypting the street, house number and city are replaced by either an empty String, a replacement value, or (in case of a number) by 0. It is possible to customize this behavior, as shown in the next example.

Example 4: Replacement behavior when the key is deleted

An event store contains valuable information that you might want to keep for analytics. When deleting sensitive data, you may want to keep the year of birth in the case of date of birth. This can be done with a custom implementation of the ReplacementValueProvider:

In the unit test, you can provide this class to the FieldEncrypter:

As you can see, the date of birth changed into Jan 1st of the year of birth.

Conclusion

The Data Protection Module can help you to implement security regulations like GDPR by restricting access to sensitive data in distributed systems. In some jurisdictions, crypto shredding alone may not be feasible because, in theory, the encrypted data can still be decrypted. They state that brute force cracking a key may just be a matter of hardware, time, and technological progress. In that case, you may want to think about putting your sensitive data in a secure database instead of saving them in the events. You could use the replacement feature in the data protection module to add a reference to it. GDPR, for instance, does not prescribe which deletion method should be used. 

The Data Protection Module is commercial software. For information about the software, pricing, and licensing, send an email to sales@axoniq.io.

For more information on the Axon products, check https://axoniq.io/product-overview 

Subscribe to blog notifications