How Buttercup creates an encrypted password archive
I’ve had a great time writing the Buttercup password manager with Sallar, and we’ve both learnt alot about how the system behind a credentials vault should work; But even before the thousands of lines of code, I had an idea about how I’d make a password manager.
TLDR; This is a long discussion on how Buttercup functions: How the encryption works, what data structures and used, how information is handled and moved around within the application. It may take some minutes to read through!
Buttercup is designed to store credentials, like any decent manager, but in a way that makes it easy to remotely access and avoid conflicts when being used from multiple locations or interfaces. The internal system uses a list of commands (delta changes) that when run, make up an in-memory object that constitutes an archive (groups and entries).
Credentials (entries) are stored within groups in the archive, which can be nested. Credentials have properties (title, username, password etc.) and meta data (notes, URLs, user-entered key-value pairs). An archive when in memory may look like the following:
At the system’s core
There are two main classes that make up a usable Buttercup archive: The archive class, and Westley.
The archive class, formerly named “Buttercup”, represents an instantiated archive instance ready to take commands and return credentials. With an instance you can do things like
The archive instance, behind the scenes, holds an instance of a class named
Westley. This class is responsble for holding both the history array (of delta commands) and the dataset (the in-memory archive object discussed earlier).
These two classes work together to provide the core of an archive’s functionality. When you open an archive instance with the Buttercup application on desktop or mobile, you make use of a third class called
Workspace. This instance holds three things:
- An archive instance
- The password
- An archive datasource (discussed later)
var Buttercup = require("buttercup");
Entries and groups
Like most password managers - and I feel that at least here, being similar is a good thing - Buttercup uses a simple data architecture to store information. An archive is entirely made up of groups and entries, both of which can be moved around within each other recursively.
Take for instance the following example:
var generalGroup = archive.createGroup("General"),
Entries are created and moved within groups. Groups are nestable, so they can be created on the root level (
archive.createGroup()) or inside an existing group (
Datasources, local and remote
Archives can be located anywhere, and I wanted Buttercup to address that head-on. It’s up to the user where and how they store their encrypted archive, and it’s simple enough to provide interfaces to allow them to connect to any archive datasource easily.
Buttercup comes with some basic datasources out of the box:
- Text (read from various sources etc.)
- File (local filesystem, USB disk etc.)
- WebDAV (cloud/remote storage that supports the WebDAV protocol)
- ownCloud (the best open-source cloud storage system, IMHO)
Providing the datasource is valid and the underlying data exists, proving this and a password to a
Workspace instance will yield an
Archive instance. Implementing support for new datasources is trivial as they all support the same interface.
Operations and command generation
When an archive is being worked with (items being created, edited and destroyed), commands are generated and stored in the delta history so that the next time the archive is opened, the same state can be reached.
var groupA = archive.createGroup("Group A");
The first part of the command is the name of the operation:
cgr means create-group and
sep means set-entry-property. Commands operate on items (groups or entries), which have IDs. Items are identified by UUIDs (v4) in Buttercup archives so they can be referenced directly from the deltas (flat structure).
groupA.delete() command in the example - We can delete a group without deleting the entries inside of it. Since the in-memory archive is a tree, all leaves on a branch are removed by deleting the parent. As the archive grows in size (history length) it can eventually be flattened so that created-and-deleted items can be removed from the history altogether.
Let’s see an example going the other direction:
cgr 0 4b38b5d0-8cbd-44fc-a1f3-51d18c5b929e
The tree structure that this set of deltas forms is this:
The sequence of actions here is quite straightforward:
- Create a group, in root, title it “Item 1”
- Create a group, in root, title it “Item 2”
- Create an entry in “Item 2”, title it “The only entry”
- Move “Item 2” into “Item 1”
This process is how Buttercup interacts with the saved state of archives.
Saving an archive to an encrypted state
The process of storing the credentials of a user is perhaps one of the most important aspects of how a password manager works.
From memory to disk, Buttercup starts by optimising the history list - it strips all superfluous commands from early on in the history, and ensures the resulting archive remains the name. For instance, if an archive has 1500 lines of historical commands, Buttercup will flatten the first 1000 and leave the last 500 untouched.
Once the archive has been cleaned up, it’s convered into a text file (in memory) and compressed using GZIP. This drastically reduces the size of the archive.
After compression, Buttercup performs text encryption using the AES Cipher-Block-Chaining algorithm with a 256bit key. The key is salted and prepared with many rounds of PBKDF2 before being passed to the encryption method, and a SHA-256 HMAC is prepared over all of the stored & encrypted data.
Before writing the encrypted contents to their destination, a header prefixed to the file contents. This is the final phase of encryption, and decryption obviously occurs in reverse order.
In the very near future
Buttercup will be a cross-platform solution to password archive management, and NodeJS can help us cover the desktop environments. Where NodeJS current falters is the mobile application realm, where we’ll most likely build native applications and frameworks for Buttercup. Keep an eye out for my posts on building Buttercup in Objective-C and possibly Java.