Crypt

1. Backblaze Rclone Mount
2. Rclone S3
3. Rclone Backblaze B2

The Backblaze Personal Backup client was started in January of 2007 as a ‘C’ and C source tree that compiled on Windows, Macintosh, and Linux (originally CentOS but later when we standardized on Debian for the server side we now compile the client on Debian).

Rclone config reconnect. Re-authenticates user with remote. This reconnects remote: passed in to the cloud storage system. To disconnect the remote use 'rclone config disconnect'. This normally means going through the interactive oauth flow again. Rclone config reconnect remote: flags Options-h, -help help for reconnect. Backblaze B2 via rclone cheatsheet. Create B2 remote; Tasks. List buckets; Synchronize local files to bucket; Synchronize local files to bucket - checksum; Verify local files against bucket; Reference; Create B2 remote. Where 'Key ID' is account. Where 'Application key' is key. Using b2 as the remote name and used in all task examples (e.g.

Rclone crypt remotes encrypt and decrypt other remotes.

A remote of type crypt does not access a storage systemdirectly, but instead wraps another remote, which in turn accessesthe storage system. This is similar to how alias,union, chunkerand a few others work. It makes the usage very flexible, as you canadd a layer, in this case an encryption layer, on top of any otherbackend, even in multiple layers. Rclone's functionalitycan be used as with any other remote, for example you canmount a crypt remote.

Accessing a storage system through a crypt remote realizes client-sideencryption, which makes it safe to keep your data in a location you donot trust will not get compromised.When working against the crypt remote, rclone will automaticallyencrypt (before uploading) and decrypt (after downloading) on your localsystem as needed on the fly, leaving the data encrypted at rest in thewrapped remote. If you access the storage system using an applicationother than rclone, or access the wrapped remote directly using rclone,there will not be any encryption/decryption: Downloading existing contentwill just give you the encrypted (scrambled) format, and anything youupload will not become encrypted.

The encryption is a secret-key encryption (also called symmetric key encryption)algorithm, where a password (or pass phrase) is used to generate real encryption key.The password can be supplied by user, or you may chose to let rclonegenerate one. It will be stored in the configuration file, in a lightly obscured form.If you are in an environment where you are not able to keep your configurationsecured, you should addconfiguration encryptionas protection. As long as you have this configuration file, you will be able todecrypt your data. Without the configuration file, as long as you rememberthe password (or keep it in a safe place), you can re-create the configurationand gain access to the existing data. You may also configure a correspondingremote in a different installation to access the same data.See below for guidance to changing password.

Encryption uses cryptographic salt,to permute the encryption key so that the same string may be encrypted indifferent ways. When configuring the crypt remote it is optional to enter a salt,or to let rclone generate a unique salt. If omitted, rclone uses a built-in unique string.Normally in cryptography, the salt is stored together with the encrypted content,and do not have to be memorized by the user. This is not the case in rclone,because rclone does not store any additional information on the remotes. Use ofcustom salt is effectively a second password that must be memorized.

File content encryption is performed usingNaCl SecretBox,based on XSalsa20 cipher and Poly1305 for integrity.Names (file- and directory names) are also encryptedby default, but this has some implications and is thereforepossible to turned off.

Configuration

Here is an example of how to make a remote called secret.

To use crypt, first set up the underlying remote. Follow therclone config instructions for the specific backend.

Before configuring the crypt remote, check the underlying remote isworking. In this example the underlying remote is called remote.We will configure a path path within this remote to contain theencrypted content. Anything inside remote:path will be encryptedand anything outside will not.

Configure crypt using rclone config. In this example the cryptremote is called secret, to differentiate it from the underlyingremote.

When you are done you can use the crypt remote named secret justas you would with any other remote, e.g. rclone copy D:docs secret:docs,and rclone will encrypt and decrypt as needed on the fly.If you access the wrapped remote remote:path directly you will bypassthe encryption, and anything you read will be in encrypted form, andanything you write will be unencrypted. To avoid issues it is best toconfigure a dedicated path for encrypted content, and access itexclusively through a crypt remote.

Important The crypt password stored in rclone.conf is lightlyobscured. That only protects it from cursory inspection. It is notsecure unless configuration encryption of rclone.conf is specified.

A long passphrase is recommended, or rclone config can generate arandom one.

The obscured password is created using AES-CTR with a static key. Thesalt is stored verbatim at the beginning of the obscured password. Thisstatic key is shared between all versions of rclone.

If you reconfigure rclone with the same passwords/passphraseselsewhere it will be compatible, but the obscured version will be differentdue to the different salt.

Rclone does not encrypt

• file length - this can be calculated within 16 bytes
• modification time - used for syncing

Specifying the remote

When configuring the remote to encrypt/decrypt, you may specify anystring that rclone accepts as a source/destination of other commands.

The primary use case is to specify the path into an already configuredremote (e.g. remote:path/to/dir or remote:bucket), such thatdata in a remote untrusted location can be stored encrypted.

You may also specify a local filesystem path, such as/path/to/dir on Linux, C:pathtodir on Windows. By creatinga crypt remote pointing to such a local filesystem path, you canuse rclone as a utility for pure local file encryption, for exampleto keep encrypted files on a removable USB drive.

Note: A string which do not contain a : will by rclone be treatedas a relative path in the local filesystem. For example, if you enterthe name remote without the trailing :, it will be treated asa subdirectory of the current directory with name 'remote'.

If a path remote:path/to/dir is specified, rclone stores encryptedfiles in path/to/dir on the remote. With file name encryption, filessaved to secret:subdir/subfile are stored in the unencrypted pathpath/to/dir but the subdir/subpath element is encrypted.

The path you specify does not have to exist, rclone will createit when needed.

If you intend to use the wrapped remote both directly for keepingunencrypted content, as well as through a crypt remote for encryptedcontent, it is recommended to point the crypt remote to a separatedirectory within the wrapped remote. If you use a bucket based storagesystem (e.g. Swift, S3, Google Compute Storage, B2, Hubic) it is generallyadvisable to wrap the crypt remote around a specific bucket (s3:bucket).If wrapping around the entire root of the storage (s3:), and use theoptional file name encryption, rclone will encrypt the bucket name.

Changing password

Should the password, or the configuration file containing a lightly obscuredform of the password, be compromised, you need to re-encrypt your data witha new password. Since rclone uses secret-key encryption, where the encryptionkey is generated directly from the password kept on the client, it is notpossible to change the password/key of already encrypted content. Just changingthe password configured for an existing crypt remote means you will no longerable to decrypt any of the previously encrypted content. The only possibilityis to re-upload everything via a crypt remote configured with your new password.

Depending on the size of your data, your bandwith, storage quota etc, there aredifferent approaches you can take:

• If you have everything in a different location, for example on your local system,you could remove all of the prior encrypted files, change the password for yourconfigured crypt remote (or delete and re-create the crypt configuration),and then re-upload everything from the alternative location.
• If you have enough space on the storage system you can create a new cryptremote pointing to a separate directory on the same backend, and then userclone to copy everything from the original crypt remote to the new,effectively decrypting everything on the fly using the old password andre-encrypting using the new password. When done, delete the original cryptremote directory and finally the rclone crypt configuration with the old password.All data will be streamed from the storage system and back, so you willget half the bandwith and be charged twice if you have upload and download quotaon the storage system.

Note: A security problem related to the random password generatorwas fixed in rclone version 1.53.3 (released 2020-11-19). Passwords generatedby rclone config in version 1.49.0 (released 2019-08-26) to 1.53.2(released 2020-10-26) are not considered secure and should be changed.If you made up your own password, or used rclone version older than 1.49.0 ornewer than 1.53.2 to generate it, you are not affected by this issue.See issue #4783 for moredetails, and a tool you can use to check if you are affected.

Example

Create the following file structure using 'standard' file nameencryption.

Copy these to the remote, and list them

The crypt remote looks like

The directory structure is preserved

Without file name encryption .bin extensions are added to underlyingnames. This prevents the cloud provider attempting to interpret filecontent.

File name encryption modes

Off

• doesn't hide file names or directory structure
• allows for longer file names (~246 characters)
• can use sub paths and copy single files

Standard

• file names encrypted
• file names can't be as long (~143 characters)
• can use sub paths and copy single files
• directory structure visible
• identical files names will have identical uploaded names
• can use shortcuts to shorten the directory recursion

Obfuscation

This is a simple 'rotate' of the filename, with each file having a rotdistance based on the filename. Rclone stores the distance at thebeginning of the filename. A file called 'hello' may become '53.jgnnq'.

Obfuscation is not a strong encryption of filenames, but hindersautomated scanning tools picking up on filename patterns. It is anintermediate between 'off' and 'standard' which allows for longer pathsegment names.

There is a possibility with some unicode based filenames that theobfuscation is weak and may map lower case characters to upper caseequivalents.

Obfuscation cannot be relied upon for strong protection.

• file names very lightly obfuscated
• file names can be longer than standard encryption
• can use sub paths and copy single files
• directory structure visible
• identical files names will have identical uploaded names

Cloud storage systems have limits on file name length andtotal path length which rclone is more likely to breach using'Standard' file name encryption. Where file names are less thn 156characters in length issues should not be encountered, irrespective ofcloud storage provider.

An alternative, future rclone file name encryption mode may toleratebackend provider path length limits.

Directory name encryption

Crypt offers the option of encrypting dir names or leaving them intact.There are two options:

True

Encrypts the whole file path including directory namesExample:1/12/123.txt is encrypted top0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0

False

Only encrypts file names, skips directory namesExample:1/12/123.txt is encrypted to1/12/qgm4avr35m5loi1th53ato71v0

Modified time and hashes

Crypt stores modification times using the underlying remote so supportdepends on that.

Hashes are not stored for crypt. However the data integrity isprotected by an extremely strong crypto authenticator.

Use the rclone cryptcheck command to check theintegrity of a crypted remote instead of rclone check which can'tcheck the checksums properly.

Standard Options

Here are the standard options specific to crypt (Encrypt/Decrypt a remote).

--crypt-remote

Remote to encrypt/decrypt.Normally should contain a ':' and a path, e.g. 'myremote:path/to/dir','myremote:bucket' or maybe 'myremote:' (not recommended).

• Config: remote
• Env Var: RCLONE_CRYPT_REMOTE
• Type: string
• Default: '

--crypt-filename-encryption

How to encrypt the filenames.

• Config: filename_encryption
• Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION
• Type: string
• Default: 'standard'
• Examples:
  • 'standard'
    • Encrypt the filenames see the docs for the details.
  • 'obfuscate'
    • Very simple filename obfuscation.
  • 'off'
    • Don't encrypt the file names. Adds a '.bin' extension only.

--crypt-directory-name-encryption

Option to either encrypt directory names or leave them intact.

NB If filename_encryption is 'off' then this option will do nothing.

• Config: directory_name_encryption
• Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
• Type: bool
• Default: true
• Examples:
  • 'true'
    • Encrypt directory names.
  • 'false'
    • Don't encrypt directory names, leave them intact.

--crypt-password

Password or pass phrase for encryption.

NB Input to this must be obscured - see rclone obscure.

• Config: password
• Env Var: RCLONE_CRYPT_PASSWORD
• Type: string
• Default: '

--crypt-password2

Password or pass phrase for salt. Optional but recommended.Should be different to the previous password.

NB Input to this must be obscured - see rclone obscure.

• Config: password2
• Env Var: RCLONE_CRYPT_PASSWORD2
• Type: string
• Default: '

Advanced Options

Here are the advanced options specific to crypt (Encrypt/Decrypt a remote).

--crypt-server-side-across-configs

Allow server-side operations (e.g. copy) to work across different crypt configs.

Normally this option is not what you want, but if you have two cryptspointing to the same backend you can use it.

This can be used, for example, to change file name encryption typewithout re-uploading all the data. Just make two crypt backendspointing to two different directories with the single changedparameter and use rclone move to move the files between the cryptremotes.

• Config: server_side_across_configs
• Env Var: RCLONE_CRYPT_SERVER_SIDE_ACROSS_CONFIGS
• Type: bool
• Default: false

--crypt-show-mapping

For all files listed show how the names encrypt.

If this flag is set then for each file that the remote is asked tolist, it will log (at level INFO) a line stating the decrypted filename and the encrypted file name.

This is so you can work out which encrypted names are which decryptednames just in case you need to do something with the encrypted filenames, or for debugging purposes.

• Config: show_mapping
• Env Var: RCLONE_CRYPT_SHOW_MAPPING
• Type: bool
• Default: false

--crypt-no-data-encryption

Option to either encrypt file data or leave it unencrypted.

• Config: no_data_encryption
• Env Var: RCLONE_CRYPT_NO_DATA_ENCRYPTION
• Type: bool
• Default: false
• Examples:
  • 'true'
    • Don't encrypt file data, leave it unencrypted.
  • 'false'
    • Encrypt file data.

Backend commands

Here are the commands specific to the crypt backend.

Run them with

The help below will explain what arguments each command takes.

See the 'rclone backend' command for moreinfo on how to pass options and arguments.

These can be run on a running backend using the rc commandbackend/command.

encode

Encode the given filename(s)

This encodes the filenames given as arguments returning a list ofstrings of the encoded results.

Usage Example:

decode

Decode the given filename(s)

This decodes the filenames given as arguments returning a list ofstrings of the decoded results. It will return an error if any of theinputs are invalid.

Usage Example:

Backing up a crypted remote

If you wish to backup a crypted remote, it is recommended that you userclone sync on the encrypted files, and make sure the passwords arethe same in the new encrypted remote.

This will have the following advantages

• rclone sync will check the checksums while copying
• you can use rclone check between the encrypted remotes
• you don't decrypt and encrypt unnecessarily

For example, let's say you have your original remote at remote: withthe encrypted version at eremote: with path remote:crypt. Youwould then set up the new remote remote2: and then the encryptedversion eremote2: with path remote2:crypt using the same passwordsas eremote:.

To sync the two remotes you would do

And to check the integrity you would do

File formats

File encryption

Files are encrypted 1:1 source file to destination object. The filehas a header and is divided into chunks.

Header

• 8 bytes magic string RCLONEx00x00
• 24 bytes Nonce (IV)

The initial nonce is generated from the operating systems cryptostrong random number generator. The nonce is incremented for eachchunk read making sure each nonce is unique for each block written.The chance of a nonce being re-used is minuscule. If you wrote anexabyte of data (10¹⁸ bytes) you would have a probability ofapproximately 2×10⁻³² of re-using a nonce.

Chunk

Each chunk will contain 64kB of data, except for the last one whichmay have less data. The data chunk is in standard NaCl SecretBoxformat. SecretBox uses XSalsa20 and Poly1305 to encrypt andauthenticate messages.

Each chunk contains:

• 16 Bytes of Poly1305 authenticator
• 1 - 65536 bytes XSalsa20 encrypted data

64k chunk size was chosen as the best performing chunk size (theauthenticator takes too much time below this and the performance dropsoff due to cache effects above this). Note that these chunks arebuffered in memory so they can't be too big.

This uses a 32 byte (256 bit key) key derived from the user password.

Examples

1 byte file will encrypt to

• 32 bytes header
• 17 bytes data chunk

49 bytes total

1MB (1048576 bytes) file will encrypt to

• 32 bytes header
• 16 chunks of 65568 bytes

1049120 bytes total (a 0.05% overhead). This is the overhead for bigfiles.

Name encryption

Backblaze Rclone Mount

File names are encrypted segment by segment - the path is broken upinto / separated strings and these are encrypted individually.

File segments are padded using PKCS#7 to a multiple of 16 bytesbefore encryption.

They are then encrypted with EME using AES with 256 bit key. EME(ECB-Mix-ECB) is a wide-block encryption mode presented in the 2003paper 'A Parallelizable Enciphering Mode' by Halevi and Rogaway.

This makes for deterministic encryption which is what we want - thesame filename must encrypt to the same thing otherwise we can't findit on the cloud storage system.

This means that

• filenames with the same name will encrypt the same
• filenames which start the same won't have a common prefix

This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both ofwhich are derived from the user password.

After encryption they are written out using a modified version ofstandard base32 encoding as described in RFC4648. The standardencoding is modified in two ways:

• it becomes lower case (no-one likes upper case filenames!)
• we strip the padding character =

base32 is used rather than the more efficient base64 so rclone can beused on case insensitive remotes (e.g. Windows, Amazon Drive).

Key derivation

Rclone uses scrypt with parameters N=16384, r=8, p=1 with anoptional user supplied salt (password2) to derive the 32+32+16 = 80bytes of key material required. If the user doesn't supply a saltthen rclone uses an internal one.

scrypt makes it impractical to mount a dictionary attack on rcloneencrypted data. For full protection against this you should always usea salt.

SEE ALSO

• rclone cryptdecode - Show forward/reverse mapping of encrypted filenames

What is the problem you are having with rclone?

Mounting Backblaze B2 and copying files to it
Mount seems to work but copying files fails

What is your rclone version (output from rclone version)

rclone v1.53.1

• os/arch: windows/amd64
• go version: go1.15

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 x64

Which cloud storage system are you using? (eg Google Drive)

Backblaze B2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

The rclone config contents with secrets removed.

A log from the command with the -vv flag

...gist.github.com/UsefulVid/9f35dcc41bf71796d3f0a9e45e6297ea

edit:

Looks like to be solved with:
rclone.exe mount --vfs-cache-mode writes Backblaze Y: -o volname=local

Rclone S3

edit2:
Still does not work
I used
rclone.exe mount --vfs-cache-mode writes Backblaze:usefulviddemo1 Y: -o volname=local
But I can not delete files:

Rclone Backblaze B2

Even with
rclone.exe mount --vfs-cache-mode full Backblaze:usefulviddemo1 Y: -o volname=local

it looks like it works but no files arrive at my bucket.