What is Reddit's opinion of
Tarsnap?
From 3.5 billion Reddit comments
➔ Tarsnap website
By popularity on Reddit, this Service is:
71 reviews of this app found across Reddit:
Definitely Tarsnap! Tarsnap is an awesome tool and service for truly secure backup. It's run by our own /u/perciva, it's relatively inexpensive, very reliable (based on S3) and secure, everything is encrypted with your key before it ever leaves your computer. Author used to be a security officer for FreeBSD and he's behind scrypt.
Using something you're comfortable with is more important than technical excellence though so another great, and more straight-forward, way is to listen to Jamie Zawinski and do exactly what he says.
Personally, I keep my projects and configuration files backed up with Tarsnap (I have a small script I can run whenever I have a decent connection to the Internet), periodically download stuff from the "cloud" locally (ssh accounts, e-mail, etc) and then mirror my whole local home directory onto an external drive (just bluntly with rsync --delete-before).
"If your files are not stored in three places, they don't exist." Or something like that.
As keeperofdakeys said, you're not going to be able to restore the whole system (without a ton of work) from anything you rsync to an ntfs partition, as it won't retain permissions. If something catastrophic does happen, it'll far easier to reinstall CentOS, install the packages you had previously installed and restore configuration and data.
I use, and highly recommend tarsnap to back up my servers. I run a script nightly, that dumps all databases, dumps a list of installed packages (dpkg --get-selections on Debian, rpm -qa on RedHat), then backs up /var/www, /etc, /usr/local and my home directory (including db dumps, python virtualenvs, etc). I've been using it for well over a year of daily backups and have only spent $6.40924039529889 (yes, they bill in attodollars).
It uses strong encryption, is extremely efficient in regards to bandwidth, io and storage and is written and maintained by Colin Percival, FreeBSD's security officer. Read his about page, he's one smart mofo.
The creator of awesome backup program Tarsnap actually pays bounties for typos:
https://www.tarsnap.com/bugbounty.html
The logic is it means that everything is taken seriously and the comments are important.
I wouldn't use openssl enc having looked at it - it's looks more like a crappy example of the API than a suitable serious encryption tool. The most obviously awful bit:
if (!EVP_BytesToKey(cipher,dgst,sptr, (unsigned char *)str, strlen(str),1,key,iv))
That hardcoded 1 near the end is the iteration count to the (obsolete) key derivation function, which is what turns your passphrase into an encryption key (using MD5, by default). Quoth RFC2898, 14 years ago:
> For the methods in this document, a minimum of 1000 iterations is recommended.
For perspective, the old Xeon in my home server can pump out over 5 million MD5's per second per core. The old GPUs gathering dust on the shelf next to it can pump out over 5 billion per second, each.
I tend to use scrypt for this sort of thing:
> On modern hardware and with default parameters, the cost of cracking the password on a file encrypted by scrypt enc is approximately 100 billion times more than the cost of cracking the same password on a file encrypted by openssl enc; this means that a five-character password using scrypt is stronger than a ten-character password using openssl.
As for "like tarsnap", one of the nicest bits about tarsnap, on top of it actually using modern crypto primitives, is the key management system. You can allocate each machine its own unique passphraseless append-only keys which can only create new archives, while keeping keys to delete or restore separate, either passphrased or even air-gapped. How easily can you replicate that with S3?
And of course, deduplication, mumble mumble, even if S3's 10x cheaper.
Plain disk secure erase:
% geli onetime $device % dd if=/dev/zero of=${device}.eli bs=1M (wait 8 hours)
Encrypted disk secure erase:
% geli kill $device (wait 80 milliseconds)
Granted, metadata backups (/var/backups/ by default on FreeBSD) may need destroying too depending on circumstances. I suggest making them to a geli onetime provider or a memory disk and scrypting them before hitting stable storage.
> the AWS EC2 images are funded by a single developer who receives money only via patreon
Most of my funding comes from my day job at Tarsnap. They've been very generous in allowing me to spend time working on FreeBSD in ways which are at times only marginally relevant to how Tarsnap uses EC2. (In case the joke isn't obvious: I'm the founder of Tarsnap.)
I'm sure the Foundation would be happy to provide me with funding, but there's two issues there:
The Foundation generally works on a "project funding" model, which isn't a great fit for this sort of ongoing "deal with stuff as it arises" work.
Via Tarsnap, I'm actually one of the Foundation's major donors, so it would be a bit weird to have them paying me...
That said, the Foundation is funding work which is helping here -- a lot of the work going into getting the ARM instances working well is being done by Andrew under Foundation sponsorship.
> adds the salt
Careful, do not simply append the salt to the password like H(password + salt) or H(salt + password) as that is insecure for certain choices of H. But yes, that is the correct idea.
Furthermore a single iteration is undesireable. A password hash is required to be slow for security, hence why multiple iterations (a la PBKDF2) are normally used.
Please use scrypt or even bcrypt instead of attempting to implement this yourself.
The OpenSSL command line utility is not a very good choice for file encryption. It uses a very weak process to stretch the passphrase into an encryption key and lacks authentication.
gpg or scrypt are better alternatives.
They're both proprietary software, and not open source. I wouldn't trust either of them to be secure or private.
If you're looking for a private and secure backup solution, consider using Tarsnap.
> I researched paid solutions from Google and other companies, and the cheapest storage service I could find was something like 0.01$/mo
Does this include Amazon S3? Take a look at something like s3ql. It (in some sense) implements Tarsnap features but is open source and doesn't need to run on Amazon servers.
edit: yep, Amazon S3 is 0.03 / month per GB
Recent comment about GPG from Colin Percival:
> Avoid if possible. The code was written by a colony of drunk monkeys, in an era before anyone understood the basics of modern cryptography; I'm really not sure which is worse between gnupg and OpenSSL. Of course, GPG is the standard for encrypted email, just like SSL/TLS is the standard for web sites, so you may have no choice... > > (FYI: https://vuxml.freebsd.org/freebsd/pkg-gnupg.html )
Such a shame tarsnap is bound to a (relatively expensive) online service.
I have used tarsnap for off-site backups and borg for on-site backups. Both do encryption, compression, and smart incremental backups and are suitable for backing up your whole disk (if you are merely backing up personal documents that rarely change, then they will take essentially zero space to do each incremental backup). Tarsnap is not free and stores data on Amazon S3 servers which come with nice guarantees about (almost) never losing your data. Borg is free but you must arrange a place to store your data.
Anything you can do with a homebrew system using git or dd or rsync can be done better with borg, or various other dedicated tools (e.g. someone mentioned duplicity). Since you already have a place to store your data tarsnap may be less useful to you. (I suspect tarsnap has slightly better de-duplication algorithms though.)
I don't know much about backblaze but tarsnap was written by the former FreeBSD security officer. It is completely secure, so much so if you lose your key you are SOL. You may have to install cygwin on Windows to get it uploaded, and use the features like --checkpoint-bytes.
Tarsnap at 25 cents/GB seems excessive in comparison.
My problem is bandwidth my 20Gbps down is only 1MB up. It took me ages on a free trial of Backblaze to upload even a modest amount. My trial was over before I had 25% uploaded and that was running 24/7.
For any backup service another big cost is the per computer cost. Many services only allow uploads for one PC, sure you can pile everything on it but it's tedious.
No, rolling hash's purpose is content-based chunk splitting.
For example, imagine you encrypt files in chunks for the purpose of deduplication.
File: ABCDEFGHIJKL
First, you need to split a file into chunks. If you use fixed-size chinks, say, 3 byte chunks for our example:
Chunks: ABC, 'DEF', GHI, JKL
Now, the file changes and you want to encrypt only the chunks that changed. If something was appended it's simple:
File: ABCDEFGHIJKLmno
Chunks: ABC, 'DEF', GHI, JKL, mno
Same when the addition is aligned with our 3-byte chunk size:
File: ABCDEFmnoGHIJKL
Chunks: ABC, DEF, mno, GHI,JKL`
However, when it's not aligned, you'll get a lot of different chunks:
File: mnABCDEFGHIJKLo
Chunks: mnA, BCD, EFG, HIJ, KLo
See how all chunks changed? Dedup is thrown out of the window.
Rolling hashes produce variable-chunk size based on content hash. For example, it can split a file like this:
File: ABCDEFGHIJKL
Chunks: AB, C, DEFG, HIJ, KL
After change it may produce something like this:
File: mnABCDEFGHIJKLo
Chunks: mn AB,C,DEFG,HIJ,KLo`
Only two chunks changed, others are the same.
More reading: https://en.wikipedia.org/wiki/Rolling_hash
https://www.tarsnap.com/download/EuroBSDCon13.pdf
The problem is that attackers get some information about chunks based on content length. Some software use secret parameters for the rolling hash function to defend against this. For example, Tarsnap uses HMAC of secret keys for various parameters.
Amazon Glacier sounds like the kind of service that you'd need for that amount of data.
You'd pay:
| What | Price |
|---|---|
| Initial 5 TB backup | $0.00 |
| Monthly for 5 TB | $20.48/mo |
| Retrieval of 5 TB | $460.80 |
Plus the request costs that are based on file count.
Tarsnap ("Online backups for the truly paranoid") is great but it'll cost you 25¢ per GB-month, so about $1,260/mo for 5 TB at worst case. Its backup program is open source and does block detection, so it might be cheaper if you have a lot of redundant data.
That's pretty much the limit of what you'll get out of a single S3 PUT. With really good networking (i.e., sitting in EC2 about 100 microseconds away from an S3 front-end node) you can get up to around 150 Mbps; but if you look at the TCP traffic at that point, it's being throttled by the receive window, not by the congestion window. Basically, the S3 server code can't ingest a stream of data any faster than that.
The answer is to use parallel uploads -- either upload several files at once, or use multipart uploads.
Source: I run a service which pushes lots of data into S3 and I've spent a lot of time looking at S3 performance over the years...
Pois é, extensão de browser ou App de celular serve. Mas armazenar o e-mail criptografado no servidor e decriptar quando você está acessando o website, sem nenhum tipo de extensão de browser, não protege seus dados. Isto porque quando você acessa uma página, é o próprio servidor que te manda o código que é executado no seu computador: o servidor poderia simplesmente te mandar um código que, após decriptar (e mostrar o e-mail na tela), manda de volta o e-mail para lá.
Este artigo fala sobre esta impossibilidade: nem na teoria, numa situação mais ideal possível, isto poderia ser seguro.
E eis o problema: as pessoas quando acessam e-mail, dropbox e afins, elas estão querendo ter acesso de qualquer browser, só colocando a senha. Infelizmente isto não é compatível com impedir que o servidor tenha acesso a esses dados.
(PS: para os nerds, o tarsnap é um serviço que é tipo um Dropbox, mas você criptografa do lado do cliente, e ele não tem acesso aos seus dados. O sq3l é outra implementação da mesma idéia e é software livre, rodando em cima do Amazon S3)
Tarsnap doesn't fit requirement 1 (based in Canada), but does match the rest. And if Colin Percival can't get security right no one can.
It does not, however, have a nice UI. That said, backups are something you want running in the background as a service, not something you want showing up in your face. The client is designed to run in the background.
Only if your master password is weak.
The algorithm is basically:
key = SCRYPT(master_password, salt=username, N=2^15, r=8, p=2) generated_password = HMAC-SHA256(site_parameters, key)
scrypt is pretty much best-in-class when it comes to key derivation - the cost to crack it ramps up very rapidly with passphrase complexity. Estimate in the 2009 paper is $43 billion to recover a 10 character printable-ASCII password (95^10 combinations, about as complex as 6 random common words) within a year.
They're not going to spend a billion dollars to recover your password when they can just bug your computer or smack you with a hammer.
Your proposed solution sounds brittle and convoluted. I'm using this scheme for a website, which I think is decent:
Site has a 32-byte "site salt" (stored on the web server). Each user has a unique, 32-byte "user salt" (stored in the database). Plaintext message (stored on the web server) is encrypted by scrypt using the following: "site_salt:password:user_salt". The resultant ciphertext is stored in the user's 'password' column. scrypt is instructed to take 0.5 seconds to encrypt/decrypt.
The password can be up to 1000 characters in length, and no characters are disallowed. That's very important. There's absolutely no technical reason to arbitrarily restrict passwords. The salts are also pure bytes (values of 0-255), not just alphanumerics.
If your platform doesn't have an scrypt library, use bcrypt.
Good info here - http://stackoverflow.com/a/401684/1135604
Personally I use tarsnap. It is a de-duplicated (only uploads new information), locally compressed, locally encrypted backup system that uses the Amazon S3 service to keep data safe. It is a cli only program though, but it operates like tar. Currently I have my normal key with a passphrase, and a second key that can only add archives with no passphrase that is used for the cron service. Due to de-duplication, I just need to put the date in each archive name, and it only uploads new data. It is also quite cheap for large amounts of data.
You can use scrypt utility from Tarsnap. It uses scrypt to derive a key and uses it to encrypt a file using AES in counter mode.
Hi,
although, I have never used it myself, what you describe/want to achieve would be solved with https://www.tarsnap.com/ .
I always shied away from their super exact pricing structure, that left me fearing to pay more than I expected.
Still, to me it sounds pretty much exactly like what you want! :)
Yeah, The email alias thing is awesome. If you have your own domain you can create any type of alias you want. say like git@yourdomain and then restrict it from forwarding to your main email if you wanted too!
Borg is awesome, I really like it. I have also been using Tarsnap
I wouldn't use age(1) if the goal is to encrypt files strictly to yourself, as it's meant to be asymmetric, encrypting data to other people. Not that it's wrong, but I would recommend the <code>scrypt(1)</code> encryption utility instead (which also uses the scrypt KDF).
Tarsnap is great, but it's anything but cheap. Their pricing is 25 cents per GB/month, plus 25 cents per GB for bandwidth. For comparison, rsync.net is 3 cents per GB/month, and no bandwidth charges.
I have no affiliation with rsync.net other than as a satisfied customer.
Yay, hash function terminology! I'm sure you're enjoying yourself.
Both scrypt and HKDF are "key derivation functions," but:
- scrypt is for passwords (low entropy inputs);
- HKDF is for high-entropy keying material that's however not uniformly distributed.
So scrypt has a work factor, HKDF doesn't. Both take a salt, but in HKDF the salt is optional. And "hash functions" that have neither (you work out what it means in this context) are often put to the same use as HKDF.
I'd suggest that password KDFs should be labeled as "PBDKFs," and non-password ones as just "KDFs," but maybe a better term would be useful for the latter.
You don't mention what OS you're on, so have you looked into tarsnap?
It's written and maintained by Colin Percival, the FreeBSD security officer.
As an old graybeard programmer, I keep anything I deem worthy in local git repositories which are backed up daily, automatically via tarsnap using a cron job. Now, this is only for stuff I've transcribed from my notebooks, which mostly contain junk. So most of my writing is junk. Did I just admit that?
For anything I want others to see after I'm dead, I print it out in a nice font on good paper and keep it in an expensive binder.
From the scrypt paper, the cost to crack an 8 character password in 1 year:
PBKDF2 (100ms): $18k bcrypt (95ms): $130k scrypt (65ms): $4.8M
Seeing as the entire point of these algorithms is to increase the cost of an attack, I'd say there's a pretty good reason not to use PBKDF2 if you can help it.
It's probably not feasible, with that much data. How much upstream bandwidth do you have? Considering you're posting in /r/HomeServer we'll assume you have a residential internet connection. The most common of which these days would be a cable modem. Another assumption would be a speed plan of something like 50Mbps down, 3Mbps up. If you never had speed drops, or used it for ANY other surfing/etc, at a sustained rate, your 3Mbps pipe would take 225 DAYS to upload 7TB of data.
Now, once you've done that, you'd have to start the process over to start pushing whatever deltas you've created between the time you started this process and 225 days later, which is probably significant, but is completely dependent on your data and use-cases for your storage.
Please correct me if my math is wrong. Unfortunately, it's a lot easier to hoard data than it is to move it around easily :( Your best bet is going to be to build yourself a new storage system and migrate your data locally; however painful that may be to your wallet. You could always sell your old setup once you're done, to mitigate the costs...
This is all predicated on the assumptions of you:
- wanting to save ALL of that 7TB
- having nowhere else to put that data
- having the kind of bandwidth I'm expecting
Even if you have a LOT more upstream bandwidth (and downstream to match it), it's still not going to be fast. You could probably use a service like Backblaze, or tarsnap if you think you could get your data on/off fast enough to not waste a lot of money leaving it there for ages.
Handy Bandwidth Calculator: http://www.ibeast.com/content/tools/band-calc.asp
Certainly leads it some credence, but WTF?
From Colin Percival's paper on scrypt, a much newer key derivation function meant to replace PBKDF2:
> PBKDF2-HMAC-SHA256 with an iteration count of 86,000; [snip] ...the parameters are chosen such that the running time on one core of a 2.5 GHz Intel Core 2 Duo processor is less than 100 ms
So a single core on an old Intel Core 2 does 86,000 iterations in 1/10th of a second. Roughly as fast as a Titan X GPU?
Something odd here. Maybe he's dividing the cracking rate by the number of password hashes to give an average rate-per-hash across the entire database or something.
I think an example of a business that enables you to have security from them is Tarsnap. It's backed by Amazon storage and there's a third party managing it, but the encryption is all done locally, by a source you can review.
(unfortunately it's not properly open source -- you can view the source but it's still proprietary; free alternatives to Tarsnap include s3ql and s3backer.
Hi, I just posted our server last week with a fresh map. We are strictly vanilla, 20+ age limit. We meet all your requirements listed.
- Difficulty is set to 3
- 100% Legit - I'm the only admin, with just a handful of players, spawn isn't even built yet, we're still gathering resources.
- Purely Vanilla - Stock 1.8 minecraft
- Good Hosting - Currently on a 4gb digitalocean droplet. Will upgrade to a dedicated if we start seeing any lag. Currently it's been smooth. I'm monitoring resources with collectd and grafana.
Hourly backups (daily's go to tarsnap and are archived for 3 months)
I'm also a sysadmin
World border is at 10,000 blocks and we'll expand when/if needed
We have a teamspeak server, most of us play nights/weekends and are in the process of planning out some group events.
Feel free to apply at our website
Unless cloud backup services are out of the question, I would recommend Tarsnap. It was written, and is still maintained by the guy who used to be the FreeBSD Security Officer.
Failing that, something with RSync onto a LUKS/dmcrypt partition, perhaps?
SHA-256 and Bitcoin are not directly connected. SHA-256 is used in plenty of stuff apart from (and before) Bitcoin. It wasn't created for Bitcoin, and Bitcoin could have settled with any good hashing algorithm. It's just that... SHA-2 was the only good and proven one back in 2009. Scrypt wasn't presented until May 2009.
edit: And Scrypt was created for a specific purpose (Tarsnap). You can read a little more about it here.
Contrary to what others are saying ("there’s nothing clearly spelled out in their documentation"), I find the documentation to be pretty clear. In principle, if you wanted to manually decrypt your files, you could using the instructions. I haven't tried myself, so I can't say for sure it's true. But I don't see any obfuscation on the part of Arq.
If you want a solution that is open source and fully in your control, you can use tarsnap. The source code is available, you can compile it yourself and point it to whatever storage resource you want. They guy who wrote it (and his brother) is one of the best cryptographers out there. It's a bit more work and less user friendly, so that's a tradeoff you'll have to decide on yourself.
That said, I agree that it would be nice to have a third party audit and verify a number of these things.
Most modern incremental backup software does this. Arq, Kopia, and more handle it with compression.
Archivers like tar, zip, etc, solve a different problem. Where incremental backups (usually) store a single copy of every file (and delta revisions), archivers typically represent a complete copy of the source at the time of the snapshot. They don’t have the advantage of just referring to a delta, but instead have to store every block of every file in the snapshot.
There are some “hybrids” out there like Tarsnap, or Zpaq but most of them usually attempt to build on top of existing archiving tools. Please not that I have no idea if any of the above tools will eat your data. I’ve never used them, though I hear good things about tarsnap.
Tarsnap is quite great for UNIX backups even if it's pricing model is way more complex than it needs to be.
How frequently will your 2TB of data change? If it is all mostly static tarsnap should be cost effective.
<code>scrypt</code> always refers to a memory-hard key derivation function.
Colin Percival's encrypted backup utility which uses scrypt as a KDF is called tarsnap.
not sure if this helps, but this is the most hassle free way i have found nextcloud-snap. (well, for NC at any rate) for everything else, it's all going to have to be about balance, an obvious solution as pointed out elsewhere is an encrypted folder synced to cloud. for the easy way to do this maybe check out tarsnap? there are caveats to every method, you just need to consider them first.
If that's the case, I would seriously rip off the scrypt utility for how it does encryption & decryption. Here is the C source. It does everything correctly including converting weak passwords into strong password hashes and then using AES-CTR+HMAC to (stream) encrypt a file. The next best thing in my opinion is probably what FiloSottile age encryption tool does, which just happens to have some compatibility with scrypt.
Download the scrypt tool and try doing an scrypt enc foo.txt to try it out for yourself.
If you're encrypting before uploading then I highly doubt it will matter if your provider is ordered by the government.
If you're really worried about this look into tarsnap
I'm a relative newcomer to Tarsnap, but am already a huge fan.
I funded my brand-new account over two months ago with $10 and my current balance is: $9.368557018353598117.
It's incredibly geeky, but the instructions on the website (particularly the general advice, tips, and examples) are very good. What I love about it is:
- Rock-solid and stable
- Easy to test (restore a file or two...or all of them any time you want)
- Works like any true UNIX utility - does one thing and one thing well, completely unobtrusive
- Cheap!
I also back up to a NAS (Synology), but I feel way more secure with Tarsnap and when I make a mistake and need a file back (it happens!), I use Tarsnap to recover it, mostly to prove to myself that I can.
What OS and how much data?
I back everything up to a FreeBSD box that's sat in my basement for nearly a decade and once a night it backs itself up to tarsnap.
We really need more information to recommend anything for you.
Tarsnap. https://www.tarsnap.com/ Colin Percival designed SCrypt. If he's not capable of good security design, we're pretty screwed. If he's not trustworthy we're equally screwed.
Windows compatibility via Cygwin or Windows Subsystem for Linux (Win10). Android compatibility via compiling it from source & using a terminal emulator app.
http://latacora.singles/2018/04/03/cryptographic-right-answers.html (last question/answer. Latacora is a pretty well respected security firm, and the people they polled are also well respected.)
And then verify your signature.
Also bear in mind that it's not secure to print unencrypted keys/seeds. I suggest encrypting it with application like scrypt with custom options for good key stretching and then encrypt it (with different password!) with PGP in ASCII armor format, and then print it.
That sounds like a job for a --dry-run flag - might be worth making a feature request.
Tarsnap considers this an important enough use-case to include it on the very short general usage page:
> Print how much data would be uploaded (without uploading anything or spending any money) > > tarsnap -c -f mybackup --dry-run --print-stats /usr/home
Shame the service is so expensive :(
Memorize long password and use CPU/RAM-intensive KDF.
Example: https://www.tarsnap.com/scrypt.html
Do not use default options. Example with more secure options:
$ sudo apt-get install scrypt $ scrypt enc -M 1073741824 -t 200 secret.txt encrypted.scrypt
Might check out tarsnap. It does the hard work for you with dedup, encryption, compression, placement and retrieval. It costs just a little more than using s3 by itself:
> "quite simple" is not how you describe or evaluate crypto functions. scrypt is quite simple too.
The point is that basic cryptographic operations are easy to scale up.
Case in point: a small box that does 28 trillion SHA256 operations per second for $1500. Imagine what it would look like if they'd had to include 1GB of memory for every 4 execution units.
> That table is very non-specific. It doesn't even indicate which hash function was used, let alone whether there are any assumptions about hardware acceleration.
It's from the scrypt paper, Stronger Key Derivation via Sequential Memory-Hard Functions. Section 8, page 12 onwards.
> if the difference between SHA256 PBKDF2 and scrypt is less than 20,000 than I think SHA256 is probably a good call if your system has SHA256 hardware.
Even if it was only 1,000x, that puts the crossover point where PBKDF2 meets the security of scrypt at about 4 billion iterations. That's a lot even for an ASIC.
It may depend on the type of data. A big point they advertise is deduplication + compression which they give an example of reducing 96TB worth to 16GB = $4/month. Personally I'm not looking to archive hundreds of gigabytes of mkvs but rather personal documents.
Good job doing the implementation, however the mistake you are doing is thinking that bruteforce is done by personal computers, when is done with ASICS, as bruteforcing any significant amount of entropy requires one, which is extremely expensive as you increase the memory requirements, and extremely cheap with SHA. This is why if you build and ASIC that actually bruteforces something, you will need much more than 2.5 million times the money and therefore you need much less than 128 bits of entropy to achieve the same difficulty (in term of dollars) than a 128bits with SHA...
Remember : difficulty is the amount of work necessary to bruteforce, which is counted in dollars, or other unit of value.
and notice: I can have an ASIC that does trillions of SHA for pretty cheap nowaday, like 100$ per Thash right ? Well if I wanted a warp wallet ASIC that does 2 million times less, so about 500k tries per seconds, let me tell you that you would need some crazy amount of money, and that is true even if you replaced the SHA256 industry specialisation by warpwallet specialisation. The explanation to this is in the link that I sent previously https://www.tarsnap.com/scrypt/scrypt.pdf
Notice that with your interpretation of key stretching anyone could be just doing 2.5M rounds of SHA256 and that would be the same... which is -of course- wrong.
2.5 million rounds of SHA is incredibly less secure than Warpwallet
Yes, they are theoretical. I based them on calculations of attack cost from scrypt paper. Measured how much memory it took for 1 sec of scrypt and how much rounds there were for 1 sec of PBKDF2-HMAC-SHA256, then matched them with Colin's estimates in the table on page 14.
"When used for interactive logins, it is 35 times more expensive than bcrypt and 260 times more expensive than PBKDF2; and when used for file encryption — where, unlike bcrypt and PBKDF2, scrypt uses not only more CPU time but also increases the die area required — scrypt increases its lead to a factor of 4000 over bcrypt and 20000 over PBKDF2."
From the scrypt paper.
> That's not a big number in terms of password security. Using a GPU you can crack 8 million passwords per second.
Using a raw hash is considered insecure. Best practices say to use a key derivation function (KDF). See Table 1 of this PDF for a comparison of some hashes and KDFs.
> A "standard" 8-character that's case sensitive and includes numbers & special characters has an entropy of: 62^8
The average person is not going to pick a password with 6 bits of entropy per character, and if they do they will never remember the password.
> But don't just take my word on it of course. XKCD's system would be awesome, had he only kept it to himself. The hack tools are onto his technique now.
This is only true for XKCD style passwords that are chosen by a human. A human should never pick a password.
I prefer to use a password manager. In cases where a human must remember a password, generating an XKCD style password using a computer and using a mnemonic to memorize it is a fairly secure strategy in practice.
You think passing an encoded SHA2 hash to bcrypt is something you should never ever do, but you don't bat an eye at the alternative interpretation of using a plain unsalted single-round SHA2 hash directly?
Here's a cryptographer recommending it if you like, that's always good:
> [pdf] While our estimated costs and NIST’s estimates of passphrase entropy suggest that bcrypt’s 55-character limitation is not likely to cause problems at the present time, implementors of systems which rely on bcrypt might be well-advised to either work around this limitation (e.g., by “pre-hashing” a passphrase to make it fit into the 55-character limit) or to take steps to prevent users from placing too much password entropy in the 56th and subsequent characters
(72 bytes is the effective hard limit, 55 is a softer limit based on a suggestion that additional bytes contribute less entropy to the resulting hash).
And here's another one:
> Using a secure hash function to preprocess the password is secure; it can be shown that if bcrypt(SHA-256(password)) is broken, then either the password was guessed, or some security characteristic of SHA-256 has been proven false.
Note passing a raw hash to bcrypt is very bad, because bcrypt isn't NULL-byte safe and many hashes will end up truncated. Hence the base64.
Tarsnap. It's unfortunately relatively expensive.
S3QL looks interesting, albeit a bit scary (do you really want your online backups to have the complexity of a fully-blown POSIX filesystem?), and the deduplication looks like it leaves something to be desired (simple block-level, not shift-resistant).
I'm currently using Attic, though it doesn't have native S3/etc support, it's fast and has very effective deduplication.
The secret_string is your master password. It's the same thing. https://www.tarsnap.com/scrypt.html
You could use the Hash module of the node.js implementation (https://www.npmjs.com/package/scrypt), since it's designed for this sort of thing. Just concatenate your password and the application name, and hash that.
Tarsnap is the gold standard, but it's pretty much a souped up Unix tar(1) command on the client side, and is quite pricey. Cyphertite is the first one that springs to mind that supports Windows, but I have no idea how good they are at that.
Both provide client source code, as well as detailed descriptions of how they work. Wish that was more common :(
I'd strongly suggest https://www.tarsnap.com/ as backup solution.
https://www.tarsnap.com/bugbounty.html https://www.tarsnap.com/crypto.html
If that isn't what you'd like to do [e.g. completely self managed and/or regulatory concerns] I'd suggest using GPG and then storing the files on a pair of NAS that you swap out from a safety deposit box. [e.g. NAS A is the on-site nightly backup, swap it with NAS B the bank on Friday afternoons]
I agree with the first half of this (Not so much the latter half). My bookmarks are full of services I saw and I want to try out at one point, but, I've yet to have the time to do so. I'm willing to share one bookmark I have (Out of ~200):-
It's under my 'to test' folder, why? It looks cool, but, it's going to require me dedicating an hour or more to test it, so, I'm waiting for a good time to get back to it. Do I bookmark Reddit? Of course not, I can remember Reddit, I come here every day. Am I going to bookmark an obscure backup tool I found once on the internet and want to try out at some point? Yes.
As far as offsite backup goes tarsnap is my favourite. On mac it should be fine, but on Windows it does have to run in cygwin. It's probably the best for in China since it is heavily encrypted. And it's super cheap. If you're backing up less than a gig you can pay less than a quarter a month for storage.
If you have some sort of network storage or an external hard drive you could just write a batch script to xcopy the files to that and run it via the scheduler. I think Windows also has a backup function, but I think it focuses on backing up all the user data.
edit: If you want cloud backups I like tarsnap. The only catch is it runs in cygwin on Windows. You pay only for the bandwidth and storage you use and it's pretty cheap. The charge in picodollars per byte-month! For small amounts of data it's a pretty good deal - and it does de-duplication locally so you can store multiple snapshots of the data and save on both the bandwidth and storage costs. For 60 megs it would be slightly more than a penny a month for storage.
OpenStack Swift is all Python, and it fullfills the "distributed file storage system", but it doesn't really have any capabilities for trustless backups -- though you could build those on top of them, something like Tarsnap does with S3: https://www.tarsnap.com/
For the best security, I would go with https://www.tarsnap.com/ No idea on all of its features and suspect it doesn't have versioning and shouldn't have dedup (since encrypted data can't be deduped).
Just a couple of minor points:
> DO NOT USE: Insecure versions of crypt ($1$, $2$, $2a$, $2x$, $3$).
$2a$ is only insecure on vulnerable versions of the crypt library. Since the vulnerability was fixed, $2a$ is secure again. $2y$ is simply an alias of $2a$, but one reason you may want to refer to it as $2y$ is if you explicitly want it to fail to run on an older, vulnerable version. Just as an aside: If I understand the vulnerability correctly, if you never allowed 8-bit characters in passwords either when generating or authenticating (and there are arguments for and against doing this) then you were never vulnerable.
> DO USE: Secure versions of crypt ($2y$, $5$, $6$)
bcrypt ($2y$ / $2a$) are significantly more "secure" (referring to difficulty of brute-forcing) than $5$ and $6$, which use the SHA-2 family of hashes (SHA-256 and SHA-512). Despite using many rounds, these hash algorithms still lend themselves quite well to a big speed-up from GPU parallelisation (unlike bcrypt, which is designed to need more memory which removes most of the GPU benefit). These forms of crypt mainly exist for political reasons, because SHA-2 is NIST-approved.
If you need something more "secure" than bcrypt, the direction you should be looking is toward scrypt (which isn't implemented by unix crypt) rather than something like $5$ or $6$.