How to securely hash passwords?
If I hash passwords before storing them in my database, is that sufficient to prevent them being recovered by anyone?
I should point out that this relates only to retrieval directly from the database, and not any other type of attack, such as bruteforcing the login page of the application, keylogger on the client, and of course rubberhose cryptanalysis (or nowadays we should call it "Chocolate Cryptanalysis").
Of course any form of hash will not prevent those attacks.
It's interesting that (if I read it right) after offering chocolate/guessing there was almost an identical number of folks revealing their password. Does that mean us IT professionals are just more susceptible to chocolate? ;)
This question made it to the blog: http://security.blogoverflow.com/2013/09/about-secure-password-hashing/
ideally this all should be pointless, because nobody should reuse passwords and then passwords by themselves are worthless. And if someone reuses passwords, they are simply searching for a punishment. If the attacker has pwned database, then they already can snatch all data, not just passwords, so if the password is unique, it won't matter how is it hashed, because it has no value…
@SargeBorsch what? I'm sorry but I'm not able to understand your comment in anything resembling something sensible, but anyway it probably belongs more in the [chat]?
More recent similar discussion In 2018, what is the recommended hash to store passwords: bcrypt, scrypt, Argon2?
Note: This answer was written in 2013. Many things have changed in the following years, which means that this answer should primarily seen as how best practice used to be in 2013.
We need to hash passwords as a second line of defence. A server which can authenticate users necessarily contains, somewhere in its entrails, some data which can be used to validate a password. A very simple system would just store the passwords themselves, and validation would be a simple comparison. But if a hostile outsider were to gain a simple glimpse at the contents of the file or database table which contains the passwords, then that attacker would learn a lot. Unfortunately, such partial, read-only breaches do occur in practice (a mislaid backup tape, a decommissioned but not wiped-out hard disk, an aftermath of a SQL injection attack -- the possibilities are numerous). See this blog post for a detailed discussion.
Since the overall contents of a server that can validate passwords are necessarily sufficient to indeed validate passwords, an attacker who obtained a read-only snapshot of the server is in position to make an offline dictionary attack: he tries potential passwords until a match is found. This is unavoidable. So we want to make that kind of attack as hard as possible. Our tools are the following:
Cryptographic hash functions: these are fascinating mathematical objects which everybody can compute efficiently, and yet nobody knows how to invert them. This looks good for our problem - the server could store a hash of a password; when presented with a putative password, the server just has to hash it to see if it gets the same value; and yet, knowing the hash does not reveal the password itself.
Salts: among the advantages of the attacker over the defender is parallelism. The attacker usually grabs a whole list of hashed passwords, and is interested in breaking as many of them as possible. He may try to attack several in parallels. For instance, the attacker may consider one potential password, hash it, and then compare the value with 100 hashed passwords; this means that the attacker shares the cost of hashing over several attacked passwords. A similar optimisation is precomputed tables, including rainbow tables; this is still parallelism, with a space-time change of coordinates.
The common characteristic of all attacks which use parallelism is that they work over several passwords which were processed with the exact same hash function. Salting is about using not one hash function, but a lot of distinct hash functions; ideally, each instance of password hashing should use its own hash function. A salt is a way to select a specific hash function among a big family of hash functions. Properly applied salts will completely thwart parallel attacks (including rainbow tables).
Slowness: computers become faster over time (Gordon Moore, co-founder of Intel, theorized it in his famous law). Human brains do not. This means that attackers can "try" more and more potential passwords as years pass, while users cannot remember more and more complex passwords (or flatly refuse to). To counter that trend, we can make hashing inherently slow by defining the hash function to use a lot of internal iterations (thousands, possibly millions).
We have a few standard cryptographic hash functions; the most famous are MD5 and the SHA family. Building a secure hash function out of elementary operations is far from easy. When cryptographers want to do that, they think hard, then harder, and organize a tournament where the functions fight each other fiercely. When hundreds of cryptographers gnawed and scraped and punched at a function for several years and found nothing bad to say about it, then they begin to admit that maybe that specific function could be considered as more or less secure. This is just what happened in the SHA-3 competition. We have to use this way of designing hash function because we know no better way. Mathematically, we do not know if secure hash functions actually exist; we just have "candidates" (that's the difference between "it cannot be broken" and "nobody in the world knows how to break it").
A basic hash function, even if secure as a hash function, is not appropriate for password hashing, because:
- it is unsalted, allowing for parallel attacks (rainbow tables for MD5 or SHA-1 can be obtained for free, you do not even need to recompute them yourself);
- it is way too fast, and gets faster with technological advances. With a recent GPU (i.e. off-the-shelf consumer product which everybody can buy), hashing rate is counted in billions of passwords per second.
So we need something better. It so happens that slapping together a hash function and a salt, and iterating it, is not easier to do than designing a hash function -- at least, if you want the result to be secure. There again, you have to rely on standard constructions which have survived the continuous onslaught of vindicative cryptographers.
Good Password Hashing Functions
PBKDF2 comes from PKCS#5. It is parameterized with an iteration count (an integer, at least 1, no upper limit), a salt (an arbitrary sequence of bytes, no constraint on length), a required output length (PBKDF2 can generate an output of configurable length), and an "underlying PRF". In practice, PBKDF2 is always used with HMAC, which is itself a construction built over an underlying hash function. So when we say "PBKDF2 with SHA-1", we actually mean "PBKDF2 with HMAC with SHA-1".
Advantages of PBKDF2:
- Has been specified for a long time, seems unscathed for now.
- Is already implemented in various framework (e.g. it is provided with .NET).
- Highly configurable (although some implementations do not let you choose the hash function, e.g. the one in .NET is for SHA-1 only).
- Received NIST blessings (modulo the difference between hashing and key derivation; see later on).
- Configurable output length (again, see later on).
Drawbacks of PBKDF2:
- CPU-intensive only, thus amenable to high optimization with GPU (the defender is a basic server which does generic things, i.e. a PC, but the attacker can spend his budget on more specialized hardware, which will give him an edge).
- You still have to manage the parameters yourself (salt generation and storage, iteration count encoding...). There is a standard encoding for PBKDF2 parameters but it uses ASN.1 so most people will avoid it if they can (ASN.1 can be tricky to handle for the non-expert).
bcrypt was designed by reusing and expanding elements of a block cipher called Blowfish. The iteration count is a power of two, which is a tad less configurable than PBKDF2, but sufficiently so nevertheless. This is the core password hashing mechanism in the OpenBSD operating system.
Advantages of bcrypt:
- Many available implementations in various languages (see the links at the end of the Wikipedia page).
- More resilient to GPU; this is due to details of its internal design. The bcrypt authors made it so voluntarily: they reused Blowfish because Blowfish was based on an internal RAM table which is constantly accessed and modified throughout the processing. This makes life much harder for whoever wants to speed up bcrypt with a GPU (GPU are not good at making a lot of memory accesses in parallel). See here for some discussion.
- Standard output encoding which includes the salt, the iteration count and the output as one simple to store character string of printable characters.
Drawbacks of bcrypt:
- Output size is fixed: 192 bits.
- While bcrypt is good at thwarting GPU, it can still be thoroughly optimized with FPGA: modern FPGA chips have a lot of small embedded RAM blocks which are very convenient for running many bcrypt implementations in parallel within one chip. It has been done.
- Input password size is limited to 51 characters. In order to handle longer passwords, one has to combine bcrypt with a hash function (you hash the password and then use the hash value as the "password" for bcrypt). Combining cryptographic primitives is known to be dangerous (see above) so such games cannot be recommended on a general basis.
scrypt is a much newer construction (designed in 2009) which builds over PBKDF2 and a stream cipher called Salsa20/8, but these are just tools around the core strength of scrypt, which is RAM. scrypt has been designed to inherently use a lot of RAM (it generates some pseudo-random bytes, then repeatedly read them in a pseudo-random sequence). "Lots of RAM" is something which is hard to make parallel. A basic PC is good at RAM access, and will not try to read dozens of unrelated RAM bytes simultaneously. An attacker with a GPU or a FPGA will want to do that, and will find it difficult.
Advantages of scrypt:
- A PC, i.e. exactly what the defender will use when hashing passwords, is the most efficient platform (or close enough) for computing scrypt. The attacker no longer gets a boost by spending his dollars on GPU or FPGA.
- One more way to tune the function: memory size.
Drawbacks of scrypt:
- Still new (my own rule of thumb is to wait at least 5 years of general exposure, so no scrypt for production until 2014 - but, of course, it is best if other people try scrypt in production, because this gives extra exposure).
- Not as many available, ready-to-use implementations for various languages.
- Unclear whether the CPU / RAM mix is optimal. For each of the pseudo-random RAM accesses, scrypt still computes a hash function. A cache miss will be about 200 clock cycles, one SHA-256 invocation is close to 1000. There may be room for improvement here.
- Yet another parameter to configure: memory size.
OpenPGP Iterated And Salted S2K
I cite this one because you will use it if you do password-based file encryption with GnuPG. That tool follows the OpenPGP format which defines its own password hashing functions, called "Simple S2K", "Salted S2K" and "Iterated and Salted S2K". Only the third one can be deemed "good" in the context of this answer. It is defined as the hash of a very long string (configurable, up to about 65 megabytes) consisting of the repetition of an 8-byte salt and the password.
As far as these things go, OpenPGP's Iterated And Salted S2K is decent; it can be considered as similar to PBKDF2, with less configurability. You will very rarely encounter it outside of OpenPGP, as a stand-alone function.
Recent Unix-like systems (e.g. Linux), for validating user passwords, use iterated and salted variants of the crypt() function based on good hash functions, with thousands of iterations. This is reasonably good. Some systems can also use bcrypt, which is better.
The old crypt() function, based on the DES block cipher, is not good enough:
- It is slow in software but fast in hardware, and can be made fast in software too but only when computing several instances in parallel (technique known as SWAR or "bitslicing"). Thus, the attacker is at an advantage.
- It is still quite fast, with only 25 iterations.
- It has a 12-bit salt, which means that salt reuse will occur quite often.
- It truncates passwords to 8 characters (characters beyond the eighth are ignored) and it also drops the upper bit of each character (so you are more or less stuck with ASCII).
But the more recent variants, which are active by default, will be fine.
Bad Password Hashing Functions
About everything else, in particular virtually every homemade method that people relentlessly invent.
For some reason, many developers insist on designing function themselves, and seem to assume that "secure cryptographic design" means "throw together every kind of cryptographic or non-cryptographic operation that can be thought of". See this question for an example. The underlying principle seems to be that the sheer complexity of the resulting utterly tangled mess of instruction will befuddle attackers. In practice, though, the developer himself will be more confused by his own creation than the attacker.
Complexity is bad. Homemade is bad. New is bad. If you remember that, you'll avoid 99% of problems related to password hashing, or cryptography, or even security in general.
Password hashing in Windows operating systems used to be mindbogglingly awful and now is just terrible (unsalted, non-iterated MD4).
Up to now, we considered the question of hashing passwords. A close problem is about transforming a password into a symmetric key which can be used for encryption; this is called key derivation and is the first thing you do when you "encrypt a file with a password".
It is possible to make contrived examples of password hashing functions which are secure for the purpose of storing a password validation token, but terrible when it comes to generating symmetric keys; and the converse is equally possible. But these examples are very "artificial". For practical functions like the one described above:
- The output of a password hashing function is acceptable as a symmetric key, after possible truncation to the required size.
- A Key Derivation Function can serve as a password hashing function as long as the "derived key" is long enough to avoid "generic preimages" (the attacker is just lucky and finds a password which yields the same output). An output of more than 100 bits or so will be enough.
Indeed, PBKDF2 and scrypt are KDF, not password hashing function -- and NIST "approves" of PBKDF2 as a KDF, not explicitly as a password hasher (but it is possible, with only a very minute amount of hypocrisy, to read NIST's prose in such a way that it seems to say that PBKDF2 is good for hashing passwords).
Conversely, bcrypt is really a block cipher (the bulk of the password processing is the "key schedule") which is then used in CTR mode to produce three blocks (i.e. 192 bits) of pseudo-random output, making it a kind of hash function. bcrypt can be turned into a KDF with a little surgery, by using the block cipher in CTR mode for more blocks. But, as usual, we cannot recommend such homemade transforms. Fortunately, 192 bits are already more than enough for most purposes (e.g. symmetric encryption with GCM or EAX only needs a 128-bit key).
How many iterations ?
As much as possible ! This salted-and-slow hashing is an arms race between the attacker and the defender. You use many iterations to make the hashing of a password harder for everybody. To improve security, you should set that number as high as you can tolerate on your server, given the tasks that your server must otherwise fulfill. Higher is better.
Collisions and MD5
MD5 is broken: it is computationally easy to find a lot of pairs of distinct inputs which hash to the same value. These are called collisions.
However, collisions are not an issue for password hashing. Password hashing requires the hash function to be resistant to preimages, not to collisions. Collisions are about finding pairs of messages which give the same output without restriction, whereas in password hashing the attacker must find a message which yields a given output that the attacker does not get to choose. This is quite different. As far as we known, MD5 is still (almost) as strong as it has ever been with regards to preimages (there is a theoretical attack which is still very far in the ludicrously impossible to run in practice).
The real problem with MD5 as it is commonly used in password hashing is that it is very fast, and unsalted. However, PBKDF2 used with MD5 would be robust. You should still use SHA-1 or SHA-256 with PBKDF2, but for Public Relations. People get nervous when they hear "MD5".
The main and only point of the salt is to be as unique as possible. Whenever a salt value is reused anywhere, this has the potential to help the attacker.
For instance, if you use the user name as salt, then an attacker (or several colluding attackers) could find it worthwhile to build rainbow tables which attack the password hashing function when the salt is "admin" (or "root" or "joe") because there will be several, possibly many sites around the world which will have a user named "admin". Similarly, when a user changes his password, he usually keeps his name, leading to salt reuse. Old passwords are valuable targets, because users have the habit of reusing passwords in several places (that's known to be a bad idea, and advertised as such, but they will do it nonetheless because it makes their life easier), and also because people tend to generate their passwords "in sequence": if you learn that Bob's old password is "SuperSecretPassword37", then Bob's current password is probable "SuperSecretPassword38" or "SuperSecretPassword39".
The cheap way to obtain uniqueness is to use randomness. If you generate your salt as a sequence of random bytes from the cryptographically secure PRNG that your operating system offers (
CryptGenRandom()...) then you will get salt values which will be "unique with a sufficiently high probability". 16 bytes are enough so that you will never see a salt collision in your life, which is overkill but simple enough.
UUID are a standard way of generating "unique" values. Note that "version 4" UUID just use randomness (122 random bits), like explained above. A lot of programming frameworks offer simple to use functions to generate UUID on demand, and they can be used as salts.
Salts are not meant to be secret; otherwise we would call them keys. You do not need to make salts public, but if you have to make them public (e.g. to support client-side hashing), then don't worry too much about it. Salts are there for uniqueness. Strictly speaking, the salt is nothing more than the selection of a specific hash function within a big family of functions.
Cryptographers can never let a metaphor alone; they must extend it with further analogies and bad puns. "Peppering" is about using a secret salt, i.e. a key. If you use a "pepper" in your password hashing function, then you are switching to a quite different kind of cryptographic algorithm; namely, you are computing a Message Authentication Code over the password. The MAC key is your "pepper".
Peppering makes sense if you can have a secret key which the attacker will not be able to read. Remember that we use password hashing because we consider that an attacker could grab a copy of the server database, or possible of the whole disk of the server. A typical scenario would be a server with two disks in RAID 1. One disk fails (electronic board fries - this happens a lot). The sysadmin replaces the disk, the mirror is rebuilt, no data is lost due to the magic of RAID 1. Since the old disk is dysfunctional, the sysadmin cannot easily wipe its contents. He just discards the disk. The attacker searches through the garbage bags, retrieves the disk, replaces the board, and lo! He has a complete image of the whole server system, including database, configuration files, binaries, operating system... the full monty, as the British say. For peppering to be really applicable, you need to be in a special setup where there is something more than a PC with disks; you need a HSM. HSM are very expensive, both in hardware and in operational procedure. But with a HSM, you can just use a secret "pepper" and process passwords with a simple HMAC (e.g. with SHA-1 or SHA-256). This will be vastly more efficient than bcrypt/PBKDF2/scrypt and their cumbersome iterations. Also, usage of a HSM will look extremely professional when doing a WebTrust audit.
Since hashing is (deliberately) expensive, it could make sense, in a client-server situation, to harness the CPU of the connecting clients. After all, when 100 clients connect to a single server, the clients collectively have a lot more muscle than the server.
To perform client-side hashing, the communication protocol must be enhanced to support sending the salt back to the client. This implies an extra round-trip, when compared to the simple client-sends-password-to-server protocol. This may or may not be easy to add to your specific case.
In the context of SRP, password hashing necessarily occurs on the client side.
Use bcrypt. PBKDF2 is not bad either. If you use scrypt you will be a "slightly early adopter" with the risks that are implied by this expression; but it would be a good move for scientific progress ("crash dummy" is a very honourable profession).
Ah ! Not so fast. There is now an open competition for defining new algorithms; deadline for candidate submission is March 31st. The submission reports for some candidates will probably explain at length why their candidate is better than scrypt. Thus, two months from now, we shall have enough information to be able to reach a conclusion on "scrypt or not scrypt" (e.g. there have been suggestion that scrypt's "memory hardness" is more expensive than previously envisioned on multi-core servers).
There is an attack against scrypt that can help an attacker up to a factor of 4, which is not critical. scrypt can still be considered to fare at least as good as bcrypt, and substantially better when the attacker has FPGA. It is still harder to configure and using it for a busy server with high peak loads requires some care. So far, PHC has a few candidate functions which try to do "memory hardness" like scrypt, but are arguably better at it.
So you are saying that, right now, `scrypt` is as good as or better than `bcrypt`, and both are better than anything else at the moment, including (marginally) `PBKDF2`, at least until these new candidate functions prove themselves 5 years down the road? Did I get all that right?
You make a pepper look as something only useful with a HSM, which it is not. Its purpose is to have different salts on different places, thus forcing the attacker to compromise all of them. Typically, the salt is stored with the username in the db but the pepper is stored in the login server. Thus making some leaks resistant to offline guessing: A broken RAID disk from the database server is leaked, but the pepper was stored in the web server; or the db was obtained through a SQL injection but the configuration file is not.
"A salt is a way to select a specific hash function among a big family of hash functions." Uh, no? The hash function used is the same for all passwords. A salt is a randomly generated sequence of characters that's appended to the password in some way, e.g. `hash(password + salt)`.
@nyuszika7h given function `f(x)`, `g(x) = f(x + 1)` is a function as well, shifted in x. A crypto-secure hash, when each input is salted, is another crypto-secure hash. Technically, appending the salt to the password isn't the only way to salt the password -- you could salt the password using any transformation you like, but for a few good reasons which a comment is not the place to discuss, `hash(pwd + salt)` is either indistinguishable from or superior to other alternatives. This q covers that a bit.
@ThomasPornin since "Input password size is limited to 51 characters" is it a good practise to hash the password before supplying it to bcrypt in all cases given that we're never going to know what the password is?
@ThomasPornin do I overcome the bcrypt limitation "Output size is fixed: 192 bits." by increasing my iteration count/work factor? Or is that limitation still a concern?
bcrypt produces 192 bits, no more, for all work factors. When your bcrypt library outputs a _character string_, that string encodes the 192 bits and some other parameters (salt and work factor), and uses a Base64-derivative to get printable characters, but cryptographically speaking there are still only 192 bits -- which is fine for password verification, but possibly not for use as a KDF. This is "fixed" by modifying the algorithm, but this is not recommended at all (and if you knew enough to do such a modification securely, you would not ask that question).
I thought client-side hashing is bad because then the hash basically becomes the password and loses its value?
@DavidGrinberg: doing _all_ the hashing in the client side is bad for the reason you indicate (i.e. what you store in the database is what grants access when sent to the server, so it is plaintext-equivalent). However, doing _most_ of the hashing on the client side is reasonable (i.e. client computes bcrypt(password), server stores SHA-256(bcrypt(password))).
Has any of this changed in 2016? It's been over 5 years for scrypt. =) I'd also be interested to know if the "Not as many available, ready-to-use implementations for various languages" con still applies.
As someone with a background in CS with just over 15 years professional experience as a developer, I'd just like to say that this is my all-time favorite SE answer. Clear, concise, unbiased. Thanks for answering and reminding me why this stuff is so *hard*.
@ThomasPornin: This sentence is missing one or several words (after one simple): "Standard output encoding which includes the salt, the iteration count and the output as one simple to store character string of printable characters."
It is a verb-less sentence as part of an enumeration. The output is a character string, consisting only of "printable" characters, and is thus simple to store. That string contains the encoding of the password hashing function, along with the salt and the iteration count: all three elements are encoded into a single string.
Excellent commentary. Just a note though... Modern (as of say 2014) GPU cards actually excel at scrypt compared to CPU. This should not detract from someone using it, as even a GPU doing scrypt is thousands of times slower than bcrypt or SHA2-512.
@ThomasPornin, The example for pepper need not be so convoluted. **Pepper is useful for normal applications.** Consider a parsing bug on a webapp that allows us complete read access to a server's database. We can do `select*from user` and get the full list of hash together with their salt for us to crack at offline pleasure. Since we are brutespearing only highprofile admin accounts, salt is not going to be an obstacle to us at all. But ooops, without the pepper we couldn't do no nothing. And full read-access to the database alone does not in anyway allow us to get access to the pepper.
thought of downvoting because for long explaination :) just kidding..... upvoted because for `conclusion` part.....
The password hash Argon2, winner of PHC: https://github.com/P-H-C/phc-winner-argon2
The author of this answer, Thomas Pornin, has designed a password-hashing function named Makwa: https://www.bolet.org/makwa/ Makwa was a competitor and a finalist in the Password Hashing Competition. Thus, the author is well aware that that Argon2 has won the PHC. I guess he will update this answer when he finds the time.
@ThomasPornin This answer on SO was written right around the same time this answer was, and makes an argument *against* using peppers. What is your take on it? Who's right or wrong? and why?
(1) Is `scrypt` now recommended? Has it passed the test of time? (2) You talk about client-side hashing, but don't mention how that means the hash really *is* the password, since you don't control the clients.
@ThomasPornin When you say `iteration` you mean applying the hash function multiple times?
@CanSürmeli The iteration is a specific way to specify a work factor. It does indeed mean applying *some function*, possibly a hash function, multiple times. It is a way of increasing the work factor in a linear fashion (bcrypt uses a exponentially increasing work factor instead). What one iteration includes depends on the password hashing scheme though.