www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - SHA-3 is KECCAK

reply "Xinok" <xinok live.com> writes:
With the recent discussions regarding std.crypt/std.hash, I 
thought it would be worth announcing that the SHA-3 competition 
has ended and the winner is KECCAK.

http://csrc.nist.gov/groups/ST/hash/sha-3/winner_sha-3.html
Oct 03 2012
next sibling parent reply "Chris Cain" <clcain uncg.edu> writes:
On Wednesday, 3 October 2012 at 11:35:42 UTC, Xinok wrote:
 With the recent discussions regarding std.crypt/std.hash, I 
 thought it would be worth announcing that the SHA-3 competition 
 has ended and the winner is KECCAK.

 http://csrc.nist.gov/groups/ST/hash/sha-3/winner_sha-3.html

I would love to see D support SHA3 ASAP. I cringe at the fact that many still use MD5.
Oct 03 2012
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
17-Jan-2014 18:06, Kagamin пишет:
 On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:
 For any use where security isn't an actual concern, sure. If it's just
 to casually verify that a file transferred successfully (like an
 alternative to a checksum)

Doesn't TCP take care of that?

As long as 16 bit CHECKSUM feels sufficient for you. -- Dmitry Olshansky
Jan 17 2014
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-01-17 15:06, Kagamin wrote:

 Doesn't TCP take care of that?

Doesn't TCP only verify that a package is received intact, not a complete file? -- /Jacob Carlborg
Jan 17 2014
prev sibling next sibling parent "Uranuz" <neuranuz gmail.com> writes:
I support the idea that it would be useful to have an 
implementation or binding for Keccak and may be other crypto hash 
functions. Now all what I found are pycrypto implementation an 
official implementation written with C at 
http://keccak.noekeon.org/. But I was unable to deal with the 
interface of this library. There are a little of comments in the 
code and I couldn't find concrete examples how to use it. Phobos 
simple interface like in std.digest.md would be more suitable for 
casual use. Very often in not needed to tweek all of these 
parameters for algorithm, but needed that it will just work.

I will appreciate for some guidelines if someone tried to bind 
this C implementation for using in D.
Jan 16 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 3 October 2012 at 12:39:15 UTC, Chris Cain wrote:
 On Wednesday, 3 October 2012 at 11:35:42 UTC, Xinok wrote:
 With the recent discussions regarding std.crypt/std.hash, I 
 thought it would be worth announcing that the SHA-3 
 competition has ended and the winner is KECCAK.

 http://csrc.nist.gov/groups/ST/hash/sha-3/winner_sha-3.html

I would love to see D support SHA3 ASAP. I cringe at the fact that many still use MD5.

MD5 is good enough for most cases. AFAIK, keccak uses weird bit fiddling. Wasn't it considered a bad practice since DES because a specialized hardware would give a considerable speedup, which will help in brute force attacks?
Jan 17 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 11:37:26 UTC, Kagamin wrote:
 MD5 is good enough for most cases.

For any use where security isn't an actual concern, sure. If it's just to casually verify that a file transferred successfully (like an alternative to a checksum), then it's fine to use. But don't use it to secure anything against an attacker at this point.
 AFAIK, keccak uses weird bit fiddling. Wasn't it considered a 
 bad practice since DES because a specialized hardware would 
 give a considerable speedup, which will help in brute force 
 attacks?

Actually, the idea is that it _should_ be implemented in specialized hardware to make it faster. And improving brute force attacks in this manner will only provide a multiplicative increase in speed, and that's not a concern. The overall strategy of using brute force isn't going to be turned from infeasible to feasible because of that. It's still completely infeasible to find two different messages s.t. their SHA-3 hash is equal.
Jan 17 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:
 On Friday, 17 January 2014 at 11:37:26 UTC, Kagamin wrote:
 MD5 is good enough for most cases.

For any use where security isn't an actual concern, sure. If it's just to casually verify that a file transferred successfully (like an alternative to a checksum), then it's fine to use. But don't use it to secure anything against an attacker at this point.

There's no successful preimage attack on MD5, which is the only deadly attack on a hash function. SHA3 is just more convenient than MD5 because when you want to change the hash function, you don't have to ditch the whole system, only change its parameters.
 AFAIK, keccak uses weird bit fiddling. Wasn't it considered a 
 bad practice since DES because a specialized hardware would 
 give a considerable speedup, which will help in brute force 
 attacks?

Actually, the idea is that it _should_ be implemented in specialized hardware to make it faster.

That's rather inconvenient, that you can't have an efficient implementation of the algorithm on common hardware. MD5 family has no such flaw.
 And improving brute force attacks in this manner will only 
 provide a multiplicative increase in speed, and that's not a 
 concern. The overall strategy of using brute force isn't going 
 to be turned from infeasible to feasible because of that. It's 
 still completely infeasible to find two different messages s.t. 
 their SHA-3 hash is equal.

You assume that Moore's law doesn't and won't work. Write asserts for assumptions.
Jan 17 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:
 For any use where security isn't an actual concern, sure. If 
 it's just to casually verify that a file transferred 
 successfully (like an alternative to a checksum)

Doesn't TCP take care of that?
Jan 17 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 14:04:29 UTC, Kagamin wrote:
 On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:
 On Friday, 17 January 2014 at 11:37:26 UTC, Kagamin wrote:
 MD5 is good enough for most cases.

For any use where security isn't an actual concern, sure. If it's just to casually verify that a file transferred successfully (like an alternative to a checksum), then it's fine to use. But don't use it to secure anything against an attacker at this point.

There's no successful preimage attack on MD5, which is the only deadly attack on a hash function.

Your first part is true, but the second part is not. A generic collision attack can cause issues depending on what you're trying to do. For general use, MD5 is not safe enough based on that alone. The wikipedia article on MD5 covers this reasonably well enough: http://en.wikipedia.org/wiki/MD5 Do note that SHA1 is similar enough to MD5 that many are recommending against SHA1 as well (and I recommend following that advice). It's not as broken as MD5 but since we have SHA2 there's no good reason not to use the better version. For general messages requiring any degree of security against an attacker, use SHA2 (this does not necessarily apply to hashing passwords which have their own concerns, obviously). Stay away from MD5 and no new projects should use SHA1 (although my understanding is that no one is saying "jump from the ship" for SHA1 right now). These are generally accepted practices in the cryptography community. If you want to roll the dice with MD5, by all means, but don't spread acceptance of MD5 because someone might follow your advice against the clear recommendations of the cryptography community. As far as I'm concerned, it's unethical to recommend MD5 for security purposes at this point.
 SHA3 is just more convenient than MD5 because when you want to 
 change the hash function, you don't have to ditch the whole 
 system, only change its parameters.

I'm sorry but no. The reason we wanted a SHA3 is because we needed a message digest algorithm that is different enough from MD5, SHA1, and SHA2 so that an attack discovered on those algorithms should not be applicable to SHA3 as well (as was the case with MD5 and SHA1). The things you describe are just bonuses.
 That's rather inconvenient, that you can't have an efficient 
 implementation of the algorithm on common hardware. MD5 family 
 has no such flaw.

You seem to not know enough about this topic to discuss it. SHA3 is fast on common hardware, as it is designed. Specialized hardware can be made and that's fine, intentional, and even beneficial. Most cryptography is designed that way. DES was, AES is (AES instructions exist on many modern hardware, in fact), and SHA3 is as well. This is not a flaw and if you think it is, go talk with cryptographers instead of me because you're clearly well beyond my skill level (that said, since you're discussing the viability of MD5, you're not anywhere close to as competent on this matter as you think you are, no offense).
 You assume that Moore's law doesn't and won't work. Write 
 asserts for assumptions.

I'm not sure what you mean here. No such assumptions need to be made. You are aware of how large the space of 256-bit message digests are, right? Multiplicative (specialized hardware) increases in performance makes no difference because of the ridiculous size of such a space. Exponential increases in performance (like Moore's law) are flawed because of how long it takes for those improvements to accumulate. The way you defeat such a thing is not via multiplicative increases nor exponential increases over a long time, but by instantaneous, SIGNIFICANT breaks. Even effectively cutting a 256-bit message digest down to the complexity of 128-bit (which is a *massive* break) is still not enough, for comparison.
Jan 17 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 14:06:57 UTC, Kagamin wrote:
 Doesn't TCP take care of that?

For a packet, yes. In general you can assume that if a transfer completes under TCP then it is very likely correct. That's the way TCP is designed. If you want to check the entire file at the end, MD5 could theoretically be done as a sanity check. I'd still use SHA-2 (in the form of a digital signature, obviously) minimally if your intention is to ensure it hasn't been tampered with, but the speed, simplicity, and availability of MD5 can be arguments for its use as long as you're not actually concerned about security.
Jan 17 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 14:04:29 UTC, Kagamin wrote:
 There's no successful preimage attack on MD5, which is the only 
 deadly attack on a hash function.

For one amusing example of why MD5 is broken (from years ago using a video game console, of all things), consider: http://www.win.tue.nl/hashclash/Nostradamus/
Jan 17 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
Here, let me use an analogy. Note, this analogy comes from me so 
it's not necessarily endorsed by any professional 
cryptographers... but it explains where my concerns are coming 
from.

Let's say that everyone has something very precious ... like a 
diamond. Let's imagine a multiverse. Each universe can 
"represent" an algorithm. One is for MD5, another is for SHA1 and 
so on. In each universe, there exists a planet that holds a vault 
that has your diamond. Some universes are similar to others (for 
instance, the SHA1 universe has some passing resemblance to the 
MD5 universe).

When you say "specialized hardware can be made to speed up the 
process", I think "I can visit ten planets in this universe 
looking for the vaults instead of just one at a time if I invest 
in a special ship". Can you see why this doesn't concern me in 
the slightest?

When you say "Moore's law", I think "every 18 months I can know 
which half of the remaining universe contains the diamonds". 
Again, can you see why this doesn't concern me?

The universe is _just too big_ for this stuff to matter. If you 
can't find the vaults containing the diamonds, then these things 
don't bother me nor should it bother you.

For the MD5 universe, people have found a way of discovering the 
planet the diamonds are on very quickly (something like being 
able to find which millionth of the remaining universe contains 
the diamond each attempt ... a very fast solution). They've 
overcome the challenge of finding the diamonds (collision 
resistance) and now we wonder who is going to invent the 
blowtorch capable of opening the vaults up (preimage attack). All 
any cryptographer is saying is move the diamonds you care about 
into another universe before the MD5 blowtorch is invented. It 
may never get invented (we don't have any idea whether it's 
possible or if it is possible, when it might materialize) but 
moving it into another universe where the diamonds haven't even 
been found yet is simply safer. Ideally you don't want a universe 
that is too similar (such as MD5 and SHA1) because it might make 
the planet easier to be found. Also note that despite their 
similarities, no planets with diamonds have been found in the 
SHA1 universe. Yet still cryptographers are recommending moving 
on to SHA2 for new projects. Just food for thought.

Plus knowing which planet the diamond is on is dangerous enough 
since it can be used in certain circumstances to do some 
nefareous things (already showed the example where someone can 
lie about a prediction for the future). It's only in special 
circumstances, but ignoring subtleties isn't recommended in 
cryptography since people almost always figure out a way to use 
things incorrectly.

To summarize, do not use MD5 intending it to be secure against 
attackers. Just don't. I'd also like to say that you really 
shouldn't suggest to anyone that it's a reasonable approach 
either. Obviously, I can't force you to, but do know that 
generally people (not just me) are going to argue against you on 
this, so you can save yourself a lot of trouble by just never 
mentioning MD5 as being acceptable again. At this point I think 
there's been enough discussion on the matter that exceptionally 
few will wander into this topic and be mislead into accepting MD5 
for security purposes, so I'm going to retire from this 
discussion. If you use MD5 for its "security", just remember this 
for later: You were warned™. (Also, remember that people have 
been saying "Don't use MD5" since 1996, so when you do get bit by 
it, not only "You were warned" but "you are 20 years out of date, 
find a new job because you're fired" is also likely)
Jan 17 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 14:53:04 UTC, Chris Cain wrote:
 Your first part is true, but the second part is not. A generic 
 collision attack can cause issues depending on what you're 
 trying to do. For general use, MD5 is not safe enough based on 
 that alone.

No technology is safe, and SHA3 doesn't solve this problem. Cryptography is particularly not fool-proof. Remember issues in RSA? The factorization math behind it is bulletproof, yet, it's still considered hackable.
 The wikipedia article on MD5 covers this reasonably well enough:
 http://en.wikipedia.org/wiki/MD5

 Do note that SHA1 is similar enough to MD5 that many are 
 recommending against SHA1 as well (and I recommend following 
 that advice). It's not as broken as MD5 but since we have SHA2 
 there's no good reason not to use the better version.

 For general messages requiring any degree of security against 
 an attacker, use SHA2 (this does not necessarily apply to 
 hashing passwords which have their own concerns, obviously). 
 Stay away from MD5 and no new projects should use SHA1 
 (although my understanding is that no one is saying "jump from 
 the ship" for SHA1 right now). These are generally accepted 
 practices in the cryptography community. If you want to roll 
 the dice with MD5, by all means, but don't spread acceptance of 
 MD5 because someone might follow your advice against the clear 
 recommendations of the cryptography community. As far as I'm 
 concerned, it's unethical to recommend MD5 for security 
 purposes at this point.

Ethics don't help in cryptography, only real factors play role. Your insistence on blind following to recommendations and expecting them to magically protect security - by encouraging thoughtless behavior such voodooism is way more troublesome.
 SHA3 is just more convenient than MD5 because when you want to 
 change the hash function, you don't have to ditch the whole 
 system, only change its parameters.

I'm sorry but no. The reason we wanted a SHA3 is because we needed a message digest algorithm that is different enough from MD5, SHA1, and SHA2 so that an attack discovered on those algorithms should not be applicable to SHA3 as well (as was the case with MD5 and SHA1). The things you describe are just bonuses.

This works both ways: an attack on SHA3 doesn't affect other hash functions. The older algorithms are more trustworthy exactly because they are well studied, have no dark corners and evolve gradually by fixing discovered weaknesses.
 That's rather inconvenient, that you can't have an efficient 
 implementation of the algorithm on common hardware. MD5 family 
 has no such flaw.

You seem to not know enough about this topic to discuss it. SHA3 is fast on common hardware, as it is designed. Specialized hardware can be made and that's fine, intentional, and even beneficial. Most cryptography is designed that way. DES was, AES is (AES instructions exist on many modern hardware, in fact), and SHA3 is as well. This is not a flaw and if you think it is, go talk with cryptographers instead of me because you're clearly well beyond my skill level (that said, since you're discussing the viability of MD5, you're not anywhere close to as competent on this matter as you think you are, no offense).

It's even worse that new algorithms force hardware vendors to add specialized instructions which are of no use for other software. On Friday, 17 January 2014 at 15:13:27 UTC, Chris Cain wrote:
 On Friday, 17 January 2014 at 14:04:29 UTC, Kagamin wrote:
 There's no successful preimage attack on MD5, which is the 
 only deadly attack on a hash function.

For one amusing example of why MD5 is broken (from years ago using a video game console, of all things), consider: http://www.win.tue.nl/hashclash/Nostradamus/

They didn't predict the result of elections, not sure what you want to prove by a lie.
Jan 18 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 15:00:38 UTC, Chris Cain wrote:
 On Friday, 17 January 2014 at 14:06:57 UTC, Kagamin wrote:
 Doesn't TCP take care of that?

For a packet, yes. In general you can assume that if a transfer completes under TCP then it is very likely correct. That's the way TCP is designed. If you want to check the entire file at the end, MD5 could theoretically be done as a sanity check. I'd still use SHA-2 (in the form of a digital signature, obviously) minimally if your intention is to ensure it hasn't been tampered with

It's hard and expensive to use digital signatures in public projects and doesn't protect from tampering. In fact, direct tampering in such setup is cheaper than a collision attack, not even speaking, that a collision attack doesn't work here, only a preimage attack.
Jan 18 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Saturday, 18 January 2014 at 12:48:28 UTC, Kagamin wrote:
 ..snip almost everything..

I'm not going to be doing this. Everything you say suggests you have no clue about anything about cryptography and you're going to be bit sooner rather than later. Don't spread your information, just keep it to yourself so that the damage is limited. Thanks.
 They didn't predict the result of elections, not sure what you 
 want to prove by a lie.

You're correct. It was an example of a collision attack which means they can put up a single hash and it will match many possible documents that include all the names of the candidates. If you had _read_ the source, you may have understood that. In fact, if you have read anything on cryptography at all before, you'd be staying away from MD5, but ignorance is bliss I suppose. Just don't spread your ignorance to others because you could easily cost people their livelihood. If you lose your job because of this, then that's on you, so I won't feel badly about it. But I am concerned about an innocent third party walking in thinking you might have a clue of what you're talking about, when you clearly don't. Take care.
Jan 18 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Saturday, 18 January 2014 at 15:17:52 UTC, Chris Cain wrote:
 You're correct. It was an example of a collision attack which 
 means they can put up a single hash and it will match many 
 possible documents that include all the names of the candidates.

Consequence of a 2-document collision is big enough, that a 12-document collision doesn't pose any notably bigger threat. Nostradamus is all they could come up with, which is laughable. If they upgrade this to an actual partial 3.6-bit preimage attack, and one collision takes 2 days as they say, for a comparison, with such partial preimage attack, full preimage attack on a 32-bit hash like crc32 will take 2^^(32-3.6)*2 days = 1959870 years in an ideal case.
 If you had _read_ the source, you may have understood that. In 
 fact, if you have read anything on cryptography at all before, 
 you'd be staying away from MD5, but ignorance is bliss I 
 suppose.

Isn't it you, who insist on ignorance to how a collision attack works and how it doesn't work? You insist on a magical approach to cryptography, that MD5 magic doesn't work and SHA3 magic works, but you should know that magic is a delusion, and delusion leads to failures and damage, so by spreading delusions, it's you who cause damage, not me.
Jan 19 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Sunday, 19 January 2014 at 15:09:46 UTC, Kagamin wrote:
 Isn't it you, who insist on ignorance to how a collision attack 
 works and how it doesn't work? You insist on a magical approach 
 to cryptography, that MD5 magic doesn't work and SHA3 magic 
 works, but you should know that magic is a delusion, and 
 delusion leads to failures and damage, so by spreading 
 delusions, it's you who cause damage, not me.

Excuse me? Straw men arguments don't work here. Take that crap elsewhere. You keep asserting I say things that I don't say like crap like "You assume that Moores law doesn't work". Get out of here with that. The fact that MD5 is weak against a collision attack means that it strictly provides weaker guarantees than the stronger SHA1, SHA2, and SHA3. No magic is necessary to know that with cryptography, attackers look for the weakest point in a chain to attack. Intentionally using something that provides weaker guarantees is foolish. And indeed a collision attack can cause problems, depending on what you want to do. I can't predict (nor can you) where some one will use a hash function expecting it to be secure, so the right thing to do is just suggest people use the stronger hash functions. Furthermore, when talking about anything related to cryptography, we discuss things in terms of how much of a safety buffer we get. You're right that nothing is "perfectly safe" but when our estimations of safety suggest it should remain safe for at least 20 years, then we're fine with it. That's why we don't use MD5. Because it has known flaws AND because it no longer has the "shield" of collision resistance. For a hash function, if it is "collision resistant" that strictly means that no pre image attack exists either. You'd have to break through two walls to break the hash function. With MD5, there is no such safety buffer. It's one discovery away from being destroyed. Ergo, don't use it because it can turn from "no pre image attacks exist" to "here's the pre image attack" overnight. Preempting your counterargument: Indeed such a thing can happen with SHA2, but the the likelihood of such a thing is essentially non existent. If we haven't even discovered a single collision ever, we don't find the likelihood of generating collisions at will likely nor do we find the likelihood of generating a specific collision likely either. Finally, I have never suggested anyone work off of "this stuff is magic so be ignorant of it while working with it". Quite the opposite. You should be well educated with this stuff prior to working with it or else you risk exposing confidential information which can be quite damaging. Part of this is _don't deliberately ignore the recommendations of cryptography experts_. What you suggest people do, "despite being a non-expert, I suggest you do use MD5 because those experts clearly don't know what they're talking about" is dangerous. And stop with the straw men attacks because I'm sick of it. Actually, I'm sick of all of your crap. You can have the last word and if it's "ignore the cryptography experts because I'm smarter than them", then so be it. You might want to also let your employer know as well so they can give you a raise *wink*.
Jan 19 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Sunday, 19 January 2014 at 15:09:46 UTC, Kagamin wrote:
 Consequence of a 2-document collision is big enough, that a 
 12-document collision doesn't pose any notably bigger threat. 
 Nostradamus is all they could come up with, which is laughable. 
 If they upgrade this to an actual partial 3.6-bit preimage 
 attack, and one collision takes 2 days as they say, for a 
 comparison, with such partial preimage attack, full preimage 
 attack on a 32-bit hash like crc32 will take 2^^(32-3.6)*2 days 
 = 1959870 years in an ideal case.

By the way, I didn't respond to this, but I suppose I should. https://code.google.com/p/tweakcrc/ http://blog.fortinet.com/trash-crc32/ Hence why you should keep your butt out of cryptography discussions until you take any class on it at the very least. Your calculation is garbage and shows how you haven't a single clue about what you are talking about. Again, you can have the last word, but I'm certain I'm done here regardless. If you can't see the fact that you aren't smarter than the experts by now (with the above) then I don't think I can convince you with my words. Experience is the best teacher sometimes. I just hope that I never accidentally use a program you write expecting my data to be secure.
Jan 19 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
One more thing, if you're curious about taking your first step 
into actually learning the basics of cryptography, you can use 
this:
https://www.coursera.org/courses?search=cryptography

Dan Boneh is a well respected expert in cryptography and his 
class is very well structured (it wasn't the first one I took, 
but it's definitely comparable to an actual college course on 
cryptography). Do not take it expecting that you'll even be 
remotely capable of going against the recommendations by experts 
by the end (in fact, you'll learn that such things is idiocy if 
you pay any attention to it).

If you do take it, do the assignments and homework. Despite such 
things having the reputation of being just a bother, the 
assignments are necessary to fully understand the material for 
advanced topics like this. They are tough but they'll teach you 
pretty quickly that looking at, for instance, the number of bits 
in crc32 is meaningless for creating attacks.
Jan 19 2014
prev sibling next sibling parent "Uranuz" <neuranuz gmail.com> writes:
I don't feel myself confident about crypto and security 
questions, but I need to make password hashing and generating of 
session Id. And make it difficult to pick up password with bruto 
force or dictional with single "usual" computer. I'm slightly 
disappointed that then more I read different articles on IT 
forums then less I understand something. And there are several 
opposite ideas that stunning me.
  1. All security systems, cipher, etc can be hacked If someone 
wants it
  2. Do not reinvent the wheel. All have been invented already.
  3. If you use standart implementation it's high risk than it was 
cracked already.
  4. Is it really essential to someone tho crack you security.

About md5 I have read that it's already cracked. It's vulnerable 
to length extension attack. As I feel SHA 2 is better (but it's 
not my opinion - it's just subjective feeling). And may be more 
modern algorithm isn't hacked until now. Higher variety of 
standart implemented hash algorithms can enable to combine them 
in different manner to get not standart implementation of hash. 
As I think it can increse security against attacks with rainbow 
tables.

I don't know if I rigth or not. The reason why I asked is that 
I'm implenenting authentication on site written in D. So I want 
to make password hash generation function enough secure to forget 
about it for ~5 years or more. Because there only a litle of hash 
functions implemented in std.digest and they are not so strong by 
security reasons. It makes it not very useful.

P.S. Sorry for my English.
Jan 21 2014
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:
 I don't feel myself confident about crypto and security 
 questions, but I need to make password hashing and generating 
 of session Id. And make it difficult to pick up password with 
 bruto force or dictional with single "usual" computer. I'm 
 slightly disappointed that then more I read different articles 
 on IT forums then less I understand something. And there are 
 several opposite ideas that stunning me.
  1. All security systems, cipher, etc can be hacked If someone 
 wants it
  2. Do not reinvent the wheel. All have been invented already.
  3. If you use standart implementation it's high risk than it 
 was cracked already.
  4. Is it really essential to someone tho crack you security.

 About md5 I have read that it's already cracked. It's 
 vulnerable to length extension attack. As I feel SHA 2 is 
 better (but it's not my opinion - it's just subjective 
 feeling). And may be more modern algorithm isn't hacked until 
 now. Higher variety of standart implemented hash algorithms can 
 enable to combine them in different manner to get not standart 
 implementation of hash. As I think it can increse security 
 against attacks with rainbow tables.

 I don't know if I rigth or not. The reason why I asked is that 
 I'm implenenting authentication on site written in D. So I want 
 to make password hash generation function enough secure to 
 forget about it for ~5 years or more. Because there only a 
 litle of hash functions implemented in std.digest and they are 
 not so strong by security reasons. It makes it not very useful.

 P.S. Sorry for my English.

I don't have any significant expertise on this subject, but I did find this highly rated article useful and interesting: http://www.codeproject.com/Articles/704865/Salted-Password-Hashing-Doing-it-Right Note that it recommends against md5
Jan 21 2014
prev sibling next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:
 I don't feel myself confident about crypto and security 
 questions, but I need to make password hashing and generating 
 of session Id. And make it difficult to pick up password with 
 bruto force or dictional with single "usual" computer. I'm 
 slightly disappointed that then more I read different articles

MD5, SHA2, SHA3, etc, none of these are valid for password hashing. Not because of being able to generate a collision, because that doesn't matter, but because it can be brute forced easily. Use bcrypt or scrypt. If you really can't do that, then hash with a salt at least thousand times (but if done improperly this can make it actually less secure). You should always use a built-in thing though, ideally bcrypt or scrypt.
Jan 21 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:
 I'm slightly disappointed that then more I read different 
 articles on IT forums then less I understand something. And 
 there are several opposite ideas that stunning me.
  1. All security systems, cipher, etc can be hacked If someone 
 wants it

A bit true and untrue at the same time. Let's look at it from the perspective of "feasible attacks" first: It's impossible to prove one way or the other. On the other hand, the only way we can verify something as secure is we have many "Really Smart People" look at it and see if they can come up with anything to attack it (and often there are often huge incentives provided to discover these weaknesses). It stands to reason if many motivated RSP can't discover a weakness, the likelihood of some joe-shmoe discovering a weakness is extremely low. It's possible but just not likely. If we accept "infeasible attacks" also, then this statement is absolutely true. If you have some motivated individual who is willing to sit around until after the sun burns out (this is NOT an exaggeration but actual calculation), then they can eventually brute force a single AES256 instance. Since I'll be long dead and so will my children and my children's children (etc), I'm not concerned about such attacks. With all that said, if you use a system that has NOT been reviewed by many motivated RSP, then you've (likely) got a ticking time bomb. That's the reason for the idea below ("Do not reinvent the wheel"). It's sometimes really difficult to discern whether you're using an accepted standard way of doing things or not, however. Even using the "accepted standard primitives" can be done in non-standard/weak ways. Consider using AES256 poorly: 1. Take any password 2. hash it with MD5 3. Use that hash as the key (pad the extra bits with 0s) to encrypt something using AES256 in ECB mode That's not strong for many reasons. But in particular step 1 is very vulnerable to attacks. If you have anything less than a 8 character password, such a scheme is very easy to break. As a general rule, passwords should be no less than 12 characters long (my passwords are stored in a password manager with 20+ characters in length and my computer is encrypted with a password of length 20+ as well) or that'll be the weak point in the system. MD5 is trash for a variety of reasons but the fact that collisions have been discovered means that it's more likely that multiple strings will hash to the same hash value. So despite being 128-bits long, it has the apparent value of something much lower. It's much more likely, for instance, that your 50-character password will hash the same as a 7-character password. Unless you specifically chose your password in such a way to avoid such a thing, it's impossible to know whether you've chosen an unlucky password where such a thing is true. Since step 3 involves padding the extra bits with 0s, that means AES256 is no more powerful than AES128. (this would be true even if you alternated between 0 and 1 or any number of other patterns, by the way) Furthermore since MD5 has been proven to not actually have the value of 128-bit, the actual encryption power is even lower than that. Furthermore, using ECB mode makes it insecure to certain types of attacks too. http://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_codebook_.28ECB.29 (look at the image ... you can discover what a message is without actually decrypting it! Yes, this works with text too, but it requires a bit more analysis that just looking at it.) Despite AES256 itself being very secure against attacks, the scheme used makes it very vulnerable to a variety of attacks. The only way to know whether it's good or not is to research yourself, or, better, ask a group of experts to break it. (The "ask a group to break it" really reminds me of Andre's "Destroy" philosophy) One last thing to consider for this: there are usually side-channel attacks that can circumvent most security systems. A famous attack is known as the "rubber hose attack": Take some guy who has information you want (such as a password to access a system or decrypt something important) and beat him with a rubber hose until he tells you what you need to know. http://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis So if you also start expanding your scope to things like side-channel attacks, the difficulty of implementing a truly secure system goes WAAAY up. The probability of all of those side-channel attacks being used is fairly low, however. Some of them are actually pretty important, though. Like timing attacks. In general, though, it is possible to get a system secure enough that such things become too unlikely to worry about. It's absolutely worth putting forth the effort to secure things despite it being very hard to nearly impossible to make things "perfectly" secure.
  2. Do not reinvent the wheel. All have been invented already.

Do not reinvent the wheel: absolutely true. All have been invented already: Not necessarily. If there exists a solution to the problem you're trying to solve (and it's likely that such a solution does exist), use the existing solution as long as you have good reason to believe it to be strong against attackers (a solution described by some random guy on a forum that isn't providing citations is an example of something you shouldn't necessarily trust ... the strength of someone's solution has no relation to how smart they think they are). At the very least you should be using cryptographic primitives to create your solution (such as using AES or one of the SHAs, particularly SHA2). You should also "peer review" your solution with others when possible (even using accepted primitives can result in weaknesses). Do _not_ hide your solution and hope no one discovers how it's done as your form of security ("security through obscurity"). That is extremely unreliable and will almost certainly result in a successful attack if you are guarding anything of value.
  3. If you use standart implementation it's high risk than it 
 was cracked already.

Very false. Look at my response to #1 again. The standard implementations have been reviewed by many very intelligent experts in the field and they'll have some pretty good tricks up their sleeve to discover potential weaknesses. A more recent example, the Dual_EC_DRBG algorithm of RSA infamy, was theorized to be weak in November 2007 because of the constants used. Although no attack was actually discovered at the time, the fact that a RSP saw the likelihood of a flaw in it shows that these guys are _very_ good at spotting weaknesses before they become an actual issue. Indeed recently it was shown to actually be weak against the NSA (who paid the RSA to include the algorithm in their standards). This type of idea usually come from amateurs who have relatively low experience in cryptography. It's an idea that, on the surface or in theory, looks pretty reasonable. But experience teaches you that this is false in practice. This whole concept is derived from "use something that no one has seen yet so they haven't discovered the weakness yet" ... or, alternatively, "Security through Obscurity". An example is "hide a key to your house under a rock in front of your front door instead of carrying it around." Or "use a lock that has a single pin but the pin is located on the horizontal instead of the vertical". Using something non-standard hoping to avoid issues will NOT actually help you avoid issues. At best you can hope to delay an attack whereas things being open means that, despite many motivated RSP having looked at it, the system remains secure from attacks. Look at this wikipedia article for a full explanation of why "avoid the standard implementations"/"security through obscurity" is bad: http://en.wikipedia.org/wiki/Security_through_obscurity FWIW, if you're "smart enough" you might be able to come up with a system using Security through Obscurity on parts of it that doesn't significantly negatively impact actual security of your system. Unfortunately, no one is able to accurately gauge if you are "smart enough". However, here's a reasonable heuristic: Do you have 4 years of specialized training/experience in cryptography alone? If no, then you're probably not "smart enough". If yes, then you likely know the risks inherent to rolling your own solution without others reviewing it and you'll know whether its worth the risks to implement it so it's up to you. If you have no experience in cryptography but think that it's bogus that you'd actually need experience to do cryptography and think you can do it anyway: You're almost absolutely not "smart enough" but you're probably "dumb enough" to ignore my advice (or anyone's for that matter) anyway, so it's a waste of time to discuss it. Do what you will, but hopefully after you fail a few times (and you will) you'll get the hint.
  4. Is it really essential to someone tho crack you security.

I don't understand. Could you rephrase this? Okay, all of that was a bit too much, but hopefully you'll find it helpful.
Jan 21 2014
prev sibling next sibling parent "Uranuz" <neuranuz gmail.com> writes:
 4. Is it really essential to someone tho crack you security.


It's beacuse I wrote it at night. I mean that developer should estimate security level that needed and do not reinvent complicated security system where it's not essential. But question is how to evaluate security level itself and what type of protection is needed for concrete data to be protected. Thanks for responses. Now I'm trying to use OpenSSL library that has complete set of crypto primitives. But I'll try to take bcrypt, scrypt into consideration
Jan 22 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:
 I don't feel myself confident about crypto and security 
 questions, but I need to make password hashing and generating 
 of session Id. And make it difficult to pick up password with 
 bruto force or dictional with single "usual" computer.

The article linked by John Colvin is comprehensive enough to give you understanding of the problem, if outlines all mistakes in password hashing schemes and how to solve them properly. The dictionary attack is the most dangerous, as indicated by the article, slow hash like bcrypt can be of some help, it may win you a little time, but that time can be enough for the administrator to lock the system. Slow hash makes brute force slower, because it has to compute hash a lot. So for password hashing there's no reason to choose md5, because it's the fastest :) There's no reason to implement bcrypt on your own (like repetitive hashing), because it should be well optimized in order to not let the attacker to compute the hash faster by a more optimal implementation. Keyed hash is a very strong protection if you keep the key secret. The task of protecting the key is more administrative than cryptographic. If the attacker can't retrieve the key, he can only brute force the password remotely on your system, but since he will use your code, you have the chance to detect the attack. What to do once you detected it is up to you. Another possibility to consider is two-factor authentication for the case when the password is indeed stolen.
 About md5 I have read that it's already cracked. It's 
 vulnerable to length extension attack.

The length extension attack is used against digital signatures, it's useless against password hashing (that's why it's not considered in the article). The attack is prevented by prepending salt instead of appending.
Jan 22 2014
prev sibling next sibling parent "Uranuz" <neuranuz gmail.com> writes:
Thanks for responses. I considered to use scrypt for now, because 
as described is's harder to hack it with brute force using GPU, 
ASIC, FPGA than bcrypt. But it has some limitations too. I've 
tried to tweak it to use more memory and less computation time 
but failed. Another question for me is how would I support this 
type of password authentication. After some years computation 
powers will grow and I will need to modify my hased keys stored 
in DB to adequately resist this. So I need some protocol where 
some parameters could be tweaked I'll be able to "rehash" or make 
some additional iterations with existing password hashes with no 
need to disturb users that "we have some security updates and ask 
users to change their passwords". So this part of reply is 
interesting for with from described point of view.

 Consider using AES256 poorly:
 1. Take any password
 2. hash it with MD5
 3. Use that hash as the key (pad the extra bits with 0s) to 
 encrypt something using AES256 in ECB mode

I need kind of function with the these properties. KDF(passwd, salt, jobFactor1) KDF(passwd, salt, jobFactor2) where jobFactor1 - initial number representing amont of job needed to calculate hash, and jobFactor2 ( > jobFactor1 ) is index that will be actual in the future. For old hashes I would do something like this KDF_diff(KDF_old, jobFactor2 - jobFactor1) But for new users (or for users that wants to change password) I will call original KDF(passwd, salt, jobFactor2) with new job factor. Also would be suitable implementation where KDF_diff is same as KDF and simply used as KDF(KDF_old, salt, jobFactor2 - jobFactor1). It can be simply achieved by using some SHA2, SHA3, MD5 or other hash function used in cycle. But I want to believe that some other elegant solution exists in combination with memory-hard function like scrypt. Also I found some info about new Key Derivation Functions. But seems that they are not enough checked for security and have enough tested implementation. http://lyra-kdf.net/ http://eprint.iacr.org/2013/525.pdf Also I found site about password hashing functions competition. It will be interesting to read what D community thinks about it. https://password-hashing.net/faq.html
Jan 27 2014
prev sibling parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 27 January 2014 at 09:08:04 UTC, Uranuz wrote:
 Thanks for responses. I considered to use scrypt for now, 
 because as described is's harder to hack it with brute force 
 using GPU, ASIC, FPGA than bcrypt. But it has some limitations 
 too. I've tried to tweak it to use more memory and less 
 computation time but failed. Another question for me is how 
 would I support this type of password authentication. After 
 some years computation powers will grow and I will need to 
 modify my hased keys stored in DB to adequately resist this. So 
 I need some protocol where some parameters could be tweaked 
 I'll be able to "rehash" or make some additional iterations 
 with existing password hashes with no need to disturb users 
 that "we have some security updates and ask users to change 
 their passwords". So this part of reply is interesting for with 
 from described point of view.

Just a quick thought of the hash migration might be to store the algorithm version (lets say 0 for scrypt). Then when you do an opCmp on the hash password or how ever you're comparing. And it checks out it'll update and save it on login. If it gets too out of date then maybe ask them to login to change their password.
Jan 27 2014