# Explaining entropy

Introductory Chemistry course, non-science majors. I am trying to explain entropy without using the word randomness. Example: a deck of 52 cards has multiple states of card order. That we may like a particular order, say suit order, does not mean suit order has less entropy than does any other particular order. To reduce the entropy of a deck of cards one would have to reduce the number of cards. So a deck of 51 cards has less entropy, less possible arrangements, than a deck of 52.

Any comments will be gratefully studied.

• The meaning of entropy in chemistry (thermodynamics) and the meaning in cryptography are significantly different; if you have a question about the 'in thermodynamics' meaning, you're probably better off asking the chemistry stack exchange Apr 26, 2018 at 14:25
• If you find a way to bridge the gap between the chemist's view of entropy, and the information theorist/cryptographer's one, kudos! For the later, it is (among other possible definitions) a quantity of information, in bit, necessary to represent a variable with a certain distribution. If there are $s$ equiprobable states, that's $\log_2(s)$; for a shuffled deck of 52 cards, $\log_2(52!)$ ≈225.6 bit. In chemistry, uh.. is that even on-topic?
– fgrieu
Apr 26, 2018 at 14:27
• Also, with the cryptographical meaning, "to reduce the entropy of a deck of cards", all you need to do is display the order of the cards. That means that the probability of that specific card order is 1, while the probability of any other card order is 0, and so we've reduced the (informational) entropy to 0. The (chemistry) entropy meaning doesn't have any related 'give out information to reduce entropy' effect. Apr 26, 2018 at 14:32
• Please don’t get me wrong, but this is a question and answer site – and for a question, your post really lacks in the realms of question mark usage. Maybe you could edit your “question” a bit? Thanks in advance. Apr 27, 2018 at 3:47
• Comments are not for extended discussion; this conversation has been moved to chat. Apr 27, 2018 at 3:48

Unfortunately, order and random are two ends of a scale. And where you are on that scale is called entropy. If you're asking on a cryptography forum, we have to look at this from an information angle.

## Thermodynamic entropy from an information perspective

If all the balls are at rest because the stuff is cold, they are very ordered. It is easy to describe their positions and thus the state of the material.

If the stuff is hot and all the balls are flying around everywhere (randomly), it's very difficult to totally describe the state of the gas molecules. You'd have to write down all of the current 3D positions of them and their respective velocity vectors. That's a lot of writing and thus lots of entropy.

## Entropy from a poker and information perspective

Consider the two decks. It's very easy to describe this one:-

It's simply C,S,H,D,1-10,J,Q,K in code I just made up. You might be able to shorten it. This illustrates that an unsorted deck doesn't have a lot of entropy.

Now attempt to describe this shuffled deck:-

You'd have to write down each card individually, there's no other way unless you can find some sort of mathematical relationship between each card. If this is a fully random shuffle, the state contains $$log_2(52!)$$ bits, or about 226 bits of binary information. It cannot be completely described with less.

To answer your question of removing cards - not necessarily. You could just slightly order the cards, forming relationships between them. Say all Diamonds are ordered as new. That would reduce the entropy whilst keeping the same number of cards.

## Summary

Entropy = information necessary to describe a system or state.

In cryptography, if you know the information, entropy = 0. So cryptographic entropy is the enumerated amount of uncertainty. And we measure that in bits.

• The specific arrangement of cards in the second photograph has zero entropy to anyone who knows it, just like in the first photograph. A uniform random permutation of a deck in canonical order has equal probability of yielding each photograph. The uniform distribution on permutations has $\log_2 52!$ bits of entropy, not any one particular ordering. It is a property of a distribution, not a particular outcome. Apr 26, 2018 at 18:01
• To demonstrate a deck of cards with maximum entropy, you'd want to only show the backs of the cards Apr 26, 2018 at 18:15