Is there a way to generate sound one-time pads out of public information?

Is there a way to 'mangle' a public data-source (for example, the current date in YYYYMMDD or the top New York Times headline) to form a one-time pad that will sufficiently hide the pad's source? What are the other issues such a scheme would be running into?

Disclaimer: I am not an expert cryptographer and am just interested in the matter - I do not intend to implement this cryptosystem.

-

Quite Short answer: No, because a scheme can only be a One-Time-Pad if the entire pad is perfectly random and secret.
Concise answer: It sounds like you're trying to build a stream cipher. The security of it really comes down to how much of the scheme you think can be kept secret. If I listen in to your wifi and hear you requesting a certain resource, then I can work out what your data-source was, calculate the pad myself and decrypt your data. This tends to be bad!

Full answer: Any scheme 'like' the One-Time-Pad (OTP) breaks down into two sections: first we form a pad and second we combine the pad with the plaintext. The second of these tends to be through the xor operation (denoted $\oplus$), but this doesn't really matter for this question.

So, the first section: generating a pad. For the OTP, we assume the pad is perfectly random and secret. If so, then the scheme is perfectly secure. However, this means we somehow have to transmit a large amount of secret information to the other person before we can send them the message, which in most cases just means we pass the difficult problem of key distribution upstream.

So, how do we get around this problem? We take a small key that can be more easily distributed, and expand it with a method that everyone involved knows to form a stream cipher. This is a function that tries to expand a small key into a suitably large key stream to pad the message with, and we have some pretty tough requirements of it. In particular, we demand that the key stream appears random. That is, someone seeing even a large amount of the keystream should not be able to differentiate it from truly random data (within 'reasonable' computation effort). If they cannot differentiate it from random data, we decide that the output ciphertext is sufficiently secure.

Before I try and pull things together, I'd like to point you to Kerckhoffs's principle. Basically, for any real level of security in the modern world, we require that a cryptographic algorithm should be secure even when the attacker knows everything about it except for the key. Note that 'the key' is a vague notion - one can draw the algorithm-key boundary pretty well wherever they like - but good practise says the key should be as small as possible (eg algorithm=RandomAccessMachineSimulator; key=MyCryptoAlgorithm) without compromising the security of the algorithm.

So, how does your situation work? Well, turning a large amount of not-very-random data into a small about of 'quite random' data is a question of [extracting the entropy][Note 5]. A naive way of doing it would be to simply pump all your input data through a key derivation function like PBKDF2, but I won't go into this in much detail chiefly because there's a bigger problem: What do you keep secret? - Keeping the entire algorithm secret: If you want to keep everything secret from an attacker, then how do you tell the person you're messaging how to decrypt it? Somehow you must privately give your ally all the required details. At this point though, why not just give them an AES key that you can then use to communicate securely? - The data-source is secret: This makes a bit more sense. All the 'fiddly' computer bits (eg the entropy extraction) can be exchanged insecurely (eg email), then you send them the data-source (URL?) in secret. However, (assuming that for some reason this time we can't just send a key instead) anyone listening to your HTTP requests will hear all URLs loaded, and by testing each one of them will find your key very efficiently.

[Note 5] - Is there already a good description of this somewhere?

-

What you've described is generally called a "book cipher" or "Ottendorf cipher", where the "key" is knowing which publication is being referenced, as well as the algorithm for recovering information from it.

A hundred years ago they were quite secure because not only were books fairly rare, but trying every book against an unknown cipher was very time consuming. Some historically famous codes, such as the Beale Ciphers, used this method, and they withstood a lot of very motivated attackers for a very long time.

But with the digitization of vast quantities of books and other print resources by huge organizations such as Google, (and security agencies) they are now pretty much a historical footnote.

-