Without going deep into math, is there a way to make sure that non-open source programs like WhatsApp, FaceTime, Zoom, etc. are really using end-to-end encryption instead of just 'regular encryption' (i.e. the server has the cryptographic keys)?
$\begingroup$ Check in the user folder for (public-)key of the people you're contacting? $\endgroup$– DannyNiuApr 8, 2020 at 3:35
7$\begingroup$ @DannyNiu even if there is a key, the software could just not use it, use it incorrectly, leak it to a central server, silently allow a central server to replace it or have generated it in an unsafe (and possibly backdoored) way in the first place. $\endgroup$– ManfPApr 9, 2020 at 17:06
$\begingroup$ You might find this Turing Award Lecture interesting. cs.cmu.edu/~rdriley/487/papers/… $\endgroup$– Thorbjørn Ravn AndersenApr 11, 2020 at 9:13
Is there's a way to make sure that non-open source programs are really using end-to-end encryption?
Only by deep reverse-engineering. Which is hard, and might be illegal. Plus the apps are a moving target: they change like weekly. And, using end-to-end encryption is not proof that no interception is possible: this encryption (or its key generation) could be weak, accidentally, deliberately, perhaps surreptitiously. In the end, there's no practical way¹.
True story: I was the lead engineer for this² modem capable of end-to-end encryption with negotiation of session keys using Smart Cards, released in 1990, sold in France with some success. A director of our company insisted that we get approval from the authorities for these features, because that was raised in a conversation with Apple France, our main distributor. Soon after (mid 1991) I had a guy introducing himself with a military rank in my office, explaining that the law required that for crypto devices sold to the general public, there must be a way to find the key with a test requiring at most 240 steps. He required a detailed description of the protocol. I argued that changing it would break interoperability. The guy proposed addition of a side channel leaking the session key, so that if one end was using the rigged device, the session key would be available for one intercepting the line signal and knowing the method. I had the distinct impression this was standard procedure. In the end, I created a new packet type that older devices ignored, containing the session key RSA-encrypted, sent at moreless regular intervals, and new models did that. I remember the guy came for a demo. I insisted not to know the production private key, for my own safety. I still have the source code for that device, in 8051 assembly.
Note: the main feature of that device's point to point crypto was that, when you had made a set of Smart Cards, and inserted one in your modem, you could be confident that only someone holding another Smart Card from the set could connect. This feature was not compromised, as French law did not ask.
¹ No practical way to be confident that there is effective end-to-end encryption. It's sometime possible to prove that there is no end-to-end encryption. For example, if data goes thru a central server thru a https link protected using a server certificate, and the user messages are revealed by breaking that https encryption using a local proxy (a standard hack on one's device), then we are sure there is no end-to-end encryption.
² Scroll down and click on the wide picture or right arrow for more, including internals.
I have no connection with this website.
$\begingroup$ @fgrieu, when was this, roughly? That device in the pictures doesn't really look like the latest fashion. $\endgroup$– ilkkachuApr 8, 2020 at 10:30
1$\begingroup$ @ilkkachu : I had a look at the revision history. Release was autumn 1990, and the rigging occurred july-august 1991. The product was sold for at least another two years, though I made my best to not promote it, and we made/sold cheaper versions without crypto or Smart Card. $\endgroup$ Apr 8, 2020 at 11:17
7$\begingroup$ @Joshua: because the device is a modem, the packet with the encrypted session key is never accessible in the computer using the device. It exists in the modem, and on the phone line modulated per the V22bis standard. It would be observable only with appropriate equipment, and removing it requires modifying the firmware in the EPROM socketted in the modem, thus reverse engineering. $\endgroup$ Apr 9, 2020 at 7:02
1$\begingroup$ @FabianRöling No, this is something nations of every description have done or attempted to do at some point, with varying degrees of success. See the story of Crypto AG, where the U.S. and West German governments conspired to sell backdoored crypto devices to enemies and allies alike, for decades. $\endgroup$– XanderApr 10, 2020 at 20:55
In the original question:
non-open source programs like WhatsApp, FaceTime, Zoom, etc
I think what you're really asking for is whether you can trust that something is using end-to-end encryption that you didn't personally install and host.
I don't think this is practical, even for open source software. The reason being that if you use someone else's service running jitsi or BigBlueButton, you're assuming they're running exactly what's provided on the public repository. Who really knows whether this is true.
So in my opinion, your best option is to find a service that uses some form of external verification (audit) that you trust, or run it yourself.
1$\begingroup$ This is another very interesting point you raised here! One open source project may provide the full code so that anyone can see, but provide an installation package using different code. Who would know, right? $\endgroup$– FulalasApr 10, 2020 at 22:42
Agreed there is no way to be sure, in the sense that in general it's hard to be completely sure what is going on in any electronic device. Even if you dismantle it, and somehow observe what is going on, how do you know the device you are using to inspect it has not been hacked? Even supposing you built it yourself from scratch, how do you know someone didn't compromise it while you were asleep? You can take paranoia to almost any level.
In practice though, a reasonable test would be to test it externally, that is capture a sample of the packets of data exchanged and then examine the bits to see if the data is encrypted as advertised. Provided the external specification is published, or even if it is not, this should not be impossible. Of course this would only show that the captured packets were properly encrypted, it would be possible that the device sends un-encrypted packets on Sundays. In general, to be reasonably confident that cryptographic software does what it claims, you do need to examine the source code.
1$\begingroup$ Two problems: 1) as advertised often does not apply: the method may not be be published at all or with sufficient detail/accuracy/freshness. 2) it is possible and practiced to make crypto devices that seem to behave as advertised for any external observer, but actually are weakened. One method generates session keys with a weakened generator: only one knowing the (key to the) weakening method stands a chance to prove that the system is breakable. Another conceivable method would leak session key not with extra packets as in my modem, but by timing variation. $\endgroup$ Apr 10, 2020 at 7:29
$\begingroup$ The question is specifically about end-to-end encryption. There is no possible way to distinguish between "data encrypted as advertised" that is decrypted on the server and data that is passed through the server without encryption, without having access to the server. There will be no difference in the data exchanged between the end devices whether the session key is generated in a way that the server can know it or not. $\endgroup$ Apr 10, 2020 at 15:22
$\begingroup$ @JörgWMittag, exactly what I had in mind! Thanks for clarifying! :) $\endgroup$– FulalasApr 10, 2020 at 22:38