Let's suppose that P=NP. That is, every problem whose solution can be quickly verified can also be solved quickly, regardless of what that means at a formal level. So, not only does P=NP, but there are practical polynomial-time algorithms for NP-complete problems. Also, the proof is either constructive or non-constructive. That is, an algorithm can be found that we would eventually find fast enough to start using, even if we couldn't prove it. Then it becomes much harder to keep a secret--a massive problem. What scares me would not be our inability to hide information, but our inability to reveal it.
In a complex society, we need to trust others, and what we can independently verify is not enough. In practice, we establish trust with institutions when they repeatedly supply us with accurate information. I don't see an alternative method, so if we lose the ability to verify that we are receiving information from a specific institution, then an impostor could exploit our trust, which must not be allowed. Therefore, we cannot be sufficiently well-informed to have a complex society. P=NP would destroy current authentication and therefore pose a fundamental risk if other solutions are not found.
Could other solutions be found? If P=NP, we have a world without privacy, but, cutting our losses in that regard, we could try to build authentication around that fact. Instead of entities giving us the information we need to identify them, we could just squeeze it out of them. One idea is that upon receiving a message, we would send out our message containing some information to be sent back to us with the original message, so we would know that our message had been received and that the recipient at least wanted to send the original message. Our message is spyware, enhanced by our efficient algorithm for NP-complete problems, which we use to verify the original message's source.