Some languages like PL/I and Oracle Database SQL indeed use ||
for string concatenation.
One reason is maybe that +
might be confusing when talking about fundamental cryptography, since there is a lot of math involved. The mathematical notation for 'OR' would be reversed caret $\lor$ and the exclusive 'OR', better known as 'XOR' is a circled plus $\oplus$.
But I don't think that there is a specific reason for using ||
for a string concatenation. If anything then I would presume that someone used it once early and then it has become accustomed until it has become a standard for cryptography.