I've seen the word 'cost' being used several times in describing algorithms e.g. algorithm A costs more to compute than algorithm B. What does 'cost' mean in this sense?
There is not one commonly accepted definition of "cost" in cryptography, other than the definition of "cost" in a common dictionary; this is not jargon or slang to my knowledge.
Here is the most logical definition from Merriam Webster:
b: the outlay or expenditure (as of effort or sacrifice) made to achieve an object
I'd say that "object" is not a particular object here, it should be read as "objective".
Usually "cost" refers to CPU time, commonly counted using the number of CPU cycles as unit for a particular CPU.
Depending on the context it may also refer to memory usage requirements or even energy expenditure. For hardware it may also relay to transistor count, die size or other costly elements within chip design.
In general it means that it's too inefficient to compute (it would take far too long).
In the context of your question it means that algorithm B is faster than algorithm A.