The answer to this question says that you should assume an attacker can do one billion operations per second: Key Size for Symmetric Homomorphic Encryption Over the Integers
Is that a single attacker with a simple desktop computer, or is that someone with a bot net or super computer?
A single PC CPU core is running at roughly 3 GHz. Both hash functions and stream ciphers exist that operate at around 1 cycle per byte (with long messages). A brute force attack on a key requires computing at least one block, which we can call 128 bits, or 16 bytes.
So about 200 million tries per second per core, or call it billion per CPU. A GPU has about ten times the computational power of a whole CPU, so that would be ten billion per second. (Those are almost an order of magnitude more than Bitcoin hash rates, but a fast cipher might be close.)
So a billion operations per second is something even your typical script kid can probably do. That allows breaking 50-bit security level in a week or two, and 56-bit in a year. A small cluster or botnet could multiply that by a couple of orders of magnitude. 64 bits is not necessarily secure against an adversary with some resources to throw at you.
After that the math becomes fuzzier. Can a state-level adversary break 80-bit resistance? Maybe. If not right now, probably in a few years or decades. 90 bits? Maybe not, but possibly in the future. 100 bits? Remotely possible to reach in the future. 112-128 bits? The top end is essentially enough against any classical adversary.
I'd say it depends of value of the assets. If you need to protect data worth let's say $10m, you can assume someone could throw at least money up to that value to get it. Converting money into computational power with AWS or Azure is pretty trivial today, and it can be done by virtually anyone.
And of course state-level parties might be interested in your data for whatever reason they like (for example NSA, interested even in your private photos - see Can They See My Dick?), and they might throw computational power of much higher value that traditional cyber-criminal could justify in given case.
So in short: always assume adversary might have supercomputer or powerful botnet.
A single core of a single modern standard CPU is capable of several billion elementary operations per second, where billion = $10^9\approx 2^{30}$, and elementary operation can be addition, rotation, XOR of a word (of say 64 or perhaps 128 bits), or an AES round (on CPUs with hardware support for that). There are Intel x64 CPUs with 18 cores, single-board computers with 8 Intel x64 CPUs, and clusters with several million cores thus capable of many quadrillion elementary operations per second, where quadrillion = $10^{15}\approx 2^{50}$.
Thus one billion operations per second is rather for a single commodity computer than several, though that depends heavily on what operation is: a SHA-256 uses a few thousands elementary operations (for each 64-byte padded block).
And then some attackers can be expected to use ASICs or FPGAs to improve on efficiency w.r.t. general purpose CPUs; that's popular for bitcoin mining (which in essence is a partial preimage attack on SHA-256∘SHA-256). Or it could be GPUs (not competitive for bitcoin mining, but still popular for password cracking).
