We can use thermodynamics to explore this possibility. If we treat the computation of a SHA256 hash as a simple operation, with a single bit of entropy required to calculate, then the energy expended to perform the hash is equal to the Landuaer limit: k*T*ln2. The blackbody temperature of outer space (caused by the cosmic microwave background left over from the big bang) is: 2.7K. So if we built a computer that was operating at 2.8 K and was radiating waste heat into outer space at 2.7K, we could theoretically perform each hash at 1.38E-23 J/K * 2.7K = 3.7E-23 J/operation, or inversely, 2.7*10^13 GH/J, or around 13 orders of magnitude better than the current Bitcoin miners! So plenty of room for improvement.
If we assume a Moore's law type improvement in efficiency of doubling every 18 months, then the time it will take before someone develops a thermodynamically limited ASICminer will be 65 years or so. I expect that DNA-based and other soft-matter computers will be much more energy efficient than current CPUs. I think it's not crazy to believe that a soft-matter computer (ie DNA annealing computer) may improve the energy efficiency of Bitcoin mining by a few orders of magnitude.
So now if we harnessed the entire power of the sun to perform bitcoin operations, with around 3.8*10^26W of power, in theory we could perform: 3.8*10^26W/ 3.7E-23 J/operation =10^49 operations per second. The current Bitcoin mining network performs around 2*10^16 Hash/s, and so there are around 33 orders of magnitude left before the entire power of the sun is consumed calculating Bitcoin hashes.
OK, now what if someone built a hyper-efficient computer? What would happen to the difficulty of mining? Well, currently, the "target" to create a new block while mining on the Bitcoin network is: 0000000000000001F52C00000000000000000000000000000000000000000000
This number means that if a miner stumbles upon a hash that is less than this value, he/she wins the block reward. So around in in every 15 and 1/16 hexadecimals will be rewarded, which is (15 1/16)^17 = 7*10^18 hashes. So we expect each block to be solved in around: 7*10^18 hashes/2*10^16 Hash/s = 350 seconds, or 6 minutes or so. Now if we want to keep the block generation rate of the Bitcoin network the same, but with massively increased hashrate, the target just need to be readjusted. So what would the target have to be if we are using all of the power of the sun in a thermodynamically limited computer that radiates energy into space? Well, the target would have to be 10^33 times more difficult, or 33ln10/(ln16)=27 hexadecimals further to the right. So the target would have to be approximately: 000000000000000000000000000000000000000001F52C00000000000000000
And so, the Bitcoin system is pretty much OK with any foreseeable improvements in computational speed.
If you know how a quantum computer might affect these calculations (I don't), then please comment below!