Keeping humans relevant in the shadow of TL10 AI.

[Fred Brackin]

One idea I've had is that programming ability does not increase at the same speed as device capacity. Also, the bigger and more complicated your program needs to be the longer and harder it is to get the bugs out. AI would be hardest of all of course.

The consequences of this might be that even though people gained 2 levels of complexity between TLs 9 and 10 that msotly means they moved to computers 2 levels smaller to run their reliable TL9 programs. If they did go for the extra levels it was so they could run 100x as many of those programs.

The result of the difficulty of AI debugging might actually be that AIs have shorter lives than humans before their programs get corrupted and have fatal crashes. This could be a problem even with lower Complexity and IQ AIs.

[RyanW]

Originally Posted by AlexanderHowl

Recent estimations have there being around 5,000 alien civilizations in the Milky Way Galaxy currently (https://www.astrobio.net/alien-life/...ons-are-there/).

The critical numbers are still very much guesses. If we find evidence of life having developed independently on Mars, Europa, the cloudtops of Venus, etc. (or a separately arising, possibly extinct line of life on Earth) the numbers firm up a lot. As it stands, our evidence can only really support the claim that a self-replicating molecule appeared once. It could be that the specific conditions that caused it are so improbable, that the fact that it occurred even once is a total fluke. Once it happens, life adapts and spreads, but maybe that starting event had simply never happened before and will never happen again.

I personally think (though I wouldn't go so far as to say I believe) that's not the case, but it does mean that 1 is a perfectly valid solution to the Drake Equation.

[Flyndaran]

In four and a half billion years, this planet produced a single technological species. So regardless of how common complex planetary life is, intelligent life is orders of magnitude rarer.
And we still have no idea what conditions are truly required for life to start in the first place.
Anything other than saying that it is possible is pure guesswork, and not science.

[AlexanderHowl]

AI devolution could also be the result of trying to copy emergent AI (or any attempt to digitally copy biological intelligence in the form of ghosts). Copying emergent AI (or creating ghosts) would rapidly become illegal if they accumulated -5 CP of mental disadvantages every week as their coding devolved. I imagine that copying emergent AI (or creating ghosts) would become the work of mad scientists as no corporation would think it worth the investment after the first few public disasters.

[Andreas]

In discussions like these, it seems to me that the impact of Ultra-Tech AIs gets way too much focus.

We already have expert systems which can perform well for a wide range of tasks (and in many cases outperform even the most skilled humans). With the hardware improvements assumed in UT, as well as decades or even centuries of further software development, the capabilities of such expert systems would be massivly increased.

For example, why look at UT AIs for things like pilot and gunner performance in space combat? There are already computer programs which perform really well in simulated atmospheric dogfighting (and space combat is in many respects a simpler problem than that). Expert systems would surely be used for far more complicated tasks than those.

[Flyndaran]

The problem isn't whether there's any sensible reason for something that's within the realm of possibility. It's that as long as it's something that someone really really wants, it will get created eventually.

[vicky_molokh]

Originally Posted by David Johnston2

And there's a reason for that. This reason: https://www.youtube.com/watch?v=RdXjxlFM6Jo. This is the product of "translating" without understanding. Nobody assumed that you couldn't mechanize the process of looking up isolated words in a bilingual dictionary. You just can't have the result reliably convey the meaning of a sentence because the machine doesn't understand what things really mean. And in closing I'd just like to say Мій судно на повітряній подушці повно вугрів.

They're not fully isolated though. While the interconnections between them aren't handled perfectly*, nowadays they're handled to a degree that makes the auto-translations more useful than a series of dictionary-pair replacements. But the bit about not understanding the meaning while translating really gets into a Chinese Room problem. Notably, that the speech centres of a human mind probably don't wouldn't qualify as fully comprehending the meanings either, but when acting in concert with the rest of the brain produce capability to participate in meaningful exchange of information about experienced phenomena.

* == Google seems to be terrible at flexion in languages, but I suppose it's understandable given the English starting point.

[David Johnston2]

Originally Posted by Flyndaran

The problem isn't whether there's any sensible reason for something that's within the realm of possibility. It's that as long as it's something that someone really really wants, it will get created eventually.

Ah but "created" isn't really the issue. A few curiosities created as proof of concept or for specialized purposes aren't going to change the world that much. It has to go into general distribution before you need to wrestle with the impact.

But, once again, humanity is at more risk from the non-volitional AIs because the truth is, there aren't all that many jobs that actually call for creativity and rebelliousness. For the others an IQ 10+ AI that will never get out of line is better than a human.

[AlexanderHowl]

Which generally means one of three things occur. First, the rich replace the non-rich with non-volitional AI until the entire economy collapses because there are insufficient consumers left. Second, the rich replace the non-rich with non-volitional AI until the non-rich rebel and restructure society to avoid non-volitional AI except in the most dangerous jobs. Third, the rich replace the noj-rich with non-volitional AI until the non-rich rebel and restructure society to give the unemployed a basic wage that allows them to consume enough to support the economy. The first is the trajectory of our current society, the second is the more likely alternative, and the third is the less likely alternative.

[Andreas]

Originally Posted by AlexanderHowl

Which generally means one of three things occur. First, the rich replace the non-rich with non-volitional AI until the entire economy collapses because there are insufficient consumers left. Second, the rich replace the non-rich with non-volitional AI until the non-rich rebel and restructure society to avoid non-volitional AI except in the most dangerous jobs. Third, the rich replace the noj-rich with non-volitional AI until the non-rich rebel and restructure society to give the unemployed a basic wage that allows them to consume enough to support the economy. The first is the trajectory of our current society, the second is the more likely alternative, and the third is the less likely alternative.

An economy doesn't require a sufficient number of consumer in order not to collapse. You wouldn't even necessarily need a single dedicated consumer, you could just have a huge number of AIs that (other than mantain themselves and their infrastructure) just stockpile produced goods according to the orders left behind by their long dead masters.

Rather, what is necessary for an economy not to collapse is obviously a minimum level of production.