How close are we to NAIs?

[whswhs]

Originally Posted by WingedKagouti

ChatGPT and other algorithms are basically predictive language models that look at a large set of data, trying to find the most apropriate output given the current input (which includes the previous output in the session).

On one hand, they might be able to pass the Turing test. On the other hand, they show the limitations of the Turing test as an actual measure of "intelligence."

[Prince Charon]

Originally Posted by whswhs

On one hand, they might be able to pass the Turing test. On the other hand, they show the limitations of the Turing test as an actual measure of "intelligence."

IIRC, there are humans in real life who have failed a Turing Test (given via text) for various reasons, including a tester or testers being unwilling to believe that an actual human knew that much about $topic, and could answer that quickly.

[ravenfish]

I don't think ChatGPT is ready for the Turing Test yet, bearing in mind that the Turing test involves a sustained attempt to find out which participant is the AI rather than mere casual interaction.

Knowing one's own limitations may be the mark of a great mind, but even ordinary humans practice it to a limited degree, whereas ChatGPT seems to be utterly without this ability. When asked a question to which it does not know the answer- for example, a question that doesn't have an answer because it assumes the existence of something that the questioner made up- it produces a result that is superficially plausible but clearly incorrect to someone familiar with the subject.

When I told it to "Tell me about 'One More Day' by the Edsels", I was told that this song was originally performed be the Heartbeats in 1957 and covered by the Edsels in 1963, followed by a description of the Edsels and a plausible description of the song. I was momentarily concerned that I had accidentally asked it about a song that really existed (I freely admit my ignorance about mid-twentieth century music, and I could find a complete discography for neither group), but, when I repeated the question, I learned that the song actually was by the Edsels, and was released in 1961, so I am now fairly confident that the ignorance was in the algorithm and not in me.

Similarly, when I told it to "Tell me about Euler's Last Theorem", it told me that this is another name for Fermat's Last Theorem, followed by a description of the latter; a request for information about the American small-clawed otter produced what seems to be based on a description of the Asian small-clawed otter, complete with a mangled version of the scientific name of the latter and the intriguing statement that the American small clawed otter lives in Southeast Asia (perhaps it's an expat).

[Interestingly, in a few cases it did recognize that I was asking about something nonexistent. When I asked it about Beethoven's Thirteenth Symphony, I was told that he only wrote nine.]

Basically, ChatGPT is a cool toy and very impressive for what it does, but is (as of yet) neither capable of seriously passing for human nor of serving as a practical source of information.

[whswhs]

Yeah, I asked it to explain two obscure lines from a Joni Mitchell song, and it gave me a lucid explanation—but it had those lines as part of her early song "For Free," not the later song "Don't Interrupt the Sorrow" where they actually occur. So the context was entirely wrong. It all gives the impression of confabulation, and indeed, I can imagine that the brain might be doing something like that when it confabulates.

[DaibhidC]

Originally Posted by ravenfish

[Interestingly, in a few cases it did recognize that I was asking about something nonexistent. When I asked it about Beethoven's Thirteenth Symphony, I was told that he only wrote nine.]

That is interesting. I would guess, from a very non-AI-understanding perspective, that it was in posession of the fact "Beethoven wrote nine symphonies", and was therefore able to extrapolate that there wasn't a thirteenth, but didn't have any facts from which it could extrapolate the non-existence of the other things, so instead defaulted to trying to extrapolate what they might be.

[Astromancer]

For me there's a category of "good enough A.I " as in "it's good enough to do a job I find useful". An A.I. might be competent enough to weed the strawberry patch (which I will never be, at least not if the goal is growing strawberries) without being able to drive a car on a real highway with other drivers including humans.

Clever toys are fun. But does ChatGPT show progress toward useful A.I.s good enough to do valuable work? People are using the Facial Recognition software from airport security to create farming programs to tend crops. You'll still need skilled hardworking farmers but inputs of water and expensive agrochemicals will be dramatically reduced.

[WingedKagouti]

Originally Posted by Astromancer

Clever toys are fun. But does ChatGPT show progress toward useful A.I.s good enough to do valuable work?

Think of ChatGPT et al. as large databases of knowledge that can be searched with normal sentences and will provide an answer in the same format.

If you yourself is knowledgeable in a field, you will be able to put in a sensible question and should be able to figure out if the answer makes sense in the given context. And you need to analyze the reply, because sometimes the LLM (Large Language Model) will output something nonsensical or factually false while trying to use language that sounds authoritative.

The first part of the breakthrough LLMs present is the ability to analyze a sentence with reasonable accuracy and translate that into something a computer can understand. The second part is the ability to take a query and put together a human readable reply using various language rules (and a massive database of examples). Or put together components of an image for the algorithms focused on that.

[Astromancer]

Originally Posted by WingedKagouti

Think of ChatGPT et al. as large databases of knowledge that can be searched with normal sentences and will provide an answer in the same format.

If you yourself is knowledgeable in a field, you will be able to put in a sensible question and should be able to figure out if the answer makes sense in the given context. And you need to analyze the reply, because sometimes the LLM (Large Language Model) will output something nonsensical or factually false while trying to use language that sounds authoritative.

The first part of the breakthrough LLMs present is the ability to analyze a sentence with reasonable accuracy and translate that into something a computer can understand. The second part is the ability to take a query and put together a human readable reply using various language rules (and a massive database of examples). Or put together components of an image for the algorithms focused on that.

So a clever researcher could go fishing through a large datafile and gain some possibly useful leads.

[WingedKagouti]

Originally Posted by Astromancer

So a clever researcher could go fishing through a large datafile and gain some possibly useful leads.

One of the most useful things I've seen done with it, was a coworker using it to get suggestions for potential solutions to a frustrating task. He did have to go through several suggested solutions before getting something that worked.

[Prince Charon]

Originally Posted by WingedKagouti

One of the most useful things I've seen done with it, was a coworker using it to get suggestions for potential solutions to a frustrating task. He did have to go through several suggested solutions before getting something that worked.

I think that's something that others have done when using it for writing (as well as editing the results, but that mostly doesn't apply to asking it for solutions), since the results are often variable in quality. Probably also true for creating character sheets, or backstories based on character sheets.