• Wildmimic@anarchist.nexus
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    ·
    14 hours ago

    Intelligence needs something to anchor it in the real world, or it becomes useless. In the biological world, that is a world model, based on previous experiences - and this experience needs at least a modicum of sentience to become useful as we can see all around us. Animals more aware of their own existence are also those who are a lot better at intelligent behavior.

    The “intelligent” output we get from LLMs are NOT a sign of intelligence, it is a statistic. This is useful, no doubt about it - having computers that are able to output the average answer one would get if asking someone who knows their shit is a very good thing, as is being able to walk across a large amount of different (pre-existing) approaches. But conflating it with actual intelligence which would be able to correct hallucinations before they are uttered (because the hallucination isn’t logical at all in most cases) and which would be resistant against breaking them by asking them to do irrational things is not the way to go.

    There is no representation of the huge amount of rejections to answer a question or do a task that many people are not qualified to work on at all. An actually intelligent being realizes when it reaches the limits of it’s knowledge (or it’s training data). Intelligent beings can then try to extrapolate from existing knowledge, but they know that they actually do not know, and often communicate that (putting social things aside for the moment).

    This is the point where LLMs break down with 100% chance if pushed - they do not say: “i do not know”, or “i do not have enough data for that”, or “that doesn’t make sense”. An output is obligatory, regardless of how much an intelligent being would say “oh well, what you want from me doesn’t make sense at all”.

    And it’s as easy as asking an LLM to modify their way of communicating: If you tell it to use a symbol/word again after that same symbol/word, and continue to ask it to do this every time it fails to do so (under the guise of having reading or hearing issues) will break these things every time, sending them spiraling. An intelligent being would say “You know what, fuck you, you are messing with me”.

    This is actual proof of a complete lack of anything resembling real intelligence - intelligence would allow analysis of the situation and detection of unreasonable requests.

    Edit: they trained a model they called “Othello-GPT” with championship datasets, containing only valid moves, and tried to make the point that these things have an internal world model by providing it with game states and asking it for a valid move. It still messed up 5% of the time with real world data, meaning there is no consistent reasoning going on here.

    • robert@piefed.ca
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      11 hours ago

      What are you are basically saying here, is that LLMs are functionally equivalent to intelligent systems but are not themselves intelligent. Most people will not find this satisfying because intelligence is defined in functional terms. Is something is flexible enough to solve a wide range of problems with human or above-human level accuracy then it is intelligent. Whether or not the system is a program, a human or an animal or some slime mold, or a bunch of statistics that system fits the functional criterion of an intelligent system.

      That said, you don’t need to go so far as to ascribe nonfunctional mental characteristics to these systems. These are mental characteristics that cannot be described in functional terms, like eg pain or emotions; you can mimic pain or emotions (which is a functional behaviour) without actually feeling them (the intrinsic, nonfunctional part).

      All I’m saying here is that LLMs fit the functional criterion of an intelligent system, which is the only criterion of intelligence we actually have.

      Does that make sense?

      • Wildmimic@anarchist.nexus
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        7 hours ago

        They fit the functional criterion of an intelligent system if you cut out the whole process of bringing real innovation instead of applying previous approaches to a wide variety of subjects. As i said, this isn’t nothing, it’s very useful. But it has already been shown that transformer models utterly fail at creating anything resembling original research, which is where intelligence really gets going. This is NOT REPLICABLE by current systems, and will never be unless humans find a new mechanism to enable it - the self-improvement of transformer models is visibly not happening (very much the opposite - too much synthetic data leads to model collapse. Haven’t heard yet of human intelligence collapsing because of more human input) (Edit: this is because LLM-derived training data only enforces already existing pathways through the statistical sieve LLMs are, it cannot create new pathways and actually reduces the capabilities of these models because other valid pathways are getting de-incentivized.)

        And a lot of this part of intelligence where humans break previous boundaries is bound to those “nonfunctional mental characteristics” you mention - may it be experiences that cause people to think a certain way that enables these advancements, or emotions that let them follow approaches that are less promising at first glance than the average, but appealing to this individual because of previous experiences. These characteristics are what enable people at the frontier of human knowledge to expand this knowledge.

        If given enough time, humans would have solved these mathematical problems on their own, we just take longer to traverse the space of already existing approaches to apply them, because we are not machines (and i heavily suspect OpenAI of simply stealing the approaches used in the cited case). But for real intelligence, we must be able to discard previous assumptions. Even if we take away everything else, LLMs are utterly incapable of escaping their training material, and this training material misses a whole lot of what actually encompasses intelligence.

        Another Edit, because i am someone who really thinks iteratively, going over my arguments again and again, refining them as i go over them again - thats part of intelligence too (and another thing that will never become replicated with the current technology used, because it would only lead to model collapse): one of the most important part of intelligence that is completely missing in training data is the refusal to do something (outside of static filters) because the premise is faulty or the available data isn’t enough to reach a solution. These things will try to come to a conclusion by brute force, and the option of “i do not know enough” does never ever come up. This acknowledgement of missing parts in training data // knowledge is what drives humans to fill these gaps, and without an LLM that actively says:“I do not know fuck about this, i need to consult external sources or even do original research” (which as i said, they are incapable of) these things are the illusion of intelligence, but not the real thing.

        Source for the inability of LLMs to do original research: https://arxiv.org/html/2601.03315v1 - This study actually provides full information about inputs, artifacts and outputs. I cite: “From these attempts, we document six recurring failure modes: bias toward training data defaults, implementation drift under execution pressure, memory and context degradation across long-horizon tasks, overexcitement that declares success despite obvious failures, insufficient domain intelligence, and weak scientific taste in experimental design.” To be fair, one of those studies was actually fine at the end - but a dataset of 4 studies showing a 25% success rate is not an attractive proposition at all, the real value could plummet very hard compared to 2/4 or 3/4, which i would expect from an really intelligent system.

        Edit to my edit (damn this thing is getting long): Of these failure modes i can only see possible improvement in the areas of memory and context degradation. Execution pressure will always be a thing, overexcitement is baked inzo these models by only allowing positive responses into the training data - refusals or proposition for change are not part of this data. Insufficient domain intelligence is the part about “i do not fucking know” which is completely pruned out of LLMs, weak scientific taste in experimental design is an unavoidable issue when using LLMs that can’t step away from training data defaults.

        I really have to add one more thing: These studies were things that were doable completely digital, no real world experimentation needed at all. As soon as the real world comes in, i postulate a horrendous decline in usable results, because these systems do neither have real intelligence, nor an “world model” as proposed by the other poster.