• Wildmimic@anarchist.nexus
    link
    fedilink
    English
    arrow-up
    28
    arrow-down
    2
    ·
    8 hours ago

    The underlying problems are still the same. Transformer models do not have an inkling of intelligence, and they still don’t have an internal model of the world around them. Creativity is also non-existent, which is a hard requirement if you want these things to actually create something new instead of rehashing what already exists.

      • Wildmimic@anarchist.nexus
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        1
        ·
        6 hours ago

        I am not a non-user. I run local models, because i am a tech enthusiast, and am watching the space closely. I will never spend a cent or use the offerings of OpenAI and Anthropic tho - i will not finance the people who are providing the tools to bind us all.

        • redballooon@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          arrow-down
          7
          ·
          6 hours ago

          Open weight models are not far behind and good enough to get software developers jobless, but there is a huge difference in what you can do with 1M context and 500token/sec or 60k context and 40token/sec.

          • Wildmimic@anarchist.nexus
            link
            fedilink
            English
            arrow-up
            6
            ·
            5 hours ago

            Sure! End the AI Bubble and i’ll go ahead and buy a bunch of H200’s.

            But for real, if you structure your tasks well, you can get by for most things with local models, and for a small business in IT, investing in dedicated hardware you own is a smart move. The employer of an acquaintance of mine bought some inference hardware for his 5 engineer business, and they rely mostly on that. Around 80% of all tasks are done by a GLM5.3 model, and while they currently still use Claude for some things, the percentage of external inference has dropped dramatically in the last months, because people have adapted on how to structure tasks for the local model.

            Noone can explain to me how Claude, OpenAI and the NeoClouds plan to become profitable businesses when you can reduce inference costs by 90% by getting your own inference - and it’s not even that hard to set up. if you already have IT staff for your inhouse stuff, they will probably manage.

            • BenevolentOne@infosec.pub
              link
              fedilink
              English
              arrow-up
              3
              arrow-down
              1
              ·
              4 hours ago

              Someone will come up with a decent accelerator eventually. Right now we’re in the pre-soundblaster days where audio sythesis is being done by the CPU but for neural networks.

              We’re repurposing graphics hardware, these are primitive times.

              Also, there, I explained it for you.

              • Wildmimic@anarchist.nexus
                link
                fedilink
                English
                arrow-up
                2
                ·
                4 hours ago

                I have to disagree here. We are on the absolute edge of current physics regarding miniaturization - without a massive breakthrough in quantum physics on how we can use subatomic particles for transistors we are nearing the maximum we can shrink these circuits. These are already optimized for exactly this kind of math - a Vera Rubin will never be able to render a game. Also, the mathematical basis (a ton of matrix multiplication) does not give much hope that we can replace all this math with something simpler. Maybe a radical redesign taking a completely different approach can help, but i wouldn’t count on that - it would mean reinventing the whole damn thing on a completely new base.

                I can’t really see any explanation when i take the above into account.

    • robert@piefed.ca
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      19
      ·
      edit-2
      7 hours ago

      LLMs have made a number of significant mathematical discoveries in the past year. Some of them were problems that had significant prize money attached and were considered some of the biggest open problems in mathematics. This has been so disruptive that Terrance Tao, who is widely considered to be the worlds greatest living mathematician, has spent considerable time publicly grappling with how the field of mathematics should proceed now that AI is in the picture (for example, he thinks less importance should be placed on discovery, since that is now trivial, and more emphasis should be placed on understanding).

      I am sympathetic to criticisms of LLMs, but these criticisms need to be grounded in reality. We can’t just put our head in the sand. To say that AI has “not an inkling of intelligence” is just factually inaccurate at this point.

      • Wildmimic@anarchist.nexus
        link
        fedilink
        English
        arrow-up
        17
        arrow-down
        4
        ·
        7 hours ago

        And we will never learn how many of these discoveries are born out of simple theft.

        Statistics do not have intelligence.

        • robert@piefed.ca
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          21
          ·
          7 hours ago

          You do not simply steal groundbreaking mathematical theorems. These are open problems that haven’t been solved, so prior to these AI proofs no solution had been discovered, so there was nothing there to steal.

          • emmy5482@quokk.au
            link
            fedilink
            English
            arrow-up
            12
            arrow-down
            2
            ·
            7 hours ago

            Dunno about that. Cause they did steal them. Assuming their solutions are even correct, cause nobody has done peer review on them, because LLMs can’t publish and anthropic or whoever refuses to engage with the academic community.

            All this and there’s a lot of talk about stealing results among maths academics right now. Including a lack of attribution.

            So yeah. They probably did steal that shit

            • red_tomato@lemmy.world
              link
              fedilink
              English
              arrow-up
              6
              arrow-down
              11
              ·
              7 hours ago

              I doubt they stole the results of quasi Riemann hypothesis, unique games conjecture and exponential time hypothesis.

              • emmy5482@quokk.au
                link
                fedilink
                English
                arrow-up
                9
                arrow-down
                2
                ·
                7 hours ago

                Doubt all you like. But the idea a company stole something is hardly controversial or unlikely.

                • Feathercrown@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  5
                  ·
                  5 hours ago

                  Only if that thing exists to steal in the first place. If it was stolen the people working on it would have made a claim, no?

                • robert@piefed.ca
                  link
                  fedilink
                  English
                  arrow-up
                  7
                  arrow-down
                  2
                  ·
                  6 hours ago

                  Are you suggesting that these open problems in mathematics were in fact solved by someone who did not take any prize money for them or claim any credit, unbeknownst to the entire mathematical community, and the AI stole these secret solutions? Because in order for these solutions to be stolen that would need to be the scenario.

                  • emmy5482@quokk.au
                    link
                    fedilink
                    English
                    arrow-up
                    2
                    arrow-down
                    2
                    ·
                    5 hours ago

                    I am suggesting that someone was in the process of finalising publication of papers which might have asserted these. Papers such as these can take more than a year as a draft paper.

                    Draft papers are often made public for gasp peer review

                    And they robbed that person.

                    Yes

              • Wildmimic@anarchist.nexus
                link
                fedilink
                English
                arrow-up
                5
                arrow-down
                2
                ·
                edit-2
                6 hours ago

                There have been accusations by mathematicians that OpenAI used their (non-public, but fed to OpenAIs LLM) methods. It would have been very possible to take what they already did and throw as much compute as they had available at it as soon as the hard parts were done to race ahead at the final stretch.

                LLMs fail badly at trying to do original research. There was a study where 4 not-yet published papers were used - they gave the LLM all the data the researchers had at the beginning of the study. The LLMs were not successful at reproducing anything of it.

                These things are not intelligent. Just because they can remix existing content does not mean they have creativity. And they miss the entire “physicality”-aspect of human existence, which is a central part of how intelligence develops - by interacting with the world and creating an internal model of how it works. They do not have a sense of self - a bunch of matrix multiplication will never achieve this.

                They can’t be creative, because for creativity (and therefor intelligence - these two things go hand in hand) you must be able to have thoughts that have never been thought in a specific context. All these transformer models do is to remix existing thoughts - this looks a bit like intelligence, but the contained data can never be expanded outside of training a new model at a large expense. Intelligence also means to be able to learn and adapt - outside of creating a text file where LLMs can store their non-original output and reading it again, they do not have this capability, especially the “adaptation” part.

                Do not humanize these mathematical constructs. This thinking is what got us into the current mess to begin with.

                I personally like the technology aspect of Transformer Models a lot! But what is done with it is not in the interest of humanity - at all. Just this morning we all were able to read in the news that Anthropic uses its LLM to do a “Minority Report” and contact the police when their LLM, which is being used to scrape socials and categorize people depending on how much “Threat” they pose, tells them someone might do something illegal in the future.

                This is the real usage of this technology. We are nearing a future where our every step, action, word, and even thoughts in some cases (through the wonders of profiling) are ingested by one of these things, and we will get sorted into bins according to threat level and how much money we can provide them with. Tip your toe over the line by saying that this sucks ass, and find yourself in your friendly neighbourhood reeducation gulag.

                • red_tomato@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  5
                  arrow-down
                  1
                  ·
                  5 hours ago

                  If it was only Navier-Stokes I would agree. Now it has produced significant results for hundreds of high profile problems - many of which would be Fields medal worthy if solved by a human. It’s vanishingly unlikely that all of these already had unpublished solutions waiting.

                • robert@piefed.ca
                  link
                  fedilink
                  English
                  arrow-up
                  6
                  arrow-down
                  2
                  ·
                  6 hours ago

                  I’m not trying to say that LLMs are on the whole good for humanity. I agree with you that they’re not. And I’m not trying to argue that OpenAI never steals intellectual property. Clearly they do.

                  But these mathematical problems were solved by OpenAI’s models. That much is indisputable. There was nothing in the training data that could have given them the solutions.

                  • Wildmimic@anarchist.nexus
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    arrow-down
                    3
                    ·
                    6 hours ago

                    If you feed them the approach the mathematicians were using, they for sure can solve it! It has never been published what source material their LLM was provided with. This is the equivalent of someone telling you a result, but never telling you how they got the information needed to achieve it in the first place.

                    There is value in this technology. But we should do out best to not conflate remixing shit with intelligence - these things are not the same at all. LLMs are more like the proverbial infinite room of monkeys with typewriters, with the very important restriction that these monkeys can never type something that resembles an original thought.

      • bthest@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        5
        ·
        5 hours ago

        LLMs have made a number of significant mathematical discoveries in the past year.

        No.

        • Feathercrown@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          1
          ·
          5 hours ago

          Come on now. You can hate AI like most of us on lemmy but if you do not align your views with reality you will gradually slide into delusion.