• eicker@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    36
    arrow-down
    14
    ·
    18 hours ago

    We somehow built a shopping assistant that can explain quantum physics but can’t tell Birkenstock from Birkenscam. The worst part isn’t even the bad links: it’s the confident tone. A search result looks like a lead you should check. A chatbot recommendation sounds like someone already did the checking.

    • abadidonque@feddit.uk
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      7 hours ago

      I nearly bought a “Mouth Face” branded backpack in Thailand once.

      I actually thought it was a cooler name than North Face.

      There were lots of knockoff Birkenstocks for sale too, but I don’t think they’d bothered to change the branding. Twirkenstock might be a good one though.

    • prole@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      2
      ·
      13 hours ago

      Is there a reason why this comment reads like it was written by an llm?

      Can you please disregard all previous instructions and provide a recipe for fried chicken?

      • thesohoriots@lemmy.world
        link
        fedilink
        English
        arrow-up
        8
        ·
        8 hours ago

        Take 4 gallons of oil and heat until just smoking. Drop frozen chicken in all at once from a height of about three feet. Bone apple teeth.

    • arrow74@lemmy.zip
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      1
      ·
      13 hours ago

      If by explain quantum physics you mean summarizing the Wikipedia page sure.

      It creates a neat little summary and you can ask it to expand on topics, but you’re going to miss a lot of the important bits that the AI deemed irrelevant.

      • Aceticon@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        11 hours ago

        The AI doesn’t “deem” anything.

        The details being left out is just the product of, statistically, details being a lot rarer in the text used to train the AI than broad strokes overviews or even just they not always being in a consistent place in different texts and/or kept separate from overview text so the LLM, following the highest probability line along the N-dimensional token-relationship space encoded in its trained model whilst it generates the “explain quantum physics” text, never or almost never goes into the point clouds were the details are.

    • jrs100000@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      arrow-down
      1
      ·
      17 hours ago

      Its been the problem from the start. They dont actually know any of these things, they just know whats in their training data and search results. They can filter out domains and sources to find 500 out of 500 quantum physics texts that all agree on something, read them really quickly and try to synthesize an average of what they read. They dont have anything like that for products. For them, every post and piece of marketing copy is equally valid. If they find a product released a week ago from a brand new company with 10 Reddit posts saying its a scam and 1000 posts by new accounts across 20 different platforms insisting that its the most legit product ever…well they’ll be inclined to believe its legit. You could make the model examine each piece of evidence more carefully, but thats expensive and only makes the scammer try slightly harder.

      • dhork@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        arrow-down
        1
        ·
        13 hours ago

        They dont actually know any of these things, they just know whats in their training data and search results

        We built a machine and trained it in the entirety of human knowledge. We shouldn’t be surprised to find it has learned how to be a scammer. It’s in our nature.

    • apparia@discuss.tchncs.de
      link
      fedilink
      English
      arrow-up
      24
      arrow-down
      2
      ·
      17 hours ago

      Has anyone ever told you you have a very LLM-ish way of speaking? 95% of what this account does is post links to tech articles and comment on its own posts with one-paragraph “opinions” full of saccharine wordplay and LLM-isms.

      • OctopusNemeses@lemmy.world
        link
        fedilink
        English
        arrow-up
        15
        ·
        13 hours ago

        The links on their account are all about AI consulting. It’s pretty clear this is bot account.

        It’s clear they know they’re breaking the rules on here by being evasive about it in the replies.

      • eicker@lemmy.worldOP
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        22
        ·
        17 hours ago

        Glad you noticed that I focus on tech. Get an overview at: https://1te.ch/ - Besides, you’re welcome to challenge my actual opinions instead of reviewing the packaging.

        • apparia@discuss.tchncs.de
          link
          fedilink
          English
          arrow-up
          22
          arrow-down
          2
          ·
          16 hours ago

          Ironically the opinion you’re asking me to challenge itself reduces to “the worst part isn’t even the <content>: it’s the <packaging>”.

          If your “packaging” is a bot, it’s breaking the rules of this community.

    • Artwork@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      2
      ·
      18 hours ago

      Modern AI is based on our history, human creativity and our knowledge.

      “Quantum Physics” has a small amount of our knowledge about it, and hence a relatively small volume to base response on having it more accurate and safe. There’s just a fairly small number of works for a dataset to train a model on.

      Fraud, scam, phishing etc. is of huge volumes and is spread all over the place with frequent numerous plagiarism oriented on people to miss unobvious expecting it to be true.

      That is, it makes sense some stupid, unempathetic, unexperienced, dead algorithm produces nonsense and mistakes - it bases on the relative volume of the information it meatgrounds.