• brucethemoose@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    5 hours ago

    Generative ML inference isn’t so dramatically bandwidth limited anymore, either, so the “demand” for HBM isn’t as clear.

    • boonhet@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      3
      ·
      3 hours ago

      Eh the really big models want a lot of bandwidth still, otherwise you just don’t make use of all the compute power you have.