• 1 Post
  • 400 Comments
Joined 2 years ago
cake
Cake day: November 14th, 2023

help-circle





  • That is not really true. Yes, there are jump instructions being executed when you run interference on a model, but they are in no way related to the model itself.

    The model is data. It needs to be operated on to get information out. That means lots of JMPs.

    If someone said viewing a gif is just a bunch of if-else’s, that’s also true. That the data in the gif isn’t itself a bunch of if-else’s isn’t relevant.

    Executing LLM’S is particularly JMP heavy. It’s why you need massive fast ram because caching doesn’t help them.










  • I got a sff P330 Xeon with integrated graphics for ~$500 two years ago that includes case power supply etc. Far faster than an n100 and even lower power than if you added a GPU to an n100.

    I just plugged in a kilowatt to check:

    My Lenovo sff workstation running Plex idles at 15 watts- which is 90% of the time. Streaming 4k 52Mbs hevc (This Flash Gordon is my torture test that caused me to upgrade 2 years ago) it’s 18 watts! I was so surprised that I went back and unplugged the Ethernet thinking I put the killawatt on the wrong server.