• 0 Posts
  • 428 Comments
Joined 2 years ago
cake
Cake day: August 27th, 2023

help-circle



  • Right, should say deep neural networks. Perceptrons hit a brick wall because there’s some problems they cannot handle. Multi-layer networks stalled because nobody went ‘what if we just pretend there’s a gradient?’ until twenty-goddamn-twelve.

    Broad applications will emerge and succeed. LLMs kinda-sorta-almost work for nearly anything. What current grifters have proven is that billions of dollars won’t overcome fundamental problems in network design. “What’s the next word?” is simply the wrong question, for a combination chatbot / editor / search engine / code generator / puzzle solver / chess engine / air fryer. But it’s obviously possible for one program to do all those things. (Assuming you place your frozen shrimp directly atop the video card.) Developing that program will closely resemble efforts to uplift LLMs. We’re just never gonna get there from LLMs specifically.



  • Neural networks will inevitably be a big deal for a wide variety of industries.

    LLMs are the wrong approach to basically all of them.

    There’s five decades of what-ifs, waiting to be defictionalized, now that we can actually do neural networks. Training them became practical, and ‘just train more’ was proven effective. Immense scale is useful but not necessary.

    But all hype has been forced into spicy autocomplete and a denoiser, and only the denoiser is doing the witchcraft people want.










  • Which does cause problems now that Google search is shit.

    Every time ‘new tool makes old skills rusty’ is treated as novel, I’m reminded of The Gentleman’s Magazine:

    Instead of simply reproducing the operations of man’s intelligence, the arithmometer relieves that intelligence from the necessity of making the operations. Instead of repeating responses dictated to it, this instrument instantaneously dictates the proper answer to the man who asks it a question. It is not matter producing material effects, but matter which thinks, reflects, reasons, calculates, and executes all the most difficult and complicated arithmetical operations with a rapidity and infallibility which defies all the calculators in the world. The arithmometer is, moreover, a simple instrument, of very little volume and easily portable. It is already used in many great financial establishments, where considerable economy is realized by its employment.

    It will soon be considered as indispensable, and be as generally used as a clock, which was formerly only to be seen in palaces, and is now in every cottage.

    This was a crank-powered adding machine. Numbers used levers instead of buttons because buttons hadn’t been invented yet. There were already people who expected it the next version would do everything for us - and people who thought that would be bad, somehow.






  • Oh hey, found an old comment:

    I admire the concept behind Denuvo.

    Programs bounce around between a ton of different code segments, and it doesn’t really matter how they’re arranged within the binary. Some code even winds up repeated, when repetition is more efficient than jumping back and forth or checking a short loop. It doesn’t matter where the instructions are, so long as they do the right thing.

    The machine code still tends to be clean, tight, and friendly toward reverse-engineering… relatively speaking. Anything more complex than addition is an inscrutable mess to people who aren’t warped by years of computer science, but it’s just a puzzle with a known answer, and there’s decades of tools for picking things apart and putting them back together. Scene groups don’t even need to unravel the whole program. They’re only looking for tricky details that will detect pirates and frustrate hackers. Eventually, they will find and defeat those checks.

    So Denuvo does everything a hundred times over. Or a dozen. Or a thousand. Random chunks of code are decompiled, recompiled, transpiled, left incomplete, faked entirely, whatever. The whole thing is turned into a hot mess by a program that knows what each piece is supposed to be doing, and generally makes sure that’s what happens. The CPU takes a squiggly scribbled path hither and yon but does all the right things in the right order. And sprinkled throughout this eight-ton haystack are so many more needles, any of which might do slightly different things. The “attack surface” against pirates becomes enormous. They’ll still get through, eventually, but a crack delayed is a crack denied.

    Unfortunately for us this also fucks up why computers are fast now.

    Back in the single-digit-megahertz era, this would’ve made no difference to anything, besides requiring more RAM for these bloated executables. 8- and 16-bit processors just go where they’re told and encounter each instruction by complete surprise. Intel won the 32-bit era by cranking up clock speeds, which quickly outpaced RAM response times, leading to hideously clever cache-memory use, inside the CPU itself. Cache layers nowadays are a major part of CPU cost and an even larger part of CPU performance. Data that’s read early and kept nearby can make an instruction take one cycle instead of one thousand.

    Sending the program-counter on a wild goose chase across hundreds of megabytes guarantees you’re gonna hit those thousand-cycle instructions. The next instruction being X=N+1 might take literally no time, if it happens near a non-math instruction, and the pipeline has room for it. But if you have to jump to that instruction and back, it’ll take ages. Maybe an entire microsecond! And if it never comes back - if it jumps to another copy of the whole function, and from there to parts unknown - those microseconds can become milliseconds. A few dozen of those in the wrong place and your water-cooled demigod of a PC will stutter like Porky Pig. That’s why Denuvo in practice just plain suuucks. It is a cache defeat algorithm. At its pleasure, and without remedy, it will give paying customers a glimpse of the timeline where Motorola 68000s conquered the world. Hit a branch and watch those eight cores starve.