• 1 Post
  • 169 Comments
Joined 3 years ago
cake
Cake day: July 4th, 2023

help-circle
  • Its a bit of both.

    Absolutely, for sure, a decent amount of telemetry is for simply making decisions about what people actually use.

    “Should we improve (thing), or drop it and stop supporting it?”

    Well, lets just track how many people actually use it first for a bit and then decide.

    Youd be surprised how often users beg for features and then stop using them after 1 week lol.

    But sometimes a random feature you thought no one uses much turns out to be actually quite popular.

    This same goes for optimizing. Your highest traffic parts of your website are there you wanna focus the most on stuff being optimized to save money and improve user experience.

    Do a tonne of companies track stuff just to sell it as data for training AI?

    Yeah, they do. And its gross.

    But there is a huge amount of telemetry thats just developers wanting to genuinely improve the user experience, catch bugs, etc.



  • Theres not really any fooling here. Theres tonnes of interesting examples you can find.

    Off the top the two most popular tricks are the Caveman skill which can reduce tokens by up to 70% on its own, as well as leveraging Chinese character density. Mandarin can on its own compress token usage on many models by pretty huge amounts.

    Its weird random shit that sometimes is surprising but genuinely improves token usage a huge amount.

    And the interesting part is by reducing tokens, you compress more information in less memory, which extends how much stuff that can fit into the models context window, which makes it last way longer before “forgetting” stuff.

    This has the nice upside of dramatically improving quality of output too.

    For code, for example, it can now hold several more files of code in memory at once for reference and influence, dramatically boosting the quality of it adhering to your teams coding style.

    Thats just one example you learn on how to make the tool less stupid.

    Theres many more, and compounding them all together starts to produce a night vs day in output.

    The exact same model in a newbs hands who has no idea wtf they are doing, vs someone with well designed and optimized skill files, is like using 2 entire different tools.

    Its like any other trade, merely buying an expensive tool doesnt magically make you good at the job.

    Knowing how to use the tool is way more important


  • Its not complicated. People have become extremely insulated away from what the real work looks like of a dev over the years.

    The reality os starkly contrasted to public perception.

    Most software developers heavily use LLMs now. They sucked 5 years ago, we’re meh 3 years ago, decent 2 years ago, but over the past year and a bit have rapidly become genuinely more efficient when used right and skillfully than doing about 90% of your work load by hand.

    Bits and pieces still require doing it by hand, but the vast majority of work for the average dev now is via moderating an LLM (with skill) to success.

    Unfortunately a fuck tonne of devs lack that “with skill” part still, and what this comes out as is them costing their companies tremendously more money to do the same job.

    A loooot of companies (stupidly) hedged their bets that if they just gave their devs wild west access to using LLMs without training they’d magically just “figure it out” along the way.

    Which is nonsense, why would a dev feel compelled to conserve tokens or improve efficiency with zero incentive?

    So now companies are scrambling as they realize their devs, who just spent 12 months going hog wild with LLMs, still havent learned how to use them well and in dact have developed arguably worse poor habits that they now need to unlearn

    Thats where the industry is at now largely.

    Meanwhile companies like the one I work at predicted this as a natural thing and we’re preparing for it long in advance. When token prices shot up we already had set ourselves up with lots of training so the price increase was not nearly as noticeable.

    I think when Im fully optimized out on a project I only spend about $10~$15 a day, despite going full steam for 5 hrs or so.

    And despite that my productivity is probably higher than unskilled devs who burn through 10x~20x that. I get more work down in way less time and way less tokens.

    Training and the resource/knowledge pool go a long way here. It cannot be understated


  • If my boss comes to tell me that from now on my “productivity” will be measured in token usage rather than actual “production”

    Did your boss actually say this, or are you just going off of some memes you saw and think thats something happening often.

    Most companies care about getting work done in as few tokens as possible now, developers that can achieve the same results but with less tokens are extremely valuable.

    Not only that, but less tokens also inherently means faster.

    Any company that is blowing through tokens, without any effort put into training employees how to use the tools better, deserve to fail.


  • Start by learning all the critical things like Skills, MCP, Agents, etc.

    Then look up skills and MCP tools that reduce token usage, improve recall, improve searching, improve parsing, etc

    Then learn how to use sub agents bound to cheaper models for more expensive operations (the largest of which us always search and find ops)

    Swapping to a cheaper model for a subagent with the job to just go find a specific thing alone can reduce costs like 30%, equipping it with tools that can search and find faster can push that to 70%


  • No, I work in the industry and am vety actively entwined eith systems where we contract out to train and show companies how to use LLMs better.

    And a lot of our clients now are of the “how do we use less tokens” variety, and you walk into the project and see the way they currently operate and go “oh god”

    The average developers have absolutely zero clue wtf they are doing, they’ll burn a million tokens on something that outta take 10k.

    We often can get token usage down easily 90%+ in the first month just by on boarding and offering some basic training and helping install some basic guard rails, skills, etc.






  • by the types of people you maybe don’t want using your code anyways

    …companies? Sure I guess, if you want to angle your career trajectory towards “unemployable” by all means lol.

    Personally anyone doing this I’m going to be more likely to use their code

    I am a tech lead, if any dev under me intentionally added/used a tool to our systems because it had malicious undocumented behaviors of any kind, they would be fired immediately and any company that contacted us for reference would be informed of their behavior.

    To be clear, this is the scenario of

    Me: hey I saw you installed [tool], that thing is flagged by our systems for the maintainers having done malicious undocumented stuff in the past

    Dev: haha yeah thats why I used it

    Me: you are joking right?

    Thatd be an instant high level escalation to “strip this person of privs and get them off our system asap, and HR now has to be involved”

    You dont fuckin do shit like that in a real company if you wanna stay employed lol.


  • Most open source maintainers never “license [any] stuff you maintain for big bucks” that is often hard to do and/or goes against the philosophy of open source entirely.

    Uhhh… no this is actually very common. Usually with scaling licenses, “free for use if your company is below [threshold]”, its super common…

    And I don’t even think this is malicious behaviour as it just nukes the code of this package and nothing else if you are not being careful yourself…

    Are you even reading what you just wrote lol.

    Being “sorta” malicious is still malicious. And companies usually have zero tolerance for that shit.

    If you don’t do version control you are not a good programmer, imo

    You really underestimate how much damage this could do then, lol…


  • They only documented it after all the outcry, which is way too late.

    Documenting it post release still counts as having released undocumented behavior.

    And if its malicious (which this 100% is), then it doesn’t fuckin matter anyways lol. You now are treated akin to a trojan maintainer by companies. You’ll get flagged as “don’t ever use anything by this person”

    Super great way to get yourself flagged and lose any opportunity in the future for possibly licensing stuff you maintain for big bucks. What company would risk paying money to someone who does childish stuff like that lol


  • How to get yourself blacklisted by large sweeps of the FOSS community:

    Step 1: Include any kind of undocumented subversive behaviour in your thing.

    That’s it, doesn’t matter what the intent is, simply by demonstrating you are willing to include anything that is remotely subversive without being open about it is usually enough to get blacklisted by a lot of people, because if you did it once… who’s to say you won’t do it again, but possibly worse next time?

    People are extremely coldly receptive to anytime a FOSS dev throws a sudden undisclosed anything in their tool, let alone one that is actively malicious.

    If I’m gonna depend on work life on anything FOSS, I ain’t touching anything like that, regardless of intent, with a 200 foot pole lol.

    All it takes is one button click to get notified:




  • pixxelkick@lemmy.worldtolinuxmemes@lemmy.worldchange my mind
    link
    fedilink
    arrow-up
    37
    arrow-down
    1
    ·
    edit-2
    2 months ago

    In the latest episode we get to see the outright answer for this for Cain / Kinger

    As you can see, the circus (managed by Cain the AI) uses an oldschool Windows OS, looks to be 98?

    Kinger then uses CMD to ssh into some bash shell, I dunno if we get much view of what kind of OS it is, but it appears to be possibly a systemd flavor?

    We also get to see that the circus was programmed in Lisp in the latest 90s, which sounds about right

    Based on what we know about the laws of how the circus works, these indicate a manifestation of how these 2 characters mentally work with OSes.

    Cain works in windows 98.

    Kinger on the other hand visualizes stuff with linux!