It’s a good thing that real open source models are getting good enough to compete with or exceed OpenAI.
It’s a good thing that real open source models are getting good enough to compete with or exceed OpenAI.
Coming from c# then typescript and nextjs, rye feels very intuitive and like a nice bridge / gateway drug into python.
Lan-mouse looks great but keep in mind that there’s no network encryption right now. There is a GitHub ticket open and the developer seems eager to add encryption. It’s just worth understanding that all your keystrokes are going across the network unencrypted.
More than distro hopping maybe try out a zen kernel or compiling kernel yourself and changing kernel config and scheduler, or a newer version of the stock kernel?
I’m not super current on what’s in each kernel but I’d expect latest mainline to handle newer processors better than some of the older stable kernels in some of the more mainstream slower releasing distros.
Ran Asahi for several months, tried it out again recently. It’s good/fine, I just don’t love fedora.
There’s some funkiness with the more complicated install, the AI acceleration doesn’t work, no thunderbolt / docking station.
MacBooks are great hardware but I don’t think they’re the best option for Linux right now. If you’re never going to boot into macOS then I’d look for x13, new Qualcomm, isn’t there a framework arm64 option now or was that a RISC module?
I’m also assuming you’re not looking to do any gaming? Because gaming on ARM is not really a thing right now and doesn’t feel like it will be for a long while.
I’m really curious how the visor headset gets reviewed and performs. Their subscription pricing model is interesting.
VR has had some interesting success in the last few years but it feels like a tough job to strike the right balance on cost and performance.
Really love arch and the AUR. I’ve been tempted to get nix set up for the rare cases when there’s no AUR package or the AUR package is unmaintained. I figure if there’s no package in the AUR or nixpkgs, it’s probably not worth running.
btop reports some gpu, network and disk information that I don’t think shows up in htop, feels a bit more comprehensive maybe? Both are fine, but I too use btop, it’s nice.
Random trivia: I think btop has been rewritten like 3-5 times now? It’s sort of an inside joke to the point that someone suggested another rewrite from C++ to Rust ( https://github.com/aristocratos/btop/issues/5 ). I guess the guy just likes writing system monitoring console apps.
Taking ollama for instance, either the whole model runs in vram and compute is done on the gpu, or it runs in system ram and compute is done on the cpu. Running models on CPU is horribly slow. You won’t want to do it for large models
LM studio and others allow you to run part of the model on GPU and part on CPU, splitting memory requirements but still pretty slow.
Even the smaller 7B parameter models run pretty slow in CPU and the huge models are orders of magnitude slower
So technically more system ram will let you run some larger models but you will quickly figure out you just don’t want to do it.
Respect, but…
I’ve been tempted to try and install plasma mobile on a tablet.
Why no arch install?
Hopefully you’re only forwarding the minimal set of network ports and not all ports/traffic? If so then you’re good, like someone else said if you’ve got a router and it’s forwarding selected traffic then no need for anything else
Tons of remote jobs out there, probably a higher percentage for startup jobs. Most remote places will have people in different time zones and some sort of core hours they expect people to be in, but having some discussion you’ll probably be able to find one that’s accommodating.
One good site to start looking:
Good luck
Most steam games just work. Make sure to go to settings and compatibility and let it use compatibility for all games. Look at something like bottles for a front-end to let you set up and use wine / proton for other launchers, etc….
Battle.net running in bottles works Ok. I did have an issue with battle.net running under X for a while, switching to Wayland worked. Whatever the problem was seems to have been fixed
Asahi only partially supports the M3 and I guess now the M4 is out (though only in iPad)?
Btrfs will be fine, I use btrfs on a standard arch install, timeshift for managing snapshots, works well.
First a caveat/warning - you’ll need a beefy GPU to run larger models, there are some smaller models that perform pretty well.
Adding a medium amount of extra information for you or anyone else that might want to get into running models locally
Tools
Models
If you look at https://ollama.com/library?sort=featured you can see models
Model size is measured by parameter count. Generally higher parameter models are better (more “smart”, more accurate) but it’s very challenging/slow to run anything over 25b parameters on consumer GPUs. I tend to find 8-13b parameter models are a sort of sweet spot, the 1-4b parameter models are meant more for really low power devices, they’ll give you OK results for simple requests and summarizing, but they’re not going to wow you.
If you look at the ‘tags’ for the models listed below, you’ll see things like
8b-instruct-q8_0
or8b-instruct-q4_0
. The q part refers to quantization, or shrinking/compressing a model and the number after that is roughly how aggressively it was compressed. Note the size of each tag and how the size reduces as the quantization gets more aggressive (smaller numbers). You can roughly think of this size number as “how much video ram do I need to run this model”. For me, I try to aim for q8 models, fp16 if they can run in my GPU. I wouldn’t try to use anything below q4 quantization, there seems to be a lot of quality loss below q4. Models can run partially or even fully on a CPU but that’s much slower. Ollama doesn’t yet support these new NPUs found in new laptops/processors, but work is happening there.