I’ve used spicy auto-complete, as well as agents running in my IDE, in my CLI, or on GitHub’s server-side. I’ve been experimenting enough with LLM/AI-driven programming to have an opinion on it. And it kind of sucks.
A pet project… A web novel publishing platform. It’s very fancy: Uses yjs (CRDTs) for collaborative editing, GSAP for special effects (that authors can use in their novels), and it’s built on Vue 3 (with Vueuse and PrimeVue) and Python 3.13 on the backend using FastAPI.
The editor TipTap with a handful of custom extensions that the AI helped me write. I used AI for two reasons: I don’t know TipTap all that well and I really want to see what AI code assist tools are capable of.
I’ve evaluated Claud Code (Sonnet 4.5), gpt5, gpt5-codex, gpt5-mini, Gemini 2.5 (it’s such shit; don’t even bother), qwen3-coder:480b, glm-4.6, gpt-oss:120b, and gpt-oss:20b (running locally on my 4060 Ti 16GB). My findings thus far:
Claude Code: Fantastic and fast. It makes mistakes but it can correct its own mistakes really fast if you tell it that it made a mistake. When it cleans up after itself like that it does a pretty good job too.
gpt5-codex (medium) is OK. Marginally better than gpt5 when it comes to frontend stuff (vite + Typescript + oh-god-what-else-now haha). All the gpt5 (including mini) are fantastic with Python. All the gpt5 models just love to hallucinate and randomly delete huge swaths of code for no f’ing reason. It’ll randomly change your variables around too so you really have to keep an eye on it. It’s hard to describe the types of abominations it’ll create if you let it but here’s an example: In a bash script I had something like SOMEVAR="$BASE_PATH/etc/somepath/somefile" and it changed it to SOMEVAR="/etc/somepath/somefile"for no fucking reason. That change had nothing at all to do with the prompt! So when I say, “You have to be careful” I mean it!
gpt-oss:120b (running via Ollama cloud): Absolutely fantastic. So fast! Also, I haven’t found it to make random hallucinations/total bullshit changes the way gpt5 does.
gpt-oss:20b: Surprisingly good! Also, faster than you’d think it’d be—even when giving it a huge refactor. This model has lead me to believe that the future of AI-assisted coding is local. It’s like 90% of the way there. A few generations of PC hardware/GPUs and we won’t need the cloud anymore.
glm-4.6 and qwen3-coder:480b-cloud: About the same as gpt5-mini. Not as fast as gpt-oss:120b so why bother? They’re all about the same (for my use cases).
For reference, ALL the models are great with Python. For whatever reason, that language is king when it comes to AI code assist.
A pet project… A web novel publishing platform. It’s very fancy: Uses yjs (CRDTs) for collaborative editing, GSAP for special effects (that authors can use in their novels), and it’s built on Vue 3 (with Vueuse and PrimeVue) and Python 3.13 on the backend using FastAPI.
The editor TipTap with a handful of custom extensions that the AI helped me write. I used AI for two reasons: I don’t know TipTap all that well and I really want to see what AI code assist tools are capable of.
I’ve evaluated Claud Code (Sonnet 4.5), gpt5, gpt5-codex, gpt5-mini, Gemini 2.5 (it’s such shit; don’t even bother), qwen3-coder:480b, glm-4.6, gpt-oss:120b, and gpt-oss:20b (running locally on my 4060 Ti 16GB). My findings thus far:
SOMEVAR="$BASE_PATH/etc/somepath/somefile"and it changed it toSOMEVAR="/etc/somepath/somefile"for no fucking reason. That change had nothing at all to do with the prompt! So when I say, “You have to be careful” I mean it!For reference, ALL the models are great with Python. For whatever reason, that language is king when it comes to AI code assist.