codeinabox@programming.dev to Programming@programming.devEnglish · 2 days agoTurn off Cursor, turn on your mindallvpv.orgexternal-linkmessage-square19fedilinkarrow-up169arrow-down18
arrow-up161arrow-down1external-linkTurn off Cursor, turn on your mindallvpv.orgcodeinabox@programming.dev to Programming@programming.devEnglish · 2 days agomessage-square19fedilink
minus-squarethinkercharmercoderfarmer@slrpnk.netlinkfedilinkarrow-up1·14 hours agoYou sent me down a bit of a rabbit hole, but it turned up an interesting answer. Turns out they are nondeterministic, and why they aren’t deterministic is still an open question https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
minus-squareMichal@programming.devlinkfedilinkarrow-up1·12 hours agoInteresting, I had assumed that turning down temperature to 0, or hardcoding a seed would make LLM inference deterministic. Especial after watching this video https://youtu.be/J9ZKxsPpRFk
minus-squarethinkercharmercoderfarmer@slrpnk.netlinkfedilinkarrow-up1·11 hours agoI had thought I had seen both as well.
minus-squarethinkercharmercoderfarmer@slrpnk.netlinkfedilinkarrow-up1·14 hours agoSkipped over the opening graphic on first read but just read it. Could they have picked a creepier sample sentence.
You sent me down a bit of a rabbit hole, but it turned up an interesting answer. Turns out they are nondeterministic, and why they aren’t deterministic is still an open question https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
Interesting, I had assumed that turning down temperature to 0, or hardcoding a seed would make LLM inference deterministic.
Especial after watching this video https://youtu.be/J9ZKxsPpRFk
I had thought I had seen both as well.
Skipped over the opening graphic on first read but just read it. Could they have picked a creepier sample sentence.