Determinism means performing the same way every time it is run with the same inputs. It doesn’t mean it follows your mental model of how it should run. The article you cite talks about aggressive compiler optimizing causing unexpected crashes. Unexpected, not unpredictable. The author found the root cause and addressed it. Nothing there was nondeterministic. It was just not what the developer expected, or personally thought was an appropriate implementation, but it performed the same way every time. I think you keyed on the word “randomly” and missed “seemed to,” which completely changes the meaning of the sentence.
LLMs often act truly nondeterministically. You can create a fresh session and feed it exactly the same prompt and it will produce a different output. This unpredictability is poison for producing a quality product that is maintainable with dynamic LLM code generation in the pipeline.
I have talked with the author to confirm what he meant with this and other posts he made on compilation. He has confirmed that most (if not all) C compilers are not deterministic. He has pointed me to here as an example. He added that optimizations are not applied in deterministic order and when you add LTO it worsens the problem.
Determinism means performing the same way every time it is run with the same inputs. It doesn’t mean it follows your mental model of how it should run. The article you cite talks about aggressive compiler optimizing causing unexpected crashes. Unexpected, not unpredictable. The author found the root cause and addressed it. Nothing there was nondeterministic. It was just not what the developer expected, or personally thought was an appropriate implementation, but it performed the same way every time. I think you keyed on the word “randomly” and missed “seemed to,” which completely changes the meaning of the sentence.
LLMs often act truly nondeterministically. You can create a fresh session and feed it exactly the same prompt and it will produce a different output. This unpredictability is poison for producing a quality product that is maintainable with dynamic LLM code generation in the pipeline.
I have talked with the author to confirm what he meant with this and other posts he made on compilation. He has confirmed that most (if not all) C compilers are not deterministic. He has pointed me to here as an example. He added that optimizations are not applied in deterministic order and when you add LTO it worsens the problem.