• 0 Posts
  • 61 Comments
Joined 1 year ago
cake
Cake day: June 30th, 2023

help-circle








  • I’m an AI Engineer, been doing this for a long time. I’ve seen plenty of projects that stagnate, wither and get abandoned. I agree with the top 5 in this article, but I might change the priority sequence.

    Five leading root causes of the failure of AI projects were identified

    • First, industry stakeholders often misunderstand — or miscommunicate — what problem needs to be solved using AI.
    • Second, many AI projects fail because the organization lacks the necessary data to adequately train an effective AI model.
    • Third, in some cases, AI projects fail because the organization focuses more on using the latest and greatest technology than on solving real problems for their intended users.
    • Fourth, organizations might not have adequate infrastructure to manage their data and deploy completed AI models, which increases the likelihood of project failure.
    • Finally, in some cases, AI projects fail because the technology is applied to problems that are too difficult for AI to solve.

    4 & 2 —>1. IF they even have enough data to train an effective model, most organizations have no clue how to handle the sheer variety, volume, velocity, and veracity of the big data that AI needs. It’s a specialized engineering discipline to handle that (data engineer). Let alone how to deploy and manage the infra that models need—also a specialized discipline has emerged to handle that aspect (ML engineer). Often they sit at the same desk.

    1 & 5 —> 2: stakeholders seem to want AI to be a boil-the-ocean solution. They want it to do everything and be awesome at it. What they often don’t realize is that AI can be a really awesome specialist tool, that really sucks on testing scenarios that it hasn’t been trained on. Transfer learning is a thing but that requires fine tuning and additional training. Huge models like LLMs are starting to bridge this somewhat, but at the expense of the really sharp specialization. So without a really clear understanding of what can be done with AI really well, and perhaps more importantly, what problems are a poor fit for AI solutions, of course they’ll be destined to fail.

    3 —> 3: This isn’t a problem with just AI. It’s all shiny new tech. Standard Gardner hype cycle stuff. Remember how they were saying we’d have crypto-refrigerators back in 2016?




  • Cool cool, we’re cool. I get a little triggered when I hear people say that NN/DL models are “fancy statistics”—it’s not the first time.

    In what seems like another lifetime ago, my first engineering job was as a process engineer for an refinery-scale continuous chromatography unit in hydrocarbon refining. Fuck that industry, but there’s some really cool tech there nevertheless. Anyway when I was first learning the process, the technician I was learning from called it a series of “fancy filters” and that triggered me too—adsorption is a really fascinating chemical process that uses a lot of math and physics to finely-tune for desired purity, flowrate, etc. and to diminish it as “fancy filtration”!!!

    He wasn’t wrong, you’re not either; but it’s definitely more nuanced than that. :)

    Engineers are gonna nerd out about stuff. It’s a natural law, I think.


  • AI is a very broad term that also includes expert systems (such as Computational Fluid Dynamics, Finite Element Analysis, etc approaches.). Traditional machine learning approaches (like support vector machines, etc.) too. But yes, I agree—most commonly associated with deep learning/neural network approaches.

    That said, it’s misleading and inaccurate to state that neural networks are just statistics. In fact they are substantially more than just advanced statistics. Certainly statistics is a component—but so too is probability, calculus, network/graph theory, linear algebra, not to mention computer science to program, tune, and train and infer them. Information theory (hello, entropy) plays a part sometimes.

    The amount of mathematical background it takes to really understand and practice the theory of both a forward pass and backpropagation is an entire undergraduate STEM curriculum’s worth. I usually advocate for new engineers in my org to learn it top down (by doing) and pull the theory as needed, but that’s not how I did it and I regularly see gaps in their decisions because of it.

    And to get actually good at it? One does not simply become a AI systems engineer/technologist. It’s years of tinkering with computers and operating systems, sourcing/scraping/querying/curating data, building data pipelines, cleaning data, engineering types of modeling approaches for various data types and desired outcomes against constraints (data, compute, economic, social/political), implementing POCs, finetuning models, mastering accelerated computing (aka GPUs, TPUs), distributed computation—and many others I’m sure I’m forgetting some here. The number of adjacent fields I’ve had to deeply scratch on to make any of this happen is stressful just thinking about it.

    They’re fascinating machines, and they’ve been democratized/abstracted to an extent where it’s now as simple as import torch, torch.fit, model.predict. But to be dismissive of the amazing mathematics and engineering under the hood to make them actually usable is disingenuous.

    I admit I have a bias here—I’ve spent the majority of my career building and deploying NN models.



  • Reward models (aka reinforcement learning) and preference optimization models can come to some conclusions that we humans find very strange when they learn from patterns in the data they’re trained on. Especially when those incentives and preferences are evaluated (or generated) by other models. Some of these models could very well could come to the conclusion that nuking every advanced-tech human civilization is the optimal way to improve the human species because we have such rampant racism, classism, nationalism, and every other schism that perpetuates us treating each other as enemies to be destroyed and exploited.

    Sure, we will build ethical guard rails. And we will proclaim to have human-in-the-loop decision agents, but we’re building towards autonomy and edge/corner-cases always exist in any framework you constrain a system to.

    I’m an AI Engineer working in autonomous agentic systems—these are things we (as an industry) are talking about—but to be quite frank, there are not robust solutions to this yet. There may never be. Think about raising a teenager—one that is driven strictly by logic, probabilistic optimization, and outcome incentive optimization.

    It’s a tough problem. The naive-trivial solution that’s also impossible is to simply halt and ban all AI development. Turing opened Pandora’s box before any of our time.





  • Niche subreddits were still good. Fuck the mains, but there was a lot of really good content, even very technical, in tightly focused communities like r/LocalLLaMA, etc. In a lot of ways the format of how conversations flow there work better (and worse) than stackoverflow. Still is good content, but I really can’t bring myself to go there because of the nasty shenanigans that spez put the communities through.

    I really hoped for a while that Reddit would be the one to break the embrace, extend, extend, enshittify mold that so many great techs succumb.

    But everyone has a sellout price and so the EEEE seems to be a law of nature.