A new critical vulnerability discovered by security research firm LayerX has exposed a fundamental architectural flaw in how Large Language Models (LLMs) handle trust boundaries.
  • chisel@piefed.socialEnglish
    25·
    7 hours ago

    Bad, sure, but also a well-known fundamental issue with basically all agentic LLMs, to the point where sounding the alarm as much as this seems silly. If you give an LLM any access to your machine, it might be tricked into taking malicious actions. The same can be said to real humans. Though they are better than LLMs right now, humans are the #1 security vulnerability in any system.

    LLMs can get around this by requiring explicit permission before being able to take certain actions, but the more it asks for permission, the more the user has to just sit there and approve stuff, defeating the purpose of an autonomous agent in the first place.

    To be clear, it’s not good, and anyone running an agent with a high level of access with no oversight in a non-sandboxed environment today is a fool. But this article is written like they found an 11/10 severity CVE in the linux kernal when really this is a well-known fundamental thing that can happen if you’re an idiot and misusing LLMs to a wild degree. LLMs have plenty of legitimate issues to trash on. This isn’t one of them. Well, maybe a little.

    • XLE@piefed.socialEnglish
      151·
      7 hours ago

      According to the article, the lack of sandboxing is intentional on Anthropic’s part. Anthropic also fails to realistically communicate how to use their product.

      This is Anthropic’s fault.

      • chisel@piefed.socialEnglish
        3·
        4 hours ago

        Oh, for sure, the marketing is terrible and makes this into a bigger issue by making people over confident. I wouldn’t say the lack of sandboxing is a major problem on its own, though. If you want an automated agent that does everything, it’s going to need permissions to do everything. Though they should absolutely have configurable guardrails that are restrictive by default. I doubt they bothered with that.

        The idea is sound, but the tech isn’t there yet. The real problem is that the marketing pretends that LLMs are ready for this. Maybe Anthropic shouldn’t have released it at all, but at this point AI companies subsist on releasing half-baked products with thrice-baked promises so at this point I wouldn’t be surprised if OpenAI, in an attempt to remain relevant, tomorrow releases an automated identity theft bot to help you file your taxes incorrectly.