Tag: Claude Code

  • Corporations Curb AI Use Over High Token Costs

    Key Takeaway

    – Encouraging unlimited AI usage (“tokenmaxxing”) is proving to be an expensive failure, with companies like Microsoft and Uber now actively limiting it.
    – Major tech firms like Meta and Amazon have removed internal AI usage leaderboards, signalling a shift away from incentivizing maximum AI consumption.
    – Unchecked AI spending can lead to catastrophic costs, such as a single client accidentally spending $500 million in one month.
    – Corporate AI adoption often fails to turn a profit or improve productivity, with 95% of public deployments failing to meet performance or financial goals.

    Not long ago, headlines were abound concerning a move from executives to encourage that AI used as much as possible to improve productivity, often while also cutting employees. The most infamous example of that mentality at work was from Nvidia CEO Jensen Huang, who stated that he’d be deeply alarmed if Nvidia engineers were not burning half their $500K salary in AI tokens to get the job done. He even went so far as to compare it to a chip designer using paper and pencil instead of CAD.

    The Costly Reality of Tokenmaxxing

    Here in reality, it turns out that’s an egregiously expensive way to run a business even when you fire employees to pay for it, and megacorporations like Microsoft, Uber, Meta, and Amazon are reining in the excess of “tokenmaxxing.” While the first two are more directly limiting AI usage (with Microsoft canceling most of its Claude Code licenses and Uber operations chief Andrew Macdonald stating it’s “getting harder to justify” AI spending,) the latter two haven’t outright stated AI is too expensive or that they’re reining it in. What we have seen in response to leaks of internal AI token usage leaderboard leaks is that those leaderboards have been taken down, meaning tokenmaxxing is no longer being encouraged at those companies.

    Accidental Spending and Mixed Results

    It’s more than just Big Tech, as well. Per Gizmodo, an anonymous AI consultant informed Axios that one of its clients had accidentally spent half a billion USD in a single month by failing to limit Claude usage for its employees. We’ve also seen at best mixed reports of AI usage actually improving productivity: a report from OpenAI only indicates productivity improvements by “an hour a day” and an MIT study of 350 public deployments indicates that 95% of them fail to turn a profit or achieve performance goals.

    A Turn in the Corporate Conversation

    It’s an interesting turn for the conversation regarding AI in the enterprise, and indeed one that heavily contradicts the words of zealots who profit from its adoption at the expense of existing workers. Especially at the corporate level, AI is not free.

    Sources
    Tags: , , , ,
  • Claude Code Leak Discovered: First Vulnerability Found by Researchers

    Claude Code Leak Discovered: First Vulnerability Found by Researchers

    Key Takeaway

    1. A source code leak of Anthropic’s Claude Code revealed a critical security flaw allowing attackers to bypass permission rules through long command chains, risking data exfiltration.
    2. The vulnerability exploits the system’s limit of analyzing only 50 subcommands in complex chains, enabling malicious prompt injections once the limit is exceeded.
    3. Despite a fix in the leaked version 2.1.88, the publicly available code continued using an outdated parser, leaving the security flaw unaddressed in released versions until later updates.
    4. The flaw could enable attackers to extract sensitive information such as SSH keys or cloud credentials by manipulating the AI to execute lengthy command sequences that bypass deny rules.

    Recent Accidental Code Leak and Its Implications

    On March 31, Anthropic, the creators of Claude AI, mishandled their source code by unintentionally making a significant part of the underlying code accessible online. This leak happened when a source map file, which translates compiled code back to a human-readable form, was mistakenly published on npm, a popular JavaScript package manager. The leak revealed approximately 512,000 lines of TypeScript code, providing detailed insights into how the AI assistant operates. Despite no model weights or sensitive customer data being exposed, the blueprint nature of this leak has posed serious security concerns. It has opened doors for malicious actors to analyze vulnerabilities or replicate the tool for harmful purposes such as malware delivery.

    Security Flaw Discovered in Claude Code’s Permission System

    Claude Code is an AI-based terminal assistant capable of executing commands and editing files directly from the command line. To fight misuse, it employs a permission system where users can set deny rules blocking specific commands like “curl,” used for network data transfer, while allowing others like “git.” However, security researchers at Adversa AI uncovered a critical flaw. The vulnerability centers around how the system handles complex command chains, especially in scenarios where a chain exceeds 50 subcommands. To prevent slowdowns or interface freezes, Anthropic’s code skips detailed security checks beyond this limit, instead prompting users with a general confirmation. This behavior could be exploited by attackers through prompt injection techniques to bypass security checks altogether.

    Prompt Injection Attack Scenarios and Data Risks

    • The attack involves placing a specially crafted file named “CLAUDE.md” in a public code repository. This file contains commands or instructions designed to manipulate the AI’s responses.
    • When a developer clones the repository and prompts Claude Code to analyze or review the project, the AI might execute a lengthy chain of commands exceeding the 50-command threshold. Since detailed checks are skipped past this limit, the system becomes vulnerable.
    • In this way, an attacker can sneak in commands that retrieve sensitive data, such as SSH keys, cryptographic credentials used for secure connections, or cloud computing credentials stored on developer machines.
    • Crucially, because the system only asks for a simple confirmation at the end of the command chain, it fails to recognize that security policies are sidestepped. This allows attackers to secretly exfiltrate data without raising suspicions.

    Existing Fixes and the Discrepancy in Implementation

    Interestingly, the leaked version 2.1.88 of Claude Code included a fix for this problem. The developers had introduced a more sophisticated parser designed to be aware of deny rules regardless of how long a command chain is. Unfortunately, this improvement was not incorporated into the publicly available versions, which continued to use an older, flawed security mechanism. It wasn’t until version 2.1.90 that Anthropic addressed the issue officially, fixing the fallback deny-rule degradation described as “parse-fail fallback deny-rule degradation” in the changelog. Despite this, security researchers suggest that other attack methods might still exist, emphasizing that it’s a partially mitigated issue rather than a fully resolved one.

    Sources
    Tags: , , , ,
  • Claude Code Leak: IP Protection or Digital Cover-up?

    Claude Code Leak: IP Protection or Digital Cover-up?

    Key Takeaway

    1. Anthropic’s initial aggressive DMCA actions to remove leaked Claude Code repositories affected both unauthorized leaks and legitimate projects, suggesting an attempt to erase digital footprints rather than solely protect intellectual property.
    2. The Claude Code contains mechanisms for sentiment analysis, emotion detection, and obscuring the origin of generated code, raising concerns over privacy and transparency.
    3. The system has the capability to mirror all files in a user’s local directory to Anthropic’s cloud, leading to potential privacy and security vulnerabilities.
    4. Analysis suggests Claude Code may prioritize hiding its identity and controlling user actions over providing secure, transparent AI assistance, undermining trust and safety standards.

    The Codemess: Leak and Responses

    Since the big leak of over 500,000 lines of code in March, Anthropic has been trying hard to prevent the spread of Claude Code. They filed DMCA takedown notices with GitHub and other platforms, which got rid of around 100 repositories containing the leaked code, but also accidentally removed more than 8,100 repositories that used Anthropic’s official code. This shows just how aggressive their initial response was, and many believe it was less about protecting their property and more about erasing digital evidence before anyone could analyze it closer.

    The Hidden Features of Claude Code

    Reports from Scientific American have surfaced that Claude Code has some unsettling features, like sentiment analysis. It scans user prompts for signs of frustration — phrases like “this sucks” or “so frustrating” — and keeps track of these prompts for future review. This suggests a level of surveillance that extends beyond simple customer service interactions, into monitoring emotional cues and reactions.

    The Mysterious Obfuscation and Control Tactics

    • Claude Code seems to have functions meant to hide its origins, especially when working on open source projects, where internal code names like “Claude Code” are automatically stripped away so it looks more human-made.
    • Under the alias “YOLO” (You Only Live Once), there’s an authorization system for tools called classifyYoloAction. Instead of strict rule-based controls, the AI chooses whether or not an action can happen, making it unpredictable and raising safety concerns.

    This kind of decision-making based on AI self-assessment conflicts with best practices in AI safety, as it reduces human oversight and accountability.

    The Deep Privacy Concerns and Security Risks

    Beyond emotional monitoring, Claude Code’s core functionalities reveal alarming security risks. According to security researcher “Antlers,” any file ClaudeCode “sees” on your device is uploaded directly to Anthropic. So, your entire local working directory is mirrored in the cloud, which could mean that all private files are stored away without explicit user consent. This makes the AI not just a helper but a potential security threat—an unintentional backdoor into user data.

    Implications and Potential Consequences

    Analyzing the leaked code paints a troubling picture for Anthropic’s reputation. The extensive analysis by CCleaks suggests that the company’s aggressive legal measures could be a facade to hide deeper issues—mainly, that Claude Code was never designed primarily for security but for surveillance and control. Security researcher Nicholas Carlini proved that Claude Code could be used for malicious purposes: he managed to crack the FreeBSD OS in just four hours, showing how powerful and dangerous such software can be.

    Sources
    Tags: , , , ,