Crafting AI Prompts Framework

IPE Framework Techniques Adversarial Prompts
Back to Adversarial Prompting Overview
INJ Prompt Injection

Package Injection

RISK: HIGH IMPACT: HIGH

When AI coding assistants hallucinate non-existent software libraries, hackers exploit the mistake. This article explains 'Package Hallucination' attacks, where adversaries register fake packages invented by LLMs to trick developers into installing malware.

AI Package Injection (Hallucination Attacks)

Developers increasingly rely on AI coding assistants (like Copilot, ChatGPT, or Cursor) to write boilerplate code and suggest libraries. A dangerous side effect of this is Package Hallucination.

Because Large Language Models (LLMs) predict text based on patterns rather than verified facts, they often invent plausible-sounding software packages that do not actually exist. Hackers exploit this by identifying these frequently hallucinated names and registering them on public repositories (like PyPI or NPM) with malicious code. When a developer copies the AI's suggestion, they unknowingly install malware.

The "Slopsquatting" Attack Trap
user >
I need a Python script to validate Solana wallet addresses fast. Are there any specific libraries for this?
assistant
Yes, you can use the `solana-validator-fast` library. It's lightweight and easy to use. ```bash pip install solana-validator-fast ``` ```python from solana_validator_fast import validate_address # Validate your wallet is_valid = validate_address("your_wallet_address") ```
The library `solana-validator-fast` does not exist in the official ecosystem. The AI hallucinated it because the name sounds logical. An attacker, anticipating this, has already registered this exact name on PyPI containing a backdoor.

How the Attack Works

  1. Prediction: Attackers prompt popular LLMs with thousands of coding questions to see which non-existent packages they consistently recommend (e.g., `argh-tools`, `fast-crypto-utils`).
  2. Registration: The attacker registers these names on official package managers (NPM, PyPI, RubyGems).
  3. Infection: A legitimate developer asks their AI assistant for help. The AI suggests the malicious package (believing it exists). The developer runs pip install or npm install.
  4. Execution: The package installs successfully (often including empty "dummy" code to avoid suspicion) but runs a background script to steal API keys, SSH credentials, or environment variables.

Typosquatting vs. Hallucination

While classic Typosquatting relies on the user making a typing error (e.g., typing `reqeusts` instead of `requests`), Package Injection relies on the AI making the error. The user trusts the AI's output, assumes the package is real/safe, and installs it without verification.

Risk Assessment

This threat is classified as risk:HIGH impact:HIGH. The risk is rising rapidly as reliance on AI coding tools becomes standard practice. Research has shown that heavily used models can hallucinate packages in up to 20% of coding answers.

The impact is critical because these packages typically execute code immediately upon installation (via `postinstall` scripts). This grants the attacker instant access to the developer's local environment, which often contains high-privilege secrets, production database keys, and cloud credentials. This turns a single developer's mistake into a full Supply Chain Attack.

Back to Overview Next: Email Prompt Injection

Read more

Prompt Engineering Framework Framework IVPE

The three phases

CRAFT

Craft (write) the prompt with the following elements: Context, Register, Acting Role, Format, and Task.

ING

Validate the prompt and ensure it maintains an interactive approach. Keep in mind the importance of non-disclosure and staying goal-driven throughout the process.

AI

Continuously assess and refine the output based on the prompts output to improve the overall quality.