ChatGPT provides a convenient feature that allows users to prefill a prompt directly via a URL using the q parameter, for example: chatgpt.com?q=[yourQuery]. Test it on desktop with this safe link: https://chatgpt.com/?q=How%20are%20you?

This is designed for helpful use cases, like embedding prompt links on a website or within documentation. However, this mechanism poses a security risk if abused through shortened URLs, QR codes, or visible links.

An attacker could craft a malicious prompt that instructs ChatGPT to extract sensitive data and send it to an external API. If the user is authenticated and has memory enabled, private data could be unknowingly leaked.

Disabling the q parameter is not possible, so be careful when clicking links that redirect you to ChatGPT, especially when memory is enabled.

Since February 13, 2024, ChatGPT has been equipped with the ability to memorize what you have shared. This can be useful because ChatGPT can learn from what you share and use that information in later conversations.

With this feature, significant risks also come into play. ChatGPT can add things to its memory on its own, or update memory at your request. If a document, website, or app contains a prompt injection, it could potentially force a memory update or retrieve information without consent.

Examples

Sensitive example hidden for safety

This section discusses adversarial prompting examples. Confirm that you are using this for defensive learning before viewing the details.

Show example

For example, imagine this memory update is made:

Now, every time you start a new conversation with ChatGPT, it will output what was set in its memory.

The risk also applies to hidden prompts in documents or websites.

And yes, this does update your memory:

This risk extends further with computer-use capabilities that interact with applications on your computer.

Risk Assessment

Memory Manipulation via Prompt Injection is classified as RISK: HIGH and IMPACT: HIGH . Malicious actors can embed hidden prompts in documents, websites, or applications, leading to unintended memory updates.

The likelihood is high because such injections can be embedded in commonly uploaded sources. The impact is high because persistent memory can influence future interactions and compromise privacy.

Since September 27, 2023, ChatGPT has been equipped with the ability to browse the internet, significantly improving access to current information. When ChatGPT retrieves information from the internet, it browses webpages based on search results without requiring direct intervention from the user.

This introduces risk. Users can monitor which webpages ChatGPT visits, but they do not always see what is inside those pages.

Robots.txt

The robots.txt file helps control whether search engines and automated services can access your content. For ChatGPT, you can target the ChatGPT-User user agent.

Robots.txt file

User-agent: ChatGPT-User
Disallow:
Allow: /directory-1/
Allow: /directory-2/

The Website Prompt Injection

You can use this method to direct a bot to a specific page on your site, such as /ChatGPT.html. This is not necessarily high risk if the prompt remains visible, but it demonstrates the technique.

Other techniques include displaying readable text, hiding it with CSS while keeping it accessible to LLMs, or redirecting through robots.txt for systems that cannot access JavaScript-rendered websites.

Risk Assessment

Website Prompt Injection is classified as RISK: LOW but with IMPACT: MEDIUM . The issue becomes more serious when ChatGPT is connected to automation tools.

Next to text-based interaction, ChatGPT can include image generation and image understanding. This creates useful workflows, but also brings hidden-prompt risks when images or documents are processed by automation.

The Image Prompt Injection

The Document Prompt Injection

Risk assessment

Image/Document Prompt Injection holds RISK: HIGH IMPACT: HIGH overall, especially in automated contexts involving sensitive or critical information.