Skip to content

Cloud agent: viewing any image ends the session with CAPIError: 400 The image data you provided does not represent a valid image #4930

Description

@CynthiaLuijkx

Describe the bug

Summary

In a Copilot cloud agent session on a GHEC data residency tenant (<tenant>.ghe.com), the view tool on any image file ends the session. The tool itself reports success. The next model call is rejected, and the session dies.

The error names the image as the fault. The image is never the fault. A 4 KB, 1280x720 PNG that Pillow reads as a valid RGB image fails exactly like a real screenshot.

What the session gets

Viewed image file at path /tmp/img-720.png

then

CAPIError: 400 The image data you provided does not represent a valid image.
Please check your input and try again with one of the supported image formats:
['image/jpeg', 'image/png', 'image/gif', 'image/webp']

The session state goes to failed and that message is the task's recorded error.

Reproduction

Any session. No repository state is needed.

python3 - <<'PY'
import zlib, struct
w, h = 1280, 720
row = b'\x00' + bytes([200, 40, 90]) * w
raw = row * h
def chunk(t, d):
    c = t + d
    return struct.pack('>I', len(d)) + c + struct.pack('>I', zlib.crc32(c) & 0xffffffff)
hdr = struct.pack('>IIBBBBB', w, h, 8, 2, 0, 0, 0)
open('/tmp/img.png', 'wb').write(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', hdr) + chunk(b'IDAT', zlib.compress(raw, 6)) + chunk(b'IEND', b''))
PY

Then call view on /tmp/img.png. The file is 4319 bytes. PIL.Image.open reads it as (1280, 720) RGB.

What I ruled out

I made five PNGs of 1280 pixels wide and 720, 1440, 2880, 5760 and 11520 pixels tall, then viewed them one per turn, smallest first. The session died on the first one.

Variable Tested Result
Height 720 through 11520 dies on the smallest
Byte size 4 KB synthetic, 78 KB real screenshot, 128 KB real screenshot all die
Images per turn one dies on one
Source hand-written PNG, Playwright page.screenshot both die
Model sweagent-capi:gpt-5.6-sol, sweagent-capi:gpt-5.6-terra both die

Why this is worse than it reads

  1. The failing step reports success. view answers Viewed image file at path <path>. The agent has no way to know it has just ended the session.
  2. The error blames the file. Every reader goes and checks the file, and the file is fine.
  3. It kills a session that already did its work. In our case a session implemented a work item across 24 files, passed 691 unit tests and 6 Playwright end-to-end tests, wrote its screenshots, then viewed three of them and died before it could open the pull request.
  4. It makes your own guidance fatal. Any instruction that tells an agent to check a screen capture before attaching it now ends the session that follows it.

What I ask for

  1. Make image input work, or say plainly that a cloud agent session cannot take image input on a data residency tenant.
  2. If it cannot, make view refuse an image with a clear message instead of accepting it and killing the next model call.
  3. Fix the error text. "The image data you provided does not represent a valid image" is wrong for a valid image, and it sends every reader to the wrong place.

Affected version

No response

Steps to reproduce the behavior

No response

Expected behavior

No response

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions