gopeed/internal/protocol/gblob/README.md

# gblob Protocol

## Background

Gopeed extensions can resolve a download target into something that is not a normal remote URL.
Typical examples are:

- text generated by an extension
- binary data assembled in JavaScript
- a stream produced incrementally inside the extension runtime

Returning a plain `http` or `https` URL is not enough for these cases.
The `gblob` protocol provides an internal object-URL style transport so extensions can hand generated content to the downloader in a browser-like way.

The extension-facing model is intentionally close to the browser:

- `URL.createObjectURL(new Blob(...))`
- `URL.createObjectURL(writableStream)`
- `URL.revokeObjectURL(url)`

The returned URL looks like:

```text
gblob:<id>
```

`gblob` is a process-local protocol. The source only exists inside the current Gopeed process.

## Design Goals

- make extension-generated data download as a normal task
- support both eager `Blob` data and incremental `WritableStream` data
- avoid large in-memory buffering by spooling to disk
- keep pause/continue behavior intuitive
- allow an extension to recreate a new `gblob` URL after source expiration

## High-Level Architecture

The implementation has two layers in this directory:

- [`registry.go`](./registry.go): process-local source registry and spool-file manager
- [`fetcher.go`](./fetcher.go): downloader fetcher implementation for the `gblob` scheme

The end-to-end flow is:

1. JavaScript creates a `Blob` or `WritableStream` object URL.
2. Go registers a `gblob` source and assigns it a unique `gblob:<id>` URL.
3. Source bytes are written into a local spool file.
4. The downloader resolves the `gblob` URL through the `gblob` fetcher.
5. The fetcher copies bytes from the spool file into the final download target.

## Registry

The registry owns all active `gblob` sources for the current process.

Each source tracks:

- source id and `gblob` URL
- local spool-file path
- source type: `blob` or `writable_stream`
- written byte count
- source state: `open`, `closed`, `aborted`, `failed`
- revoke state
- pin count
- optional extension session reference

Key behaviors:

- `Blob` sources are written once and immediately closed.
- `WritableStream` sources stay open until the writer closes or aborts.
- readers wait on a per-source notification channel when no new bytes are available yet.
- revocation only prevents future use; cleanup happens when the source is no longer pinned.

## Fetcher

The fetcher treats `gblob` as a local file-backed download source.

### Resolve

`Resolve()` looks up the source in the registry and builds a single-file `Resource`.

Behavior differences by source type:

- `Blob`: resolved size is the current written size
- `WritableStream`: resolved size is reported as `0` until the writer closes

`Range` is always `false`.

### Start

`Start()` opens the final target file and reads from the source spool file.

There are two resume modes:

- same `gblob` id: continue from the current target-file size
- different `gblob` id: truncate the target file and restart from byte `0`

The fetcher persists the last started source id so it can distinguish these two cases across pause/continue and task restore.

### Pause / Continue

`Pause()` stops the current copy loop by canceling the reader side.
It does not abort the `gblob` source.

If the source id stays the same, `Continue()` resumes from the existing target-file offset.
This is how normal in-process `Pause -> Continue` works.

### Source Replacement

If an extension replaces `req.url` with a new `gblob:<new-id>` before the next start, the fetcher detects the id change and restarts from scratch.

This behavior is important for recovery flows such as:

- original `gblob` source expired
- extension recreates a new `gblob` URL in `onError`
- task continues with the new source

## Lifecycle Notes

### In-process pause and continue

For the same `gblob` id:

- the partially downloaded output file is kept
- the fetcher resumes from the current file size
- `WritableStream` producers may continue writing while the task is paused

### Process restart

`gblob` sources are not persisted across process restarts.
An unfinished `gblob` task is restored without a live source behind its old URL.

The intended recovery pattern is:

1. user clicks continue
2. the old `gblob` source fails
3. extension `onError` runs
4. extension recreates a new `gblob` URL from `req.rawUrl`
5. extension updates `ctx.task.meta.req.url`
6. extension calls `ctx.task.continue()`

Because the new URL uses a different `gblob` id, the fetcher restarts from byte `0`.

## Why `rawUrl` Exists

When an extension resolves a request into `gblob:<id>`, the original site URL would otherwise be lost.
`Request.RawURL` keeps the original creation URL so lifecycle events can still reason about the original target.

Extension event URL matching prefers `req.rawUrl` when present.

This is especially useful for `onStart`, `onError`, and `onDone` on `gblob` tasks.

## Extension Usage

### 1. Create a `Blob` object URL

```js
gopeed.events.onResolve(async function (ctx) {
  const blob = new Blob(["hello world"], { type: "text/plain" });
  const url = URL.createObjectURL(blob);

  ctx.res = {
    name: "example",
    files: [
      {
        name: "hello.txt",
        size: 11,
        req: { url }
      }
    ]
  };
});
```

### 2. Create a streaming object URL

```js
gopeed.events.onResolve(async function (ctx) {
  const stream = new TransformStream();
  const writer = stream.writable.getWriter();
  const url = URL.createObjectURL(stream.writable);

  (async () => {
    await writer.write(new TextEncoder().encode("line 1\n"));
    await writer.write(new TextEncoder().encode("line 2\n"));
    await writer.close();
  })();

  ctx.res = {
    name: "stream-example",
    files: [
      {
        name: "stream.txt",
        req: { url }
      }
    ]
  };
});
```

### 3. Recover in `onError`

```js
gopeed.events.onError(async function (ctx) {
  const req = ctx.task.meta.req;
  if (!req.rawUrl) return;
  if (!req.url.startsWith("gblob:")) return;

  req.url = URL.createObjectURL(new Blob(["recovered\n"]));
  ctx.task.continue();
});
```

This creates a new `gblob` id, so the downloader restarts from byte `0`.

## Limitations

- `gblob` is not a network protocol and does not support range requests.
- unfinished `gblob` sources do not survive process restarts.
- recovery after restart depends on extension logic recreating a new source.
- the current restart behavior is full restart on source-id change, not byte-range recovery across recreated sources.
- `WritableStream` tasks with unknown total size report size `0` until the writer closes.

## Summary

`gblob` lets Gopeed extensions treat generated content as a first-class download source.
The registry handles source lifetime and disk spooling.
The fetcher turns those sources into normal download tasks with predictable pause/continue behavior.

The most important rule is simple:

- same `gblob` id means resume
- different `gblob` id means restart