# gblob Protocol

## Background

Gopeed extensions can resolve a download target into something that is not a normal remote URL.
Typical examples are:

- text generated by an extension
- binary data assembled in JavaScript
- a stream produced incrementally inside the extension runtime

Returning a plain `http` or `https` URL is not enough for these cases.
The `gblob` protocol provides an internal object-URL style transport so extensions can hand generated content to the downloader in a browser-like way.

The extension-facing model is intentionally close to the browser:

- `URL.createObjectURL(new Blob(...))`
- `URL.createObjectURL(readableStream)`
- `URL.createObjectURL(openReadable)`
- `URL.revokeObjectURL(url)`

The returned URL looks like:

```text
gblob:<id>
```

`gblob` is a process-local protocol. The source only exists inside the current Gopeed process.

## Design Goals

- make extension-generated data download as a normal task
- support eager `Blob` data, one-shot `ReadableStream` data, and resumable opener functions
- avoid large in-memory buffering by spooling to disk
- keep pause/continue behavior intuitive
- allow an extension to recreate a new `gblob` URL after source expiration

## High-Level Architecture

The implementation has two layers in this directory:

- [`registry.go`](./registry.go): process-local source registry and spool-file manager
- [`fetcher.go`](./fetcher.go): downloader fetcher implementation for the `gblob` scheme

The end-to-end flow is:

1. JavaScript creates a `Blob`, `ReadableStream`, or opener-function object URL.
2. Go registers a `gblob` source and assigns it a unique `gblob:<id>` URL.
3. Source bytes are written into a local spool file.
4. The downloader resolves the `gblob` URL through the `gblob` fetcher.
5. The fetcher copies bytes from the spool file into the final download target.

## Registry

The registry owns all active `gblob` sources for the current process.

Each source tracks:

- source id and `gblob` URL
- local spool-file path
- source type: `blob` or `writable_stream`
- written byte count
- source state: `open`, `closed`, `aborted`, `failed`
- revoke state
- pin count
- optional extension session reference

Key behaviors:

- `Blob` sources are written once and immediately closed.
- `ReadableStream` sources are bridged into the internal writable-source pipeline.
- opener-function sources are opened lazily and can be reopened from an offset.
- readers wait on a per-source notification channel when no new bytes are available yet.
- revocation only prevents future use; cleanup happens when the source is no longer pinned.

## Fetcher

The fetcher treats `gblob` as a local file-backed download source.

### Resolve

`Resolve()` looks up the source in the registry and builds a single-file `Resource`.

Behavior differences by source type:

- `Blob`: resolved size is the current written size
- `ReadableStream` / opener function: resolved size is whatever the extension declared; if unknown, it stays `0` until download completion updates the final size

`Range` is `true` only when the extension provides a resumable opener and marks the resource as ranged.

### Start

`Start()` opens the final target file and reads from the source spool file.

There are two resume modes:

- same `gblob` id: continue from the current target-file size
- different `gblob` id: truncate the target file and restart from byte `0`

The fetcher persists the last started source id so it can distinguish these two cases across pause/continue and task restore.

### Pause / Continue

`Pause()` stops the current copy loop by canceling the reader side.
It does not abort the `gblob` source.

If the source id stays the same, `Continue()` resumes from the existing target-file offset.
This is how normal in-process `Pause -> Continue` works.

### Source Replacement

If an extension replaces `req.url` with a new `gblob:<new-id>` before the next start, the fetcher detects the id change and restarts from scratch.

This behavior is important for recovery flows such as:

- original `gblob` source expired
- extension recreates a new `gblob` URL in `onError`
- task continues with the new source

## Lifecycle Notes

### In-process pause and continue

For the same `gblob` id:

- the partially downloaded output file is kept
- the fetcher resumes from the current file size
- opener-function sources can be reopened from the current offset

For non-reopenable one-shot `ReadableStream` sources:

- a normal in-process pause/continue only works while the same in-memory source is still alive
- they are not suitable for cross-session or reconstructable range resume

### Process restart

`gblob` sources are not persisted across process restarts.
An unfinished `gblob` task is restored without a live source behind its old URL.

The intended recovery pattern is:

1. user clicks continue
2. the old `gblob` source fails
3. extension `onError` runs
4. extension recreates a new `gblob` URL from `req.rawUrl`
5. extension updates `ctx.task.meta.req.url`
6. extension calls `ctx.task.continue()`

Because the new URL uses a different `gblob` id, the fetcher restarts from byte `0`.

## Why `rawUrl` Exists

When an extension resolves a request into `gblob:<id>`, the original site URL would otherwise be lost.
`Request.RawURL` keeps the original creation URL so lifecycle events can still reason about the original target.

Extension event URL matching prefers `req.rawUrl` when present.

This is especially useful for `onStart`, `onError`, and `onDone` on `gblob` tasks.

## Extension Usage

### 1. Create a `Blob` object URL

```js
gopeed.events.onResolve(async function (ctx) {
  const blob = new Blob(["hello world"], { type: "text/plain" });
  const url = URL.createObjectURL(blob);

  ctx.res = {
    name: "example",
    files: [
      {
        name: "hello.txt",
        size: 11,
        req: { url }
      }
    ]
  };
});
```

### 2. Create a one-shot `ReadableStream` object URL

```js
gopeed.events.onResolve(async function (ctx) {
  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    start(controller) {
      controller.enqueue(encoder.encode("line 1\n"));
      controller.enqueue(encoder.encode("line 2\n"));
      controller.close();
    }
  });
  const url = URL.createObjectURL(stream);

  ctx.res = {
    name: "stream-example",
    files: [
      {
        name: "stream.txt",
        req: { url }
      }
    ]
  };
});
```

### 3. Create a resumable opener object URL

```js
gopeed.events.onResolve(async function (ctx) {
  const target = "https://example.com/file.bin";

  const openReadable = async (offset) => {
    const headers = {};
    if (offset > 0) {
      headers.Range = `bytes=${offset}-`;
    }
    const response = await fetch(target, { headers });
    if (response.status !== 200 || response.status !== 206) {
      throw new Error(`unexpected response status: ${response.status}, expected ${expectedStatus}`);
    }
    if (!response.body) {
      throw new Error("empty response body");
    }
    return response.body;
  };

  ctx.res = {
    name: "range-example",
    range: true,
    files: [
      {
        name: "file.bin",
        req: { url: URL.createObjectURL(openReadable) }
      }
    ]
  };
});
```

Important:

- opener functions must return a `ReadableStream`
- do not return `response.body.getReader()`
- use opener functions for resumable / range-aware sources
- validate HTTP status codes explicitly:
  first request usually expects `200`, resumed range requests usually expect `206`

### 4. Recover in `onError`

```js
gopeed.events.onError(async function (ctx) {
  const req = ctx.task.meta.req;
  req.url = URL.createObjectURL(new Blob(["recovered\n"]));
  ctx.task.continue();
});
```

This creates a new `gblob` id, so the downloader restarts from byte `0`.

## Limitations

- `gblob` is not a network protocol. Range support only exists when the extension recreates bytes through a resumable opener.
- unfinished `gblob` sources do not survive process restarts.
- recovery after restart depends on extension logic recreating a new source.
- the current restart behavior is full restart on source-id change, not byte-range recovery across recreated sources.
- one-shot `ReadableStream` tasks with unknown total size report size `0` until completion updates the final size.

## Summary

`gblob` lets Gopeed extensions treat generated content as a first-class download source.
The registry handles source lifetime and disk spooling.
The fetcher turns those sources into normal download tasks with predictable pause/continue behavior.

The most important rule is simple:

- same `gblob` id means resume
- different `gblob` id means restart

And for extension authors:

- use `ReadableStream` for one-shot streaming content
- use opener functions for resumable content
- opener functions must return `ReadableStream`, not a reader