Files
gopeed/internal/protocol/gblob

gblob Protocol

Background

Gopeed extensions can resolve a download target into something that is not a normal remote URL. Typical examples are:

  • text generated by an extension
  • binary data assembled in JavaScript
  • a stream produced incrementally inside the extension runtime

Returning a plain http or https URL is not enough for these cases. The gblob protocol provides an internal object-URL style transport so extensions can hand generated content to the downloader in a browser-like way.

The extension-facing model is intentionally close to the browser:

  • URL.createObjectURL(new Blob(...))
  • URL.createObjectURL(writableStream)
  • URL.revokeObjectURL(url)

The returned URL looks like:

gblob:<id>

gblob is a process-local protocol. The source only exists inside the current Gopeed process.

Design Goals

  • make extension-generated data download as a normal task
  • support both eager Blob data and incremental WritableStream data
  • avoid large in-memory buffering by spooling to disk
  • keep pause/continue behavior intuitive
  • allow an extension to recreate a new gblob URL after source expiration

High-Level Architecture

The implementation has two layers in this directory:

  • registry.go: process-local source registry and spool-file manager
  • fetcher.go: downloader fetcher implementation for the gblob scheme

The end-to-end flow is:

  1. JavaScript creates a Blob or WritableStream object URL.
  2. Go registers a gblob source and assigns it a unique gblob:<id> URL.
  3. Source bytes are written into a local spool file.
  4. The downloader resolves the gblob URL through the gblob fetcher.
  5. The fetcher copies bytes from the spool file into the final download target.

Registry

The registry owns all active gblob sources for the current process.

Each source tracks:

  • source id and gblob URL
  • local spool-file path
  • source type: blob or writable_stream
  • written byte count
  • source state: open, closed, aborted, failed
  • revoke state
  • pin count
  • optional extension session reference

Key behaviors:

  • Blob sources are written once and immediately closed.
  • WritableStream sources stay open until the writer closes or aborts.
  • readers wait on a per-source notification channel when no new bytes are available yet.
  • revocation only prevents future use; cleanup happens when the source is no longer pinned.

Fetcher

The fetcher treats gblob as a local file-backed download source.

Resolve

Resolve() looks up the source in the registry and builds a single-file Resource.

Behavior differences by source type:

  • Blob: resolved size is the current written size
  • WritableStream: resolved size is reported as 0 until the writer closes

Range is always false.

Start

Start() opens the final target file and reads from the source spool file.

There are two resume modes:

  • same gblob id: continue from the current target-file size
  • different gblob id: truncate the target file and restart from byte 0

The fetcher persists the last started source id so it can distinguish these two cases across pause/continue and task restore.

Pause / Continue

Pause() stops the current copy loop by canceling the reader side. It does not abort the gblob source.

If the source id stays the same, Continue() resumes from the existing target-file offset. This is how normal in-process Pause -> Continue works.

Source Replacement

If an extension replaces req.url with a new gblob:<new-id> before the next start, the fetcher detects the id change and restarts from scratch.

This behavior is important for recovery flows such as:

  • original gblob source expired
  • extension recreates a new gblob URL in onError
  • task continues with the new source

Lifecycle Notes

In-process pause and continue

For the same gblob id:

  • the partially downloaded output file is kept
  • the fetcher resumes from the current file size
  • WritableStream producers may continue writing while the task is paused

Process restart

gblob sources are not persisted across process restarts. An unfinished gblob task is restored without a live source behind its old URL.

The intended recovery pattern is:

  1. user clicks continue
  2. the old gblob source fails
  3. extension onError runs
  4. extension recreates a new gblob URL from req.rawUrl
  5. extension updates ctx.task.meta.req.url
  6. extension calls ctx.task.continue()

Because the new URL uses a different gblob id, the fetcher restarts from byte 0.

Why rawUrl Exists

When an extension resolves a request into gblob:<id>, the original site URL would otherwise be lost. Request.RawURL keeps the original creation URL so lifecycle events can still reason about the original target.

Extension event URL matching prefers req.rawUrl when present.

This is especially useful for onStart, onError, and onDone on gblob tasks.

Extension Usage

1. Create a Blob object URL

gopeed.events.onResolve(async function (ctx) {
  const blob = new Blob(["hello world"], { type: "text/plain" });
  const url = URL.createObjectURL(blob);

  ctx.res = {
    name: "example",
    files: [
      {
        name: "hello.txt",
        size: 11,
        req: { url }
      }
    ]
  };
});

2. Create a streaming object URL

gopeed.events.onResolve(async function (ctx) {
  const stream = new TransformStream();
  const writer = stream.writable.getWriter();
  const url = URL.createObjectURL(stream.writable);

  (async () => {
    await writer.write(new TextEncoder().encode("line 1\n"));
    await writer.write(new TextEncoder().encode("line 2\n"));
    await writer.close();
  })();

  ctx.res = {
    name: "stream-example",
    files: [
      {
        name: "stream.txt",
        req: { url }
      }
    ]
  };
});

3. Recover in onError

gopeed.events.onError(async function (ctx) {
  const req = ctx.task.meta.req;
  if (!req.rawUrl) return;
  if (!req.url.startsWith("gblob:")) return;

  req.url = URL.createObjectURL(new Blob(["recovered\n"]));
  ctx.task.continue();
});

This creates a new gblob id, so the downloader restarts from byte 0.

Limitations

  • gblob is not a network protocol and does not support range requests.
  • unfinished gblob sources do not survive process restarts.
  • recovery after restart depends on extension logic recreating a new source.
  • the current restart behavior is full restart on source-id change, not byte-range recovery across recreated sources.
  • WritableStream tasks with unknown total size report size 0 until the writer closes.

Summary

gblob lets Gopeed extensions treat generated content as a first-class download source. The registry handles source lifetime and disk spooling. The fetcher turns those sources into normal download tasks with predictable pause/continue behavior.

The most important rule is simple:

  • same gblob id means resume
  • different gblob id means restart