Use your visitors' own AI

A site reaches the visitor's agent wallet in one of two ways: through the browser extension's window.byoa, or with a bringyourownagent:// link, after which the wallet connects to your server itself. Both end in the same consent, limits and activity log, and a site allowed one way is allowed the other way too.

The samples are short and complete, and follow the real code of the agent wallet 0.1; the whole design is in docs/wallet-0.1.md. The wallet 0.1 serves two skills: speech to text and text answers.

The manifest

The site says what it needs and why, and names no vendor. The format is the draft rules; the libraries validate it, and the wallet checks every call against it.

The wallet 0.1 has two skills. audio.transcribe takes a WAV clip (16 kHz, mono, 16-bit PCM, up to 15 seconds) as base64, the parameter is the language, the output is { text }. The wallet measures the clip itself; it never trusts a length the page states. text.generate takes a list of chat messages and optional parameters such as maxTokens and choices, and returns { text }. It needs a text source that the visitor already runs (LM Studio, Ollama) or pays for (an OpenRouter key); without one, a site that requires it gets no Allow. The Twenty Questions game at twenty-questions.appshub.net uses it.

The manifest of the voice geography game

{
  "byoa": "0.1-draft",
  "purpose": {
    "summary": "Family geography game: recognise country names spoken aloud",
    "limits": { "maxClipSeconds": 15, "maxCallsPerMinute": 40 }
  },
  "skills": {
    "audio.transcribe": { "languages": ["en", "ru", "sk"], "required": true }
  },
  "privacy": "personal-voice",
  "expectedCost": { "perSession": "0.05 USD" }
}

Check it with the libraries (Node.js; Python and .NET have the same call)sdk/

// npm install bringyourownagent   (the same library: PyPI bringyourownagent, NuGet BringYourOwnAgent)
import { validateManifest } from 'bringyourownagent';

const { ok, errors } = validateManifest(MANIFEST);
if (!ok) console.error(errors);            // [{ code, path, message }]: stable codes, JSON Pointer paths

With the extension: window.byoa

The extension adds window.byoa to every https page (and to http://localhost for tests) and fires byoa:ready. Without the extension window.byoa is simply missing, and your page goes its own way.
Call request(manifest) from a click handler. Without a click the wallet shows nothing and answers null, unless the visitor let it decide on its own for your site.
null from request and { ok: false } from call are the only “no”: declined, blocked, no fitting model, no answer in 120 seconds, no wallet running. Your page cannot tell them apart, and that is intended.
The wallet answers request only when its models are loaded, so the first call is as fast as the others; on a fresh computer that can take some seconds. Show that you are waiting.
The answer is user input: show it as text, never as HTML, and never let a score, prize or payment depend on it.

In the pagewallet/extension/page.js

// In the page. The extension adds window.byoa at document_start and fires "byoa:ready".
const MANIFEST = { /* the manifest above */ };
let grant = null;

document.querySelector('#voice').addEventListener('click', async () => {
  if (!window.byoa) return useOwnMode();          // no extension: your own path
  grant = await window.byoa.request(MANIFEST);    // in the click handler; the wallet asks the visitor
  if (!grant) return useOwnMode();                // "AI unavailable": declined, blocked, no model, no wallet
  startListening();                               // grant: { grantId, skills, expiresAt }
});

// One clip: a 16 kHz mono 16-bit PCM WAV, up to 15 s, as base64. Resolves with the text, or null.
async function transcribe(wavBase64, language) {
  const r = await window.byoa.call(grant.grantId, 'audio.transcribe',
    { audio: { mime: 'audio/wav', base64: wavBase64 } },
    { language });                                // 'ru', 'sk', 'en', ...
  if (!r.ok) return null;                         // unavailable for this call: use your own path
  return r.output.text;                           // user input: show it as text, never as HTML
}

window.addEventListener('pagehide', () => { if (grant) window.byoa.release(grant.grantId); });

A helper: microphone samples to the WAV the skill takes

// Float32 samples at 16 kHz (from an AudioWorklet, or resampled with an OfflineAudioContext)
// -> base64 of a 16-bit PCM mono WAV, the input of audio.transcribe
function wavBase64(samples) {
  const v = new DataView(new ArrayBuffer(44 + samples.length * 2));
  const tag = (at, text) => [...text].forEach((ch, i) => v.setUint8(at + i, ch.charCodeAt(0)));
  tag(0, 'RIFF'); v.setUint32(4, 36 + samples.length * 2, true); tag(8, 'WAVE');
  tag(12, 'fmt '); v.setUint32(16, 16, true); v.setUint16(20, 1, true); v.setUint16(22, 1, true);
  v.setUint32(24, 16000, true); v.setUint32(28, 32000, true); v.setUint16(32, 2, true); v.setUint16(34, 16, true);
  tag(36, 'data'); v.setUint32(40, samples.length * 2, true);
  samples.forEach((x, i) => v.setInt16(44 + i * 2, Math.max(-1, Math.min(1, x)) * 0x7fff, true));
  let bin = '';
  new Uint8Array(v.buffer).forEach((b) => { bin += String.fromCharCode(b); });
  return btoa(bin);
}

Without the extension: a link and a WebSocket

On the click, the page asks your server for a one-time session code. The server answers with the code and the link, which carries your WebSocket endpoint, the manifest itself and the code.
Still in the same click handler, the page opens the link: a browser starts an app only after a click. Show the link too, in case the visitor closed the browser's question.
The wallet starts, or the running one takes the link, checks it and shows its notice and consent window. Before the visitor's Allow it contacts nothing at all, not even your server's name: a decline looks exactly like no wallet.
After Allow, the wallet opens its own WebSocket to your endpoint and says hello with the code. Your welcome returns a pairing token and the manifest string from the link, byte for byte; if it differs, the wallet says bye and revokes your site.
Then your server sends call frames to that connection and gets result frames back: the calls go from your server to the visitor's wallet directly, with no round trip through the page.

The link, with notesDirectLink.cs

bringyourownagent://connect?v=0.1
  &endpoint=wss%3A%2F%2Fgame.example.org%2Fbyoa%2Fws      the WebSocket the wallet opens after Allow (URL-encoded)
  &manifest=eyJieW9hIjoiMC4xLWRyYWZ0IiwicHVycG9zZSI6...   base64url of the manifest's JSON bytes, no padding, at most 4096 bytes
  &session=Jf3k2N0q-Yb8wQe1sX5mTt7LrA9cDhKu               a one-time code: 128 random bits or more, URL-safe, 120 s

In the page

// In the page: <button id="wallet-link">Use my AI without the extension</button>
//              <a id="open-wallet" hidden>Open the agent wallet</a>
document.querySelector('#wallet-link').addEventListener('click', async () => {
  const { session, link } = await (await fetch('/byoa/code', { method: 'POST' })).json();
  location.href = link;                           // still in the click: a browser starts an app only after one
  const a = document.querySelector('#open-wallet');
  a.href = link; a.hidden = false;                // shown too, in case the browser's question was closed

  const until = Date.now() + 90_000;              // the wallet asks the visitor first: wait, then give up quietly
  while (Date.now() < until) {
    await new Promise((r) => setTimeout(r, 1000));
    const { state } = await (await fetch('/byoa/wait?session=' + session)).json();
    if (state === 'connected') return walletConnected();
    if (state === 'expired') break;
  }
  walletDidNotConnect();                          // no other detail: a decline looks like no wallet at all
});

A minimal server (Node.js)

// server.mjs: Node.js 18+, npm install ws. Serve it over TLS (wss://), e.g. behind a reverse proxy.
import { createServer } from 'node:http';
import { randomBytes } from 'node:crypto';
import { WebSocketServer } from 'ws';

const HOST = 'game.example.org';                              // your public host name
const MANIFEST = { /* the manifest above */ };
// The link carries base64url of these exact bytes, and every welcome echoes the same string.
const MANIFEST64 = Buffer.from(JSON.stringify(MANIFEST)).toString('base64url');
const secret = () => randomBytes(24).toString('base64url');   // 192 random bits, URL-safe
const codes = new Map();      // session code -> { expires, used, socket }: single use, 120 s
const wallets = new Map();    // pairing token -> the wallet's live socket, or null while it is away

const server = createServer((req, res) => {
  const url = new URL(req.url, `https://${HOST}`);
  const json = (body) => { res.writeHead(200, { 'content-type': 'application/json' }); res.end(JSON.stringify(body)); };
  if (req.method === 'POST' && url.pathname === '/byoa/code') {           // limit this per client address
    const session = secret();
    codes.set(session, { expires: Date.now() + 120_000, used: false, socket: null });
    return json({ session, link: 'bringyourownagent://connect?v=0.1'
      + '&endpoint=' + encodeURIComponent(`wss://${HOST}/byoa/ws`)
      + '&manifest=' + MANIFEST64 + '&session=' + session });
  }
  if (req.method === 'GET' && url.pathname === '/byoa/wait') {            // has a wallet come with this code?
    const c = codes.get(url.searchParams.get('session'));
    return json({ state: c?.used ? 'connected' : c && c.expires > Date.now() ? 'waiting' : 'expired' });
  }
  res.writeHead(404).end();
});

const wss = new WebSocketServer({ server, path: '/byoa/ws', maxPayload: 4 * 1024 * 1024 });
wss.on('connection', (ws) => {
  ws.pending = new Map();                                     // call id -> resolve
  const noHello = setTimeout(() => ws.close(1008, 'no hello'), 10_000);
  ws.on('message', (data, binary) => {
    if (binary) return;                                       // unknown frames are ignored
    let f;
    try { f = JSON.parse(data); } catch { return ws.close(1007, 'malformed'); }
    if (!ws.pairing) {                                        // nothing is sent before a valid hello
      if (f.type !== 'hello') return;
      clearTimeout(noHello);
      const c = codes.get(f.session);
      if (c && !c.used && c.expires > Date.now()) {           // after a link: a fresh code, used once
        c.used = true; c.socket = ws; ws.pairing = secret();
      } else if (!f.session && wallets.has(f.pairing)) {      // later: the wallet comes back with its pairing
        const old = wallets.get(f.pairing);
        if (old) { old.send('{"type":"bye"}'); old.close(); } // one live connection per pairing
        ws.pairing = f.pairing;
      } else {
        ws.send(JSON.stringify({ type: 'error', code: 'bad-session' }));
        return ws.close(1008, 'bad-session');
      }
      wallets.set(ws.pairing, ws);
      return ws.send(JSON.stringify({ type: 'welcome', pairing: ws.pairing, manifest: MANIFEST64 }));
    }
    if (f.type === 'result') ws.pending.get(f.id)?.(f);
    else if (f.type === 'bye') ws.close();
    // "ping" (every 30 s) and unknown types: nothing to do
  });
  ws.on('close', () => { if (wallets.get(ws.pairing) === ws) wallets.set(ws.pairing, null); });
});

let nextId = 0;
// One clip for the visitor's wallet: resolves with the text, or null (then use your own path).
export function transcribe(ws, wavBytes, language, ms = 20_000) {
  const id = ++nextId;
  return new Promise((resolve) => {
    const done = (text) => { clearTimeout(timer); ws.pending.delete(id); resolve(text); };
    const timer = setTimeout(() => done(null), ms);
    ws.pending.set(id, (r) => done(r.ok && typeof r.output?.text === 'string' ? r.output.text : null));
    ws.send(JSON.stringify({ type: 'call', id, skill: 'audio.transcribe',
      input: { audio: { mime: 'audio/wav', base64: wavBytes.toString('base64') } },
      params: { language } }));
  });
}

server.listen(8080);

A complete small server in Python (aiohttp), the one the wallet's own end-to-end test talks to: wallet/tests/e2e/fake_site.py. The wallet's side of the connection: DirectConnection.cs.

The frames: JSON text, one message each, up to 4 MBdocs/wallet-0.1.md, section 10

// wallet -> server, the first frame, after the visitor's Allow
{"type":"hello","v":"0.1","session":"<code from the link>","pairing":null,
 "wallet":{"name":"Agent wallet","version":"0.1"},"grant":{"skills":["audio.transcribe"],"expiresAt":"2026-10-12T23:04:10Z"}}
// server -> wallet: the manifest string from the link, byte for byte, and a pairing token
{"type":"welcome","pairing":"<token>","manifest":"eyJieW9hIjoiMC4xLWRyYWZ0Ii..."}
// ...or, for an unknown, used or expired code, and then a close
{"type":"error","code":"bad-session"}
// server -> wallet, one per use; at most 4 at a time on a connection
{"type":"call","id":1,"skill":"audio.transcribe","input":{"audio":{"mime":"audio/wav","base64":"UklGR..."}},"params":{"language":"ru"}}
// wallet -> server
{"type":"result","id":1,"ok":true,"output":{"text":"Peru"},"receipt":{"cost":"0.000 USD","provider":"local"}}
{"type":"result","id":2,"ok":false}
// wallet -> server every 30 s; either side, before it closes
{"type":"ping"}
{"type":"bye"}
// the wallet coming back with its pairing (only for a site with "decide on my own")
{"type":"hello","v":"0.1","session":null,"pairing":"<token>","wallet":{...},"grant":{...}}

What the server must do

Codes: at least 128 random bits, URL-safe, single use, valid for 120 seconds, a limited number a minute per visitor. The wallet ignores a code it saw in the last 10 minutes.
The endpoint is wss:// on a public host name: the wallet refuses IP addresses, localhost and local names, and refuses private addresses when it connects. For local tests the wallet has a developer switch, BYOA_WALLET_DEV_LOCAL=1, that allows ws://localhost.
Send nothing before a valid hello. Ignore frame types you do not know; close on a frame that is not JSON. Keep at most as many calls in flight as the hello's grant.limits.maxParallelCalls says (4 when it says none; the wallet's default is 8); more get ok: false.
A site that works with text may ask for several different models (a models block in the skill: how many, from different companies or not, which it requires or only recommends, why) and declare limits (maxTextChars, maxAnswerTokens, maxCallSeconds, maxParallelCalls, maxCallsPerMinute). The visitor ticks the models; the grant and the hello list them by code (m1, m2…), a call names one in params.model, and the wallet never answers with another. params.stream over the direct link is answered in chunk frames before the result, and a grant frame tells a change. See CONTRACT.md sections 3.3 and 3.4 and CONTRACT-DIRECT.md section 4.
Keep the pairing token: the wallet keeps it only when the visitor ticked “decide on my own”, and then comes back with it after a dropped connection or a restart (after 1, 2, 5, 10, then every 30 seconds). Its welcome carries your current manifest; if that asks for more than was allowed, the wallet says bye and you need a new link. A site the visitor paused is not reconnected.

On the visitor's computer

How the wallet 0.1 for Windows catches both ways in. Nothing here is needed to build a site; it is here so that you can see what runs on the visitor's side and why it is safe to rely on.

A link starts the wallet

The installer registers the bringyourownagent scheme for the current Windows user only. The scheme is never byoa://, a generic name another program could take: on Windows the last program to register a scheme gets its links, and a session code is meant for the wallet alone.

Registry keyswallet/install.ps1

# Written by wallet\install.ps1 for the current Windows user only (HKCU), removed by uninstall.ps1
HKCU\Software\Classes\bringyourownagent
    (Default)       URL:Bring Your Own Agent link
    URL Protocol    (empty)
HKCU\Software\Classes\bringyourownagent\shell\open\command
    (Default)       "%LOCALAPPDATA%\BYOA\wallet-app\Byoa.Wallet.exe" --url "%1"
HKCU\Software\Google\Chrome\NativeMessagingHosts\org.bringyourownagent.wallet
    (Default)       %LOCALAPPDATA%\BYOA\wallet-app\org.bringyourownagent.wallet.json
HKCU\Software\Chromium\NativeMessagingHosts\org.bringyourownagent.wallet
    (Default)       the same file

One wallet at a time

A second Byoa.Wallet.exe, started by a link while the wallet runs, finds the first one by a mutex and hands the link over the wallet's named pipe, then exits. The pipe is per user and opened with CurrentUserOnly on both ends; each message is a 4-byte little-endian length and UTF-8 JSON.

Abridged from Program.csProgram.cs

// Byoa.Wallet.exe --url "<link>": one wallet per user and data folder; a second one hands the link over
var url = Array.IndexOf(args, "--url");
if (url >= 0) LinkArgument = url + 1 < args.Length ? args[url + 1] : "";
using var instance = new Mutex(true, WalletPaths.MutexName(), out var first);
Background = args.Contains("--background") || LinkArgument is not null;
if (!first)
{
    // the running wallet takes the link over its per-user pipe (CurrentUserOnly), and this process ends
    if (LinkArgument is not null) LinkHandOff.Send(WalletPaths.PipeName(), LinkArgument);
    else if (!Background) AskRunningWalletToOpen();
    return 0;
}

The extension talks to a native host

Chrome starts byoa-host.exe for the extension through native messaging and lets only the extension whose ID is in allowed_origins use it; the ID is fixed by the key in the extension's manifest. The host relays messages to the wallet's pipe and starts the wallet for a request; it decides nothing. The page's origin comes from Chrome, never from the page.

The host manifest, written by the installerbyoa-host

{
  "name": "org.bringyourownagent.wallet",
  "description": "BYOA agent wallet: lets pages use your own AI within your limits",
  "path": "C:\\Users\\you\\AppData\\Local\\BYOA\\wallet-app\\byoa-host.exe",
  "type": "stdio",
  "allowed_origins": ["chrome-extension://niobopdoiopjilmlonjcgllpmgjemhpk/"]
}

Checks before anything else

Only a connect link of version 0.1, with no fragment and no repeated parameter. The endpoint must be wss:// with a public DNS name: no IP address in any spelling, no localhost, no .local, .home.arpa, .internal or .lan, no single-label name.
The manifest must be base64url without padding, at most 4096 bytes, a JSON object, and valid under the draft rules (the .NET library). The session code: 22 to 256 URL-safe characters.
A link that fails a check, repeats a recent code or comes from a blocked site does nothing but leave an entry in Activity. At most 4 links are handled at once.
After Allow, the wallet resolves the host itself, refuses private, loopback, link-local and other special addresses, and connects to the address it checked: no proxy, no cookies, no redirects.

Rules for sites

Ask only after the visitor clicks something; otherwise the wallet stays silent and answers null.
Always keep your own way: no wallet, a decline, a block and a failed call look the same, and your page must work without the visitor's AI.
Treat every answer as something the visitor typed. A visitor can connect any model, or a script, to the wallet.
State your purpose honestly and ask only for what you use: the visitor reads it in the consent window, and a site that asks for more later gets a new consent showing only the difference.
Sites need no key, no account and no payment to use BYOA; the visitor pays their own provider, or nothing for a model on their own computer.

Questions and ideas: GitHub issues or the Ideas page. Try it as a visitor: get the wallet and open geo.appshub.net.

Use your visitors' own AI

1The manifest

2With the extension: window.byoa

3Without the extension: a link and a WebSocket