Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

The Keeper as a Library

Everything the bazaar command does is available from TypeScript in @liberta-labs/bazaar-keeper: build a keeper from the same config, listen to what it does, run one call, or run it all inside your own service. Use the library when the keeper is part of something larger (a trading system that also keeps its markets, a service with its own health checks and metrics), or to script one-off operations.

The package runs in Node 20 or later (it reads files, holds keys and opens sockets: it is not for browsers). Install it as in Setting Up a Keeper; the samples import from @liberta-labs/bazaar-keeper and @liberta-labs/bazaar-sdk.

Creating a Keeper

// Reads bazaar-keeper.yaml, the .env beside it and the environment, with the same rules as the CLI.
const cfg = loadConfig("bazaar-keeper.yaml");

// The CLI configures logging itself; a program does it once, before the keeper logs anything.
setupLogging(cfg.log.level, cfg.log.format);

let keeper: Keeper;
try {
  // Loads the wallets now: a keystore without a password in the config asks for it on a terminal.
  keeper = await Keeper.create(cfg);
} catch (e) {
  if (e instanceof ConfigError || e instanceof WalletError) {
    console.error(e.message); // one readable line, never a key or a password
    process.exit(2);
  }
  throw e;
}
FunctionWhat it does
loadConfig(path?)reads the YAML file (default bazaar-keeper.yaml), the .env beside it and the environment, applies the secrets rules, fills ${NAME} references and validates. Throws ConfigError
parseConfig(object)validates an object against the same schema, with the same defaults. Throws ConfigError
Keeper.create(cfg, { loadWallets? })builds the keeper and loads its wallets (a keystore may prompt for its password when stdin is a terminal). loadWallets: false makes a watch-only keeper. Throws ConfigError for a config it cannot run with (rpc.send: sequencer on a network without one, prices.endpoint: bazaar), WalletError for a wallet it cannot load
setupLogging(level, format, write?)configures the keeper’s log output: level debug/info/warning/error, format pretty/json, lines to stdout or to write. Until it is called, warnings and errors go to stderr as bare messages and the rest is dropped

ConfigError and WalletError messages are written for the person fixing the problem and never contain a key or a password. new Keeper(cfg) builds a keeper without loading its wallets yet; they load on the first connect() (and so on start() or any run.* call).

Configuring in Code

parseConfig takes the same keys as the YAML file. It does not read .env, fill ${NAME} references or apply the file’s secrets rules: the object is used as given, so take secrets from the environment or a secret store yourself.

// The same schema as the YAML file, with the same defaults and the same refusal of unknown keys.
// parseConfig does not read .env, fill ${NAME} references or apply the file's secrets rule: the
// object is used as given, so take secrets from the environment (or a secret store) yourself.
const cfg = parseConfig({
  network: "arbitrum-sepolia",
  factory: "0x0000000000000000000000000000000000000000",
  rpc: { http: env("RPC_HTTP_URL") },
  subgraph: { url: "none" },
  prices: { endpoint: "pyth", token: env("PYTH_PRO_TOKEN") }, // channel defaults to fixed_rate@200ms
  wallets: {
    default: { type: "keystore", path: "/srv/keeper/keys/default.json", password: env("KEEPER_KEYSTORE_PASSWORD") },
    fast: { type: "privateKey", key: env("KEEPER_FAST_KEY") },
  },
  jobs: {
    liquidation: { wallet: "fast" }, // {} or an object turns a job on; leaving it out keeps it off
    triggers: { wallet: "fast", minBountyUsd: 0.05 },
    unload: {},
    refresh: {},
    collect: { thresholdUsd: 10 },
  },
  budget: { maxSpendUsdPerDay: 20, minEthBalance: 0.005 },
  state: { dir: "/srv/keeper/state" },
  log: { level: "info", format: "json" },
});

Events

// A sent transaction's result, once its receipt is in.
const offAction = keeper.on("action", (r) => {
  console.log(`${r.job} ${r.market}: ${r.status}, earned $${r.rewardUsd.toFixed(4)} for $${r.costUsd.toFixed(4)} (${r.txHash})`);
});
// The moment a transaction is submitted, before its receipt.
keeper.on("sent", ([action, txHash]) => console.log(`sent ${action.job} ${action.summary}: ${txHash}`));
// lowBalance, jobPaused and endpointDown, whether or not an alert channel is configured.
keeper.on("alert", ([kind, message]) => console.warn(`alert ${kind}: ${message}`));
// Each `on` returns the function that unsubscribes.
offAction();
EventPayloadWhen
actionActionResulta sent transaction’s result, once its receipt is in (confirmed, reverted or unknown)
sent[Action, txHash]a transaction was submitted
alert[kind, message]lowBalance, jobPaused or endpointDown, whether or not an alert channel is configured or the kind is in alerts.notify (the daily summary goes to the channels only)
opportunityOpportunitya manual operation found by the running keeper’s periodic scan. The scan runs only when an alert channel is configured and opportunity is in alerts.notify; otherwise call findOpportunities yourself

on(event, handler) returns a function that unsubscribes. A handler that throws or rejects is logged and never breaks the keeper. Skipped calls are not events (they repeat every tick); they are logged.

An ActionResult has:

FieldMeaning
statusconfirmed, reverted, unknown (sent, no receipt), skipped (not sent), simulated (dry run or preview)
job, market, summarythe job (or operation), the market’s label, a one-line description
reasonwhy it was sent or skipped (earns $0.1000 for $0.0021 of gas, daily gas budget reached …)
kindthe decision: profit, loss, unpaid, forced, skip, collect, forward
wouldSendwhether the policy would send it (meaningful for simulated)
senttrue for confirmed, reverted and unknown
txHash, gasUsedthe transaction, once sent
costUsdthe estimated gas cost, then the actual one after the receipt
expectedRewardUsdwhat the simulation said the call would earn
rewardUsd, claimedUsdwhat the receipt shows it earned and claimed
errora revert, decoded (toString() gives a readable line), or null
opensAtfor a “too early” revert, when the call opens (seconds since the epoch), or null
returnedthe call’s return data from the simulation

One-Shot Runs

keeper.run holds every automatic job once and every manual operation, each naming its market on the call (a label, an address or a pair id). Each connects and syncs first when needed, so it works on a keeper that is not running as well as on one that is. Manual Operations explains what each does.

// One call, the market named on the call. A dry run simulates and reports, and sends nothing.
for (const r of await keeper.run.liquidate("ETH/USD", { dryRun: true })) {
  console.log(r.status, r.summary, r.reason);
  if (r.status === "simulated" && r.wouldSend) {
    console.log(`would earn $${r.expectedRewardUsd.toFixed(4)} for about $${r.costUsd.toFixed(4)} of gas`);
  }
}

// A manual operation returns one result. Reverts in simulation come back decoded.
const fin = await keeper.run.finalizeTermination("0x0000000000000000000000000000000000000000", { dryRun: true });
if (fin.error) console.log(`would revert: ${fin.error.toString()}`);
if (fin.opensAt) console.log(`opens at ${new Date(fin.opensAt * 1000).toISOString()}`);

The automatic jobs return ActionResult[] (one per call planned, or one skipped result saying why nothing was planned) and take { dryRun?, force? }; the list-taking ones accept the list either as an argument or in the options:

MethodLists
liquidate(market, accounts?, opts?)accounts
executeTriggers(market, orderIds?, opts?)orderIds
unload(market, opts?)
executeAdl(market, longBucketWinners?, shortBucketWinners?, opts?)longBucketWinners, shortBucketWinners
refreshPrice(market, opts?)
collect(markets?, opts?)"all" (default) or a list of markets
settleTerminal(market, accounts?, opts?)accounts
sweepExpired(market, orderIds?, opts?)orderIds

The manual operations return one ActionResult and take { dryRun? }: finalizeTermination(market), terminateStalePair(market), terminateScheduledPair(market), settleTerminationProposal(assertionId), executeInsurerTermination(market, proposalId), and under run.factory: settleDeploymentProposal(assertionId), expireStuckDeploymentProposal(pairId), settleIdentifierUpgradeProposal(assertionId), expireStuckIdentifierUpgradeProposal(), activateIdentifierUpgrade(). Ids are 32-byte hex strings (with or without 0x) or bytes.

// Automatic jobs, once. Each keeps its job's profit and loss rules unless force is set.
for (const r of await keeper.run.executeTriggers(MARKET, { dryRun: true })) show(r);
for (const r of await keeper.run.liquidate(MARKET, ["0x0000000000000000000000000000000000000001"], { force: true })) show(r);
for (const r of await keeper.run.sweepExpired(MARKET, [12n, 13n], { dryRun: true })) show(r);
for (const r of await keeper.run.collect("all")) show(r);

A manual operation sends at once unless dryRun is set, so preview it first, as the CLI does:

// Manual operations: simulate first, look at the result, then send.
const preview = await keeper.run.terminateScheduledPair(MARKET, { dryRun: true });
show(preview);
if (preview.status === "simulated" && preview.wouldSend) {
  show(await keeper.run.terminateScheduledPair(MARKET));
}

// The factory's operations sit under run.factory.
show(await keeper.run.factory.settleDeploymentProposal(ASSERTION, { dryRun: true }));

findOpportunities(keeper) returns what bazaar keeper opportunities prints: each Opportunity has operation, market (a label, for people), detail, command (the CLI command, naming the market by address), ready and opensAt, and describe() for one line.

// Every manual operation that a dry run shows would go through now, or that opens later.
for (const o of await findOpportunities(keeper)) {
  console.log(o.describe());
  if (o.ready) console.log(`   bazaar keeper ${o.command}`);
}

A one-shot run takes the same in-flight keys as the running keeper’s jobs, so in one process it never sends a call the keeper is already sending (already in flight: the keeper is sending this call; try again once it lands).

Starting and Stopping

// start() connects, then runs until stop(); it resolves once the shutdown is done.
const stopOnce = (): void => {
  // Waits up to 60 s for in-flight sends, claims earnings if jobs.collect.onShutdown, saves the snapshot.
  void keeper.stop({ timeout: 60 });
};
process.once("SIGINT", stopOnce);
process.once("SIGTERM", stopOnce);

try {
  await keeper.start();
} finally {
  await keeper.close(); // releases the price stream, the subgraph and alert clients
}
MethodWhat it does
start()connects (unless connected), then runs the tick loop, the price stream and the print path until stop(). Resolves once the shutdown has finished; rejects if connecting fails (wrong chain, bad wiring, no deployment block found)
stop({ collect?, timeout? })stops dispatching, ends the price stream at once, waits up to timeout seconds (60) for the loop and the in-flight sends, claims earnings if collect (default jobs.collect.onShutdown), and saves the snapshot. Calling it again returns the same promise
close()releases the price stream, the subgraph and alert clients. Call it after stop(), or after one-shot use
connect()checks the chain and the wiring, loads the snapshot or builds the index, reads the watched markets’ state. Once; start() and run.* call it

A keeper runs once: after stop(), start() throws this keeper has already run and stopped; create a new Keeper to start again. Calling start() while it runs throws the keeper is already running.

Status and State

MemberWhat it is
statusDict()the KeeperStatus object status.json holds: head, index, prices, each market and job, the wallets, halted (fields)
writeStatus()writes status.json now (the running keeper does every 5 s)
runningtrue while start() runs
recentthe latest 50 sent results, newest first
ledgerthe earnings ledger: ledger.summary(sinceSeconds, "job" | "market"), ledger.entries, ledger.spentToday()
storethe index: store.markets (address to Market), store.positions, store.orders
pricesthe SDK’s PriceService: prices.latest, prices.healthy, prices.onPrints(listener)
jobsthe enabled jobs by name
cfg, netthe parsed config, and the resolved network (chainId, factory, official, startBlock)
stateDirwhere the snapshot, ledger and status file live
resolveMarket(ref), findMarkets(ref)a market by label, address or pair id (one, or every one carrying a label)

stateDirFor(cfg) gives the state directory without building a keeper, and listMarkets(cfg) lists the factory’s markets with their labels without indexing anything.

Embedding in a Service

A keeper inside a long-running service, with a health endpoint and running totals for metrics:

const keeper = await Keeper.create(cfg);

// Running totals for a metrics system, fed by the "action" event (sent transactions only).
const totals = { sent: 0, reverted: 0, rewardUsd: 0, gasUsd: 0 };
keeper.on("action", (r: ActionResult) => {
  totals.sent += 1;
  if (r.status === "reverted") totals.reverted += 1;
  totals.rewardUsd += r.rewardUsd;
  totals.gasUsd += r.costUsd;
});

// GET /health: 200 while the keeper is running, has live prices and is allowed to send.
// GET /status: what `bazaar keeper status` reads (the same object as status.json).
const server = createServer((req, res) => {
  const s = keeper.statusDict();
  if (req.url === "/status") {
    res.writeHead(200, { "content-type": "application/json" });
    res.end(JSON.stringify({ ...s, totals }));
    return;
  }
  const healthy = keeper.running && s.prices.healthy && !s.halted && s.block - s.indexed < 64;
  res.writeHead(healthy ? 200 : 503, { "content-type": "application/json" });
  res.end(JSON.stringify({ healthy, block: s.block, indexed: s.indexed, prices: s.prices, halted: s.halted }));
});
server.listen(9464, "127.0.0.1");

let stopping: Promise<void> | null = null;
const shutdown = (): void => {
  stopping ??= keeper.stop().finally(() => server.close());
};
process.once("SIGINT", shutdown);
process.once("SIGTERM", shutdown);

try {
  await keeper.start(); // rejects if connecting fails (wrong chain, bad wiring, no deployment block)
} catch (e) {
  console.error(`the keeper could not start: ${e instanceof Error ? e.message : String(e)}`);
  process.exitCode = 1;
  server.close();
} finally {
  await keeper.close();
}

Points to keep in mind:

  • One keeper per set of wallets, in one process (Running in Production).
  • Handle SIGTERM by calling stop(), and give the process time to finish (up to timeout, plus the shutdown collect).
  • Configure logging with setupLogging before creating the keeper; pass write to route lines into your own logger.
  • Event handlers run on the keeper’s event loop. Keep them short; hand slow work (a database write, a network call) to a queue rather than awaiting it in the handler.
  • The bazaar_sdk.* loggers never log keys or full URLs; your handlers should not either (pass URLs through the SDK’s redactUrl).