Modernizing Steem.js — Day 3: Rewriting the Serializer Without bytebuffer

Part 3 of my 7-part Steem.js modernization series. So far we've built a safety net + modern tooling and swapped the crypto core to @noble. Today we tackle the other half of "what makes a transaction valid on-chain": binary serialization.

Why serialization matters

Before a transaction can be signed, it has to be turned into the exact sequence of bytes the blockchain expects — operation IDs, varint-encoded lengths, little-endian integers, 64-bit amounts, asset symbols, and so on. The signature is computed over those bytes. If the byte layout is off by even one byte, the signature is meaningless and the chain rejects the transaction.

The old serializer was built on bytebuffer — a capable library, but one that carries its own Long (64-bit integer) implementation and assumes a Node-ish environment. Like the crypto stack, it's a poor fit for edge and Deno.

Enter bytebuffer-lite

Rather than rewrite the entire serializer's logic, I wrote a drop-in replacement for just the piece it depended on: src/auth/serializer/src/bytebuffer-lite.js. It's a tiny, in-repo Uint8Array + DataView writer/reader that reproduces the exact bytebuffer surface the serializer uses — but with native BigInt doing the 64-bit math instead of a bundled Long class.

It implements precisely what the serializer calls and nothing more:

// Native BigInt powers the 64-bit paths — no bundled Long class
writeUint64(value) {
  this.view.setBigUint64(this.offset, BigInt(value), /* littleEndian */ true);
  this.offset += 8;
}
readUint64() { return Long.fromBigInt(this.view.getBigUint64(...)); }

writeVarint32(value) { /* LEB128, identical wire bytes */ }
writeVString(str)    { /* varint length prefix + UTF-8 bytes */ }

It also ships a minimal Long shim (isLong, fromString, fromNumber, shiftLeft/Right, and, or, toInt) so object_id.js and validation.js — which expected bytebuffer.Long — keep working untouched.

Keeping the wire format frozen

The whole point is that none of the operation definitions changed. serializer/src/operations.js and ChainTypes.js — the files that define the binary layout of every chain operation — stayed exactly as they were. Only the byte plumbing underneath them was replaced.

The proof is, again, the golden vector:

txHex = d2042e16000080009265010005616c69636503626f620d746573742d7065726d6c696e6b102700

That serialized transaction hex (a vote op with fixed inputs) reproduced byte-for-byte on the new bytebuffer-lite backend. Combined with the full signTransaction golden test, this confirms the serializer + signer pipeline is unchanged end-to-end.

Bonus: portable debug guards

The old serializer read process.env.npm_config__graphene_serializer_hex_dump directly for debug dumps — which throws on runtimes with no process. Those reads are now guarded with typeof process !== 'undefined', so the serializer never assumes a Node environment.

Where we are

After Phase 3, the two pillars of on-chain validity — signing and serialization — are both running on modern, dependency-light, edge-safe code, and both are locked down by byte-exact golden vectors.

Tomorrow: Phase 4 — transports and runtime globals: native fetch, lazy WebSockets, and dropping Bluebird for a tiny native-promise helper.

Links

• 🛠️ Code (fork): https://github.com/blazeapps007/steem-js (BlazeDevelopment branch)
• 📖 Documentation: https://blazeapps007.github.io/steem-js/

Support Secure Steem Development

If you value proactive engineering, UX polish, and performance optimizations for the STEEM ecosystem, please consider supporting my witness: blaze.apps

🗳️ Vote Here:
Vote for blaze.apps Witness