Capy-Bearer Documentation

Description

brb_encode serializes an any value as BRRB version 2, also called BRRB2. Use brb_decode to restore the value.

BRRB is a tree format for maps, lists, strings, finite numbers, booleans, and none. It has no schema, compression, checksum, encryption, or object-reference table.

Document layout

All hexadecimal byte sequences below use spaces for readability. The spaces are not part of the document.

Offset  Size in bytes  Field    Value
------  -------------  -------  -----------------------------------
0       4              Magic    ASCII BRRB, hex 42 52 52 42
4       1              Version  02
5       Variable       Root     Exactly one node, as specified below

The root can have any public node type. There is no document-length field, terminator, padding, or alignment. A decoder must consume the whole input after the root. Concatenated documents are not one valid document.

For a byte stream with several messages, add framing outside BRRB. For example, send a message length followed by that many document bytes. Both endpoints must agree on the outer framing.

Unsigned variable-length integers

Scalar lengths, child counts, and key lengths use unsigned base-128 variable-length integers, also called unsigned LEB128. They do not use a fixed byte order or a signed integer transform.

Each byte contributes its low seven bits. The first byte contains the least significant group. Bit 7 means that another byte follows.

To encode a value, emit its low seven bits. Set bit 7 if more bits remain, then shift the value right by seven. Repeat until no bits remain.

Decimal value         Hex bytes
--------------------  -----------------------------
0                     00
1                     01
127                   7f
128                   80 01
255                   ff 01
16384                 80 80 01
18446744073709551615  ff ff ff ff ff ff ff ff ff 01

The range is 0 through 2^64 - 1. Use the shortest encoding. A multi-byte encoding cannot end with a zero payload group. Thus 80 00 is not a valid encoding of zero.

A decoder rejects truncated values, more than ten bytes, and overflow. In a ten-byte value, the last byte must be 01.

Node layout

Every node has the following fields, in this order. Scalar nodes also carry the child-count field, which must be zero.

Field           Size in bytes            Meaning
--------------  -----------------------  --------------------------------------
Flags           1                        00 normally, 01 for a list
Type            1                        One ASCII type tag from the table below
Scalar length   Variable-length integer  Number of scalar payload bytes
Scalar payload  Scalar length            Raw bytes, without a terminator
Child count     Variable-length integer  Number of immediate child entries
Child entries   Variable                 Exactly child-count entries, in sequence

The child count does not include descendants of those children.

Each child entry contains a key followed by a complete node:

Field       Size in bytes            Meaning
----------  -----------------------  ----------------------------------
Key length  Variable-length integer  Number of key bytes
Key         Key length               Raw key bytes, without a terminator
Child node  Variable                 The same node layout, recursively

There is no child-node length field. Parse a node's children to find the next sibling. All lengths count bytes, not Unicode characters.

Public type tags

Tag  Hex  Meaning      Encoder scalar payload         Children
---  ---  -----------  -----------------------------  ------------
M    4d   Map or list  Empty                          Zero or more
S    53   String       Exact string bytes             Zero
I    49   s64          Signed decimal integer         Zero
U    55   u64          Unsigned decimal integer       Zero
F    46   f64          Text representation of an f64  Zero
B    42   Boolean      ASCII 0 or 1                   Zero
N    4e   none         Empty                          Zero

The type tag distinguishes an empty string, an empty map, an empty list, and none. They are different values even when their payloads are empty.

Strings and keys

Strings and keys have no escape sequences or required character encoding. Text applications normally use UTF-8, but BRRB also preserves arbitrary bytes, including zero bytes. An empty key is valid.

Do not apply JSON escaping, URL encoding, or a trailing zero byte to a payload. For example, a three-byte string 41 00 42 has scalar length 03.

Numbers and Capy integer types

An F payload contains text, not binary IEEE 754 bits. The encoder uses general floating-point formatting with max_digits10 precision, which is 17 significant digits for f64. Exponent notation can occur. For example, 1.5 has payload 31 2e 35.

The decoder requires a finite result and must consume the whole payload. The current decoder uses strtod. It accepts more numeric spellings than the encoder emits, including leading whitespace and hexadecimal floating-point text. Trailing whitespace or zero bytes cause rejection.

Use ordinary finite decimal text for interoperability. Do not depend on a particular exponent spelling or treat the encoded bytes as a canonical numeric representation. The encoder can emit non-finite floating-point text, but brb_decode rejects it. NaN and infinity do not round-trip.

I preserves the full signed 64-bit range. U preserves the full unsigned 64-bit range. Each payload contains decimal digits, with an optional leading minus sign for I. The decoder requires at least one digit and complete input consumption. It rejects overflow, plus signs, whitespace, decimal points, exponents, and negative U values.

Capy boxes signed integer types as s64 and unsigned integer types as u64. Their tags are I and U. Floating-point types use F. Strings always use S, even when their bytes contain decimal digits. Copies, cloning, and public or local BRRB preserve these numeric kinds and values.

JSON encoding emits I and U as numeric tokens. Standard JSON decoding preserves integer-form tokens within signed and unsigned 64-bit ranges. Decimal, exponent, and out-of-range integer tokens use f64 and can round. Use BRRB when exact 64-bit integer values and kinds must survive a boundary.

Both endpoints need support for I and U. Earlier decoders reject these tags. Rebuild the runtime and core together before exchanging the corrected numeric values.

Booleans and none

The encoder emits only 0 and 1 for booleans. The decoder also accepts false, true, (false), and (true). These are case-sensitive ASCII strings.

A public N node must have flags 00, scalar length zero, and child count zero. It represents none, including a missing value that the caller passes to the encoder.

Maps and lists

Maps and lists both use M. Bit 0 of the flags byte distinguishes a list. The encoder writes 00 for a map and 01 for a list. Bits 1 through 7 are reserved. Write them as zero.

A map has an empty scalar payload. Its children have explicit string keys. The encoder writes map entries in insertion order.

A list also has an empty scalar payload. It still stores a key for every child. The encoder writes keys 0, 1, through child_count - 1 as ASCII decimal strings, in numeric order. Keys 10 and above follow 9, not 1.

For interoperable lists, use consecutive keys without signs or leading zeros. The decoder inserts children by key. A missing or out-of-order new index can turn the result into a map. The list flag does not repair malformed indexes.

The decoder accepts map entries in any order. If a key repeats, the last child replaces the earlier child. The parser still counts every child node toward its node limit.

Decoder acceptance and limits

The current decoder has these limits:

• At most 262144 nodes in the document, including the root and all scalar nodes.

• At most 64 node levels. The root has depth zero, so the deepest allowed node has depth 63.

• Every length and count must use a valid, shortest unsigned variable-length integer.

• Each payload must fit within the remaining input.

• The root must end exactly at the end of the document.

There is no separate BRRB document-byte limit. Available memory and the caller's transport or storage limits still apply. Set a byte limit before accepting untrusted documents.

The decoder rejects unknown type tags, unsupported versions, malformed numbers or booleans, scalar children, and a list flag on a scalar node. It rejects every nonzero flag on N.

The decoder also accepts forms that the encoder never emits. The current decoder ignores reserved flag bits on M, S, I, U, F, and B. It also ignores a nonempty scalar payload on M. Do not emit these forms. Acceptance does not make them supported extensions.

The Capy brb_decode function returns a result. Invalid input has ok: false and error. Valid data has ok: true and the decoded value, including none.

The encoder does not enforce the decoder's node or nesting limits. Keep encoded trees within these limits when the receiver uses brb_decode. Do not encode cyclic values. The encoder recursively follows every child and has no cycle check.

Value sharing and unsupported values

BRRB stores values, not shared object identities. If two map entries refer to one any object, the encoder writes the object twice. After decoding, those entries contain separate objects. Changes to one do not change the other.

Public brb_encode converts a callable to N. It does not serialize executable code. Native raw-pointer values become empty S nodes. These values cannot round-trip through the public format.

The runtime also has a private local variant with tag C, hex 43, for callable handles. Its flags and child count are zero. Its eight-byte scalar contains two little-endian unsigned 32-bit fields, a local closure pointer followed by a callable type identifier.

Public brb_decode rejects C. These fields have meaning only within the originating runtime workspace. Do not use private local documents for files, network messages, or another process.

Complete byte examples

Each row is a complete document, including the five-byte header.

Value                  Hex bytes
---------------------  -----------------------------------
none                   42 52 52 42 02 00 4e 00 00
Empty string           42 52 52 42 02 00 53 00 00
Empty map              42 52 52 42 02 00 4d 00 00
Empty list             42 52 52 42 02 01 4d 00 00
false                  42 52 52 42 02 00 42 01 30 00
true                   42 52 52 42 02 00 42 01 31 00
Number 1.5             42 52 52 42 02 00 46 03 31 2e 35 00
String Ada             42 52 52 42 02 00 53 03 41 64 61 00
String bytes 41 00 42  42 52 52 42 02 00 53 03 41 00 42 00

For {"name": "Ada"}, the document is 21 bytes:

42 52 52 42 02       magic and version
00 4d 00 01          map, empty scalar, one child
04 6e 61 6d 65       key length 4, key "name"
00 53 03 41 64 61 00 string "Ada", no children

For a list with true and none, the document is 22 bytes:

42 52 52 42 02       magic and version
01 4d 00 02          list, empty scalar, two children
01 30                key length 1, key "0"
00 42 01 31 00       boolean true, no children
01 31                key length 1, key "1"
00 4e 00 00          none, no children

For a string of 128 ASCII x bytes, the scalar length needs two bytes:

42 52 52 42 02       magic and version
00 53 80 01          string, scalar length 128
78 ... 78           exactly 128 payload bytes
00                   no children

This document is 138 bytes. The ellipsis is explanatory text, not wire data.