Adapters

Note

rlmesh.adapters is experimental: it may change or disappear. Pin versions; see Compatibility.

rlmesh.adapters derives the preprocessing and postprocessing between an environment and a model from declarative descriptions, instead of a hand-written adapter per pair.

The split is asymmetric. An environment tags its observation and action spaces: it names the semantic role of each entry plus the few facts the spaces cannot carry (image layout, rotation encoding, an explicit value range). A model fully specifies the payload it ingests and the action it emits. resolve() matches the two by role and produces an Adapter; widths, dtypes, and keys come from the gymnasium spaces. See Adapters for a guided walkthrough.

Install it with the NumPy backend:

pip install "rlmesh[numpy]"

Note

Adapters are entirely opt-in: the core Gymnasium loop never imports this package. Resolution and plan application run in the native rlmesh-adapters core; this package keeps the host-language half: spec construction and serialization, entrypoint-trust gating, custom callables, and the custom-adapter base class.

Resolution

rlmesh.adapters.resolve(env_tags, observation_space, action_space, model_spec, *, trust_entrypoints=False, check_inverse=True)

Derive an Adapter for an env/model pair.

Parameters:

• env_tags (EnvTags) – The environment’s observation/action tags.

• observation_space (object) – The environment’s gymnasium observation space (or any space object RLMesh can parse, e.g. an rlmesh.spaces space or a native SpaceSpec).

• action_space (object) – The environment’s gymnasium action space.

• model_spec (ModelSpec) – The model’s declared input/output format.

• trust_entrypoints (bool) – Allow module:callable strings in custom inputs to be imported. Leave False for specs from untrusted sources; in-process callables are always allowed.

• check_inverse (bool) – Round-trip each two-armed CustomEncoding on a probe at resolve time to catch a mispaired encode/decode. Set False for an intentionally non-invertible encoding.

Returns:

Adapter applying observation preprocessing and action postprocessing.

Raises:

AdapterResolutionError – If a model input or action component has no usable counterpart in the env tags and spaces.

Return type:

Adapter

rlmesh.adapters.resolve_from_contract(contract, model_spec, *, trust_entrypoints=False, check_inverse=True)

Derive an Adapter from an env contract and a model spec.

Reads the env’s tags from its contract metadata (published under ENV_METADATA_KEY by a server set up with rlmesh.adapters.tag()) and its observation/action spaces from the contract, then resolves as in resolve().

Parameters:

• contract (EnvContract) – The environment contract (e.g. remote_env.env_contract).

• model_spec (ModelSpec) – The model’s declared input/output format.

• trust_entrypoints (bool) – See resolve().

• check_inverse (bool) – See resolve().

Raises:

AdapterResolutionError – If the contract carries no tags, or resolution fails.

Return type:

Adapter

rlmesh.adapters.tag(env, tags, *, validate=True)

Merge env tags into env.metadata (under ENV_METADATA_KEY) and return it.

With validate=True (default), check the tags against the env’s observation/action spaces via the native join, raising AdapterResolutionError if a role or width cannot be reconciled.

Parameters:

• env (EnvT)

• tags (EnvTags)

• validate (bool)

Return type:

EnvT

Environment Tags

An environment publishes EnvTags in its contract metadata (via tag() or EnvServer(env, tags=...)), so a client can resolve an adapter from the handshake alone.

class rlmesh.adapters.EnvTags

Bases: object

Declarative tags of an environment’s observation and action.

observation is either a mapping from observation path to its tag (dotted paths traverse nested Dict spaces), or – when the observation is a single leaf – a bare tag/layout, mirroring action being one layout. A bare tag is normalized to {".": tag}, where "." is the reserved path for the flat/root observation, so the stored observation is always a mapping.

observation

Observation tags keyed by observation path (always a mapping after construction; a bare tag is normalized to ".").

Type:

collections.abc.Mapping[str, rlmesh.adapters.specs.env_tags.ImageTag | rlmesh.adapters.specs.env_tags.StateTag | rlmesh.adapters.specs.env_tags.StateLayout | rlmesh.adapters.specs.env_tags.TextTag]

action

Layout of the action vector accepted by step.

Type:

rlmesh.adapters.specs.action.ActionLayout

class rlmesh.adapters.ImageTag

Bases: object

A camera image entry in an environment observation.

role

Semantic role used for matching, e.g. image/primary.

Type:

str

layout

Axis layout of the stored image.

Type:

Literal[‘hwc’, ‘chw’]

upside_down

Whether the image is rendered rotated 180 degrees relative to the canonical upright orientation.

Type:

bool

class rlmesh.adapters.StateTag

Bases: object

A numeric proprioception entry in an environment observation.

role

Semantic role used for matching, e.g. proprio/eef_pos.

Type:

str

encoding

Rotation encoding when the role is a rotation.

Type:

Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’] | None

range

Optional (low, high) value range, supplying the bounds where the space leaves this leaf unbounded. If the space declares finite bounds that disagree with it, resolution errors rather than silently overriding them.

Type:

tuple[float, float] | None

Tip

For a flat numeric leaf whose fixed index ranges carry distinct meaning, tag it with a StateLayout of StateField slices instead of a StateTag.

class rlmesh.adapters.StateLayout

Bases: object

An ordered split of one flat numeric observation leaf into role fields.

The observation-side mirror of ActionLayout: fields are laid out in order, offsets accumulate, and the native join requires the field widths to sum to the leaf width. Use it when an env returns a flat Box whose fixed index ranges carry distinct semantics (e.g. Metaworld):

StateLayout(StateField(EEF_POS, 3), StateField(GRIPPER, 1))

fields

State fields in vector order.

Type:

tuple[rlmesh.adapters.specs.env_tags.StateField, …]

class rlmesh.adapters.StateField

Bases: object

One contiguous field of a flat numeric observation leaf.

The observation-side mirror of ActionComponent: a slice of dim elements carrying a role, with offsets implied by order within a StateLayout. A field with no role is a skip – it advances the offset but produces no feature, used to step over elements the model never consumes.

role

Semantic role matched against model state components, or None to skip this slice.

Type:

str | None

dim

Number of elements this field occupies.

Type:

int

encoding

Rotation encoding when the field is a rotation.

Type:

Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’] | None

range

Optional (low, high) value range for this field’s slice, supplying the bounds where the space leaves it unbounded. If the space declares finite bounds for the slice that disagree with it, resolution errors rather than silently overriding them.

Type:

tuple[float, float] | None

class rlmesh.adapters.TextTag

Bases: object

A text entry (typically the task instruction) in an observation.

role

Semantic role used for matching.

Type:

str

Model Spec

class rlmesh.adapters.ModelSpec

Bases: object

Declarative description of a model’s input payload and action output.

inputs

Input features keyed into the model payload dict.

Type:

tuple[rlmesh.adapters.specs.model_inputs.ImageInput | rlmesh.adapters.specs.model_inputs.StateInput | rlmesh.adapters.specs.model_inputs.TextInput | rlmesh.adapters.specs.model_inputs.InlineCustomInput | rlmesh.adapters.specs.model_inputs.EntrypointCustomInput, …]

action

Layout of the action vector produced by the model.

Type:

rlmesh.adapters.specs.action.ActionLayout

class rlmesh.adapters.ImageInput

Bases: object

An image input expected by a model.

key

Key of the entry in the model input payload.

Type:

str

role

Semantic role matched against env image features.

Type:

str

height

Target image height, or None to keep the env height.

Type:

int | None

width

Target image width, or None to keep the env width.

Type:

int | None

layout

Axis layout the model expects.

Type:

Literal[‘hwc’, ‘chw’]

dtype

NumPy dtype name the model expects.

Type:

str

normalize

Scale 8-bit pixel values into [0, 1] before casting.

Type:

bool

lead_dims

Number of leading singleton axes to add (batch/time).

Type:

int

upside_down

Whether the model was trained on images rotated 180 degrees relative to the canonical upright orientation.

Type:

bool

resample

Resize algorithm the model’s training pipeline used: "bilinear_aa" (antialiased triangle filter, PIL-compatible) or "bilinear" (4-tap half-pixel-center bilinear, OpenCV/torch-compatible).

Type:

str

stack

Number of consecutive observations to stack on a new leading axis (frame history). 1 (default) means no stacking. Stacking is applied host-side by the adapter, which buffers processed frames (padding with the first frame at the start of an episode) and clears them on reset – the env still sends one frame per step.

Type:

int

size

Convenience for square targets – sets both height and width. Pass size or height/width, not both.

Type:

dataclasses.InitVar[int | None]

class rlmesh.adapters.StateInput

Bases: object

A numeric state input expected by a model.

key

Key of the entry in the model input payload.

Type:

str

components

Pieces concatenated (in order) to form the value.

Type:

tuple[rlmesh.adapters.specs.model_inputs.StateComponent, …]

pad_to

Zero-pad the concatenated vector to this length.

Type:

int | None

dtype

NumPy dtype name of the resulting value.

Type:

str

reshape

Optional target shape for the resulting value.

Type:

tuple[int, …] | None

container

Emit a NumPy array or a plain Python list.

Type:

Literal[‘array’, ‘list’]

For a single-piece state, pass role (and optionally encoding / dim / index) instead of components – e.g. StateInput("state", role=EEF_POS) is shorthand for StateInput("state", components=(StateComponent(EEF_POS),)).

class rlmesh.adapters.StateComponent

Bases: object

One piece of a model state vector, sourced from an env state feature.

role

Semantic role matched against env state features.

Type:

str

encoding

Rotation encoding the model expects for this piece.

Type:

Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’] | rlmesh.adapters.specs.custom_encoding.CustomEncoding | None

dim

Optional number of leading elements to keep from the source.

Type:

int | None

index

Optional single element to select after any conversion.

Type:

int | None

optional

Zero-fill this piece when the env does not declare the role, instead of failing resolution. The fill width comes from index (one), dim, or encoding; one of them must be set so the width is known without an env feature.

Type:

bool

range

Optional (low, high) the model expects this piece in; when the env declares its own range, the value is affinely mapped from the env range to this one (symmetric to action ranges).

Type:

tuple[float, float] | None

class rlmesh.adapters.TextInput

Bases: object

A text input expected by a model.

key

Key of the entry in the model input payload.

Type:

str

role

Semantic role matched against env text features.

Type:

str

container

Emit a plain string or a single-element list.

Type:

Literal[‘str’, ‘list’]

default

Value used when the observation omits the feature; when None the key is omitted from the payload instead.

Type:

str | None

Action Layout

The action layout is a shared vocabulary. An environment tags the action vector its step accepts; a model declares the action vector it emits. The resolver converts between them per component.

class rlmesh.adapters.ActionLayout

Bases: object

Ordered action components plus optional clipping bounds.

Components are passed positionally, mirroring the observation-side StateLayout:

ActionLayout(ActionComponent(DELTA_POS, 3), ActionComponent(GRIPPER, 1))

components

Action components in vector order.

Type:

tuple[rlmesh.adapters.specs.action.ActionComponent, …]

clip

Optional (low, high) clip applied to the final vector.

Type:

tuple[float, float] | None

class rlmesh.adapters.ActionComponent

Bases: object

One contiguous slice of an action vector.

role

Semantic role used for matching, e.g. action/gripper.

Type:

str

dim

Number of action dimensions occupied by this component.

Type:

int

encoding

Rotation encoding when the component is a rotation.

Type:

Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’] | rlmesh.adapters.specs.custom_encoding.CustomEncoding | None

range

Optional (low, high) range of the component values.

Type:

tuple[float, float] | None

scale

Optional multiplier applied to the model value for this role.

Type:

float | None

invert

Negate the model value for this role (equivalent to scale=-1 but explicit; the common gripper-sign correction).

Type:

bool

threshold

Subtract this from the value, recentering the decision boundary – typically paired with binary so the snap splits at threshold instead of zero.

Type:

float | None

binary

Whether the component encodes a binary decision (resolved adapters snap the value to a definite side after range mapping: >= 0 opens (+1), below closes (-1); a value exactly on the boundary opens rather than emitting an undefined 0).

Type:

bool

scale, invert, and threshold are env-side corrections: they declare the env actuator’s convention and are applied to the incoming model value after the declared formats (rotation, range) are bridged, in the order scale, invert, threshold, then binary. Declared once on the env, every model evaluated against it inherits the correction.

Escape Hatches

When a pairing needs logic a declarative spec cannot express, three mechanisms compose, most local first. A custom input computes one payload key from the raw observation while the rest stays spec-driven. A custom encoding handles a rotation convention the native crate does not ship. A custom adapter subclasses AdapterBase to add stateful behavior, typically by wrapping a resolved adapter and overriding only the stateful part.

Custom inputs

InlineCustomInput runs an in-process callable that maps the raw observation to one payload key; it is local only. EntrypointCustomInput names a module:callable string that is imported only when you pass resolve(..., trust_entrypoints=True), so it can travel in a contract. A custom input receives the environment’s own keys, not roles, and returns the entire payload key, so it does no role-matching, dim/index, or range-mapping, and is observation-side only.

class rlmesh.adapters.InlineCustomInput

Bases: object

A custom input computed in-process by a user callable.

Local only: the callable cannot be serialized, so a model spec carrying an InlineCustomInput cannot be published in contract metadata. Use EntrypointCustomInput for a spec that must travel.

key

Key of the entry in the model input payload.

Type:

str

transform

Callable taking the raw observation mapping.

Type:

collections.abc.Callable[[collections.abc.Mapping[str, Any]], Any]

class rlmesh.adapters.EntrypointCustomInput

Bases: object

A custom input computed by a module:callable entrypoint.

Serializable and publishable. The entrypoint is imported only when resolve(..., trust_entrypoints=True); otherwise resolution refuses to import it.

key

Key of the entry in the model input payload.

Type:

str

entrypoint

A module:callable string.

Type:

str

Custom encodings

Rotation encodings are a closed vocabulary (the RotationEncoding set listed under Vocabulary). You cannot register one from Python, because a spec is data that travels in a contract and resolves on a remote client with no code. For a convention that is general and stable, like a published model’s rot6d_rowmajor, add it first-party: a few lines on the native RotationEncoding enum plus the Python Literal. It then works on both the observation and action sides, serializes into the contract, and is conformance-tested once.

For a bespoke or proprietary convention, declare a CustomEncoding on the nearest native base encoding (rot6d or a quaternion) and supply the host-side repacking. resolve lowers the field to its base for the native core, so role-matching, range-mapping, and the env-to-base conversion are unchanged; the adapter applies your transforms at the boundary: from_base after the native conversion on the observation side, to_base before it on the action side. Define the encoding once and reference it from both arms:

ROT6D_MINE = adapt.CustomEncoding(
    base="rot6d", from_base=rot6d_to_mine, to_base=mine_to_rot6d, name="rot6d_mine"
)

The packing must preserve the base width, and an observation custom encoding must be the sole component of a single-piece StateInput (the offset of a field interior to a multi-piece state is env-dependent). At resolve time the two arms are round-tripped on a probe to catch a mispaired encode/decode; pass resolve(..., check_inverse=False) to skip. The transforms are in-process callables, so the spec is local; a serializable module:callable form is planned.

When the constraints do not fit (a width-changing repack, a rotation interior to a multi-field state, or non-rotation feature engineering), drop to a custom AdapterBase or replace a whole payload key with an InlineCustomInput. What none of these do is attach a custom encoding to a role in the spec itself: the vocabulary stays closed so specs remain pure data that resolve on a remote client with no code. Reach for the boundary wrapper for a one-off; upstream the encoding once you want it attached to a role and reused.

Custom adapters

Subclass AdapterBase for stateful behavior a spec cannot describe (for example temporal ensembling across action chunks, or a width-changing rotation repack interior to a multi-field state). The usual shape wraps a resolved adapter and overrides only the stateful part. Override reset() to clear episode state and wire it to the model worker’s on_reset.

A pair override replaces the adapter for one specific (model, environment) pairing entirely, for cases like control-space conversion against a robot’s kinematic model. There is no special machinery: keep a registry keyed by the pair and consult it before resolving.

class rlmesh.adapters.AdapterBase

Bases: ABC, Generic[ActionT]

Base class for env-to-model adapters.

rlmesh.adapters.resolve() derives the spec-driven implementation (Adapter); subclass this directly to plug a fully custom pairing into anything built around adapters instead. Implement the two transforms; wrap_predict comes for free. Custom adapters may hold state across steps (e.g. cache proprio from transform_obs for use in transform_action) – that is their power over declarative specs. Override reset() to clear any such state at episode boundaries.

abstractmethod transform_obs(raw_obs)

Convert a raw env observation into the model input payload.

raw_obs is a mapping for a Dict space, or a bare array/leaf for a flat (non-Dict) space.

Parameters:

raw_obs (Mapping[str, Any] | object)

Return type:

dict[str, Any]

abstractmethod transform_action(raw_action)

Convert a model action output into the env action.

Parameters:

raw_action (object)

Return type:

ActionT

reset(env_index=None)

Clear episode-scoped state, optionally for a single lane.

env_index identifies the vector lane whose episode rolled, or None for a whole-vector reset. The default does nothing (resolved adapters are stateless). Stateful custom adapters override this and wire it to the model worker’s per-lane reset so a single lane’s autoreset never wipes the other still-running lanes’ state.

Parameters:

env_index (int | None)

Return type:

None

property is_stateful: bool

Whether the adapter carries per-stream state across steps.

A stateful adapter must keep affinity to its lane (one instance per (session, route, env_index)) and so cannot yet be shared across the lanes of a vector env. Custom adapters default to stateful (the safe assumption); Adapter derives this from its frame history. The per-lane affinity manager that makes vectorized stateful adapters correct is not implemented yet.

describe()

Return a human-readable summary of the adapter.

Return type:

str

wrap_predict(predict_fn)

Wrap a model predict function with both transforms.

The returned callable takes a raw env observation – a mapping, or a bare array/leaf for a flat (non-Dict) env – and returns an env-ready action, suitable for rlmesh.numpy.Model.

Parameters:

predict_fn (Callable[[dict[str, Any]], object])

Return type:

Callable[[Any], ActionT]

Adapter Objects

final class rlmesh.adapters.Adapter

Bases: AdapterBase[object]

A resolved env-to-model adapter; build instances with resolve().

Declarative plans run in the native rlmesh-adapters core; custom inputs run their host-language transforms on the raw Python observation, exactly as before.

Errors

exception rlmesh.adapters.AdapterResolutionError

Bases: ValueError

Raised when env tags and a model spec cannot be reconciled.

__init__(*args, **kwargs)

classmethod __new__(*args, **kwargs)

Vocabulary

Semantic roles are an open vocabulary of wire strings matched verbatim between independently authored tags and specs. The well-known conventions that ship with RLMesh are re-exported from the package (single-sourced from the native crate): the domain-agnostic roles IMAGE_PRIMARY, IMAGE_SECONDARY, INSTRUCTION, JOINT_POS, JOINT_VEL; the arm-manipulation observation roles IMAGE_WRIST, EEF_POS, EEF_ROT, GRIPPER_POS; and the action roles ACTION_DELTA_POS, ACTION_DELTA_ROT, ACTION_GRIPPER. Bimanual _2 variants exist for the per-arm roles EEF_POS, EEF_ROT, GRIPPER_POS, ACTION_DELTA_POS, ACTION_DELTA_ROT, and ACTION_GRIPPER.

Rotation widths follow the declared encoding. rlmesh.adapters.ROTATION_DIMS maps each encoding to its dimension count:

Encoding	Dims	Convention
quat_xyzw	4	quaternion, scalar-last
quat_wxyz	4	quaternion, scalar-first
axis_angle	3	rotation vector
rot6d	6	first two columns of the rotation matrix, concatenated
rot6d_rowmajor	6	same two columns flattened row-major
euler_xyz	3	roll-pitch-yaw, extrinsic XYZ

rot6d is the standard 6D rotation; rot6d_rowmajor exists for checkpoints trained on the row-major interleaving. See Adapters for when to add an encoding versus reach for a custom encoding.