Adapters¶
Note
This is the autodoc API reference. For the guided tour see Adapters; for the full role registry, every leaf option, and the conversion policy see Adapter Reference.
rlmesh.adapters derives the preprocessing and postprocessing between an environment and a model from declarative descriptions, instead of a hand-written adapter per pair.
The split is asymmetric. An environment tags its observation and action spaces: it names the semantic role of each entry plus the few facts the spaces cannot carry (image layout, rotation encoding, an explicit value range). A model fully specifies the payload it ingests and the action it emits. resolve() matches the two by role and produces an Adapter; widths, dtypes, and keys come from the gymnasium spaces. See Adapters for a guided walkthrough.
Install the NumPy backend for direct adapter calls and the examples below:
pip install "rlmesh[numpy]"
Note
Adapters are entirely opt-in: the core Gymnasium loop never imports this package. Resolution and
plan application run in the native rlmesh-adapters core; this package keeps the host-language half:
spec construction and serialization, entrypoint-trust gating, custom callables, and the
custom-adapter base class.
Resolution¶
- rlmesh.adapters.resolve(env_tags, observation_space, action_space, model_spec, *, trust_entrypoints=False, check_inverse=True)[source]¶
Derive an
Adapterfor an env/model pair.- Parameters:
env_tags (EnvTags) – The environment’s observation/action tags.
observation_space (object) – The environment’s gymnasium observation space (or any space object RLMesh can parse, e.g. an
rlmesh.spacesspace or a nativeSpaceSpec).action_space (object) – The environment’s gymnasium action space.
model_spec (ModelSpec) – The model’s declared input/output format.
trust_entrypoints (bool) – Allow
module:callablestrings in custom inputs to be imported. Leave False for specs from untrusted sources; in-process callables are always allowed.check_inverse (bool) – Round-trip each two-armed
CustomEncodingon a probe at resolve time to catch a mispaired encode/decode. Set False for an intentionally non-invertible encoding.
- Returns:
Adapter applying observation preprocessing and action postprocessing.
- Raises:
AdapterResolutionError – If a model input or action component has no usable counterpart in the env tags and spaces.
- Return type:
- rlmesh.adapters.resolve_from_contract(contract, model_spec, *, trust_entrypoints=False, check_inverse=True)[source]¶
Derive an
Adapterfrom an env contract and a model spec.Reads the env’s tags from its contract metadata (published under
ENV_METADATA_KEYby a server set up withrlmesh.adapters.tag()) and its observation/action spaces from the contract, then resolves as inresolve().- Parameters:
contract (EnvContract) – The environment contract (e.g.
remote_env.env_contract).model_spec (ModelSpec) – The model’s declared input/output format.
trust_entrypoints (bool) – See
resolve().check_inverse (bool) – See
resolve().
- Raises:
AdapterResolutionError – If the contract carries no tags, or resolution fails.
- Return type:
- rlmesh.adapters.tag(env, tags, *, validate=True)[source]¶
Merge env tags into
env.metadata(underENV_METADATA_KEY) and return it.With
validate=True(default), check the tags against the env’s observation/action spaces via the nativejoin, raisingAdapterResolutionErrorif a role or width cannot be reconciled.- Parameters:
env (EnvT)
tags (EnvTags)
validate (bool)
- Return type:
EnvT
Model Spec¶
- class rlmesh.adapters.ModelSpec[source]¶
Bases:
objectDeclarative description of a model’s input payload tree and action output.
inputis a recursive tree whose container type is the payload container the model’spredictreceives: a bare leaf (a single tensor/string), adict[str, subtree], or atupleof subtrees. A leaf is anImage,State,Concat,Text, orCustom. Placement (tree position) is the payload position – model leaves carry nokey, and a role may be reused across leaves (one env camera can feed several input slots).- input¶
The model input tree.
- Type:
InputNode
- class rlmesh.adapters.Image[source]¶
Bases:
objectAn image input expected by a model.
There is no
key– placement in the input tree is the payload position.- role¶
Semantic role matched against env image features.
- Type:
str
- height¶
Target image height, or None to keep the env height.
- Type:
int | None
- width¶
Target image width, or None to keep the env width.
- Type:
int | None
- layout¶
Axis layout the model expects (
"hwc"the default, or"chw"). This is the model author’s declaration: when it differs from the env’s layout the adapter transposes silently to match, so a model that wants"chw"must say so – an omittedlayoutmeans"hwc"and the env’s frame is fed through unchanged. (channelsguards the channel count, not the axis order.)- Type:
Literal[‘hwc’, ‘chw’]
- channels¶
Channel count the model expects (e.g. 3 for RGB, 1 for grayscale). When set, a resolve error if the env image differs; the adapter does not convert between channel counts.
- Type:
int | None
- dtype¶
NumPy dtype name the model expects.
- Type:
str
- normalize¶
Whether (and into what range) to map 8-bit pixel values before casting:
False(off, the default),True(the conventional[0, 1]), or a(low, high)pair for a model trained on a different range (e.g.(-1.0, 1.0)). One field, so an on/off flag can never disagree with a range;Falseis an authoritative off-switch.- Type:
bool | tuple[float, float]
- lead_dims¶
Number of leading singleton axes to add (batch/time).
- Type:
int
- upside_down¶
Whether the model was trained on images rotated 180 degrees from the canonical upright orientation (a true rotation – rows and columns reversed – not a vertical flip). Declared on both ends: the adapter rotates only when the env and model disagree, so an env that already renders upside-down pairs with a model that also sets
upside_downand no rotation happens.- Type:
bool
- resample¶
Resize algorithm the model’s training pipeline used:
"bilinear"(4-tap half-pixel-center bilinear, OpenCV/torch- compatible; the default, which most trained policies match) or"bilinear_aa"(antialiased triangle filter, PIL-compatible).- Type:
str
- allow_upscale¶
Permit a target larger than the env’s native resolution (interpolating detail that is not there). Off by default: an upscaling target is a resolve error unless this is set.
- Type:
bool
- fit¶
How to reconcile a target whose aspect ratio differs from the env image:
"stretch"(distort),"crop"(cover + center-crop), or"pad"(letterbox) – or a sequence of them in preference order. The resolver picks, per env, the first that does not need a disallowed upscale, so one spec can crop a large camera and letterbox a small one. Required only on an aspect mismatch; absent it, an aspect-changing resize is a resolve error.- Type:
Literal[‘stretch’, ‘crop’, ‘pad’] | collections.abc.Sequence[Literal[‘stretch’, ‘crop’, ‘pad’]] | None
- optional¶
Zero-fill a black frame when the env does not provide this camera, instead of failing resolution. Needs
height,width, andchannelsso the blank can be sized.- Type:
bool
- stack¶
Number of consecutive observations to stack on a new leading axis (frame history).
1(default) means no stacking. Stacking is applied by the adapter from an episode-keyed rolling buffer (padding with the first frame at the start of an episode, cleared onreset) – host-side on the local path, natively in the core on the served path. Either way the env still sends one frame per step.- Type:
int
- size¶
Convenience for square targets – sets both
heightandwidth. Passsizeorheight/width, not both.- Type:
dataclasses.InitVar[int | None]
- class rlmesh.adapters.State[source]¶
Bases:
objectA single-part numeric state input expected by a model.
The 1-part case: one role packed into the value, sourced from an env state feature. Use
Concatto pack several roles into one tensor. AStateis also a validConcatpart (its part fields –role,encoding,dim,index,optional,range– are taken; its container fields must stay default when used as a part).There is no
key– placement in the input tree is the payload position.- role¶
Semantic role matched against env state features.
- Type:
str
- encoding¶
Rotation encoding the model expects for this part. A single encoding, or a sequence of them in preference order (most-preferred first) — the resolver picks the env’s native encoding when it appears here (no conversion), else converts into the first entry. A
CustomEncodingis a single host-side packing (not a set).- Type:
Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’] | collections.abc.Sequence[Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’]] | rlmesh.adapters.specs.custom_encoding.CustomEncoding | None
- dim¶
Optional number of leading elements to keep from the source.
- Type:
int | None
- index¶
Optional single element to select after any conversion.
- Type:
int | None
- optional¶
Zero-fill this part when the env does not declare the role, instead of failing resolution. The fill width comes from
index(one),dim, orencoding; one of them must be set so the width is known without an env feature.- Type:
bool
- range¶
Optional
(low, high)the model expects this part in; when the env declares its own (derived or tagged) range, the value is affinely mapped from the env range to this one (symmetric to action ranges). With no env source range there is nothing to map from, so it is a no-op – it does not clamp or rescale on its own.- Type:
tuple[float, float] | None
- pad_to¶
Zero-pad the resulting vector to this length.
- Type:
int | None
- dtype¶
NumPy dtype name of the resulting value.
- Type:
str
- reshape¶
Optional target shape for the resulting value.
- Type:
tuple[int, …] | None
- container¶
Emit a NumPy array or a plain Python list.
- Type:
Literal[‘array’, ‘list’]
- class rlmesh.adapters.Concat[source]¶
Bases:
objectA multi-part numeric state input: several roles packed into one tensor.
The multi-part state leaf. Parts are concatenated in order; each part is a bare role string (sugar for a role-only
State) or aStatecarrying part fields. A single-role state isStatedirectly; this is the >1-part case. Both serialize to the same{"type": "state", ...}wire form.There is no
key– placement in the input tree is the payload position.- parts¶
Roles (or
Stateparts) concatenated in order.- Type:
tuple[str | rlmesh.adapters.specs.model_inputs.State, …]
- pad_to¶
Zero-pad the concatenated vector to this length.
- Type:
int | None
- dtype¶
NumPy dtype name of the resulting value.
- Type:
str
- reshape¶
Optional target shape for the resulting value.
- Type:
tuple[int, …] | None
- container¶
Emit a NumPy array or a plain Python list.
- Type:
Literal[‘array’, ‘list’]
- class rlmesh.adapters.Text[source]¶
Bases:
objectA text input expected by a model.
There is no
key– placement in the input tree is the payload position.- role¶
Semantic role matched against env text features.
- Type:
str
- container¶
Emit a plain string or a single-element list.
- Type:
Literal[‘str’, ‘list’]
- default¶
Value used when the observation omits the feature; when None the input is omitted from the payload instead.
- Type:
str | None
Action Layout¶
The action layout is a shared vocabulary. An environment tags the action vector its step accepts; a model declares the action vector it emits. The resolver converts between them per actuator.
- class rlmesh.adapters.Action[source]¶
Bases:
objectOrdered action actuators plus optional clipping bounds.
Actuators are passed positionally, mirroring the observation-side
Split:Action(Actuator(DELTA_POS, 3), Actuator(GRIPPER, 1))
- components¶
Action actuators in vector order.
- Type:
tuple[rlmesh.adapters.specs.action.Actuator, …]
- clip¶
Optional
(low, high)clip applied to the final vector.- Type:
tuple[float, float] | None
Action chunking is no longer a spec knob: the execution horizon is chosen by the runtime (
execution_horizononConfigureRoute), and a chunked policy declares apredict_chunkcorner rather than anexecute_horizon.
- class rlmesh.adapters.Actuator[source]¶
Bases:
objectOne contiguous slice of an action vector.
- role¶
Semantic role used for matching, e.g.
action/gripper.Nonemakes the actuator opaque: it occupiesdimdims of the action with the constantfill, matched by no model output – the action-side mirror of a role-lessField. Use it for dims the env requires but no model produces (e.g. a control-mode selector). An opaque actuator carries onlydimandfill.- Type:
str | None
- dim¶
Number of action dimensions occupied by this component.
- Type:
int
- fill¶
Constant emitted for each dim of an opaque (role-less) actuator, and the fallback for an
optionalroled actuator. Defaults to0.0; inert (must stay0.0) on a roled, non-optional actuator.- Type:
float
- optional¶
On a roled actuator, make the role optional – if no model output declares it, fill the actuator’s
dimdims withfillinstead of failing resolution (the action-side mirror of a model input’soptionalzero-fill). A model that does output the role drives it normally. Meaningless on a role-less actuator (already always filled).- Type:
bool
- encoding¶
Rotation encoding when the component is a rotation.
- Type:
Literal[‘quat_xyzw’, ‘quat_wxyz’, ‘axis_angle’, ‘rot6d’, ‘rot6d_rowmajor’, ‘euler_xyz’] | rlmesh.adapters.specs.custom_encoding.CustomEncoding | None
- range¶
Optional
(low, high)range of the component values.- Type:
tuple[float, float] | None
- scale¶
Optional multiplier applied to the model value for this role.
- Type:
float | None
- invert¶
Negate the model value for this role (equivalent to
scale=-1but explicit; the common gripper-sign correction).- Type:
bool
- threshold¶
Subtract this from the value, recentering the decision boundary – typically paired with
binaryso the snap splits atthresholdinstead of zero.- Type:
float | None
- binary¶
Whether the component encodes a binary decision (resolved adapters snap the value to a definite side after range mapping:
>= 0opens (+1), below closes (-1); a value exactly on the boundary opens rather than emitting an undefined0).- Type:
bool
- clip¶
Clamp this actuator’s mapped value to its declared
range. The per-component safety clamp the globalAction.clipcannot give a mixed-range action: a global clip applies one bound to every dim, so it is wrong when dims have different ranges (e.g. delta-pos in[-1, 1]but rotation in[-pi/2, pi/2]).clip=Truerequiresrange.- Type:
bool
scale,invert, andthresholddeclare a side’s actuator convention. They can be set on either side and compose as literal transforms applied after the declared formats (rotation, range) are bridged – model-side first (the model’s own output convention), then env-side – each in the order scale, invert, threshold, thenbinary. So an env declares its quirk once for every model to inherit, and a model whose output differs from a shared env it cannot edit declares the bridge on its own actuator (e.g. a gripper-sign flip asinvert=True).clipis the exception: it stays env-side only, clamping to the env actuator’srange.
Escape Hatches¶
When a pairing needs logic a declarative spec cannot express, three mechanisms compose, most local first. A custom input computes one payload slot from the raw observation while the rest stays spec-driven. A custom encoding handles a rotation convention the native crate does not ship. A custom adapter subclasses AdapterBase to add stateful behavior, typically by wrapping a resolved adapter and overriding only the stateful part.
Custom inputs¶
A Custom input runs host-language code that maps the raw observation to one payload slot. Custom(transform=fn) runs an in-process callable and is local only; Custom(entrypoint="module:callable") names a string that is imported only when you pass resolve(..., trust_entrypoints=True), so it can travel in a contract. A custom input receives the environment’s own keys, not roles, and returns the entire payload slot, so it does no role-matching, dim/index, or range-mapping, and is observation-side only.
- class rlmesh.adapters.Custom[source]¶
Bases:
objectA custom input computed by host-language code.
Exactly one of
transform(an in-process callable) orentrypoint(amodule:callablestring) is set:transform: local only – the callable cannot be serialized, so a model spec carrying it cannot be published in contract metadata.entrypoint: serializable and publishable as wire form, but imported only whenresolve(..., trust_entrypoints=True); otherwise resolution refuses to import it. Travels on the wire under the keytransform.
There is no
key– placement in the input tree is the payload position.- transform¶
An in-process callable taking the raw observation mapping.
- Type:
collections.abc.Callable[[collections.abc.Mapping[str, Any]], Any] | None
- entrypoint¶
A
module:callablestring.- Type:
str | None
Custom encodings¶
Rotation encodings are a closed vocabulary (the RotationEncoding set listed under Vocabulary). You cannot register one from Python, because a spec is data that travels in a contract and resolves on a remote client with no code. For a convention that is general and stable, like a published model’s rot6d_rowmajor, add it first-party: a few lines on the native RotationEncoding enum plus the Python Literal. It then works on both the observation and action sides, serializes into the contract, and is conformance-tested once.
For a bespoke or proprietary convention, declare a CustomEncoding on the nearest native base encoding (rot6d or a quaternion) and supply the host-side repacking. resolve lowers the field to its base for the native core, so role-matching, range-mapping, and the env-to-base conversion are unchanged; the adapter applies your transforms at the boundary: from_base after the native conversion on the observation side, to_base before it on the action side. Define the encoding once and reference it from both arms:
ROT6D_MINE = adapt.CustomEncoding(
base="rot6d", from_base=rot6d_to_mine, to_base=mine_to_rot6d, name="rot6d_mine"
)
The packing must preserve the base width, and an observation custom encoding must be the sole part of a single-part State (the offset of a field interior to a multi-part Concat is env-dependent). At resolve time the two arms are round-tripped on a probe to catch a mispaired encode/decode; pass resolve(..., check_inverse=False) to skip. The transforms are in-process callables, so the spec is local; a serializable module:callable form is planned.
When the constraints do not fit (a width-changing repack, a rotation interior to a multi-field state, or non-rotation feature engineering), drop to a custom AdapterBase or replace a whole payload slot with a Custom input. What none of these do is attach a custom encoding to a role in the spec itself: the vocabulary stays closed so specs remain pure data that resolve on a remote client with no code. Reach for the boundary wrapper for a one-off; upstream the encoding once you want it attached to a role and reused.
Custom adapters¶
Subclass AdapterBase for stateful behavior a spec cannot describe (for example temporal ensembling across action chunks, or a width-changing rotation repack interior to a multi-field state). The usual shape wraps a resolved adapter and overrides only the stateful part. Override reset() to clear episode state and wire it to the model worker’s on_episode_end.
A pair override replaces the adapter for one specific (model, environment) pairing entirely, for cases like control-space conversion against a robot’s kinematic model. There is no special machinery: keep a registry keyed by the pair and consult it before resolving.
- class rlmesh.adapters.AdapterBase[source]¶
Bases:
ABC,Generic[ActionT]Base class for env-to-model adapters.
rlmesh.adapters.resolve()derives the spec-driven implementation (Adapter); subclass this directly to plug a fully custom pairing into anything built around adapters instead. Implement the two transforms;wrap_predictcomes for free. Custom adapters may hold state across steps (e.g. cache proprio fromtransform_obsfor use intransform_action) – that is their power over declarative specs. Overridereset()to clear any such state at episode boundaries.- abstractmethod transform_obs(raw_obs)[source]¶
Convert a raw env observation into the model input payload.
raw_obsis a mapping for a Dict space, or a bare array/leaf for a flat (non-Dict) space. The return is the model input payload tree (a dict, a list, or a bare leaf – whatever the model spec’s input tree declares).- Parameters:
raw_obs (Mapping[str, Any] | object)
- Return type:
Any
- abstractmethod transform_action(raw_action)[source]¶
Convert a model action output into the env action.
- Parameters:
raw_action (object)
- Return type:
ActionT
- reset()[source]¶
Clear episode-scoped state at an episode boundary.
The base adapter holds no episode-scoped state; a stateful custom adapter overrides this to clear its per-episode state. It is driven on the single-env local loop, so there is no per-lane bookkeeping to do (the served path stacks natively with episode-keyed buffers in the core).
- Return type:
None
- wrap_predict(predict_fn)[source]¶
Wrap a model predict function with both transforms.
The returned callable takes a raw env observation – a mapping, or a bare array/leaf for a flat (non-Dict) env – and returns an env-ready action, suitable for
rlmesh.numpy.Model.Action chunking is no longer driven here: the execution horizon is a runtime decision (
execution_horizononConfigureRoute, owned by the runtime driver) and the served engine emits the chunk. This direct wrapper applies one action per step; in-process chunk replay lives inrlmesh._models._chunk.ChunkReplay, driven byrunwith a locally chosen horizon.- Parameters:
predict_fn (Callable[[dict[str, Any]], object])
- Return type:
Callable[[Any], ActionT]
Adapter Objects¶
- final class rlmesh.adapters.Adapter[source]¶
Bases:
AdapterBase[object]A resolved env-to-model adapter; build instances with
resolve().Declarative plans run in the native
rlmesh-adapterscore; custom inputs run their host-language transforms on the raw Python observation, exactly as before.
Errors¶
Vocabulary¶
Semantic roles are an open vocabulary of wire strings matched verbatim between independently authored tags and specs. The well-known conventions that ship with RLMesh are re-exported from the package (single-sourced from the native crate): the domain-agnostic roles IMAGE_PRIMARY, IMAGE_SECONDARY, INSTRUCTION, JOINT_POS, JOINT_VEL; the arm-manipulation observation roles IMAGE_WRIST, EEF_POS, EEF_ROT, GRIPPER_POS; and the action roles ACTION_DELTA_POS, ACTION_DELTA_ROT, ACTION_GRIPPER. Bimanual _2 variants exist for the per-arm roles EEF_POS, EEF_ROT, GRIPPER_POS, ACTION_DELTA_POS, ACTION_DELTA_ROT, and ACTION_GRIPPER.
Rotation widths follow the declared encoding. rlmesh.adapters.ROTATION_DIMS maps each encoding to its dimension count:
Encoding |
Dims |
Convention |
|---|---|---|
|
4 |
quaternion, scalar-last |
|
4 |
quaternion, scalar-first |
|
3 |
rotation vector |
|
6 |
first two columns of the rotation matrix, concatenated |
|
6 |
same two columns flattened row-major |
|
3 |
roll-pitch-yaw, extrinsic XYZ |
rot6d is the standard 6D rotation; rot6d_rowmajor exists for checkpoints trained on the row-major interleaving. See Adapters for when to add an encoding versus reach for a custom encoding.