Framework Backends¶

Note

This is the autodoc API reference. For how backends decode values at the Python boundary see Framework Backends.

Each backend keeps the same environment, model, and sandbox behavior as the shared RLMesh client APIs, but decodes tensor leaves to its own array type. Space wrappers returned from a backend’s clients also sample values compatible with that type. The Torch and JAX backends are experimental.

NumPy¶

Use the NumPy backend for examples, notebooks, and model code that already works with arrays. Install it with:

pip install "rlmesh[numpy]"

Concrete API	Shared behavior	Backend-specific behavior
`rlmesh.numpy.RemoteEnv`	Serving and Clients single clients	Observations, actions, and render frames use arrays.
`rlmesh.numpy.RemoteVectorEnv`	Serving and Clients vector clients	Batched values use NumPy-compatible containers.
`rlmesh.numpy.Model`	Models	`predict_fn` receives NumPy-decoded observations.
`rlmesh.numpy.SandboxEnv`	Sandbox single sandbox sessions	Owned sandbox client is `rlmesh.numpy.RemoteEnv`.
`rlmesh.numpy.SandboxModel`	Sandbox	Runs a model policy in its own container (experimental).
`rlmesh.numpy.SandboxVectorEnv`	Sandbox vector sandbox sessions	Owned sandbox client is `rlmesh.numpy.RemoteVectorEnv`.

Conversion Semantics¶

asarray(tensor) returns a writable copy of the tensor bytes, matching Gymnasium where reset/step observations are writable (so obs /= 255.0 works). For a zero-copy, read-only view that shares the tensor buffer, use numpy.from_dlpack(tensor) or the buffer protocol.
from_array(array) always copies: it makes the array C-contiguous and serializes its bytes into a fresh RLMesh tensor.

Value Helpers¶

rlmesh.numpy.ensure_available()[source]¶

Raise if NumPy is not installed.

Return type:: None

rlmesh.numpy.asarray(tensor)[source]¶

Return a writable NumPy array containing an RLMesh tensor’s data.

The returned array owns a fresh copy of the tensor bytes, so it is writable and matches Gymnasium, where reset/step observations are writable (idioms such as obs /= 255.0 work). For an opt-in zero-copy view that shares the tensor buffer, use the buffer protocol or DLPack directly (for example numpy.from_dlpack(tensor)), treating the result as read-only.

Parameters:: tensor (Tensor) – RLMesh tensor value to convert.
Returns:: A writable NumPy array with a copy of the tensor data.
Return type:: object

rlmesh.numpy.from_array(array)[source]¶

Encode a NumPy array or scalar as an RLMesh value.

Parameters:: array (object) – NumPy array or scalar to encode.
Returns:: Tensor for non-scalar arrays, or a primitive for scalar values.
Return type:: Tensor | None | bool | int | float | str | bytes

rlmesh.numpy.space_from_spec(spec)[source]¶

Create a NumPy-adapted space wrapper for a native space spec.

Parameters:: spec (SpaceSpec)
Return type:: Space[None | bool | int | float | str | bytes | object | list[None | bool | int | float | str | bytes | object | list[NumpyValue] | tuple[NumpyValue, …] | dict[str, NumpyValue]] | tuple[None | bool | int | float | str | bytes | object | list[NumpyValue] | tuple[NumpyValue, …] | dict[str, NumpyValue], …] | dict[str, None | bool | int | float | str | bytes | object | list[NumpyValue] | tuple[NumpyValue, …] | dict[str, NumpyValue]]]

RemoteEnv¶

final class rlmesh.numpy.RemoteEnv[source]¶

NumPy-backed remote client for a single RLMesh environment.

Observations, rewards, and actions are decoded into Python primitives, NumPy arrays, or nested containers of those values. Use this client when a model or notebook expects NumPy values at the environment boundary.

Parameters:

address – Endpoint address such as "tcp://127.0.0.1:5555", "127.0.0.1:5555", or "unix:///tmp/env.sock".
host – TCP host helper used when address is omitted.
port – TCP port helper used when address is omitted.
path – Unix socket path helper used when address is omitted.
transport – Explicit transport selector.

Examples

>>> from rlmesh.numpy import RemoteEnv
>>> env = RemoteEnv("127.0.0.1:5555")
>>> observation, info = env.reset(seed=42)
>>> observation, reward, terminated, truncated, info = env.step(0)
>>> env.close()

RemoteVectorEnv¶

final class rlmesh.numpy.RemoteVectorEnv[source]¶

NumPy-backed remote client for a vectorized RLMesh environment.

A vector client connects one model process to an endpoint that owns multiple environment instances. Batched observations, rewards, terminations, and truncations decode into NumPy-compatible values.

Parameters:

address – Endpoint address such as "tcp://127.0.0.1:5555".
host – TCP host helper used when address is omitted.
port – TCP port helper used when address is omitted.
path – Unix socket path helper used when address is omitted.
transport – Explicit transport selector.

Examples

>>> from rlmesh.numpy import RemoteVectorEnv
>>> envs = RemoteVectorEnv("127.0.0.1:5555")
>>> observations, infos = envs.reset(seed=42)
>>> actions = [envs.single_action_space.sample() for _ in range(envs.num_envs)]
>>> observations, rewards, terminations, truncations, infos = envs.step(actions)
>>> envs.close()

Model¶

class rlmesh.numpy.Model[source]¶

NumPy-backed model: predict works in NumPy values.

The NumPy-typed ModelBase: wrap a predict callable (Model(fn, spec=...)) or subclass and override predict; run(env, seeds=[...]) returns a typed RunResult. See ModelBase.

Examples

>>> from rlmesh.numpy import Model
>>> Model(lambda observation: 0).run("127.0.0.1:5555", seeds=[0]).mean_reward
0.0

Sandbox¶

final class rlmesh.numpy.SandboxEnv[source]¶

Owned NumPy-backed sandbox session for one environment.

The sandbox starts an isolated environment process, connects a NumPy remote client to it, and stops the owned container when closed.

Parameters:

source – A gym id / gym:// / hf:// source built from source, or a prebuilt rlmesh-serving image (docker://img / bare img:tag).
build – Optional SandboxBuild – build-from-source infrastructure (base image, packages, rlmesh pin, …); ignored for a prebuilt image.
runtime – Optional SandboxRuntime – docker run settings (gpus / devices / volumes); prebuilt-image source only.
**params – Environment construction params – the binding forwarded to the factory’s make (validated in the container before construction).

Examples

>>> from rlmesh.numpy import SandboxEnv, SandboxBuild
>>> env = SandboxEnv(
...     "CartPole-v1", build=SandboxBuild(packages=["gymnasium==1.3.0"])
... )
>>> observation, info = env.reset(seed=42)
>>> env.close()

final class rlmesh.numpy.SandboxVectorEnv[source]¶

Owned NumPy-backed sandbox session for vectorized environments.

The sandbox starts multiple isolated environment instances and exposes them through the same vector client interface as a separately served endpoint.

Parameters:

source – A gym id / gym:// / hf:// source built from source, or a prebuilt rlmesh-serving image (docker://img / bare img:tag).
num_envs – Number of environment instances to create.
vectorization_mode – Vectorization mode requested inside the sandbox.
build – Optional SandboxBuild – build-from-source infrastructure; ignored for a prebuilt image.
runtime – Optional SandboxRuntime – docker run settings (gpus / devices / volumes); prebuilt-image source only.
**params – Environment construction params – the binding forwarded to the factory’s make (validated in the container before construction).

Examples

>>> from rlmesh.numpy import SandboxVectorEnv
>>> envs = SandboxVectorEnv("CartPole-v1", num_envs=2)
>>> observations, infos = envs.reset(seed=42)
>>> envs.close()

Torch (experimental)¶

Use the Torch backend for model code that already works with Torch tensors, especially when you want a zero-copy view over the wire buffer. Install it with:

pip install "rlmesh[torch]"

Concrete API	Shared behavior	Backend-specific behavior
`rlmesh.torch.RemoteEnv`	Serving and Clients single clients	Observations, actions, and render frames use tensors.
`rlmesh.torch.RemoteVectorEnv`	Serving and Clients vector clients	Batched values use Torch-compatible containers.
`rlmesh.torch.Model`	Models	`predict_fn` receives Torch-decoded observations.
`rlmesh.torch.SandboxEnv`	Sandbox single sandbox sessions	Owned sandbox client is `rlmesh.torch.RemoteEnv`.
`rlmesh.torch.SandboxVectorEnv`	Sandbox vector sandbox sessions	Owned sandbox client is `rlmesh.torch.RemoteVectorEnv`.

Value Helpers¶

rlmesh.torch.ensure_available()[source]¶

Raise if Torch is not installed.

Return type:: None

rlmesh.torch.as_tensor(tensor, *, copy=False)[source]¶

Return a Torch tensor view or copy of an RLMesh tensor.

Warning

Without copy=True the returned tensor shares memory with the RLMesh tensor, and Torch will let you write through it even though the export is flagged read-only (DLPack consumers may ignore that flag). Mutating the view corrupts the RLMesh tensor for every other view of the same data. Treat shared views as read-only, and pass copy=True for any tensor you intend to modify.

Parameters:

tensor (Tensor | bool | int | float) – RLMesh tensor or scalar primitive to convert.
copy (bool) – If True, copy tensor data before creating the Torch tensor.

Returns:

Torch tensor sharing the RLMesh tensor’s memory via DLPack, or an independent copy when copy=True.

Return type:

object

rlmesh.torch.from_tensor(tensor)[source]¶

Encode a Torch tensor as an RLMesh value.

Parameters:: tensor (object) – Torch tensor to encode.
Returns:: Tensor for non-scalar tensors, or a primitive for scalar values.
Return type:: Tensor | bool | int | float

rlmesh.torch.space_from_spec(spec)[source]¶

Create a Torch-adapted space wrapper for a native space spec.

Parameters:: spec (SpaceSpec)
Return type:: Space[None | bool | int | float | str | bytes | object | list[None | bool | int | float | str | bytes | object | list[TorchValue] | tuple[TorchValue, …] | dict[str, TorchValue]] | tuple[None | bool | int | float | str | bytes | object | list[TorchValue] | tuple[TorchValue, …] | dict[str, TorchValue], …] | dict[str, None | bool | int | float | str | bytes | object | list[TorchValue] | tuple[TorchValue, …] | dict[str, TorchValue]]]

RemoteEnv¶

final class rlmesh.torch.RemoteEnv[source]¶

Experimental Torch-backed remote client for one environment.

Tensor leaves decode to Torch tensors while Python primitives and nested containers are preserved.

Parameters:

address – Endpoint address such as "tcp://127.0.0.1:5555".
host – TCP host helper used when address is omitted.
port – TCP port helper used when address is omitted.
path – Unix socket path helper used when address is omitted.
transport – Explicit transport selector.

RemoteVectorEnv¶

final class rlmesh.torch.RemoteVectorEnv[source]¶

Experimental Torch-backed remote client for vectorized environments.

Parameters:

address – Endpoint address such as "tcp://127.0.0.1:5555".
host – TCP host helper used when address is omitted.
port – TCP port helper used when address is omitted.
path – Unix socket path helper used when address is omitted.
transport – Explicit transport selector.

Model¶

class rlmesh.torch.Model[source]¶

Experimental Torch-backed model: predict works in Torch values.

The Torch-typed ModelBase; see it for the wrap-a-callable / subclass-and-override-predict construction and run(env, seeds=[...]) -> RunResult eval.

Sandbox¶

final class rlmesh.torch.SandboxEnv[source]¶

Experimental Torch-backed owned sandbox session for one environment.

Parameters:

source – A gym id / gym:// / hf:// source built from source, or a prebuilt rlmesh-serving image (docker://img / bare img:tag).
build – Optional SandboxBuild – build-from-source infrastructure; ignored for a prebuilt image.
runtime – Optional SandboxRuntime – docker run settings (gpus / devices / volumes); prebuilt-image source only.
**params – Environment construction params – the binding forwarded to the factory’s make (validated in the container before construction).

final class rlmesh.torch.SandboxVectorEnv[source]¶

Experimental Torch-backed owned sandbox session for vectorized environments.

Parameters:

source – A gym id / gym:// / hf:// source built from source, or a prebuilt rlmesh-serving image (docker://img / bare img:tag).
num_envs – Number of environment instances to create.
vectorization_mode – Vectorization mode requested inside the sandbox.
build – Optional SandboxBuild – build-from-source infrastructure; ignored for a prebuilt image.
runtime – Optional SandboxRuntime – docker run settings (gpus / devices / volumes); prebuilt-image source only.
**params – Environment construction params – the binding forwarded to the factory’s make (validated in the container before construction).

JAX (experimental)¶

Use the JAX backend for model code that already works with JAX arrays; decoded arrays are immutable, so there is no mutation hazard. Install it with:

pip install "rlmesh[jax]"

Concrete API	Shared behavior	Backend-specific behavior
`rlmesh.jax.RemoteEnv`	Serving and Clients single clients	Observations, actions, and render frames use arrays.
`rlmesh.jax.RemoteVectorEnv`	Serving and Clients vector clients	Batched values use JAX-compatible containers.
`rlmesh.jax.Model`	Models	`predict_fn` receives JAX-decoded observations.
`rlmesh.jax.SandboxEnv`	Sandbox single sandbox sessions	Owned sandbox client is `rlmesh.jax.RemoteEnv`.
`rlmesh.jax.SandboxVectorEnv`	Sandbox vector sandbox sessions	Owned sandbox client is `rlmesh.jax.RemoteVectorEnv`.

Conversion Semantics¶

asarray(tensor) imports over DLPack. XLA shares RLMesh’s 64-byte-aligned buffers zero-copy and copies otherwise; JAX arrays are immutable either way, so there is no mutation hazard.
from_array(array) moves the array to CPU if needed, blocks until ready, and copies the elements into a fresh RLMesh tensor.
int64, uint64, and float64 values require JAX 64-bit mode (jax.config.update("jax_enable_x64", True)); without it JAX itself demotes those dtypes.
Requires jax >= 0.4.24, the first release with DLPack bool support. ensure_available enforces the floor at runtime.

Value Helpers¶

rlmesh.jax.ensure_available()[source]¶

Raise if JAX is not installed or is older than the supported floor.

Return type:: None

rlmesh.jax.asarray(tensor)[source]¶

Return a JAX array for an RLMesh tensor.

Parameters:: tensor (Tensor) – RLMesh tensor value to convert.
Returns:: JAX array imported over DLPack. XLA shares 64-byte-aligned buffers and copies otherwise; either way the result is immutable.
Return type:: object

rlmesh.jax.from_array(array)[source]¶

Encode a JAX array as an RLMesh value.

Parameters:: array (object) – JAX array to encode.
Returns:: Tensor for non-scalar arrays, or a primitive for scalar values.
Return type:: Tensor | None | bool | int | float | str | bytes

rlmesh.jax.space_from_spec(spec)[source]¶

Create a JAX-adapted space wrapper for a native space spec.

Parameters:: spec (SpaceSpec)
Return type:: Space[None | bool | int | float | str | bytes | object | list[None | bool | int | float | str | bytes | object | list[JaxValue] | tuple[JaxValue, …] | dict[str, JaxValue]] | tuple[None | bool | int | float | str | bytes | object | list[JaxValue] | tuple[JaxValue, …] | dict[str, JaxValue], …] | dict[str, None | bool | int | float | str | bytes | object | list[JaxValue] | tuple[JaxValue, …] | dict[str, JaxValue]]]

RemoteEnv¶

final class rlmesh.jax.RemoteEnv[source]¶

Experimental JAX-backed remote client for one environment.

Tensor leaves decode to JAX arrays while Python primitives and nested containers are preserved.

Parameters:

address – Endpoint address such as "tcp://127.0.0.1:5555".
host – TCP host helper used when address is omitted.
port – TCP port helper used when address is omitted.
path – Unix socket path helper used when address is omitted.
transport – Explicit transport selector.

RemoteVectorEnv¶

final class rlmesh.jax.RemoteVectorEnv[source]¶

Experimental JAX-backed remote client for vectorized environments.

Parameters:

address – Endpoint address such as "tcp://127.0.0.1:5555".
host – TCP host helper used when address is omitted.
port – TCP port helper used when address is omitted.
path – Unix socket path helper used when address is omitted.
transport – Explicit transport selector.

Model¶

class rlmesh.jax.Model[source]¶

Experimental JAX-backed model: predict works in JAX values.

The JAX-typed ModelBase; see it for the wrap-a-callable / subclass-and-override-predict construction and run(env, seeds=[...]) -> RunResult eval.

Sandbox¶

final class rlmesh.jax.SandboxEnv[source]¶

Experimental JAX-backed owned sandbox session for one environment.

Parameters:

source – A gym id / gym:// / hf:// source built from source, or a prebuilt rlmesh-serving image (docker://img / bare img:tag).
build – Optional SandboxBuild – build-from-source infrastructure; ignored for a prebuilt image.
runtime – Optional SandboxRuntime – docker run settings (gpus / devices / volumes); prebuilt-image source only.
**params – Environment construction params – the binding forwarded to the factory’s make (validated in the container before construction).

final class rlmesh.jax.SandboxVectorEnv[source]¶

Experimental JAX-backed owned sandbox session for vectorized environments.

Parameters:

source – A gym id / gym:// / hf:// source built from source, or a prebuilt rlmesh-serving image (docker://img / bare img:tag).
num_envs – Number of environment instances to create.
vectorization_mode – Vectorization mode requested inside the sandbox.
build – Optional SandboxBuild – build-from-source infrastructure; ignored for a prebuilt image.
runtime – Optional SandboxRuntime – docker run settings (gpus / devices / volumes); prebuilt-image source only.
**params – Environment construction params – the binding forwarded to the factory’s make (validated in the container before construction).

Where next¶

Framework Backends for how decoding works at the Python boundary.
Serving and Clients, Models, and Sandbox for the shared client behavior each backend reuses.

Framework Backends¶

NumPy¶

Conversion Semantics¶

Value Helpers¶

RemoteEnv¶

RemoteVectorEnv¶

Model¶

Sandbox¶

Torch (experimental)¶

Memory Sharing and Mutation¶

Value Helpers¶

RemoteEnv¶

RemoteVectorEnv¶

Model¶

Sandbox¶

JAX (experimental)¶

Conversion Semantics¶

Value Helpers¶

RemoteEnv¶

RemoteVectorEnv¶

Model¶

Sandbox¶

Where next¶