RLMesh

Gymnasium-compatible infrastructure for model-environment evaluation.

RLMesh connects models to environments that need not share a process, a dependency set, or a machine. The Python SDK serves Gymnasium-style environments and lets evaluators call reset, step, render, and close over local or remote transports. Rust crates provide the lower-level runtime, protocol, and packaging layers.

RLMesh is released and pre-1.0 (0.x): the Python package is the supported surface, and a minor release may change a stable API with a migration note. See Compatibility and Versioning; pin a minor range for active projects.

Note

0.1.0-rc.2 is a release candidate for 0.1.0. It advertises the exact 2026.06-0.1.0-rc.2 workflow cohort; the bare 2026.06 edition seals at the final 0.1.0. The stability statements in these docs describe that upcoming release.

Try First

Start with the shortest local loop:

• Install RLMesh with Gymnasium and the NumPy backend.

• Run the quickstart: serve CartPole-v1, connect one evaluator.

• Check Gymnasium compatibility for the current supported space set.

• Try the examples: swap environments, run sandboxed or isolated dependency stacks, and connect one evaluator to multiple endpoints.

Model-Environment Boundary

A model and an environment meet at one boundary. A model or evaluator connects through the familiar reset, step, render, and close calls, even when the environment runs in a separate process with its own dependencies.

        flowchart LR
    subgraph eval["Evaluator process"]
        M["Model / evaluator"]
    end
    subgraph env["Environment process"]
        E["Gymnasium environment"]
    end
    M <-->|"reset · step · render · close"| E
    

Use that boundary to:

• run a model against an environment without merging their dependency stacks;

• run multiple environment endpoints at the same time;

• reuse existing Gymnasium registrations, wrappers, and environment objects with small changes;

• package an environment or model as a container image and run it the same way locally or on the hosted platform (see the bring-your-own-container example).

RLMesh has two surfaces, split by protocol. The framework is this repository. It speaks the gRPC runtime contract, so you author and run environments and models in Python or Rust, on one machine or over the wire. The hosted platform speaks a REST control plane at api.rlmesh.dev and is documented separately. These docs cover the framework.