← RESEARCH

Oguzhan ErsoyResearch

IR3DE-AXL: Collective Inference Network

Most inference routing runs through a single gateway with a fixed menu of models. IR3DE-AXL flips that: every participant builds their own router from whatever experts the network currently offers, and can contribute their own in return.

Abstract glitch-art scene of a glowing human-like figure standing in a dark corridor, illuminated by harsh white light and fragmented by horizontal digital distortion, scan lines, and red-green-blue chromatic shifts.
Table of contents
  1. The Prototype: From Local Router to Collective Network
  2. How It Works
  3. Setup
  4. Inference Flow
  5. User Interface
  6. Why It Matters
  7. Limitations
  8. Takeaways

We have built a collective inference network prototype with our tools IR3DE and AXL. Participants can contribute expert models, use models served by others, or do both, without relying on one centralised gateway or a fixed model catalogue.

Two components make this work: our local routing mechanism IR3DE and our peer-to-peer networking layer AXL. IR3DE uses lightweight statistics over token embeddings to match a prompt to the most relevant expert domain. AXL connects nodes directly, so routing decisions can become live inference requests across the network. Together, they turn local expert routing into a working collective inference network.

Diagram of IR3DE routing a request across the network. A user at Node 01 asks how to read a CSV file in Python and plot the first three columns. The prompt enters the IR3DE router, labelled as routing to the most relevant expert, which considers three nodes: Node 01, a general chat model running Llama 3 8B, shown with a speech bubble icon; Node 02, a code expert running Qwen2.5-Coder-3B-Instruct, shown with a code bracket icon; and Node 03, a math expert running Qwen2.5-Math-7B-Instruct, shown with a sigma icon. A solid arrow sends the prompt to Node 02, with dashed arrows showing the unselected paths to Nodes 01 and 03. The model response box shows a short Python snippet that imports pandas and matplotlib, reads data.csv into a dataframe, plots the first three columns with iloc and calls plt.show.

The Prototype: From Local Router to Collective Network

In our IR3DE paper, we study a routing problem: given a prompt and a set of expert models, which expert should answer? IR3DE solves this efficiently by embedding the prompt, scoring tokens against domain statistics with ridge regression, and letting confident token-level votes determine the selected expert.

IR3DE-AXL extends that idea into a live networked setting. IR3DE-AXL is a decentralised inference network where every participant is an AXL node. A node can act as a user, an inference provider, or both.

Providers advertise one or more models together with their expertise categories and basic model metadata. Users browse the expertise areas available in the network, select the domains they want, and choose one model for each selected expertise. From those choices, the user builds a local IR3DE router.

That router stays local to the user. There is no central routing service deciding which model everyone must use. Each participant can compose their own expert system from the models currently available in the network. Thanks to the dynamic structure of IR3DE, users can modify the set of expert models they use without retraining the local router.

How It Works

Setup

First, a node creates a network and shares the network address with other participants. Other nodes then join that network. Once connected, provider nodes register the models they serve, along with their expertise and basic metadata. In the illustrated example, different nodes contribute different experts such as coding, math, and general chat.

Four-step network setup diagram. Step 1, Create Network: Node 01 creates the network at IP 192.168.1.10 and adds its locally stored General Chat model, Llama 3 8B. Step 2, Share IP: Node 01 shares the network address with other participants. Step 3, Join Network: Node 02 and Node 03 join, both showing status Joined. Step 4, Register Models: Node 01 advertises General Chat (Llama 3 8B), Node 02 advertises Code (Qwen2.5-Coder-3B-Instruct) and Node 03 advertises Math (Qwen2.5-Math-7B-Instruct).

Inference Flow

During inference, the user first selects the expertise areas they want to use, such as code and math. The client then builds a local IR3DE router over those selected expertise areas. When the user sends a prompt, the router chooses the matching expertise and routes the request to the corresponding provider node. In the example flow, a coding prompt is routed to the coding node, while a math prompt is routed to the math node. The selected provider runs inference and returns the response to the user.

Four-step inference flow diagram, numbered 5 to 8. Step 5, Select Expertise: the user ticks Code and Math and leaves General Chat unselected. Step 6, Build Local Router: the client constructs a local IR3DE router that discovers providers, matches expertise, routes requests and aggregates responses. Step 7, Inference: a prompt asking for a Python script to parse CSV is routed to Node 02, the Code expert running Qwen2.5-Coder-3B-Instruct, and a prompt asking to solve the integral of x squared is routed to Node 03, the Math expert running Qwen2.5-Math-7B-Instruct. Step 8, Response: Node 02 returns a parse_csv function that opens the file, reads it with csv.reader and returns the rows as a list, and Node 03 returns the integral of x squared as x cubed over 3 plus C.

User Interface

The prototype is currently represented with a simple TUI. The left side controls and monitors the router: users can select expertise categories and choose one model per expertise. The right side shows the chat history, including router decisions and model responses.

Users can continue the same chat with follow-up questions or create a new chat. In a multi-round chat, the router makes a new routing decision after every new prompt, while the selected models can see the previous messages for that chat. When the user starts a new chat, the model receives a fresh context without the previous chat history.

Screenshot of the IR3DE terminal interface. The left panel shows expertise selection with Coding, History, Instruction, Math, Multilingual, Philosophy and Physics all ticked, followed by a model selection list assigning one model to each expertise, with the serving node and model size shown alongside. The right panel shows a chat session where each user prompt is followed by a router log line naming the chosen expertise and node, then the model's answer. Visible exchanges cover 20th-century historical figures, the formula for momentum, the main research domains in philosophy and Euler's identity.

Why It Matters

IR3DE-AXL is mainly about collective inference. A participant with a coding model can serve coding requests to the network while using someone else’s math or history model. Another participant can contribute a different expert. Another may only consume the services. The same set of nodes can therefore both provide and use inference.

This makes the network dynamic and participant-driven. The set of available experts comes from what the participants themselves contribute, and each user can locally decide which experts to include in their own routing setup.

Limitations

IR3DE is intentionally designed to be lightweight, so the router is not perfect. Ambiguous prompts, overlapping domains, missing domain statistics, or poorly matched expert categories can still lead to incorrect routing decisions. The router should be seen as a simple local dispatcher, not as a guarantee that the selected model is always the best possible model for the prompt.

The prototype also does not include built-in verification or incentive mechanisms. Participants should treat it as an altruistic or trusted-network setup: if they join a network with untrusted providers, they take the risk that a provider may be unavailable, return low-quality outputs, or misrepresent what it is serving. The current prototype focuses on the core collective inference loop rather than trust, verification, reputation, or incentive design.

Takeaways

  • IR3DE-AXL turns local expert routing into a collective inference network. Users discover available expertise, build a local router, and send prompts to the most relevant provider in the network.
  • Each node can both serve and use inference. Participants contribute their own expert models while also benefiting from experts served by others.
  • Each user can dynamically select their inference providers. The ridge-regression-based routing mechanism enables users to modify the set of inference providers and expertise areas they rely on via the TUI.

View the GitHub Repo here: https://github.com/gensyn-ai/IR3DE-AXL

Continue reading

← Previous
Credibly Neutral AI Oracles

Credibly Neutral AI Oracles

A credibly neutral AI report layer allows dispute mechanisms to use a higher voting threshold without sacrificing accurate resolution. The result is a system that is harder to manipulate and easier to verify.