Choosing the right pattern to expose different kinds of services (RR, PP, PS) from a proxy to N clients

I am working with a robot from the manufacturer Kuka and my goal is :

to expose some of the functionalities of the robot system to remote clients through a clients-proxy architecture.

Hereafter I give a description of the system I want to build. And at the bottom there is my question(s).

Description of the robot

The robot is composed of a Controller hosted on a WIN XP embedded system, running on a standard PC. The Controller drives the arm, the track and a I/O bus. It is responsible for running programs written in the Kuka Robot Language (KRL) and this is the prescribed way of interacting with the robot cell.

XComm interface

Kuka provides the XComm interface to interact with the robot controller from the WIN XP system. It is compatible with .Net Framework 4.0. This interface is divided into services, and each service comes in two flavors : blocking call vs. async call with callback.

For instance the XComm.Var service is mainly exposing these Sync/Async interfaces :

interface SyncVar
{
    public string GetVar(string varName); // reads a variable
    public void SetVar(string varName, string varValue); // changes a variable
}
interface AsyncVar
{
    // reads a variable 
    public void GetVarAsync(string varName, AsyncCallback callback);
    // changes a variable
    public void SetVarAsync(string varName, string varValue, AsyncCallback callback); 

    // subscribe to varValueChanged event : callback will be invoked each time the monitored variable has changed
    public void InfoVar_ON(string varName, AsyncCallback callback, Timespan ts);
    // unsubscribe to varValueChanged event
    public void InfoVar_OFF(string varName); 
}

There is a dozen of such services : XComm.Motion, XComm.IO, XComm.File, ...

Objectives

The main feature I am concerned with is interacting remotely with the robot as a state machine. That is I want a remote client to be able to get the state of some variables and potentially change the state of some variables at any moment.

Additionally, I would be interested (latter) to expose other XComm services like the XComm.File for loading KRL programs to the robot and XComm.Module to run them from a remote client. So I need my architecture to be extensible to other XComm services.

Features

1) REQ-REP

The first way I would like a client to communicate with the robot is through a simple REQ-REP pattern. For instance :

C : Hey robot, what si the speed of your arm now ?
S : It is 10m/s
C : Ok, and what is the torque on your tool ?
S : It is 1000Nm
C : Oh, I see, the stone is harder than I expected, please reduce your speed now to 5m/s
...

Or :

C : Hey robot, please, move to x = 1000mm at speed 1000mm/s
S : Ok client, I will move to x = 1000mm
C (+250ms) : Hey robot, where are you now ?
S : I am at x = 230mm
C (+500ms) : Hey robot, where are you now ?
S : I am at x = 550mm
C (+750ms) : Hey robot, where are you now ?
S : I am at x = 800mm
C (+1000ms) : Hey robot, where are you now ?
S : I am at x = 1000mm
C : Ok, so now please move to y = -500mm at speed 250mm/s
...

2) PUSH-PULL

The second way I would like a client to communicate with the robot is through a simple PUSH-PULL pattern where clients can subscribe to get notified when the value of a variable has changed. Something which could look like :

C : Hey robot, let me know when you move the arm on the track, but not more than every 50ms
S : Ok client, I will let you know
...
S : Hey client, I've just move the arm and the track an my position is now E1=4000mm
...
S : Hey client, I've just move the arm and the track an my position is now E1=4010mm
...
C : Hey robot, I'm done with all this, stop notifying my when you move the arm on the track.
S : Ok client, understood.

3) PUB-SUB

The third way I would like a client to communicate with the robot is through a simple PUB-SUB pattern. I want a client to be able to subscribe to some predefined channels (ie. motion, command, IO). A channel broadcasts the state of a set of related variables as a list ok key-value pairs. This broadcast occurs every 40ms or so and is mainly meant to build dashboards and refresh them at more or less 25fps. The configuration of the channels is static but might be loaded from a configuration file when the proxy starts.

Performance

Latency

A (local) synchronous call to a XComm service is about 5ms long. It is ok but yet too much (by 5 to 10) for certain real time control applications. So I really need to be cautious that the chosen architecture will not increase this latency. So obviously I need my clients to send async requests to no cumulate the network roundtrip time of each REQ-REP cycle.

Maybe dispatching client requests through several XComm service consumers with a load balancing pattern could help reducing the latency ?

Throughput

When I am concerned with low latency, it is always for messages with small footprints for reading and writing variable states (few hundred bytes max). Let us say I want to drive the robot position at 1000fps. That is :

2 ways * 256 bytes / msg * 1000 msgs/sec = 512kB/s =  4.096Mbs

So clearly, throughput is not a problem on a basic 1Gbs LAN.

If I need to load a KRL program on the robot, I can afford to wait few 100ms or sec to transfer the file.

Reliability

I need my REQ-REP pattern to be reliable to network, server and client failures and to recover when I manually restart a server or a client. So I need a PING-PONG keep alive.

Possibly I need a "Secure" version of the Get/Set variable service when, for instance, I want to stream positions to the robot so I can resume my streaming at the point where it stopped. Something like the titanic pattern with a persistant queue somewhere on the disk ?

Architecture / Pattern

So my first bet is that a Majordomo pattern will be adapted to my situation.

1 Proxy running on WIN XP

• workers : each worker acts as a consumer for a single XComm service. It can consume sync or async.
• broker : a router frontend with a dealer backend that dispatches the client requests to the worker and the worker replies back to the clients.
• discovery : service for clients to known which services are available from the proxy and their state.

N < 10 Clients running remotely on the same LAN or over the internet

• clients can come and go at any time
• a client can Get/Set one or several variables with a single request (possibly with a safe option) and in the flavor of their choice (sync/async).
• For Async Get/Set var I need a per request callback mechanism : when the reply has arrived, the callback is invoked so the client as a built-in way of triggering and dispatching actions to be done with the replies.
• a client can Subscribe/Unsubscribe to variable monitoring (push notification)
• a client can Subscribe/Unsubscribe to a predefined channel
• a client can discover the available services from the proxy and know their state
• a client can discover which clients are connected to the proxy and their state

I want to provide a simple client API that any device will have to implement to consume the services from the proxy.

Ideally, I would like one ZSocket per client and one ZSocket for the proxy frontend to have a simpler heartbeat keepalive mechanism.

• The remote client-proxy communications will TCP.
• Potentially, local client-proxy is allowed through inproc
• The proxy-worker communications will be inproc (faster right ?).

Questions

So I have identified 3 services that my proxy should provide to clients :

• REQ-REPL to Get/Set robot variables
• PUSH-PULL to Subscribe/Unsubscribe to get notified when a certain variable has changed
• PUB-SUB to receive (partial) snapshots of the state of the robot at regular intervals
• (DISCOVERY) to know the available services

=> So my question is what's the right pattern for my clients-proxy so they talk to each others consuming (potentially) the 3 services at the same time ?

=> Do I need to provide 3 frontend/ports/endpoints, one for each service ?

=> Can I have a unique frontend dispatching to each service and routing REQ/REP/PUSH/PULL/PUB/SUB messages to the right clients ?

I feel I prefer to keep a unique frontend socket on my proxy talking to clients with a unique socket too. I feel it will be easier to manage reliability this way rather than if i need a socket per service (3 heartbeats PING/PONG). And my client will stay minimal. But I might be completely wrong.