NodeJS Bridge / Plugin

[Rhino]

Hi.

I'm hoping to discuss the potential for writing a bridge plugin or bridge interface between MQ2 and and a node js runtime.

Initial Discussion Items:
Discuss what some potential solution architecture's to achieve the bridge would look like
Outline some starting points to begin development and what some good milestones would be
Highlight any obvious pain points or major barriers to such a solution would be.

To minimize the amount of C++ development needed I was considering trying to leverage EQBCS as a proxy / bridge for issuing commands from the the NodeJS runtime to MQ2. Even if that is an achievable initial design I'll need to figure out a way for the runtime to query MQ2 for information (e.g. access TLO values, have the output from MQ2 command piped back to the node runtime, etc). It's an ambitious project but there's a lot of potential value, IMO.

Context / Goals:
So far I've been very pleased with the utility that MQ2 adds to the game experience. For writing/executing macros, however, there would be a lot of benefit in being able to script the macro's using a more tranditional language. It would eliminate the need to learn a new scripting language. It would also allow macro developers to leverage the large array of dev tools (IDEs, syntax highlighting, design time typing and error checking, etc) available for javascript which will, hopefully, increase productivity when writing macros.

Cheers,
Rhino

 

[Knightly]

I've often considered ways of adding a secondary macro language in the form of python or Lua. Not a plugin language like MQ2Py, but an actual scripting language.

It's a lot of work, but I see the value in it. Learning macro language is all well and good, but having languages applicable to other situations and getting built in libs for free is pretty nice.

As far as your idea of EQBC goes, if you were going to go down that road I would look at DanNet instead. The underlying library is zyre (zeromq) which has node bindings as well as a full port I believe. That would give you a jump start in being able to both receive TLOs and issue commands.

 

[Rhino]

Thanks Knightly, that's a really helpful suggestion for using sockets and message passing to handle the inter-process communication. I took a look a look at the DanNet code and at ZeroMQ and it looks like it could be a workable approach. A solution going that route would have an architecture that looks something like below, I think. Do your, or anyone else, have any thoughts on the benefits/tradeoffs of that approach vs trying to use shared memory to handle the IPC?

The sockets/message queue approach will likely be easier to achieve and require less coding by leveraging existing plugin code. There's also the possibility for a macro script to interact directly with multiple EQ peers.

Potential Solution Structure.
1. A stand-alone process for the node runtime is launched. I'd like to have a GUI for ease of use so I'm thinking I'll make this an Electron app. The stand-alone process uses zeromq to listen for the connection from the plugin.

2. The mq2 plugin establishes the socket connection to the stand alone process and handles bi-directional communication with the EQ client(s).
* Support for sending commands/queries from the client to the stand alone process and receiving/handling responses
* Support for sending commands/queries from the stand alone process to the to the plugin/eq client and receiving/handling responses

3. The stand alone process would have the ability to execute macro script(s). Initially this could just be a single macro script run at a given time, but it should be theoretically possible to run multiple on different threads using web/service workers.

4. A helper library abstracts the internals of the message passing/parsing and allows for easy use of the mq2 features/objects. In my case I'll be using typescript in lieu of plain javascript so this would include support for strong typing of the objects and functions.


Thoughts, flaws in my design, suggested improvements?

 

[jimbob]

You might want a helper plugin to have something more like RPC style calls instead of calling macroblock code remotely and parsing the output. Then you'd always have standard protocol communications.

 

[Knightly]

Yeah, I was thinking of it similar to jimbob, treating whatever your communications method is more as an API to macroquest than as, for example, a scripting engine. A lot of the benefits of a third party language go away when you're trying to translate that to macro script. I would just translate it to commands (and use MQ2 enhanced commands to augment anything that's missing). In that case you only need a method of execution and a method of data retrieval, both of which are readily available over the network right now.

In the (not currently available) development branch, brainiac has a good working prototype for IPC via the injector which is planned to allow plugin interaction, so I wouldn't spend too much time trying to write your own IPC (though, no telling when development will roll live).

 

[Wink-]

I would look into using something like http://getrobot.net/api/global.html . I used to hack together WoW bots (with a lot of community help) using this until the last major Warden update back in 7.x. It would be a lot of work for sure, but a dedicated smart person could figure it out!

 

[Rhino]

At least initially, I'm more interested in a "re-use as much existing MQ2 stuff as possible approach so it's not a lot of work" solution.

I've got pre-written code of my own for native event automation, process inspection, etc, in both Java and C#. And MQ2DotNet is a very promising option for better scripting/botting. If my goal was just writing bots I would start with that probably. I'd personally prefer using C# over typescript or c++.

My interest in this project was mostly to leverage the work and feature set that already exists in MQ2 while eliminating some of it's draw backs (both for myself and for the community at large). There is plenty that can be done already with MQ2 plugins and macros. I just want to make the experience even easier/better.

1) Allow for a secondary macro language using a major language that comes with features/advantages that MQ2's macro scripting lacks (IDE support, syntax highlighting, design time error checking, free 3rd party libraries, etc)
2) Leverage as much existing work from MQ2 as possible, minimizing development effort
3) If possible, avoid the complicated process involved in compiling MQ2 + Plugins as part of the development process (e.g. using an existing plugin like EQBC or Knightly's much better recommendation of MQ2DanNet)

Nice to haves:
4) Not requiring the macros/scripts to be compiled manually by the user before they can be run.
5) Being able to interface with/issue commands to multiple toons from the same macro script
6) some strong(ish) typing around the TLOs, commands, etc to allow for design time errors of the MQ2 specific stuff

C# wouldn't be a very good fit for this project or else I would use that as it's my preferred language most of the time. I've done runtime compilation and execution of C# in plenty of projects before; while it's possible to achieve #4 in C# it would be a whole bunch more development effort that I can effectively bypass by using NodeJS or a similar language+execution environment. Plus the only port of Zyre I found for C# was NetMQ/Zyre which hadn't been touched in 4 years.

I really like Knightly's suggestion of MQ2DanNet for an initial approach and there is a group investigating a more or less identical approach for python right now. Assuming that the NodeJS bindings for Zyre (https://github.com/zeromq/zyre/tree/master/bindings/nodejs) work reasonably well and are stable, I can probably piggy back on their groups research into connecting via DanNet.

 

[dannuic]

Couldn't you use C# (csx/fsx) scripts instead of compiled C# to achieve everything you want? No matter what you do, you'll have to have some kind of translation between your language and the bindings available to either MQ2 or c++, none of that comes for free.

 

[Rhino]

Yes. MQ2DotNet has an incomplete feature set though. Extending it will require rebuilding the C++ library, going through builder for live, etc which is one of the things I'm hoping to avoid.