• You've discovered RedGuides, an EverQuest multi-boxing and scripting community 🧙‍♀️⚙️. We want you to play several EQ characters at once, come join us and say hello! 👋

  • A TLP without truebox has thawed (Very Vanilla ready)
    Frostreaver
Resource icon

Release .NET / C# Plugin and "Program" API

Overview Discussion
@Timult Yeah I was aware it was being used to allow the domain and thus the dll to be unloaded dynamically. The AssemblyLoadContext in .net core can be used to do the same thing as long as the dll is loaded into a separate assembly load context with isCollectible set to true. Is that what you meant by "I've had the loader,unloader working for a while"?

I'm currently stuck on changes to the MQ2DotNetLoader.cpp plugin. Currently it hosts the .net CLR using the older `Mscoree` library approach. I was hoping to switch this to the `nethost` + `hostfxr` approach that their team recommends but I'm a noob when it comes to configuring C++ projects in VS. Their sample of the new recommended approach depends on a rather elaborate build setup to consume the `nethost.h` and `nethost.dll` from within the $(NetCoreTargetingPackRoot)`. I couldn't get their sample solution to build and I haven't figured out yet a way to get the existing MQ2DotNetLoader project to see/consume those nethost deps.

I pinged Alynel on Discord who said they'd take a look at my question when they get home. **crosses fingers**
 
@Rhino Yes, the loading/unloading from AppDomain for .NET Framework, not Core, is what I was referring. I'm a bit out of the loop with C++/CLI since I last used it over 10 years ago. But their "it just works" concept still holds true. Can't even begin to catch up with the Core changes they are making for it and what you are investigating.

My big concern for this project is sharing of Type layouts between C# and CLI. Probably the easiest option would be to generate them via code from the C++ source because manually creating and maintaining them would be a headache. Still trying to learn how all the pieces work in MQ2 as it is to figure a good way to share the data.
 
I was messing with a C++ loader for dotnetcore and it was a giant hairball, that I haven't had time to finish.
 
jimbob said:
The only real issues I see with this is that you're relying on MQ2Cast and macroblock code rather than calling native MQ2 API functions directly. This will slow down your code, and potentially reduce its reliability.
Click to expand...


MQ2Cast has been something I have a love/hate relationship with both in macros and plugins.

There are some MQ2 API Functions that aren't imported into MQ2DotNet as well, so it's a mix for now...
 
Tone said:
MQ2Cast has been something I have a love/hate relationship with both in macros and plugins.

There are some MQ2 API Functions that aren't imported into MQ2DotNet as well, so it's a mix for now...
Click to expand...

Yeah, for the most part, in order to avoid using macro block code, I "borrowed" code from the TLOs, and put them into a C++ class. You could do that here, or interface with the TLOs directly from code. It's a ton of text parsing, so I just borrow and modify the code for my needs.
 
Updates on my attempt to rework this to run .net core.
1. I was able to get the cpp loader plugin to successfully host the .net core clr. A big thank you to Alynel for helping me work through some problems on this part.
2. I have it successfully loading + starting + stopping + unloading programs using AssemblyLoadContext(s) in place of app domains.
3. I've ported the sync context and some of the other basic parts.
4. I've added my own code to the command registry to support manually signalling the cancellation token for individual async tasks.

Still todo:
Finish copying over the MQ2 Apis
Support 'Plugin' libraries
Support 'csx' scripts

Possible future items:
Currently the loader plugin depends on a `nethost.dll` that the .net core team puts out which is what it uses to locate the .net core runtime/host libraries to load. Right now I have this just dropped directly in my EQ game folder but Alynel shared this link with me and said Braniac had used this before, and that I could/should use it to make it load the nethost.dll from the MQ2 folder or a folder of our choosing. Not high on my priority list (mostly because I don't want to deal with the cpp plugin anymore than the bare minimum) but would be a nice to have item for the future.

docs.microsoft.com

SetDllDirectoryA function (winbase.h) - Win32 apps

Adds a directory to the search path used to locate DLLs for the application. (ANSI)
docs.microsoft.com
 
Last edited:
I pushed what I have for MQ2DotNetCore up to a new repo: https://gitlab.com/ryanthomas03/mq2dotnetcore

I finished copying over the MQ2 type apis and fixing a few bugs.

Still Todo:
1. Support plugin libraries
2. Support C# scripts

Plugins are pretty low on my priority list, any program can subscribe to most of the plugin events anyway and the async/await continuations run during the OnPulse handler so there isn't a lot of gain for having a separate plugin type to start. I do wan't to add support for an autoload / autostart option to start a configured list of programs once the .net core entry point is initialized.

C# scripts are going to require more research. I'm not sure what alternative to AppDomains there are for these, if any. None of the Roslyn method signatures take an AssemblyLoadContext and I'm not familiar with the InteractiveAssemblyLoader type so I'll have to research
 
Could any devs in this forum with a good working knowledge of multi-threading scenarios help me think through an architecture design change I'm considering?

Background Information:
Currently the logic in MQ2DotNet / MQ2DotNetCore uses it's own custom synchronization context to to ensure all of the async continuations run on the same MQ2 / EQ thread. It queues the continuations and then the OnPulse plugin method call processes the queue. This approach allows the use of async/await syntax which is nice but it has several drawbacks that I think could be mitigated. The calls back into the native MQ2 / MQ2 plugin dlls should happen in OnPulse on the MQ2/EQ thread to ensure thread safety, but I would like for any other logic to be able to run on a threadpool thread without issue.

Architecture Design In Consideration:
I think I can achieve my desired setup reworking the method signatures of the MQ2 apis so they all queue work to run in OnPulse and use a task completion source for async versions or an event reset slim for sync versions to communicate the result back to the caller.

I'm concerned about deadlocking in the sync scenario though. I was hoping to get some suggestions on how to account for the problem (in the code comment below) as well as some general feedback on this approach:

The only option I've been able to think of for the thread check so far is to keep a `WeakReference` to the thread that `OnPulse` is executing on and compare `Thread.CurrentThread` against it. I'm not sure how reliable that check would be or if there are better ways to accomplish it.

C#: 
// The instance of this MQ2 class would be provided for the running programs for
// them to interact with the MQ2 / EQ client.
public sealed class MQ2 : IMQ2
{
    private readonly MQ2PulseActionQueue _pulseActionQueue;

    // ... stuff ...

    public Task DoCommand(string command)
    {
        // I need a way to make sure this call isn't executing on the Thread
        // that HandlePulse(..) executes on or I will create a deadlock.

        using (var isFinishedResetEvent = new ManualResetEventSlim())
        {
            _pulseActionQueue.Enqueue(new ScheduledWork(DoCommandInternal, command, isFinishedResetEvent));
            isFinishedResetEvent.Wait();
        }
    }

    public Task DoCommandAsync(string command)
    {
        var taskCompletionSource = new TaskCompletionSource<Object>();
        _pulseActionQueue.Enqueue(new ScheduledWork(DoCommandInternal, command, taskCompletionSource));

        return taskCompletionSource.Task;
    }

    private static void DoCommandInternal(string command)
    {
        var characterSpawnIntPointer = MQ2NativeHelper.GetCharacterSpawnIntPointer();
        if (characterSpawnIntPointer == IntPtr.Zero)
        {
            return;
        }

        MQ2Main.NativeMethods.MQ2HideDoCommand(characterSpawnIntPointer, command, false);
    }
}
 
Last edited:
Sweet, I stepped away from this thread for a bit... Glad to see you got dotnetcore working... was in the middle of that cluster.

I'll go check out your repo so we aren't duplicating efforts.
 
I appreciate everyone's work on this. I was looking for almost exactly this to solve some problems I was having WRT plugins.
 
@Rhino I've gotten several requests for your work, but I'm having trouble setting up local.env.props with variables.

Between build server, server types, and dev environments, we have a lot of different folders. I was hoping to save some effort with variables like this would work, but I haven't had any luck:

Rich (BB code): 
        <MQ2InstallLiveRootFolder>$(MSBuildThisFileDirectory)..\Release</MQ2InstallLiveRootFolder>
        <MQ2SourceRootFolder>$(MSBuildThisFileDirectory)..\</MQ2SourceRootFolder>
 
@Redbot I was able to get this to build yesterday doing the following: (Note: I couldn't test because I couldn't build to EMU and I don't have a TEST/LIVE account to work on)
In the root of the mq2dotnetcore repo create a file named local.env.props

Note: now that I see you wanted variables in your variables, but not sure how to get that to work.
Most of the issues I had went away when I turned off the deployment.

For anyone else: put in this XML
INI: 
<Project>
    <!-- Shared properties, define your local values for these in a local.env.props file -->
    <PropertyGroup>
        <EQInstallRootFolder>C:\Program Files (x86)\EQ\RoF2</EQInstallRootFolder> <!-- where EQ lives-->
        <MQ2InstallLiveRootFolder>C:\Program Files (x86)\EQ\ROF RG MQ2</MQ2InstallLiveRootFolder>  <!-- where this is your mq2 folder-->
        <MQ2SourceRootFolder>C:\Users\delin\source\repos\VeryVanilla\</MQ2SourceRootFolder>   <!-- where this is your vanilla repo-->
    </PropertyGroup>
</Project>

Note, I also turned off the deployment stuff (I don't have node installed, etc) which would fail alot. You can turn that off in the directory.build.props
INI: 
        <!-- Deploy after build tasks require node.js is installed and local packages are installed via yarn/npm -->
        <DeployMQ2DotNetCoreFilesAfterBuild>false</DeployMQ2DotNetCoreFilesAfterBuild>
        <DeployProgramFilesAfterBuild>false</DeployProgramFilesAfterBuild>
 
@Redbot Do you have a branch I can look at? What project file format are you using?
 
No branch, just cloning your repo. I gave up on variables for the locan.env.props file and just tried building normally,

Code: 
3>------ Build started: Project: TestProgram, Configuration: Release Any CPU ------
3>TestProgram -> D:\Users\Trevor\work_local\VSTS\Very Vanilla Live\mq2dotnetcore\src\TestProgram\bin\Release\netcoreapp3.1\TestProgram.dll
3>'node' is not recognized as an internal or external command,
3>operable program or batch file.
3>D:\Users\Trevor\work_local\VSTS\Very Vanilla Live\mq2dotnetcore\src\TestProgram\TestProgram.csproj(29,3): error MSB3073: The command "node D:\Users\Trevor\work_local\VSTS\Very Vanilla Live\mq2dotnetcore\scripts\DeployFilesTask.js --sourcePath 'D:\Users\Trevor\work_local\VSTS\Very Vanilla Live\mq2dotnetcore\src\TestProgram\bin\Release\netcoreapp3.1\' --destinationPath 'D:\Users\Trevor\work_local\VSTS\Very Vanilla Live\Release\MQ2DotNetCore\Programs\TestProgram'" exited with code 9009.
3>Done building project "TestProgram.csproj" -- FAILED.
2>   Creating library ..\..\build\MQ2DotNetCoreLoader\Release\MQ2DotNetCoreLoader.lib and object ..\..\build\MQ2DotNetCoreLoader\Release\MQ2DotNetCoreLoader.exp
2>MQ2DotNetCoreLoader.vcxproj -> D:\Users\Trevor\work_local\VSTS\Very Vanilla Live\mq2dotnetcore\src\MQ2DotNetCoreLoader\..\..\build\MQ2DotNetCoreLoader\Release\MQ2DotNetCoreLoader.dll
2>Done building project "MQ2DotNetCoreLoader.vcxproj".
========== Build: 1 succeeded, 2 failed, 0 up-to-date, 0 skipped ==========
 
So. This took quite a few steps but I got it building with the Vanilla compile today.
Leaving the steps here that it took to build Rhino's branch.

  1. Clone the repository @ https://gitlab.com/ryanthomas03/mq2dotnetcore

  2. Download/Install .NET Core 3.1 SDK @ https://dotnet.microsoft.com/download/dotnet-core/thank-you/sdk-3.1.301-windows-x86-installer

  3. Go to the local copy of your repository, and create a file in the root named local.env.props

  4. Open local.env.props with a text editor and add the following XML:
    XML: 
    <Project>
<PropertyGroup>
<MQ2SourceRootFolder>C:\Users\delin\source\repos\VeryVanilla</MQ2SourceRootFolder>
<EQInstallRootFolder>C:\Program Files (x86)\EQ\RoF2</EQInstallRootFolder>
<MQ2InstallLiveRootFolder>"C:\Program Files (x86)\EQ\ROF RG MQ2\MQ2DotNetCore"</MQ2InstallLiveRootFolder>
</PropertyGroup>
</Project>
    1. The MQ2SourceRootFolder needs to point to your local MQ2 Code base
    2. The EQInstallRootFolder is currently unused
    3. MQ2InstallLiveRootFolder is where your copy of MQ2 that you run. Please note, if your path includes spaces like mine does, you need to quote this path.
      1. Due to my need to quote the path, I added the \MQ2DotNetCore inside this path instead of it being appended in the DeployFiles task in MQ2DotNetCore.csproj file. If you need to do this as well, you can find the project file as the root folder under the src path in Visual Studio MQ2DotNetCore. If you right click on this entry and select >Edit Project File you will see the node you need to edit on line 29. <MQ2DotNetCoreDeployFolder>$(MQ2InstallLiveRootFolder)</MQ2DotNetCoreDeployFolder>

      2. If you DO NOT plan on using the automated task to copy the file(s) to where they need to go (see the README.md) then you do not need to do this step. You can turn off the deployment task in the file directory.build.props also found in the root of the project. If you do not have node.js you will need turn off the deployment task.

      3. If you DO plan on using the automated task to copy the file(s) you will have to install node.js (There may be another way than this, but I just winged it)
        1. Install Node.js
        2. Open an Elevated PowerShell instance and cd to the repo\scripts directory
        3. You will have to install the following packages: "date-fns, fs-extra, klaw, yards, path"
          1. You do this by "npm install date-fns" etc for each of them
  5. There is a copy paste error in the file MQ2DotNetCoreLoader.cs on line 11.
    Rename the DLL referenced from ..\MQ2DotNetLoader.dll to ..\MQ2DotNetCoreLoader.dll

  6. Build > Build Solution. If you have any issues, message me or post them here.
 
You can override the DeployXXX values to false in your local.env.props file if you don't want to have it auto deploy the files on build and/or don't want to install the nodejs runtime that the deploy script uses.

[CODE lang="xml" title="Props values"]<PropertyGroup>
<EQInstallRootFolder>C:\Users\Public\Daybreak Game Company\Installed Games\EverQuest</EQInstallRootFolder>
<MQ2InstallLiveRootFolder>C:\Downloads\EQ\VeryVanilla\Live\Release</MQ2InstallLiveRootFolder>
<MQ2SourceRootFolder>C:\Source\VeryVanilla\</MQ2SourceRootFolder>

<!-- Deploy after build tasks require node.js is installed and local packages are installed via yarn/npm -->
<DeployMQ2DotNetCoreFilesAfterBuild>true</DeployMQ2DotNetCoreFilesAfterBuild>
<DeployProgramFilesAfterBuild>true</DeployProgramFilesAfterBuild>
</PropertyGroup>[/CODE]

I'm working on an update to allow for more fine grained control of logging. I'll try to include an update for the deploy script so that it works with spaces in the path directly in the local.env.props values.
 
Last edited:
I've got the spaces in the deploy task problem figured out. When the path's have a trailing backslash at the end of them, the string ends up being treated as escaping the quote around the argument which was causing it to blow up. I've update them to trim the trailing slash off to prevent this and it's working for me locally with spaces in the paths. No need to put any quotes around your values. I'll let you know when I merge my changes into master in case you want to update.

XML: 
<Project>
    <PropertyGroup>
        <EQInstallRootFolder>C:\Users\Public\Daybreak Game Company\Installed Games\EverQuest</EQInstallRootFolder>
        <MQ2InstallLiveRootFolder>C:\Downloads\EQ\VeryVanilla\Test With Spaces</MQ2InstallLiveRootFolder>
        <MQ2SourceRootFolder>C:\Source\Test With Spaces\VeryVanilla\</MQ2SourceRootFolder>
    </PropertyGroup>
</Project>
 
We just ran into this the other day. Preferring to keep quotes we just dropped the trailing backslash (or appended to it pre-script).
 
Here is my pull request with the fix for whitespace in the deploy files tasks plus my changes to support fine grained logging configurations. https://gitlab.com/ryanthomas03/mq2dotnetcore/-/merge_requests/1
I'll wait a few days before merging in case anyone on here wants/has time to review or provide feedback.

Sorry it took so long to get back to this. I had the initial changes done for the fine grained logging on Wednesday but I wasn't able to actually test it and resolve the bugs until today.
 
I myself have been busy IRL/Work so haven't looked.
I will try to get back in it next week.
 
I can't build :\

The build server is always using new folders, so we can't set a constant folder in local.env.props
 
Redbot said:
I can't build :\

The build server is always using new folders, so we can't set a constant folder in local.env.props
Click to expand...


local.env.props should support relative paths... does your gitlab let you run bash or powershell scripts before a build?
Could generate the folders to use and have a local.env.props file that you check into git / copy into the build process.
 
This is long over due; I can get side tracked from EQ quite often, but thank you for this so much. This was/is exactly what I was looking for for a very long time. Now I need to try it :).
 
Starting a project and trying to use this. One issue i found is that once you load the program you have to completely shut down eq in order to rebuild and replace the binary. I don't know a ton of c++ so before i dig in was wondering if anyone knows how to modify the dotnetcoreloader to unload the program dll when /netcoreend <program> is called? Right now it stops the program but does not unload it.

Would make things much faster to test and play with if that worked.
 
Hi all, newbie here. Back to EQ from early 2000s. Was surprised to see all the tooling available here.

I saw this plugin and decided to play around to see if I can get it working. So far I have it running and partial functionality going. I found a few forks and the dotnetcore version but nothing recent. The TLO wrapper seem broken because of a bad struct mapping and likely many other changes from the last 2 years.

Im going to keep chipping away at it for a while and see where things go.

If there are any more recent forks or private versions let me know, would love to help contribute.
 
I've made some progress last night with CppSharp to generate wrappers for interop so that there is no need to manually write or maintain it.
 
Ended up ditching CppSharp. I could only get it to generate parts which weren't of much use and ended up stripping away most of the project which became such a chore that it didnt help much with automation.
After that I added some more helper methods to the loader to avoid mapping struct offsets and fixed the type factory and most TLO. There are still loads of data type props to fix etc but it is running and I have a little test program being injected that I can debug =)

Fork with my current branch is here: https://github.com/anthony-spruyt/MQ2DotNet/tree/bugfix/bring-up-to-date-with-latest-mq

Disclaimer: I am by no means a C++ guru, very much a noob; I had some training in University back in the 2001 but mostly been in the managed world for the last 20 years.
 
aspruyt said:
Ended up ditching CppSharp. I could only get it to generate parts which weren't of much use and ended up stripping away most of the project which became such a chore that it didnt help much with automation.
After that I added some more helper methods to the loader to avoid mapping struct offsets and fixed the type factory and most TLO. There are still loads of data type props to fix etc but it is running and I have a little test program being injected that I can debug =)

Fork with my current branch is here: https://github.com/anthony-spruyt/MQ2DotNet/tree/bugfix/bring-up-to-date-with-latest-mq

Disclaimer: I am by no means a C++ guru, very much a noob; I had some training in University back in the 2001 but mostly been in the managed world for the last 20 years.
Click to expand...
There's a developer on Project Lazarus who integrated C# into MQ2 and then completely rewrote their old E3 Macro. You might be interested in looking at his work. I worked with it for a while and it was incredibly fast.
github.com

GitHub - RekkasGit/E3Next

Contribute to RekkasGit/E3Next development by creating an account on GitHub.
github.com github.com
 
Derp said:
There's a developer on Project Lazarus who integrated C# into MQ2 and then completely rewrote their old E3 Macro. You might be interested in looking at his work. I worked with it for a while and it was incredibly fast.
github.com

GitHub - RekkasGit/E3Next

Contribute to RekkasGit/E3Next development by creating an account on GitHub.
github.com github.com
Click to expand...
Awesome thanks for the info, going to check it out right now!
 
Derp said:
There's a developer on Project Lazarus who integrated C# into MQ2 and then completely rewrote their old E3 Macro. You might be interested in looking at his work. I worked with it for a while and it was incredibly fast.
github.com

GitHub - RekkasGit/E3Next

Contribute to RekkasGit/E3Next development by creating an account on GitHub.
github.com github.com
Click to expand...
Ooh wish I saw this earlier!!

This plugin https://github.com/RekkasGit/MQ2Mono is what ties it all together.

Going to have some fun with this.
 
Ive taken a good look now at both MQ2Mono and MQ2Dotnet and gotten a much better idea of how each work as well as the MQ2 native code now. MQ2DotNet has so much more work put into it though with all the datatype wrappers and factory. I have been working on getting it up to date with the latest MQ2 data type definitions and making good progress.
 
Only 14 more data types to update, must say I underestimated the effort to get this up to date severely hah. Then I need to check if there are any new TLO's and also plugin TLO's that are worth capturing.
 
aspruyt said:
Only 14 more data types to update, must say I underestimated the effort to get this up to date severely hah. Then I need to check if there are any new TLO's and also plugin TLO's that are worth capturing.
Click to expand...
Something about that statement just doesn't sound right at all. This plugin needing to individually code around all the data types of MQ.. MQ2Lua and MQ2Mono both seem to avoid needing to do that.

MQ2Lua is pretty flexible with its userdata able to point back to any mq datatype.
MQ2Mono may cheat a bit by using its MQ.Query<T>("..."); which will always result in some primitive like bool, int, string and never an MQ datatype like Spawn. May be a bit of a weakness that you have to keep doing a full TLO query each time.

Just seems like a pain in terms of extensibility if any time there is a new datatype, if you want this plugin to be able to use it, you have to add support it into this plugin as well. Is there not some generic implementation that could handle any MQ datatype?
 
You must log in or register to reply here.
Release .NET / C# Plugin and "Program" API
Next unread thread 🦁

Users who are viewing this thread

Total: 1 (members: 0, guests: 1)
Back
Top
Cart

Cart

Total -