Problem with loading dll lua modules

[adrem]

Hi Pete and John,

I was trying to make lua dll library that I could import from the FSUIPC lua environment and it crashes under certain circumstances (Crash C0000005 in the FSUIPC log). For example, it always crashes if I throw a lua error or try to call the lua print function from C code. From my understanding, the issue is that I'm linking my library statically to lua instead of linking to the lua inside FSUIPC. You can find several posts on the internet related to this, for example here. Now, of course, I can't link to the FSUIPC lua because FSUIPC doesn't expose the lua functions, so my question is: would it be too much to ask you to do that?

I can also reproduce this crash without FSUIPC if I compile a program with statically linked lua and a dll with statically linked lua that exports a single function that throws a lua error. If I instead link both the exe and the lua module to the same lua dll, everything works fine.

[Pete Dowson]

3 hours ago, adrem said:

I was trying to make lua dll library that I could import from the FSUIPC lua environment and it crashes under certain circumstances (Crash C0000005 in the FSUIPC log). For example, it always crashes if I throw a lua error or try to call the lua print function from C code.

There's no way I know you can do that. Why would you try that? In a C DLL you surely can use Windows functions directly?

3 hours ago, adrem said:

You can find several posts on the internet related to this, for example here.

Sorry, that's all beyond me. 

3 hours ago, adrem said:

FSUIPC doesn't expose the lua functions, so my question is: would it be too much to ask you to do that?

Yes, I'm sorry. But I think there is far too much complication involved. There are a lot of internal structures wrapped around the Lua threads. It would be a huge amount of work and very error prone.  In addition of course there is still a lot of development work ahead for us on the MSFS front.

The Lua facilities are provided as a way of extending FSUIPC's facilities by relatively simple scripts. I think if you are going to need to do the sorts of things you are implying then you really should be writing a complete external program, not an overly complex plug-in script.

Pete

 

[adrem]

8 hours ago, Pete Dowson said:

There's no way I know you can do that. Why would you try that? In a C DLL you surely can use Windows functions directly?

There seems to be a misunderstanding here - I just want to use the luaL_error function 😄 I'm not doing anything super complex. Whenever I call luaL_error, the plugin crashes because of the dual lua issue.

[Pete Dowson]

2 hours ago, adrem said:

I just want to use the luaL_error function 😄 I'm not doing anything super complex. Whenever I call luaL_error, the plugin crashes because of the dual lua issue.

The Lua C functions are internal to FSUIPC and many of them have been modified for fuller integration.  The LuaL_error function simply terminates the thread and adds a message to the FSUIPC or Lua log. 

The C functions are not exported. I really don't see the need. Why not implement your own error reporting function since you are making a C program?

I'm also a bit puzzled as to what you are writing a C program yet wanting to use it within the simple scripting facilities FSUIPC provides. You seem to misunderstand why the facilities are provided.

Pete

 

[adrem]

5 hours ago, Pete Dowson said:

Why not implement your own error reporting function since you are making a C program?

I'm not making a C program, just a small library to be able to access some Windows API  functionality from my lua script (that is started by the FSUIPC lua interpreter). Something like this:

int divide(lua_State* L)
{
	if (lua_tonumber(L, 2) == 0)
		luaL_error(L, "The divisor must not be 0");
	lua_pushnumber(L, lua_tonumber(L, 1) / lua_tonumber(L, 2));
	return 1;
}

extern "C"
__declspec(dllexport) int luaopen_mylualib(lua_State* L)
{
	lua_pushcfunction(L, divide);
	lua_setglobal(L, "divide");
	return 0;
}

Then I can import the dll and use it from lua like this:

require "mylualib"
divide(2, 0)

 

[Pete Dowson]

19 hours ago, adrem said:

I'm not making a C program, just a small library to be able to access some Windows API  functionality from my lua script (that is started by the FSUIPC lua interpreter). Something like this:

Why not simply use the C DLL to access the Windows functions not exposed in the Lua libraries provided, and leave all the Lua stuff in the Lua script. Which Windows functions do you need? You don't show any there.

Doesn't the 'L' in your LuaL_error call (and the others) refers to a structure created in your loaded Lua interpreter. The structure used within FSUIPC is not at all the same. I suspect trying to use it will just cause even more problems.

Why not simply return the error to the calling Lua script and use ipc.log to log the error and then terminate the thread. That's all the C function LuaL_error would do in our implementation.

Pete

 

[adrem]

Sorry to bring this up again, but while I've found an ugly workaround for the error function crashing, I've started to experience crashes under normal operation that stem from the same cause which is me having to statically link lua with my dll module.

On 1/25/2021 at 11:28 AM, Pete Dowson said:

Doesn't the 'L' in your LuaL_error call (and the others) refers to a structure created in your loaded Lua interpreter.

No, my code doesn't create a new lua state. It's the lua state that is created by the FSUIPC interpreter. My dll is just a library. It can be imported from a lua script using the require function.

The crashes happen when the lua core inside my dll tries to run the garbage collector. The problem, ultimately, is that Lua has a static variable 'dummynode_' in ltable.c, and when I include a lua core in my dll instead of linking dynamically to the lua API functions inside FSUIPC6.dll, that results in two Lua cores with their own static variables that operate on the same Lua state.

Again, you can find a lot of posts about this on the internet. The only way for dll libraries imported from lua to work properly is for those libraries to dynamically link to the same lua API functions that created the lua state. I will be very grateful if you will allow that. For now, I have to resort to forcibly running the garbage collector from lua to prevent my C code from doing it and that consumes a lot of CPU cycles.

On 1/25/2021 at 11:28 AM, Pete Dowson said:

Which Windows functions do you need? You don't show any there.

I didn't want to tell that because I basically made my own HID library instead of using the FSUIPC one 😄 I tried the FSUIPC one first, but then I encountered the same issue as described here: https://forum.simflight.com/topic/68512-hid-devices-in-lua/.

 

[John Dowson]

5 hours ago, adrem said:

The only way for dll libraries imported from lua to work properly is for those libraries to dynamically link to the same lua API functions that created the lua state.

I'll check with @Pete Dowson but I don't think we can do this as then we would either have to install the required lua.dlls with FSUIPC, or put a requirement on the user to install these if they want to use lua. Can you not include the lua core in your dll and just dynamically link to the one in the FSUIPC6.dll?

 

 

[adrem]

24 minutes ago, John Dowson said:

Can you not include the lua core in your dll and just dynamically link to the one in the FSUIPC6.dll?

Sorry, I'm not sure I'm following. How can I link to both the core I'm including and to FSUIPC6.dll and why? The only reason that I'm including a lua core in my dll module is because I can't link to the Lua API functions (the ones starting with "lua_") inside FSUIPC6.dll because FSUIPC6.dll doesn't export them.

[Pete Dowson]

I am still puzzled by your request. You are evidently capable of writing a C program, yet to seem to want to use what is and always has been intended to be simple scripting facilities to allow users to make relatively minor additions to FSUIPC's capabilities. Don't you think you are misunderstanding this, still?

Why not, instead, write a separate application which interfaces to FSUIPC in the way it was originally provided for? This would be free of the constraints you are complaining about, and be easier to start, control and amend in any way you wanted. What is the real reason you are trying to exceed the controlled environment imposed on an FSUIPC Lua plug-in?

BTW I note contradictions in what you are asking for -- "lua_" functions and "LuaL_", or just "Lua_"? And precisely which ones and why? I'd need to know in any case because some of these functions have been changed to suit the way FSUIPC's environment works.

Your examples posted earlier don't accomplish anything which can't be done within the current provisions in any case, so don't really add anything to your request. If this was because you don't want to expose details of your plans because you intend to offer it commercially, then try using email. Mine is petedowson@btconnect.com. I don't mind discussing programming with a programmer, and maybe we can arrive at a more suitable way of achieving what you want.

Pete

 

[adrem]

5 hours ago, Pete Dowson said:

This would be free of the constraints you are complaining about, and be easier to start, control and amend in any way you wanted

Not really. My 'program' is a simple library for interacting with cockpit controls of FSLabs aircraft. All it does iternally is invoke mouse macros, which would be my first problem if I tried to do it outside of FSUIPC - I wouldn't know how to implement the mouse macro functionality myself (I tried the FireMouseRectClick function from the PDK, but it crashes my sim for some reason). I intend to freely share it with other people to provide them a very easy to use keyboard and joystick binding interface; here are a couple of examples:

Bind {key = "F", onPress = Bind.cycleSwitch(FSL.OVHD_EXTLT_Strobe_Switch)}
myJoystick = Joystick.new(0x06A3, 0x0C2D)
myJoystick:onPress(5, FSL.OVHD_EXTLT_Land_L_Switch, "ON", FSL.OVHD_EXTLT_Land_R_Switch, "ON")
myJoystick:onPress(6, function() print "hello" end)
myJoystick:onAxis("Y", FSL.GSLD_FCU_DimLt_Knob)

It makes perfect sense to launch it inside the FSUIPC environment which already provides facilities for launching Lua scripts both manually and automatically - why would I want to reinvent the wheel?

Now, the Bind function is just a wrapper around event.key and I tried to make Joystick a wrapper around the com library functions first, but I encountered the exact same problem that is described here in the opening post: https://forum.simflight.com/topic/68512-hid-devices-in-lua/, so I wrote my own HID library as a C module.

6 hours ago, Pete Dowson said:

BTW I note contradictions in what you are asking for -- "lua_" functions and "LuaL_", or just "Lua_"? And precisely which ones and why?

Well, the lua API which comprises all the "lua_"' functions should theoretically be enough to write a self-contained lua application. Lauxlib.c says " This file uses only the official API of Lua. ** Any function declared here could be written as an application function." I just thought you would be more inclined to do it if there would be less functions 😄 That said, an application using lua would usually export all the "lua_" and "luaL_" functions. I actually don't know which functions I would need exactly as I don't call them directly (I use a library called sol2), but I don't understand your reluctance to just export all of the "lua_" and "luaL_" functions to enable anyone to make a dll Lua module and import it into the FSUIPC environment - it's not just about me.

By the way, why reimplement the error function to catch errors inside of it instead of using the existing interface:

if (luaL_dofile(L, "script.lua")) {
  const char* errMsg = lua_tostring(L, 1);
  ...
}

 

[Pete Dowson]

14 hours ago, adrem said:

All it does iternally is invoke mouse macros, which would be my first problem if I tried to do it outside of FSUIPC - I wouldn't know how to implement the mouse macro functionality myself

But "Mouse macros" are only a type of regular macro, and any macro can be invoked from any FSUIPC or WideFS client by simply writing its parameter and name to offsets 0x0D6C then 0x0D70 respectively.

14 hours ago, adrem said:

I intend to freely share it with other people to provide them a very easy to use keyboard and joystick binding interface; here are a couple of examples:

Not sure which are the "mouse macro" references in your example?

14 hours ago, adrem said:

It makes perfect sense to launch it inside the FSUIPC environment which already provides facilities for launching Lua scripts both manually and automatically - why would I want to reinvent the wheel?

But both FSUIPC and WideClient are prefectly capable of automatically launching application programs and the offset interface provides all you might need. On top of that the facilities now offered by Paul Henty's .NET DLL library makes everything really easy if you want to use a .Net managed interface instead of C++.

14 hours ago, adrem said:

I encountered the exact same problem that is described here in the opening post: https://forum.simflight.com/topic/68512-hid-devices-in-lua/, so I wrote my own HID library as a C module.

But that talks about the Lua HID library. Why do you need to use a separate HID access for standard joystick devices? Are you saying there's a noticeable delay in FSUIPC's native joystick detection?

14 hours ago, adrem said:

I don't understand your reluctance to just export all of the "lua_" and "luaL_" functions

Mainly because many of these have been changed to suit the internal machinations in FSUIPC. But also I don't see any justification. I think yours is a misuse of the whole idea of the Lua plug-in facilities we provide.

Pete

 

[adrem]

1 hour ago, Pete Dowson said:

Mainly because many of these have been changed to suit the internal machinations in FSUIPC

And what would be the difference from FSUIPC's point of view if people were able to call them direclty rather than through lua code?

1 hour ago, Pete Dowson said:

But also I don't see any justification

With all due respect, I think it's you who is missing the point. In my opinion, the question should be "what would be the justification to prevent people from using a standard lua feature (importing C modules)" rather than the other way round.

[Pete Dowson]

Sorry, but i think you should seriously look at the possibilities with a separate EXE interfacing to FSUIPC / WideClient.  It would be more efficient and more flexible for the users, including your good self.

Pete

 

[adrem]

1 hour ago, Pete Dowson said:

It would be more efficient and more flexible for the users, including your good self.

This is exactly the opposite of the truth 🙂 Let me give you a bit more context.

My addon comprises two things: a copilot add-on for the FSLabs A320 and a library for interacting with the cockpit controls that I've primarily made for the copilot thingy but can also be imported standalone in a regular FSUIPC Lua script (for example, to make keyboard and joystick bindings, as I've shown above). I've already uploaded it to the FSLabs forum a few months ago and there are people using it.

The majority of the code is written in Lua, with C++ only being used for parts where I can't use Lua (mainly the Windows Speech and HID APIs).

The copilot add-on can be extended with the user's own lua code. I know one real world pilot who has programmed his airline's procedures, including checklists. 

There are a lot of useful functions in the FSUIPC Lua library that people might want to use and they aren't going to be there if I move my add-on outside of FSUIPC. I would also have to make my own facility for launching Lua scripts, when there's already a perfectly working one provided by FSUIPC.

Seriously, it can't be that much work to mark a few dozen functions for export, what am I missing?

[Pete Dowson]

43 minutes ago, adrem said:

Seriously, it can't be that much work to mark a few dozen functions for export, what am I missing?

I still think you are making inappropriate use of the Lua plug-in facilities and you would find a much better solution if only your thought about it more.

I am no longer the developer of FSUIPC and I know John has got a lot on already with the current needs of most users, especially for MSFS.  So, I suggest you make a specific list of those Lua functions you need exporting -- not a general "all of this type ..." request, and either John or (maybe I) will look to see if there are any implications in terms of the changes we've made. Do this is a new thread, more appropriately entitled (eg "request for Lua functions to be exported") with perhaps some justification for each -- i.e how you will use it.

Then, if John decides to go ahead, it will be on his timescale according to workload, and in all likelihood will be subject to a "at user's risk" statement. John cannot afford any increase in support workload.

I'm closing this thread now.

Pete