Problem with reentering LUA scripts

[funkyp]

Hi Pete,

I have a problem with reentering a LUA script library which produces a weird problem that FSX/FSUIPC stops executing LUA scripts, still accepts key commands and jystick buttons via the FDSUIPC button programming and hangs, at any actions which affects the screen mode, e.g. suspending the sim for showing a dialogue windows.

The problem has become obvious after using the bufferpools=0 tweak (or similar) which results in reduction of stuttering and improving visual performance of FSX. So far I could track this down to terminating and restarting LUA scripts. As long as I don't attempt this the SIM runs normal.

Maybe this is an old one as I have read here in this older post from Guenseli:

http://forum.simflig...fsx%20%20freeze

In the readme of the newest 4.749f version I found:

24. An error in the critical section synchronisation in the Lua interpreter could cause FS to hang when attempting to "Kill" (terminate forcibly) a thread which is resident and processing events. This would have been quite rare (but probably rather less rare the more events being processed), but it is now fixed.

So I was optimisitic that my problem had something to do with that, but unfortunately persists. In the FSUIPC-log I fond eventually these two last lines:

1718943 *** LUA Error: ... (x86)\Microsoft Games\Microsoft Flight Simulator X\Modules\FsxViews.lua:6: '<eof>' expected near 'if'

1722672 *** LUA Error: ... (x86)\Microsoft Games\Microsoft Flight Simulator X\Modules\FsxViews.lua:6: '<eof>' expected near 'if'

This is the questionable code:

-- 1 : Pan right with bank
-- 2 : Pan left with bank
-- 3 : Pan reset

-- 1 : Pan right with bank
if ( ipcPARAM == 1 ) then
   PanViewHeading = 67.5/180*16384
   ipc.control(66504,PanViewHeading)
   SinHeading = math.sin(PanViewHeading/16384*math.pi)
   ACbank = ipc.readSD(0x57C)/2^16
   PanViewPitch = -SinHeading*ACbank
   ipc.control(66503,PanViewPitch+1)
   PanViewPitchOld = PanViewPitch
   while 1 do
	  ACbank = ipc.readSD(0x57C)/2^16
	  PanViewPitch = -SinHeading*ACbank
	  if math.abs(PanViewPitchOld - PanViewPitch) > 0 then
		 PanViewPitchOld = PanViewPitch
		 ipc.control(66503,PanViewPitch)
	  end
	  ipc.sleep(33)	-- slows down the loop
   end
-- 2 : Pan left with bank
elseif ( ipcPARAM == 2) then
... (and so on)
[/CODE]

I have attached the complete lua-code, log- and ini-file to this thread. The function is an alternative PAN-command which keeps your sight parallel to the horizon during banking. It has three entries: "1" and "2" for activating a fixed angle pan left or right, and "3" which resets the view to normal. The problem was provoked by quickly changing the views, e.g. during a circuit pattern.

The function principle is that starting the function with codes 1 or 2 triggers an endless loop, adjusting the view axis according to the bank angle. This loop is forcibly killed (on purpose) by each new call, typically by code 3, which resets the view direction and terminates the script normally. The weird error code pointing to line 6 (where the actual code starts) let me think of some memory corruption problem.

I have the same problem with another LUA script, which works likewise. In both the problem seams to disappear (or at least become much less frequent) if I don't apply the Bufferpools tweak and reduce CPU load by that.

Of course my programming style is not the smartest, but it allows me to take easily advantage of FSUIPC button mapping fdacilities that makes the use of such special commands very flexible and easily aircraft-specific. This would become much more complicated when checking hardcoded button events from within the script.

best regards,

Peter

FsxViews.zip

[Pete Dowson]

I have a problem with reentering a LUA script library which produces a weird problem that FSX/FSUIPC stops executing LUA scripts, still accepts key commands and jystick buttons via the FDSUIPC button programming and hangs, at any actions which affects the screen mode, e.g. suspending the sim for showing a dialogue windows.

Hmmm. Those are a very strange selection of symptoms! The problems in the past of fast forced termination and re-execution of Lua plug-ins were all due to stack overflow and caused FS to hang completely. I can't imagine what causes only parts to hang.

I'm not sure what you mean by "any actions which affect the screen mode", though -- only dialogues, not views in FS, changing scenery, gauges etc?

The problem has become obvious after using the bufferpools=0 tweak (or similar) which results in reduction of stuttering and improving visual performance of FSX.

I always use BufferPools=0 since I got a GTX 580 video card with 3Gb memory on board. Before, with a GTX 480 with 1.5Gb it created problems including video hangs. The best thing then was to use the RejectThreshold instead..

So far I could track this down to terminating and restarting LUA scripts. As long as I don't attempt this the SIM runs normal.

Do you mean forcibly terminating, or re-execution which does the same thing? Unless your plug-ins are really short, and preferably only re-executed by Repeat actions on button assignments, you should really consider using the Event system.instead, because with longer complex plug-ins you really have no idea what state things are in when their threads are forcibly terminated.

So I was optimisitic that my problem had something to do with that, but unfortunately persists. In the FSUIPC-log I fond eventually these two last lines:

1718943 *** LUA Error: ... (x86)\Microsoft Games\Microsoft Flight Simulator X\Modules\FsxViews.lua:6: '<eof>' expected near 'if'

1722672 *** LUA Error: ... (x86)\Microsoft Games\Microsoft Flight Simulator X\Modules\FsxViews.lua:6: '<eof>' expected near 'if'

Hmm. Never seen that error before. Expecting the end-of-file at line 6 seems odd. I assume there was nothing wrong with the file at that line?

This is the questionable code:

That's a very long Lua to expect to forcibly terminate and restart quickly. Have you considered splitting into three separate Lua's, one for each of the three parameters? Or keeping them together but use a timer event or a continuous loop and use LuaValue to change the ipcParam which it can act upon? You would then have the plug-in startup with an ipcReady.lua call or using the [Auto] option in the INI file.

I have attached the complete lua-code, log- and ini-file to this thread. The function is an alternative PAN-command which keeps your sight parallel to the horizon during banking. It has three entries: "1" and "2" for activating a fixed angle pan left or right, and "3" which resets the view to normal. The problem was provoked by quickly changing the views, e.g. during a circuit pattern.

I'll see if I can reproduce the problem here with what you've supplied, but I don't know when I can get to it. It might not be till I return after Christmas & NY -- after January 6th.

The function principle is that starting the function with codes 1 or 2 triggers an endless loop, adjusting the view axis according to the bank angle. This loop is forcibly killed (on purpose) by each new call, typically by code 3

As a simple change to test initially you could instead send that 3 value by LuaValue and test for it in the loop so you can tidy up and terminate normally. That would be a far preferable method in any case.

, which resets the view direction and terminates the script normally. The weird error code pointing to line 6 (where the actual code starts) let me think of some memory corruption problem.

I have the same problem with another LUA script, which works likewise. In both the problem seams to disappear (or at least become much less frequent) if I don't apply the Bufferpools tweak and reduce CPU load by that.

Certainly, if it is so affected by that tweak, it is suggestive of corruption outside of my control.

Regards

Pete

[funkyp]

Hi Pete,

thanks for your immediate reply, wow that fast, many thanks!

I'm not sure what you mean by "any actions which affect the screen mode", though -- only dialogues, not views in FS, changing scenery, gauges etc?

This means, e.g. trying to open FSUIPC config dialogue from the Addons-Dropdown menu, which terminates the 3D window and goes into the dialogue-mode. Same with calling the weather menu etc... Sometimes it happens by switching to another application and clicking into FSX 3D window.

I always use BufferPools=0 since I got a GTX 580 video card with 3Gb memory on board. Before, with a GTX 480 with 1.5Gb it created problems including video hangs. The best thing then was to use the RejectThreshold instead..

I'm using 9600GT 1Gb, Nvidia inspector reports 500 Mb (+/-) of RAM usage, so not the obvious out of VRAM reason. RejectThreshold, Usepools=0 or 4000, and even =32MB produce the same behavior, i.e. better visual performance but crashing, when I involve LUA scripting. As long as they work, I recognize some sluggishness in button processing, apparently events get lost, probably due to high CPU load. Without the tweaks my CPU chokes in the 80-90% utilization, with the tweaks I get into the 98% range (thus leaving less capacity for concurrent threads)

Do you mean forcibly terminating, or re-execution which does the same thing? Unless your plug-ins are really short, and preferably only re-executed by Repeat actions on button assignments, you should really consider using the Event system.instead, because with longer complex plug-ins you really have no idea what state things are in when their threads are forcibly terminated.

Actually I mean re-executing with some other parameter, diverting to some other action, thus forcibly terminating the thread if it has not finished normally. However, the codelets are designed to re-assemble their environment from scratch each time, so there shouldn't be problems with non-initialized variables. In order to avoid clogging the CPU there is an ipc.sleep in those parts running in a loop. Most likely, this is the state in which the re-call stumbles in.

Hmm. Never seen that error before. Expecting the end-of-file at line 6 seems odd. I assume there was nothing wrong with the file at that line?

The same part of the code has been executed several times before the freeze.

That's a very long Lua to expect to forcibly terminate and restart quickly. Have you considered splitting into three separate Lua's, one for each of the three parameters? Or keeping them together but use a timer event or a continuous loop and use LuaValue to change the ipcParam which it can act upon? You would then have the plug-in startup with an ipcReady.lua call or using the [Auto] option in the INI file.

The downside of splitting is that the drop-down box of FSUIPC button mapping get's quickly overcrowded. Therefore I try to build libraries for well defined sciopy (say an aircraft) were a number off short functions are addressed by the calling parameter. But I understand that loading and compiling can take time. Well, actually, only one piece is executed at a time (1, 2, or 3) with the exception of the while-loops which are designed to presist and update the view-pitch axis constantly regarding the bank of the aircraft until another call to the script is made. Actually meanwhile I have tried your proposal and it seems a workaround (FsxViewsEvents.zip). This one is loaded once at startup (or triggered by a key for testing purposes). However, I'm a bit concerned about the underlying problem, since re-entrance may happen at any time, by incidence and possibly shoot the sim. As an example I have a trim wheel (Saitek Multipanel) which fires sequential virtual button events (by SPAD) which are handled by a LUA script to operate the freeware Constellation's autopilot pitch reference. Or could it be, that the problem arises from the LUA-compiler which may go haywire if captured in the middle of compiling the same script? You said previously, that at each call the script is compiled new.

I'll see if I can reproduce the problem here with what you've supplied, but I don't know when I can get to it. It might not be till I return after Christmas & NY -- after January 6th.

I'd like to do anything but keeping you away from enjoying Christmas!

As a simple change to test initially you could instead send that 3 value by LuaValue and test for it in the loop so you can tidy up and terminate normally. That would be a far preferable method in any case.

Good idea, will try this.

Certainly, if it is so affected by that tweak, it is suggestive of corruption outside of my control.

So far it only occured in combination with LUA, not even with the FSUIPC button mapping to standard FSX commands.

Again, thank you very much for your effort, and please keep on going with FSUIPC (but enjoy X-max). FSUIPC is my Swiss Army Knife for FSX!

best regards,

Peter

[Pete Dowson]

The downside of splitting is that the drop-down box of FSUIPC button mapping get's quickly overcrowded.

But it is kept sorted and you don't need to scroll through, just open the drop-down then enter the first letter or two or three of the commands you are looking for. It is very quick that way.

However, I'm a bit concerned about the underlying problem, since re-entrance may happen at any time, by incidence and possibly shoot the sim. As an example I have a trim wheel (Saitek Multipanel) which fires sequential virtual button events (by SPAD) which are handled by a LUA script to operate the freeware Constellation's autopilot pitch reference. Or could it be, that the problem arises from the LUA-compiler which may go haywire if captured in the middle of compiling the same script? You said previously, that at each call the script is compiled new.

No idea really, though you could try pre-compiling it as a test (you'd need the luac component from the Lua website I think).

In order to investigate I'd need to be able to reproduce the problem. In all cases I've managed so far to cause repetitive Lua execution hangs (never seen any other result) I've found a solution. Your problem, not being a hang, surprises me. I just have no idea how it might occur.

Regards

Pete

[funkyp]

Hi Pete,

thanks again for staying online.

But it is kept sorted and you don't need to scroll through, just open the drop-down then enter the first letter or two or three of the commands you are looking for. It is very quick that way.

Yes I know, but all script commands starts with "Lua abc..." and when it comes to "LuaSetValue abc ..." you have to type the whole LUA category before it comes to my user code name.

No idea really, though you could try pre-compiling it as a test (you'd need the luac component from the Lua website I think).

Promising, and I tried it immediately, but no difference, eventually. It seems to me, though, that I had to do more "chaotic" switching between the views than with uncompiled version in order to produce the hang. It comes back again to the termination of a script and restarting it.

I tried also another solution in between: This is an endless loop, starting at the beginning with FSUIPC and evaluating the current ipcPARAM and resetting ipcPARAM to zero after handling the request. There is a problem because FSUIPC does not rewrite ipcPARAM when LuaValue is called withe same parameter on the same script (for performance reasons, I suppose). The workaround is to call "LuaValue <script> 0" upon release of the button. This again can lead to missing events if the physical buttons produce only short pulses (on/off) like the Saitek rotaries with SPAD. The workaround for that again is inverting the button logic, i.e. to excecute the desired command upon release of the button. However, I'm afraid, that I don't understand anything anymore from my own code in three weeks from now :cool:.

best regards, Peter

Edit (18.12.): Found one for the update problem: ipc.macro("LuaValue <script>",0) does it.

[Pete Dowson]

Instead of using ipcPARAM and Lua SetValue, you could use the Lua flags. You can set and clear those from FS and in your script.

Pete

[funkyp]

Hi Pete,

have modified my critical code sections now, so that re-entrance does no longer happen, and it's rock-solid now.

FSX + Bufferpools=0 + FSUIPC rocks!

I've sticked to the "LuaValue" method, as it required less code changes and it's doing the job. The procedure is now this: The script is initialized on loading FSUIPC, rsp. the A/C. A button calls "LuaVale <script> #param" on press, which captured by the event.param. In the script the parameter is copied to a local variable and ipc.macro("LuaValue <script> 0") resets the parameter state for FSUIPC main. There is no need anymore for a mandatory button release code, or inverted logic.

However, the advantage of the flag method is a better way to combine multiple button events, e.g. one code upon press and a different one upon release.

Cheers, and have a merry Christmas

Peter

[Pete Dowson]

have modified my critical code sections now, so that re-entrance does no longer happen, and it's rock-solid now.

Good. That's always going to be a better way. But I will see if I can repro your original problem to see if it is preventable. If it is due to any code I have control over I should be able to fix it. If it is deep in the Lua code I wouldn't want to fiddle with it too much. I don't really understand enough of it, yet.

[LATER]

I've spent a good half hour almost continuously and quickly pressing and releasing buttons assigned to FSXviews.lua with press parameters 1 or 2 and release parameters 3, and seeing that the Lua is continually killed and restarted, and at no time was there any problem. How often was your original 'crash' occurring? I have got Bufferpools set to 0 to encourage the problem. Maybe it's because I have 8 Gb memory on this PC? Or maybe it's because it's a very fast PC? Does it only happen in full screen mode or both full screen and windowed? What is your typical FSX frame rate? (Ah, I see 27-29 fps in the Log you included. Mine is nearer 50-60 when flying, away from the detailed airports).

I'm wondering if this is really more of a video driver problem than a corruption caused by killing and resurrecting threads. Except for the very odd "eof expected at line 6" error, your symptoms certainly seem very much more video related than anything i've encountered before due to Lua thread killing -- the latter problems have always been a case of stack overflow due to too many resources associated with the threads piling up. That certainly crashes FSX completely, or hangs it irrevocably with no responses at all. I'm reasonably sure I have all cases where that can occur sewn up now (though maybe thee's always another! ;-) )..

Unless I can reproduce it I'm going to have to assume its video driver related and suggest a possible update in that area.

Regards

Pete

[funkyp]

Hi Pete,

here some answers:

How often was your original 'crash' occurring?

Or maybe it's because it's a very fast PC?

Does it only happen in full screen mode or both full screen and windowed?

I usually took me only a few minutes to produce the crash. I think computing power margin is the key. On my rig there is not much left, maybe it would require unsane faster execution of the buttons, than manually possible, to create the problem at your side. I'd tested it in windowed mode only.

For the time being there is the workaround with the resident routines. With that I have no problems at all. The only thing what happens is, if I crank all (!) scenery sliders to the max I'm getting my NV9600GT choked at 100% GPU load, still flyable at 15fps,but producing the "polygon flashing" and mesh spikes, but no crash.

best regards,

Peter

(W7/64 HP, 4GB, NV9600GT 1GB, FSX/ACC)