Script to filter throttle spikes

[Pete Dowson]

True, I misunderstood your first post where you clearly stated that in an event triggered solution the plug-in would be always there and running on demand

Does this mean that a single thread is serving both events the same way it would with a sleep clause?

Sorry, you are mixing two separate issues. Whether you handle all throttles in one plug-in (and therefore one thread) or multiple plug-ins (multiple threads) is a programming issue, how you construct the code. It is nothing to do with whether you use events or never-ending loops.

If that's the case I think it's better to run each throttle in a separate pluggin

As I said, if you change to use the ipcPARAM method, as per my example, you have no choice but to use separate plug-ins for each throttle, because the parameter value you receive on changes is indistinguishable. You can't tell which throttle it came from.

EDIT: I put some ipc.log lines in both functions and it looks like it's actually a single thread

A single plug-in is a single thread, yes, of course. What else?

Is that the reason why there can only be one timer per pluggin? Is the timer function also running in the same thread? Logs seem to confirm that also

The timer operates for the thread. But that is no reason why there can only be one timer per thread (plug-in). I could implement as many as needed, but it makes things more complicated, adds more overhead, and is certainly not needed, because you can feed different things at different multiples of your one timer, as needed.

I have another pluggin for the throttle2 and a third one for the axes disconnects.

If you continue to use the disconnect method it would be better to keep the disconnect action for each throttle in its own plug-in. Jst add the timer event and the disconnect function to each throttle plug-in, with different bits set accordingly. It's a waste having a third unnecessary plug-in.

With ipcParam I'll get rid of the last one since I won't need to assign the throttles to the axes, but to the Lua pluggins instead

Not to the Lua plugins, they still need to be pre-loaded, but to the new control "LuaValue" for the plugins.

The updated versions of FSUIPC will be uploaded this afternoon.

Regards

Pete

[Pete Dowson]

The updated versions of FSUIPC will be uploaded this afternoon.

Updates now available. The Lua plugins package is also revised.

Regards

Pete

[dazz]

Downloading now. Thanks Pete

...but looks like the Lua Plugins and docs in the zip file are not updated

EDIT: question please. Did you add an ipc.getParam or something like that? If the ipcParam variable is tied to the plugin thread it's going to return the same value that was passed to the event.param function anyway. Is that right or will ipcParam be updated if the axis is moving while the event is being executed?

Reason I ask is because in the last version I read the axis right after the event is triggered:

nextV = ipc.readSW(0x3332)

I then use newV nextV and old1 (the previous value applied) to calculate the next valid value.

If I can do something like:

nextV = ipc.getParam()

where nextV will not necessarily be the same as the newV value passed to the function from ipcParam, then I'm fine. Otherwise I'll have to think of a different way to do this, or keep using the throttle disconnects and calibrated axes offset driven events.

Edited January 19, 2011 by dazz

[dazz]

Ok, I have the updated docs now:

old1 = -16384

function throttle1(newV)	
	ipc.log("newV " .. newV)

	nextV = ipcPARAM
	ipc.log("nextV " .. nextV)

	if not (newV > nextV and newV > old1) and not (newV < nextV and newV < old1) then nextV = newV end

	old1 = nextV
	nextV = (nextV / 2) + 8192
	ipc.writeSW(0x088C, nextV)
end

event.param("throttle1")

I can see however that newV and nextV will always be the same

[Pete Dowson]

I can see however that newV and nextV will always be the same

Why are you using it then?

You have

nextV = ipcPARAM

but, as documented, ipcPARAM is supplied to you as the parameter, newV. Why not simply use that? That line is the same as "nextV = newV", but why bother? Why not call the parameter "nextV" and be done?

Did you not even bother to try my version, using these facilities, posted here earlier?

And why is this back?

nextV = (nextV / 2) + 8192

And why not just send the result as the Axis Throttle1 Set control so you can calibrate it?

Pete

[dazz]

Why are you using it then?

You have

nextV = ipcPARAM

but, as documented, ipcPARAM is supplied to you as the parameter, newV. Why not simply use that? That line is the same as "nextV = newV", but why bother? Why not call the parameter "nextV" and be done?

Did you not even bother to try my version, using these facilities, posted here earlier?

And why is this back?

nextV = (nextV / 2) + 8192

And why not just send the result as the Axis Throttle1 Set control so you can calibrate it?

Pete

Pete, take it easy please. I appreciate your help very much and I definitely tried your version with the prevV thing, but as I explained earlier it's not needed anymore with the last version. I'm not polling the axis anymore, and I'm storing the old/previous value for calulation porposes, never to apply that again.

In an event driven approach, the new value will most likely not be the same as the previous one, so no need for checking if they are different before updating the axis. I even placed a log line in the else part to make sure it wasn't actually trying to aplly a value to the offset that was already the same one.

I know ipcPARAM is the same as the parameter passed to the function, that was my question, because I need a new value with this implementation as I said. A new value that ipc.readSW(0x3332) does actually provide unlike ipcPARAM.

The crappy calibration in the code is back there just so I could test the relevant part for me here, that is the ability to read the axis inside the function like with ipc.readSW(0x3332) in the offset event version

I'm not saying you should change anything, I'll find a way around that or will simply go back to the offset events, was just trying to figure out how it works

Thanks again for your time

EDIT: I can see now that reading the axis right after the function is called returns the same value, so it's actually not filtering anything. It works if a put a sleep before I read the axis, but I don't want to do that for obvious reasons. I'll simply think of a different way to program this

Edited January 19, 2011 by dazz

[Pete Dowson]

Pete, take it easy please.

Er, I am taking "it" easy! Sorry, what do you mean? I'm still only trying to provide helpful advice. If you don't want any, say so.

In an event driven approach, the new value will most likely not be the same as the previous one

The value supplied to the function by the event will always be different, as documented. The event is only triggered by a difference, so it can't be otherwise.

so no need for checking if they are different before updating the axis. I even placed a log line in the else part to make sure it wasn't actually trying to aplly a value to the offset that was already the same one.

Yes, but, sorry, i don't see the relevance of that to anything I said?

I know ipcPARAM is the same as the parameter passed to the function, that was my question

So, you knew the answer to that question, then? Sorry, I think you've lost me.

... because I need a new value with this implementation as I said. A new value that ipc.readSW(0x3332) does actually provide unlike ipcPARAM.

Oh dear; sorry: now I know you are still confused.

Your latest code won't work because you are only comparing old1 with the new value. In this section:

 nextV = ipcPARAM
        if not (newV > nextV and newV > old1) and not (newV < nextV and newV < old1) then nextV = newV end

since newV = ipcPARAM (by definition), and nextV = ipcPARAM too (by your code), that if contdion results in:

 if not (nextV > nextV and nextV > old1) and not (nextV < nextV and nextV < old1) then nextV = nextV end

In other words, it does nothing. To make it work you need THREE values, the new one (newV) and two previous ones -- you only have one old one now. For some reason you lost the other. This is your problem. why delete the vital second previous value?

To recap: the new value IS the parameter provided to the function, which IS the same as ipcPARAM. That value IS the latest value from the Axis, if you've assigned the axis to "LuaValue". What is the difference between that and the one which you erstwhile read from 3332, except the later is post calibration whilst the former is direct from the axis, which is why I suggested sending it as the FS control, not writing to the offset, so you can still calibrate it.?

I'm not saying you should change anything

I'm not intending to. Things are all working as they should. It's you not seeing this which worries me, and i'm trying to find out why!

I'll find a way around that

Find your way around WHAT? This is what I don't understand! What is it you are doubting? Please ask questions instead of apparently making assumptions.

Assuming you assigned the axis to "LuaValue ..." naming this Lua plugin, the values from the axis are coming into your function as the parameter, your "newV" (which = ipcPARAM). They ARE the new values, and different each time the axis changes.

or will simply go back to the offset events, was just trying to figure out how it works

It works exactly as I've been saying, and as documented. What is it you don't understand? Please ask the questions which you appear to be harbouring and not stating.

EDIT: I can see now that reading the axis right after the function is called returns the same value, so it's actually not filtering anything. It works if a put a sleep before I read the axis, but I don't want to do that for obvious reasons. I'll simply think of a different way to program this

Oh dear, what is it you are thinking? How are you assigning the axis, how are you "reading the axis right after the function". I see now that I have utterly confused you.

Why not please review the straight conversion of your routine i provided yesterday, earlier in this thread? Just use it for, say, Throttle1 (assigned to LuaValue xxxx1), and use your original for Throttle2, assigned as you did before (or vice versa). Then compare the filtered throttle movements, on screen, side by side. You'll see immediately that the new ipcPARAM event driven method is smoother and faster.

If you don't understand how things work, please ask questions. You seem to be making all sorts of incorrect assumptions at present, and I don't understand why because you don't state them.

Believe me, I'm only trying to help here, and i'd like to know what it is you don't understand because I might need to document things a little more explicitly somewhere.

Pete

[dazz]

Believe me, I'm only trying to help here, and i'd like to know what it is you don't understand because I might need to document things a little more explicitly somewhere.

I don't think you need to change anything in the docs or anywhere else, just that I didn't make myself clear enough.

You said that I need three values. That's the key here, and why I was trying to read the axis in the function, to get that third value. So I have the old value (old2), the newV value (ipcPARAM), and the third, the one I was trying to read from the 3332 offset (next).

What I end up with is something like this:

old2 = -16384

function throttle2(off, newV)
	ipc.log("old2 " .. old2)
	ipc.log("newV " .. newV.)

	nextV = ipc.readSW(0x3332)
	ipc.log("next " .. nextV)

	if not (newV > nextV and newV > old2) and not (newV < nextV and newV < old2) then nextV = newV end

	ipc.writeSW(0x0924, nextV)
	old2 = nextV

	ipc.log("Value sent -------------------------- " .. nextV)
end

ipc.writeUB(0x310A, logic.Or(ipc.readUB(0x310A), 128))
event.offset(0x3332, "SW", "throttle2")

   327196 LUA: old2 -16384
   327196 LUA: newV 0
   327196 LUA: next 0
   327227 LUA: Value sent -------------------------- 0
   331533 LUA: old2 0
   331533 LUA: newV 69
   331533 LUA: next 69
   331548 LUA: Value sent -------------------------- 69
   331580 LUA: old2 69
   331580 LUA: newV 979
   331580 LUA: next 979
   331611 LUA: Value sent -------------------------- 979
   331658 LUA: old2 979
   331658 LUA: newV 3575
   331658 LUA: next 3575
   331673 LUA: Value sent -------------------------- 3575
   331704 LUA: old2 3575
   331704 LUA: newV 6033
   331704 LUA: next 6033
   331736 LUA: Value sent -------------------------- 6033

So ipcPARAM is always the same as ipc.readSW(0x3332)... unless I add a sleep(100):

old2 = -16384

function throttle2(off, newV)
	ipc.log("old2 " .. old2)
	ipc.log("newV " .. newV.)

	ipc.sleep(100)

	nextV = ipc.readSW(0x3332)
	ipc.log("next " .. nextV)

	if not (newV > nextV and newV > old2) and not (newV < nextV and newV < old2) then nextV = newV end

	ipc.writeSW(0x0924, nextV)
	old2 = nextV

	ipc.log("Value sent -------------------------- " .. nextV)
end

ipc.writeUB(0x310A, logic.Or(ipc.readUB(0x310A), 128))
event.offset(0x3332, "SW", "throttle2")

Output log:

904961 LUA: old2 -16384
   904961 LUA: newV 0
   905055 LUA: next 0
   905071 LUA: Value sent -------------------------- 0
   905679 LUA: old2 0
   905679 LUA: newV 1403
   905773 LUA: next 3524
   905804 LUA: Value sent -------------------------- 1403
   905804 LUA: old2 1403
   905804 LUA: newV 6703
   905897 LUA: next 9443
   905929 LUA: Value sent -------------------------- 6703
   905929 LUA: old2 6703
   905929 LUA: newV 12743
   906038 LUA: next 14710
   906053 LUA: Value sent -------------------------- 12743

With the sleep it works fine, because I'm actually reading a third value to compare to the other two. Of course ipcPARAM is no solution since it's not a new value, no matter if I put the thread to sleep waiting for the axis to update, because ipcPARAM is the same thing as the function parameter.

Next:

function throttle2(off, newV)
	ipc.sleep(100)
	nextV = ipc.readSW(0x3332)
	if not (newV > nextV and newV > old2) and not (newV < nextV and newV < old2) then nextV = newV end

	...

the "if" clause only compares newV (ipcPARAM) to old2 and nextV (the 3rd value). And this is why:

I assume there will not be two consecutive spikes.

We assume for now that "old" was not a spike, because it was stored in a previous iteration after the filtering. "Old" was the value sent to the offset after filtering.

Now we have newV (ipcPARAM) and nextV.

1.- If nextV is a spike:

old = 1000

newV= 2000 *applied*

nextV = 6666

the filter will discard nextV, and the next iteration when the spike is gone:

old = 2000

newV= 2000

nextV = 2000 *applied*

If newV is a spike:

old = 1000

newV = 6666

nextV = 2000 *applied*

It can happen something like this:

old = 1000

newV = 1666 *spike* *applied*

nextV = 2000

But if the spike is small enough to stay in between old and nextV it won't hurt to allow that value to be written to the axis

Now, why do I need to get the third value from the function itself? Because I need the output to stabilize to the latest valid value. If I store two old values and just use the latest ipcPARAM as the third, let's say that I'm pushing the throttle all the way to full throttle:

old1 = 12000

old2 = 14000 *applied*

newV = 16000

The lever will stop at 14000 even though 16000 is a valid value.

If I read my third value in the function:

old = 14000 (or whatever)

newV = 16000

nextV = 16000 *applied*

So the lever will stop at full throttle as expected. This is the reason why I was polling the axes in the first place, to avoid the levers to stop too early in an event driven implementation. And this is the workaround I thought for the event driven version: reading the third value IN THE FUNCTION, after the event is sent.

I need the sleep clause because without it, the nextV is read too fast and it will actually be the one provided by ipcPARAM. Adding a delay is not an option really, so I will have to consider a new way to do this.

I hope that helps clarifying things a bit, and again your help and your time are VERY MUCH APPRECIATED :)

[Pete Dowson]

I don't think you need to change anything in the docs or anywhere else, just that I didn't make myself clear enough.

You said that I need three values. That's the key here, and why I was trying to read the axis in the function, to get that third value.

Yes, but you would get the first value from the first time the function is called, save it, get the second the next time, and the third the next time. Since the function is called EVERY time the value changes, you get precisely ONE new value each time it is called, so obviously you save the previous two -- EXACTLY as you did in your earlier examples. I don't understand why you removed the second "Old" value memory. It can't work without it.

So I have the old value (old2), the newV value (ipcPARAM), and the third, the one I was trying to read from the 3332 offset (next).

But why try reading the current value from 3322 when it is provided for you in newV = ipcPARAM? It makes no sense.

What I end up with is something like this:

Which is exactly as wrong as the last and for similar reasons.

So ipcPARAM is always the same as ipc.readSW(0x3332)

Of course it is because it IS the current value being supplied to you!

.. unless I add a sleep(100):

But that just defeats the whole point of having a function which automatically receives EVERY changed value, no matter how fast or slow they happen. All you are doing be suspending the thread for 100 mSecs is preventing however many new values might have been sent to you actually being sent!

I need the sleep clause because without it, the nextV is read too fast and it will actually be the one provided by ipcPARAM. Adding a delay is not an option really, so I will have to consider a new way to do this.

Having the Sleep simply negates the whole advantage of the provisions I have made. You might as well give up on all the new facilities if you want to do such an odd thing.

The "NEW" way to do it is the OLD way. You have two memories, Old1 and Old2. Look, it seems I have to re-print my re-write of your original to illustrate the method. I don't think you could have bothered to look at it yet, else you'd surely understand the correct solution. this is for Throttle1 (the control 66420 = Axis throttle1 Set):

new1 = -16384
new2 = -16384

function param(new3)

        nextV = new3

        if not (new2 > new3 and new2 > new1) and not (new2 < new3 and new2 < new1) then nextV = new2 end
        if not (new1 > new3 and new1 > new2) and not (new1 < new3 and new1 < new2) then nextV = new1 end

        new1 = new2
        new2 = new3

        ipc.control(66420, nextV)
end

event.param("param")

Okay? I've used "new1" and "new2" instead of "old1" and "old2", but the code is the same. I've removed the check on whether the value being sent is different or not. As you said once, it should be different more times in any case, except which the filter has filtered it.

That works fine here, like your original except faster and smoother. AND you can calibrate it in FSUIPC.

Now please explain why you don't understand it and/or rejected it, so i can understand.

Regards

Pete

[dazz]

I just explained why that version doesn't work properly: it will ignore the last ipcPARAM in the series of calls so the lever won't move all the way to the end. Nothing to do with the changes you kindly provided, it's a problem inherent to my original implementation.

It discards the max and min values of the new1, new2, and ipcPARAM. Moving the lever in one direction, ipcPARAM will allways be discarded until the next iteration

Try moving the lever fast and you will see the effect is quite apparent. As I said, that's why my first option was a loop polling the axes. It made things very simple, but obviously very inneficient too. I need a post-event reading of the axes output with this approach... or simply a different way of programming this.

[Pete Dowson]

I just explained why that version doesn't work properly: it will ignore the last ipcPARAM in the series of calls so the lever won't move all the way to the end. Nothing to do with the changes you kindly provided, it's a problem inherent to my original implementation.

It discards the max and min values of the new1, new2, and ipcPARAM. Moving the lever in one direction, ipcPARAM will allways be discarded until the next iteration

Okay, I see. But do you now understand the method? From your efforts and with the addition of the "sleep" just to read a third value, it certainly seemed as if you didn't. And that was my concern. Not the effectiveness or otherwise of the filter itself.

To fix your now-stated problem, you simply need to have a limit on the time for filtering. There's no point in filtering slow changes in any case. Keep a record of the elapsed time (you can read that using an ipc function). Have a timer event to a watchdog function, and if you see the elapsed time since the last update exceed the most reasonable filter time, say 200 msecs (or perhaps more?), send the last saved value.

BTW did you ever try the filtering in FSUIPC's calibratiion facilities (the "F" checkbox)? It's a rather more sophisticated mathematical low-pass filtering I found in a book somewhere. Unfortunately I can't remember where now, it was so many years ago.

Regards

Pete

[dazz]

Okay, I see. But do you now understand the method? From your efforts and with the addition of the "sleep" just to read a third value, it certainly seemed as if you didn't. And that was my concern. Not the effectiveness or otherwise of the filter itself.

To fix your now-stated problem, you simply need to have a limit on the time for filtering. There's no point in filtering slow changes in any case. Keep a record of the elapsed time (you can read that using an ipc function). Have a timer event to a watchdog function, and if you see the elapsed time since the last update exceed the most reasonable filter time, say 200 msecs (or perhaps more?), send the last saved value.

BTW did you ever try the filtering in FSUIPC's calibratiion facilities (the "F" checkbox)? It's a rather more sophisticated mathematical low-pass filtering I found in a book somewhere. Unfortunately I can't remember where now, it was so many years ago.

Regards

Pete

I just added the sleep when I realised that I was still getting the same value that I had already in the function param, just to make sure that I could read the new value from the axis, but the moment I saw in the logs that with a 100ms sleep each iteration took like 116ms (as expected in such a simple piece of code), I new that wasn't going anywhere (almost a 90% of the time waiting for the new value, totally unacceptable). And yes, I tried the FSUIPC built in filter and it works very well. Actually I remember having set that in the past, but totally forgot about it. Knowing all these features much better now I hope that will translate soon in some other application that doesn't involve reinventing the wheel, hehe.

I know now what to expect in terms of performance and thread management, and that will definitely help make better decisions for future scripts, know what to use and when, and of course I'll try to come up with a better idea to solve the problem next time than this one for the spikes.

Taking for granted that there would be no consecutive spikes is a big limitation for a start, but well, as I said, ever since you reminded me about the FSUIPC filter, rather than optimizing the code, I was more interested in gathering the relevant information about how the pluggins run, and how the different features work.

Thanks for the new suggestion for the problem, but I think it doesn't make sense to waste more time with this when there's already a much more capable filter available

I just printed the advanced user guide, the FSUIPC4 Offsets Status.pdf, and the Lua Library.pdf for future reference.

Maybe I could try to do something about "noise", but that's going to be much more difficult and my throttles don't suffer from that (yet) so there's no way I could test the script anyway

I have a last question if you don't mind me asking. Regarding axis calibration, when you write directly to the FS control and with no axis assigned to FSUIPC like in the event.param version (it's assigned to LuaValue pluginName), how is it that the calibration works? I tried it and it works, but I thought the axis needed to be assigned via FSUIPC for the calibration to work

[Pete Dowson]

I have a last question if you don't mind me asking. Regarding axis calibration, when you write directly to the FS control and with no axis assigned to FSUIPC like in the event.param version (it's assigned to LuaValue pluginName), how is it that the calibration works? I tried it and it works, but I thought the axis needed to be assigned via FSUIPC for the calibration to work

No, not at all. FSUIPC calibration has been a facility for many years more than the axis assignment facilities. The latter only came about as a result of needs of folks for having different devices for different aircraft types -- like a yoke for Boeing airliners, joystick for Airbuses, G-stick for helicopters, etc. Axis assignments can be auto-switched according to the aircraft loaded.

It also then provided the opportunity to add specialist axes not supported by FS, like reversers, and aileron and rudder trim.

I really don't know how the idea that FSUIPC only calibrated its own assigned axes ever arose, and it is very annoying as it seems to make folks use the axis assignments when they'd really be better off sticking with FS assignments.

FSUIPC calibration is NOT really AXIS calibration in any case, it is FS CONTROL calibration. It intercepts not the axes but the controls FS is receiving, from whatever source. It neither detects joystick axes nor knows how those controls arise, and it doesn't need to as all it is doing is mapping input ranges to output needs.

The only axception to this, which was also a much more recent addition, is the option in the axis assignments to assign "direct to FSUIPC calibration". Normal assignment sends the controls to FS (as in fact it says), NOT to FSUIPC, and FSUIPC calibration intercepts them in the same way as if they were assigned in FS. But the "direct" assignment bypasses the FS controls and calls the calibrations routines directly instead. This can be a lot more efficient (cuts down on a lot of internal activities). But I don't advise it as a blanket thing to do, in preference to the normal assignments, because it can cause some aircraft not to work -- some intercept the FS controls themselves (the add-on Airbuses, for instance), for assorted purposes like fly-by-wire, and bypassing those controls defeats them.

Regards

Pete

[dazz]

Ok, understood. Thanks Pete