[BUG] FSUIPC4 event lib gets out of sync

[wstrzalka]

Hi Pete

I think I had found a bug this time.

I do use event.com() to register callback on incoming HID reports. If I do generate HID reports slowly it works as designed. But when I do generate a lot of reports in very short period - especially when I do it faster then I'm able to consume them - it starts misbehaving. The problem is I'm only getting a few reports out of many. The reports are not lost however but are stored somewhere in FSUIPC (I believe) and when I do press another button on the device - what triggers HID report - I do get one but it's the cached one from long time ago. Then the story repeats with subsequent buttons.

So it seems like all the events coming from USB are recorded, but when they come too fast, only for some part of them there is callback function called. Then when new reports come the new report is added to the end of some queue while the oldest one (and only one although there a bunch of backlog in the buffer) is send to the callback. From now on all the reports are not from the buttons I do press... This causes a lot of mess as you may expect :)

Interesting factor is that when I do call com.readlast all the buffers/caches/queues/... are flushed (I do get pretty decent 'discarded' value returned) and everything works as expected again.

I would imagine there to be lag caused by not consuming messages fast enough, but eventually to get all the awaiting reports until buffer is empty.

 

As of reproduction - I'm reliably able to reproduce it when I do tun encoder knob quite fast (I'm not able click fast enough to trigger it with regular button). When I do have LUA logging enabled it's happening every single time, but it sometimes also happens when I do disable logging and turn my knobs reasonably fast during flight.

 

Regards 

     Wojtek

 

 

 

 

[Pete Dowson]

11 hours ago, wstrzalka said:

I do use event.com() to register callback on incoming HID reports. If I do generate HID reports slowly it works as designed. But when I do generate a lot of reports in very short period - especially when I do it faster then I'm able to consume them - it starts misbehaving. The problem is I'm only getting a few reports out of many.

What functions are you using to read the data? For com.read, data read is just stored in a buffer, which certainly isn't severely limited. The event merely signals there's something received. so you should loop reading until there's no more!

11 hours ago, wstrzalka said:

So it seems like all the events coming from USB are recorded, but when they come too fast, only for some part of them there is callback function called.

Yes, there's only one flag which triggers the call back no matter how much is in the buffer. And more can accumulate whilst you are in the function.

11 hours ago, wstrzalka said:

Interesting factor is that when I do call com.readlast all the buffers/caches/queues/... are flushed

Yes, which is exactly what it is for. It assumes you only want the latest, and deletes the earlier stuff. This is usefulwhen you just want the current state of things, not a history of events like multiple increments or decrements.

The reading of data arriving is completely asynchronous with everything else, in a separate thread. What you are describing is expected results of this implementation, which is the only way to do it without freezing things waiting. That way your data would certainly be lost.

BTW the usual way, and one I've always implemtned in things like radio stacks (I did the drivers for FlightLink gear in the early days, and PFC more recently), is to accumulate increments and decrements when they are occurring withing 200-300 mSecs of each other and send them as batch commands, like "inc by x" rather than just "inc". Send one batch at most every half second. Fast enough to make the results visually work, but not so fast that the sim clogs up.

Pete

 

 

[wstrzalka]

That was tricky one. 

I was convinced that I have either an option to get report events or to poll the reports with read. Didn't really expected a mix :)

After a fix it works as expected. 

 

Thanks Pete

 

[Pete Dowson]

26 minutes ago, wstrzalka said:

That was tricky one. 

I was convinced that I have either an option to get report events or to poll the reports with read. Didn't really expected a mix :)

That would be nice, I suppose, but very messy to implement in FSUIPC, and, I think not really worth it. To do that it would have divide up the incoming comms data stream into the requsite length chunks, and build up a queue of events which refer to this data, maybe re-buffering it, of possibly unlimited length, and deliver them one at a time. And each time it has to wait for the current actions of the plug-in to finish, so it goes dormant, and then wake it up again. That could make your actions even for dislocated for the triggering events. 

The would also be against the entire method currently used for the event system. There's only ever a maximum of one outstanding event per type of event, per plug-in.

Pete

 

 

[wstrzalka]

No problem at all until it works. It may be worth to mention in the doc however (unless it's already there as I could miss it)

[Pete Dowson]

33 minutes ago, wstrzalka said:

No problem at all until it works. It may be worth to mention in the doc however (unless it's already there as I could miss it)

I think it does mention that only one event is queued.  Here at least:

But note that, whilst FSUIPC does keep track of separate events, it does not
queue multiple identical events. If a button is pressed 20 times before you process it, you only see it once. Therefore
if you are monitoring things which can happen repetitively you will need to keep your processing short enough if
you hope to catch them all.

I know that doesn't mention the com problem at all, but then there's this in the description of event.com:

Your program doesn’t need to perform
any reads itself, thought there’s nothing stopping it—and it
may be wise if there’s more expected

And this in the com.readlast description:

This function might be useful in polling situations where the
rate at which data arrives might exceed the polling rate or
capabilities of the Lua system. HID joysticks scanning is a
prime example. Rather than process older records and get a
larger unwanted lag that is necessary it enables an efficient
way of only processing the most recently received state.

Maybe a note added there to say "but if you want to process all of the inputs piled up, loop using com.read, do not use com.readlast". Is that the sort of thing you wantr?

Pete

 

[wstrzalka]

I've been off the business for quite a long time....

So I read this in the doc before multiple times and I think I got the most confused by this

"Your program doesn’t need to perform
any reads itself, thought there’s nothing stopping it—and it
may be wise if there’s more expected"

I understood it as a assurance that I will get the messages. I'm not sure if it's because of the statement itself or my english skills :D

I think I would figure it out with the supplement proposed 

 

 

[Pete Dowson]

2 hours ago, wstrzalka said:

I think I would figure it out with the supplement proposed 

What "supplement"? Just this?

a note added there to say "but if you want to process all of the inputs piled up, loop using com.read, do not use com.readlast".

Why not read it now?

Pete