FSUIPC, client/server interface?

[iznogoud]

Hi Pete,

I was wondering (may be a stupid question) why FSUIPC would not change into a client/server application ?

I mean that I do not find very interesting for real time developpement, to be obliged to check all the variables by a call to FSUIPC at constant time interval.

If lots of applications are running, it increases the load on FSUIPC even if nothing changed.

I'd really prefer to listen to a network port for pack of data to arrive when they 're changing inside FS.

At a software connection, the server could then send all the data in one block for initialisation.

Your opinion Peter?

Regards, ;)

Michael.

[Pete Dowson]

I was wondering (may be a stupid question) why FSUIPC would not change into a client/server application ?

FSUIPC's interface was defined (not by me) in FS95 times in FS5IPC, then FS6IPC in FS98. There's a lot of history. The interface has been compatible now with application programs since FS98, continuously.

By all means write an interface to FSUIPC if you like, or, better, feel free to write a replacement to FSUIPC. But how to you make all the applications compatible "just like that"?

I'd would really like you to do this as I would rather fly more in future! ;-)

Regards,

Pete

[iznogoud]

To keep a compatibility is a good thing. But it cannot be the only reason to let things unchanged.

I really understand (and respect) all the work that has been done.

I just think about the way datas are delivered to client application. Even if the concept of FSUIPC was not changed, maybe it could be possible to think about changing the WideFS principles (or to add a sort of option)? In a way where any client could be able to receive the datas directly on the network instead of interrogating.

Could you maybe give a little more details on how datas are exchanged between wideclient and wideserver? I assume that datas are sent from/to both program. Would it be possible to catch datas directly on the network?

This would be really more close to real time handling. As I read from you, acces to variables through IPC can be a very long process.

Regards,

Michael.

There are other sims who delivered datas that way, using UDP. No doubt that if MS release next version of FS which such features, developpers will use it. But I could bet that MS won't do that. Anyway, it doesn't prevent me from thinking that for more performance, a network solution would be better.

So to answer your question, why don't I do that myself?Although I could deal with the transport layer, I do not have the necessary knowledges to find datas in FS like you do. That's why I ask.

Do not get blessed, Peter, I had no idea at all to criticise your work.

[Pete Dowson]

To keep a compatibility is a good thing. But it cannot be the only reason to let things unchanged.

It was the only reason I started on FSUIPC in the first place.

Even if the concept of FSUIPC was not changed, maybe it could be possible to think about changing the WideFS principles (or to add a sort of option)? In a way where any client could be able to receive the datas directly on the network instead of interrogating.

Anything is possible, almost -- but I am not starting on anything new, This has taken me many years full time and I'd like to make space to do some flying and maybe even get back to my model railway. I am not taking on any new FS undertakings. Perhaps, since you have ideas you will instead?

Could you maybe give a little more details on how datas are exchanged between wideclient and wideserver?

Please do a search on this, and read the WideFS docs. I have explained this all sorts of ways several times. I really haven't time at present. Sorry.

This would be really more close to real time handling. As I read from you, acces to variables through IPC can be a very long process.

Sorry, what does that mean, "very long process"? You think you can make it all work much faster? The only thing which is slow is process switching -- how can you avoid process switching?

Regards

Pete

[iznogoud]

Please do a search on this, and read the WideFS docs. I have explained this all sorts of ways several times. I really haven't time at present. Sorry.

Will do Pete.

I would have only 2 questions:

I tried to make a test module to interrogate FSUIPC from within FS.

It works using the standard FSUIPC_Open call.

Where can I find info about FSUIPC_open2 ? and what could be the problem if i only use the FSUIPC_open version? (seems this is not documented in the delphi SDK)

Is there a way to know (from within FS) when FS actualized it's datas? (like a windows message or so?)

Thanks for your help,

Michael.

[Pete Dowson]

I tried to make a test module to interrogate FSUIPC from within FS.

It works using the standard FSUIPC_Open call.

That treats FSUIPC as if it is in another process, and uses memory-mapped files and such. It is very inefficient, and it is likely to have problems if there are more than two such modules or gauges as the memory-mapped file would be shared.

Where can I find info about FSUIPC_open2 ?

Please look inside the FSUIPC SDK. Inside you weill find a file called "README.TXT". There's a section in there called "IMPORTANT NOTE FOR FS GAUGE AND MODULE WRITERS".

This is why "readme" files are called "readme" you know! ;-)

Is there a way to know (from within FS) when FS actualized it's datas? (like a windows message or so?)

Please check the main Programmer's guide in the SDK. There are offsets which tell you such things. Look at 3364 and 3365.

Regards,

Pete

[iznogoud]

--

[iznogoud]

Well Pete, I'm really sorry to disturb, but reading the doc is the first thing I did. The problem is that for non C programmer, the only header of the FSUIPC_OPEN2 function is not very clear. You know, use of pointers in delphi is something not common due to the structure of the language.

As it is not in the Delphi sdk, I do not really understand how to use it.

Anyway also read anoter readme called FSUIPC_Internal_Access.txt .

I'd like to try that way, but once again I have questions(sorry):

#define WM_IPCACCESS (WM_USER+127)

DWORD dwresult = SendMessage(hFSUIPC, WM_IPCACCESS, (WPARAM) ulSize, (LPARAM) pData);

If I understand everything, it's made through the use of windows messages.

Then, where to find the value of WM_USER?

What is the exact structure of pDATA ?. sorry to ask, but a standard call to FSUIPC_READ in delphi calls for both a pointer to receive the data and the offset value. Here, is the offset value included in the pDATA?

Michael.

[Pete Dowson]

Well Pete, I'm really sorry to disturb, but reading the doc is the first thing I did.

But you said:

Where can I find info about FSUIPC_open2 ? and what could be the problem if i only use the FSUIPC_open version? (seems this is not documented in the delphi SDK)

and these issues are addressed in the SDK -- partly in the Readme, which talks about the problems, and all the information I can offer about the internal interface is held in the ZIP. Even the full C-source codes are there.

You know, use of pointers in delphi is something not common due to the structure of the language.

As it is not in the Delphi sdk, I do not really understand how to use it.

When all this was designed I was convinced that internal modules could only be programmed in C, C++ por ASM. It seems someone has managed one with Delphi, which amazes me. But to do that you'll need to convert my C code. Sorry, I cannot help further on this.

If I understand everything, it's made through the use of windows messages.

Of course, so is the external access system.

Then, where to find the value of WM_USER?

It's defined in the standard Windows header files. Surely Delphi supports Windows? Try a search on Google, or even here. This subject has been discussed at length not long ago.

What is the exact structure of pDATA ?

As the complete sources are provided you should easily be able to find this information.

I'm really sorry, but what you are asking is beyond what I can offer. My part of the SDK is aimed firmly at C and C++ programmers. I simply cannot understake other language support. If you search through the threads here you weill find other references and names of others who should be able to help.

Regards

Pete

[iznogoud]

Hi Pete.

I began to translate into delphi your library for internal access to FS.

My only last problem is with the assembler code:

{ push eax

call next

next: pop eax

mov dwError,eax

pop eax

}

The use of asm is possible in delphi. I simply put your code like that, but I get an error message: identifier not declared: next

Seems the declaration of "next:" is not compatible with delphi asm instructions.

I do not understand a thing in asm. Would there be another way to write that part of code? could you "translate in english" what this operation do?

Thanks for your help,

Michael.

[Pete Dowson]

Seems the declaration of "next:" is not compatible with delphi asm instructions.

I do not understand a thing in asm. Would there be another way to write that part of code? could you "translate in english" what this operation do?

The "call" instruction is calling the next instruction, the one popping eax and labelled "next:" (that is merely a label). If you can use a value like "$" or similar, to represent "this place" (a standard Intel method), then you could get away with leaving the "next:" label off and calling the next instruction by "call $+5" (the call should compile into a 5 byte instruction).

Regards,

Pete

[iznogoud]

Ok, I investigate the call function and labels.

This code compiles in delphi:

asm

push eax

@next: pop eax

call @next

mov dwError,eax

pop eax

end;

Do you think this could be ok ?

(it seems I can't use the $+5 instruction)

[Pete Dowson]

Ok, I investigate the call function and labels.

This code compiles in delphi:

asm

push eax

@next: pop eax

call @next

mov dwError,eax

pop eax

end;

Do you think this could be ok ?

You need the call BEFORE the line with the @next label, as in the original, otherwise you'll just get an infinite loop!

push eax

call @next

@next: pop eax

mov dwError,eax

pop eax

Regards

Pete

[iznogoud]

Ok, Thanks! This also compiles correctly If I place the call before.

I'm gonna try the module before the end of the afternoon, I'll let you know.

[iznogoud]

Hi Pete,

I've translated all the code (probably with a mistake of course).

I've an error at execution during FSUIPC_PROCESS.

It occurs when sending the message after the asm execution.

Error code is FS6IPC_MESSAGE_FAILURE

At this moment the value in DWORD(m_pview)=30467134

Any idea on how to investigate for what can cause that error?

As it runs in FS process it's not easy to debug and the only thing I was able to do for the moment is to flush some variable content to see what's in.

[Pete Dowson]

Any idea on how to investigate for what can cause that error?

It means the data format in the data passed to FSUIPC is in error. This willl be a problem with the request type (read or write normally), request block length, offset value or data length, or terminator (a zero DWORD at the end of the data).

You can program a sequence of known reads and writes, enable IPC read and write logging in FSUIPC's Logging, and try it. If all the reads and writes operate okay (as shown by FSUIPC logging) but you still get the error, then it is "running off the end of the data", so you've omitted the zero terminator in your Process routine. That's the most likely problem in fact.

If it stops anywhere else you know it's a problem there.

As it runs in FS process it's not easy to debug

I wouldn't expect you to do that in any case. Just put a breakpoint on the SendMessage, and dump out the data you are trying to pass to FSUIPC. I can check it if you like (make sure it is in hexadecimal bytes please), but the structures are easy enough to follow, they are the original structures designed by Adam Szofran for FS95 and are defined by the "structs" in the header files.

Regards,

Pete

[iznogoud]

---

POST EDITED 15:58

---

Ok, your post made me realize some things.

In the call to FSUIPC_open2 I provide a pointer to an array of bytes, with the correct size (I hope) for my expexted following request.

As I only want to read magvar and heading I calculate the size to be magvar(2)+16+heading(4)+16+ 16more that makes 54 bytes.

please tell me if I'm wrong)

I call

FSUIPC_open2(...)

FSUIPC_read(heading...)

FSUIPC_read(magvar...)

FSUIPC_process

The problem in fact is that inside the FSUIPC_open2 call there are already some call to read/write functions, and the FSUIPC_WRITE call inside The open2 run short of memory space !!

I think there is something I really did not catch!! I might be wrong in the way I calculate the memory.

So, for test, I decided only to call for the FSUIPC_open 2 function allowing a memory of 74bytes.

And in fact now the program exits with a failure during the fsuipc_process call inside the open2 function.

I'm gonna investigate more to see where exactly if fails, and I'll make a dump of the memory space at this moment.

[Pete Dowson]

In the call to FSUIPC_open2 I provide a pointer to an array of bytes, with the correct size (I hope) for my expexted following request.

Are you only making one request? This is a lot of trouble for just one request -- you could use the gauge interface, documented in the Microsoft panels SDK, to get most trivial things, like MagVar and Heading.

If you are making many requests you should be bunching them for efficiency if possible. in any case you only want to Open and Close once -- Open when you start, Close when you finish.

The problem in fact is that inside the FSUIPC_open2 call there are already some call to read/write functions, and the FSUIPC_WRITE call inside The open2 run short of memory space !!

Well the Open2 reads and writes are processed by the Process call also inside the Open2 code, and the buffer pointer reset to the start again before exit, so those operations are not adding to yours. However, 54 bytes won't be enough. I think it writes the library version nmber, and reads the FS version and the FSUIPC version. Altogether that might need about 62 bytes (maybe only 58 actually, as the Write header takes 4 bytes less than a read header).

However, since those reads and writes call the FSUIPC_Read and FSUIPC_Write routines themselves, and there's a check in there against the size, to make sure there is space, one of those should have failed and closed the link, returning a "FALSE" to the Open2 call, and an error (set by the failing Read or Write) of "FSUIPC_ERR_SIZE".

Seems that there's something wrong in your conversion. Doesn't Delphi have a step facility in its debugging facilities, so you can single step through your code and see what is going on?

The other thing to do is compare your translation with that for the Delphi package for external programs. Much of the code for Reads and Writes and even Process will be comparable. The only big differences are the use of provided memory instead of the 31k memory mapped file and the SendMessage.

I might be wrong in the way I calculate the memory.

Well it is easy enough for you to check that. But the code you've translated should have checked that and returned a local error in any case, way before any attemnpt to talk to FSUIPC.

It's a good idea to be so "mean" with memory whilst testing, in order to highlight problems, but in the end you may as well just allocate more than you'll ever possibly need (in one Process) and forget about it.

Regards,

Pete

[iznogoud]

Hi Pete

I did make the conversion using the help of the delphi SDK of course.

But I stil have the problem.

Here is what happens.

I define an array of byte with size as 74.

I simply try to call FSUIPC_OPEN2(SIM_FS2K4, dwResult,pmem,dwsize)

with pmem pointing on the byte array and dwsize=74 (checked).

Everything goes right (calls to read x2 and to write ) until the program reaches the fsuipc_process call that is inside the open2 function.

I dumped out some data just after the assembler block.

Here is what I get:

just after the asm block, dwerror=01D0E58D

pmem=01D110E4

m_pview=01D0E58D

the array of 74 byte:

00 00 00 00

00 00 00 00

04 33 00 00

04 00 00 00

50 F9 D0 01

00 00 00 00

00 00 00 00

08 33 00 00

04 00 00 00

54 F9 D0 01

00 00 00 00

B0 04 00 00

0A 33 00 00

02 00 00 00

00 00 00 00

00 00 00 00

00 00 00 00

00 00 00 00

00 00

just after the sendmessage function dwerror=0

Hope it could get some light to you on my problem...

[Pete Dowson]

I dumped out some data just after the assembler block.

Here is what I get:

Well, it's a bit of a mess. Before I detail the errors, here's the definition of the read and write structures:

// read request structure

DWORD dwId; // FS6IPC_READSTATEDATA_ID

DWORD dwOffset; // state table offset

DWORD nBytes; // number of bytes of state data to read

void* pDest; // destination buffer for data (client use only)

// write request structure

DWORD dwId; // FS6IPC_WRITESTATEDATA_ID

DWORD dwOffset; // state table offset

DWORD nBytes; // number of bytes of state data to write

where the ID for a Read is 1 and for a write is 2.

Now look at your data:

00 00 00 00

this looks like an extra unwanted DWORD? Are you sure the pointer isn't to the next section? Otherwise everything is out by 4 bytes.

00 00 00 00

04 33 00 00

04 00 00 00

50 F9 D0 01

00 00 00 00

this group of 4 DWORDs plus 4 bytes (for data) is almost correct. But the first DWORD should be 1 (read). The offset is 3304 (correct) and the size is 4 (correct). The destination address is 01D0F950 which is probably okay, and the 4 zero bytes after is where FSUIPC will dump the result.

00 00 00 00

08 33 00 00

04 00 00 00

54 F9 D0 01

00 00 00 00

Again, similar. The read ID (1) is zero instead, but the rest is fine.

B0 04 00 00

0A 33 00 00

02 00 00 00

00 00

This is wrong too -- the ID for this Write to 330A should be 2, but it is 000004B0 (1200 decimal) instead. This may be the data to be written. I wonder if you are writing the DATA to where the ID should be?

00 00

00 00

The last 4 zeroes are needed to terminate it all.

Regards,

Pete

[iznogoud]

Hi Pete,

I've corrected lots of bugs. Main where relative to m_pnext pointer position update.

There some things not clear:

Now look at your data:

00 00 00 00

... this looks like an extra unwanted DWORD? Are you sure the pointer isn't to the next section? Otherwise everything is out by 4 bytes.

But in your source code, there is:

// Okay, now determine FSUIPC version AND FS type

m_pNext = m_pView + 4; // Allow space for code pointer

this make a 4 bytes offset at the begining of the area, and explains (for me) the extra 0 DWORD. Should I remove that??

Now the problems you noticed in the data block was due to a bad translation of that code in delphi: CopyMemory(&m_pNext[sizeof(FS6IPC_READSTATEDATA_HDR)]. m_pnext was not updated to the end of the record and he was zeroing at the begining. ok this is corrected.

Now here is what I get:

just before the asm block, m_pview=01D120F0=30482672

00 00 00 00

01 00 00 00

04 33 00 00

04 00 00 00

50 09 D1 01

00 00 00 00

01 00 00 00

08 33 00 00

04 00 00 00

54 09 D1 01

00 00 00 00

02 00 00 00

0A 33 00 00

02 00 00 00

B0 04 00 00

00 00 00 00

00 00 00 00

00 00 00 00

00 00 00 00

00 00

just after the asm block

m_pview=01D0E82D=30468141

dwerror=01D0E82D=30468141

m_pnext=01D12126=30482726

just after the sendmessage

dwerror=0

dwError<>FS6IPC_MESSAGE_SUCCESS

PROCESS FAULT, dwresult=0

Please Note that I tried with and without the 0 DWORD at the beginning and I have the error at each time.

In the send message function, what does the Wparam represent exactly?

Regards,

Michael.

[Pete Dowson]

But in your source code, there is:

// Okay, now determine FSUIPC version AND FS type

m_pNext = m_pView + 4; // Allow space for code pointer

this make a 4 bytes offset at the begining of the area, and explains (for me) the extra 0 DWORD. Should I remove that??

Ah! ApologiesI forgot all about that. Yes, it is one of the differences between the external and internal systems. Sorry.

That DWORD should contain the address (of @next) saved by that little bit of ASM code you managed to compile!

Now here is what I get:

...

just before the asm block, m_pview=01D120F0=30482672

Ah .. so does the 1st DWORD get filled in before the SendMessage?

00 00 00 00

01 00 00 00

04 33 00 00

04 00 00 00

50 09 D1 01

00 00 00 00

01 00 00 00

08 33 00 00

04 00 00 00

54 09 D1 01

00 00 00 00

02 00 00 00

0A 33 00 00

02 00 00 00

B0 04

00 00

00 00

That all looks okay, provded that 1st DWORD is filled in correctly.

just after the sendmessage

dwerror=0

dwError<>FS6IPC_MESSAGE_SUCCESS

PROCESS FAULT, dwresult=0

Please Note that I tried with and without the 0 DWORD at the beginning and I have the error at each time.

What error? This is now saying it is successful!

In the send message function, what does the Wparam represent exactly?

in C the call is:

SendMessage(m_hWnd, WM_IPCTHREADACCESS, (WPARAM) (m_pNext - m_pView - 4), (LPARAM) m_pView);

The first parameter is the Window handle, the second the message number, the third (your WPARAM) is the size of the data being passed, and the last is the pointer to the start of the data. Note that this doesn't include the special DWORD at the start added for internal access checking.

I'm not sure what you are reporting as a problem now.

Regards,

Pete

[iznogoud]

Ok Pete. We're getting closer step by step.

About error, yes, there is still one after the sendmessage function.

But lets have a look.

1. the first DWORD should contain an address saved by the asm code: but there is nothing in it after the asm. Is it the only purpose of this asm code?

2. here are the values of m_pnext and m_pview just before the call to asm:

just before the asm block, m_pnext=01D1212A=30482730

just before the asm block, m_pview=01D120F0=30482672

there is a 58bytes difference and seems correct now.

Here is the state of datas AFTER the asm code and BEFORE the sendmessage:

00 00 00 00

01 00 00 00

04 33 00 00

04 00 00 00

50 09 D1 01

00 00 00 00

02 00 00 00

0A 33 00 00

02 00 00 00

B0 04 01 00

00 00 08 33

00 00 04 00

00 00 54 09

D1 01 00 00

00 00 00 00

00 00 00 00

00 00 00 00

00 00 00 00

00 00

m_pview=01D0E88A=30468234

dwerror=01D0E88A=30468234

m_pnext=01D1212A=30482730

size before sendmessage=14492

We can see that:

the first dword is empty

the value of dwerror after the asm code

the calculation of m_pNext-m_pView-4= 14492

Could such a value be correct?

Is it normal that the m_pview pointer, that originally contained the address of the start of the memory block, is changed?

the sendmessage function returns 0 and it should return 1 if no errors.

[Pete Dowson]

Here is the state of datas AFTER the asm code and BEFORE the sendmessage:

...

00 00 00 00

...

We can see that:

the first dword is empty

the value of dwerror after the asm code

But it cannot be, if the ASM code is working then the value stored must be the address of the instruction labelled "next". Can't you see what happens with single stepping?

the calculation of m_pNext-m_pView-4= 14492

Could such a value be correct?

No, that should be 54 shouldn't it? You already told me that m_pNext-m_pView is 58! Where are you changing these to a bad value?

Is it normal that the m_pview pointer, that originally contained the address of the start of the memory block, is changed?

No. Where are you changing it?

the sendmessage function returns 0 and it should return 1 if no errors.

Oh, yes. FSUIPC_ERROR_SUCCESS is actually "1" not "0". Sorry.

You have the first DWORD incorrectly set (should be that "next" address), and evidently the length of data incorrect. Sorry, I don't know how, from here it is not possible to use your debugger. ;-)

Regards,

Pete

[iznogoud]

But it cannot be, if the ASM code is working then the value stored must be the address of the instruction labelled "next". Can't you see what happens with single stepping?

No sorry I can't. I mean, this is not a "program" that I can follow step by step, but a dll. so that I launch it on another PC with FS and collect the values using a file.

No, that should be 54 shouldn't it? You already told me that m_pNext-m_pView is 58! Where are you changing these to a bad value?

well the problem is that the m_pview value is modified by dwerror just after the asm code:

// send the request

__asm

{ push eax

call next

next: pop eax

mov dwError,eax

pop eax

}

*((DWORD *) m_pView) = dwError;

unless I'm wrong in understanding that, the value of dwerror is put into m_pView right? that's the reason of the changes.

But I might be wrong in understanding the instruction: *((DWORD *) m_pView) = dwError;

For the asm code: is it possible to replace it with standard code even if less efficient?