Format of the Microsoft real world weather file?

[in_04]

Hi Pete,

and any other reading this.

I currently develop a Weather program and it work ok with FS2004. But I found that the setting of the individual weather stations with the NewWeather interface took to long time if you plan to do this for the whole world.

So I think about a METAR to "metars_and_winds_aloft.bin" (this is the file FS2004 download, same as FS2002) Currently my biggest problem is that I guess that this file is compressed (?) I checked it and didn't find any ICAO or METAR codes in it. I know that the FS2004 weather engine has a METAR decoder build in, so I guess that the MS file include normal or maybe stripped METARS in it.

So my question is, does anybode know the compression type or maybe more about the file format?

Or could you (Pete) make a routine to read the whole world weather in FS2k4 faster than the current one?

Thanks in advance

Bye

Markus

[Pete Dowson]

So I think about a METAR to "metars_and_winds_aloft.bin" (this is the file FS2004 download, same as FS2002) Currently my biggest problem is that I guess that this file is compressed (?) I checked it and didn't find any ICAO or METAR codes in it. I know that the FS2004 weather engine has a METAR decoder build in, so I guess that the MS file include normal or maybe stripped METARS in it.

In FS2002 there was a fixed set of METAR stations, listed in the ICAO lists in the files in the Weather folder. The downloaded file contained an index to the ICAO table. The latter, if you look at the binary file, contains 16 bytes per station, with ICAO, latitude, longitude and elevation for each station (the last three in 32-bit float format).

Because stations come and go, I think the FS2004 system is a little more flexible, but it is possibly based on the same system. I think the index file may also sometimes be downloaded -- isn't it amongst the personalised data now, in Application Data or whatever? Sorry, but really haven't had time to investigate it.

Or could you (Pete) make a routine to read the whole world weather in FS2k4 faster than the current one?

I only know of a way to inject one station at a time. I suppose you could have a routine which did this in a loop, working from a file, rather than using the IPC interface to pass the data, as it obviously is not suited to such large amounts of data. I'm not sure why this would be any advantage over creating a WX file and loading that with a FLT.

What really is the application for this? Why not do as the other weather programs are doing and load stations around the aircraft, at progressively longer distances, till you've populated say 100 miles around, then just update that as you fly?

Regards,

Pete

[in_04]

I already use the FS2004 weather station file to avoid setting weather on stations FS2004 doesen't use.

Maybe the hint with the index could do the trick, but I still believe the downloaded weather is compressed.

The other suggestion to use the .wx file is also on my list, but the use of the build in download would be easyier for the users.

The plan to do a full weather set rather than a update 100 miles around is, because the use of the dynamic weather. I don't know how far FS2004 calculate the dynamic weather. So if you fly from New York to London I don't know if FS2004 start dynamic on London as soon as you start in New York or do it start the dynamics only for the weather stations nearby?

Also a big problem with the 100 miles (or more) around youre plane, are the large areas without wx stations. Esp. Australia and the Oceans, if the program updates the stations only 100 miles around the flight accross the ocean would only use the last weather station until you are 100 miles away from the destination station.

For this problem I still have an idea to solve it, but will not post it here.

Bye

Markus

[Pete Dowson]

I still believe the downloaded weather is compressed.

Oh, I don't doubt that. It will be encrypted too, because I think it has to be according to the licensing deal MS did with Jeppesen. I don't think they want their data used for anything else, that's why.

The plan to do a full weather set rather than a update 100 miles around is, because the use of the dynamic weather.

You can switch that off.

I don't know how far FS2004 calculate the dynamic weather. So if you fly from New York to London I don't know if FS2004 start dynamic on London as soon as you start in New York or do it start the dynamics only for the weather stations nearby?

I don't know, but I think it only actually populates and progresses the nearer METAR stations, but whether it is limited by distance or by array size I don't know.

Also a big problem with the 100 miles (or more) around youre plane, are the large areas without wx stations. Esp. Australia and the Oceans, if the program updates the stations only 100 miles around the flight accross the ocean would only use the last weather station until you are 100 miles away from the destination station.

That is where you have to use older (FS2002 and before) methods, using the global weather instead and changing that dynamically yourself. This is why I still included smoothing options for winds, pressure and visibility. Unfortunately the clouds may jump in and out though. I never found out how to get into the morphing part of WEATHER.DLL.

Regards,

Pete

[in_04]

Hi Pete,

I checked the update of the weather station 100 miles around the plane, but I'm not happy with this.

1. The first update of the stations took 30 or more seconds, because in US there are places where there are several hundred stations around

2. FS2004 updates the weather for every station on the screen in real time, so there is some flickering of clouds etc.

3. A update of the weather in flight give heavy stutter and fps drops to single diggits

Currently I try to minimize the overhead and reduce the number of stations, but a radius of 80 miles (or km?) is needed, because the clouds could be draw to this distance. A progrssive update isn't very usefull also, because you see clouds up to 80 miles away (if you set the distance slider in FS2004 to the rightmost position). So if I update only the stations 10 miles away and the user take an ATIS report of a station 20 miles away, he will get false informations. So I think the update radius must be 80 miles and the update should be as fast as possible. Esp. if you fly a very fast plane (like the Concorde) the update will occure quite often.

So I have some questions to you:

1. Is there a way to update the stations without redraw in FS2004?

2. Do you know the location where the complete weather is in FS2004 memory space, so I could maybe update the weather at once (I think the NewWeather structure is the one MS2004 use internal?)

3. Could you speed up the update of single stations?

4. How can I update a station without the IPC interface? You say that it be possible in youre first answer.

Bye

Markus

[Pete Dowson]

I checked the update of the weather station 100 miles around the plane, but I'm not happy with this.

1. The first update of the stations took 30 or more seconds, because in US there are places where there are several hundred stations around

2. FS2004 updates the weather for every station on the screen in real time, so there is some flickering of clouds etc.

3. A update of the weather in flight give heavy stutter and fps drops to single diggits

1. I would spread it out to much longer than 30 seconds if I were you. there's not such a big rush. Start at the aircraft and work out in a spiral. You can take minutes to reach 100 miles.

2. Please read the "important notes for FS2004 users" near the back of the FSUIPC User Guide. Those stutters are primarily due to the imposter clouds which are horrible anyway. There is no way around the drawing of the others as I could not find the way into FS's morphing mechanisms. This is the same as FS2002 and before, only better once the localised weather is entered.

3. See the "important notes for FS2004 users". Those stutters can be avoided. they don't occur here when using FSMeteo 6, for instance.

1. Is there a way to update the stations without redraw in FS2004?

2. Do you know the location where the complete weather is in FS2004 memory space, so I could maybe update the weather at once (I think the NewWeather structure is the one MS2004 use internal?)

3. Could you speed up the update of single stations?

4. How can I update a station without the IPC interface? You say that it be possible in youre first answer.

1. There should be a way to inject weather for stations as a "target" for FS to morph to. This is how FS itself would do it. However, it took me months to find the way to just set the stations the way that I do. I never managed to find the way to set target weather. Microsoft won't help with this. Maybe there will be a Weather SDK one day, but there never has been in the past.

2. No, I wish I did. but it simply isn't like FS2000/FS2002 where there was a complete table of weather structures. And, no, the NewWeather structure is nothing like the internal structure -- closer were my AWI structures, but even those are different. You can get a better idea from the .WX files.

3. It is as fast as it can be at present for the way it works and what it has to do. It is basically just creating the weather structures the way FS likes them and making a couple of calls into FS - one to pass the structure over, and the other to make it take effect. From timing tests I've done, the bulk of the time is spent inside the second of those calls, and it is FS re-populating specific weather points in a circle around the aircraft to take into account the change just supplied.

One possibility which I could try is to avoid making the call which does the populating and only do that AFTER you've set all the stations you want. This could be done by adding extra commands to the NWI. Write to my email address and we'll discuss his further and maybe do some testing.

4. Updating a station without using the IPC interface? Sorry, I don't understand. You mean loading a .WX or using the Jeppesen weather downloads? Can you clarify?

Pete

[Pete Dowson]

Hi again,

A couple of points of clarification I've just thought of:

On this:

1. The first update of the stations took 30 or more seconds, because in US there are places where there are several hundred stations around

The update rate managed by FSUIPC is really constrained by the FS Frame Rate. I cannot allow it to try to make these calls into WEATHER.DLL aynchromously. They have to be done in synch with the rest of FS, on the frame rate. Otherwise the system can get very unstable.

So if, for example, your FS is running at 20 fps it would take at least 30 seconds to update 600 stations -- probably half that as by the time you've seen the signal to allow you to change the data, you will have missed the next frame tick.

I don't think there's any way around that, even by leaving out the FS populate call as I suggested I might be able to in my last message. The only speed up would be by batching the weather stations, as FS does by loading a WX file. This brings me to this:

4. How can I update a station without the IPC interface? You say that it be possible in youre first answer.

Now here I think you are possibly mis-reading this:

"I suppose you could have a routine which did this in a loop, working from a file, rather than using the IPC interface to pass the data, as it obviously is not suited to such large amounts of data. I'm not sure why this would be any advantage over creating a WX file and loading that with a FLT. "

All that was suggesting was that possibly the IPC interface could be used to supply a filename for a set of weather stations, which would then be loaded into FS en bloc. Exactly the same in fact as available already for loading FLT + WX files. I don't know a way of injecting them into FS as a batch, so FSUIPC would have to loop reading the file and sending each WX station to FS. If this is done in one frame then it would be a stutter, certainly, maybe a long one -- though I could of course leave the "populate" call till the end.

Regards,

Pete