NaNs and length issues in wind records

Discussion of meteorological data.

Moderators: Bonnie.Jonkman, Andy.Clifton

Jennifer.Rinker
Posts: 21
Joined: Tue Jun 25, 2013 11:34 am
Organization: Duke University
Location: NC, USA

NaNs and length issues in wind records

Postby Jennifer.Rinker » Fri Jul 12, 2013 11:32 am

Hi everyone,

I'm using the rotated sonic_u data from Andy Clifton, which I downloaded from the url on May 18th. It seems there are a lot of wind records that have at least one NaN, are not 12,000 elements long, or both. In most cases, there are NaNs for all 6 height locations in a given 10-minute period, but this isn't always the case.

I have an m-file that will recurse through a specified data directory on your hard drive and return a cell with the date/time stamp, the quantity that is not "right" (i.e., Sonic_u_131m), and the error flag (1 = not 12,000 long, 2 = has one or more NaN values, 3 = both). If this is of interest to you, please message me and let me know and I'll send it to you (apparently I can't upload mfiles, and when I tried to upload a text file "the board attachment quota was reached").

I'm not absolutely certain, but I'm pretty sure the NaNs come from the quality check that Andy does on the M4 data before sending it out - if the data are below the 95% quality level, NaNs are returned for the whole record. I don't know why some records are still less than 12,000. As a reference, of the 1739 records I had available, 890 of them were flagged.

I unfortunately can't attached a text file with the records that aren't "right", but here's an example of the output after I saved the cell to a text file:

2012/10/12 00:10 Sonic_u_100m 2
2012/10/12 00:10 Sonic_u_76m 2
2012/10/12 00:10 Sonic_u_50m 2
2012/10/12 00:10 Sonic_u_15m 2
2012/10/12 00:30 Sonic_u_131m 2
2012/10/12 00:30 Sonic_u_100m 2
2012/10/12 00:30 Sonic_u_76m 2
2012/10/12 00:30 Sonic_u_50m 2
2012/10/12 00:30 Sonic_u_30m 2
2012/10/12 00:30 Sonic_u_15m 2
2012/10/12 01:10 Sonic_u_131m 3
2012/10/12 01:10 Sonic_u_100m 3
2012/10/12 01:10 Sonic_u_76m 3
2012/10/12 01:10 Sonic_u_50m 3
2012/10/12 01:10 Sonic_u_15m 3
2012/10/12 01:20 Sonic_u_131m 3
2012/10/12 01:20 Sonic_u_100m 3
2012/10/12 01:20 Sonic_u_76m 3
...etc.
Jenni Rinker, Ph.D.
Mechanical Engineering & Materials Science
Duke University/NWTC

Andy.Clifton
Posts: 83
Joined: Wed Feb 29, 2012 3:13 pm
Organization: NREL
Location: Boulder, CO
Contact:

Re: NaNs and length issues in wind records

Postby Andy.Clifton » Fri Jul 12, 2013 11:54 am

Hi Jenni,

Welcome to the forum! You should be able to just paste the code in to your post. There should be buttons that let you mark it as code. Alternatively, you can post it to e.g. dropbox or google docs and let people pick it up from there.

From your post, I presume you are working with the 20-Hz data? If you dig through some of the other posts on this forum (e.g. the ones about "version x.xx released") and the PDFs at http://wind.nrel.gov/MetData/Publications/ you'll find some more details about the calculations that are done and what data are reported. As you've realized, if there is not sufficient clean data at 20 Hz the code won't do the rotation and so those rotated data are all NaN.

There will always be occasions where we don't have 12,000 records because the tower lost power, got struck by lightning, or I took the systems offline for instrument calibration, or software updates. As an aside, the sonics struggle when the wind speeds are very low, and because the winds on site are actually frequently calm we do lose data in those circumstances. You'll see that if you plot the relationship between the cup wind speed and the number of samples from a nearby sonic anemometer.

Just as an observation, I think the original error codes would have been 5001, 5002, etc. You might want to keep those error codes for consistency? It also means that you won't get in to trouble if I change the error codes on you :)
Andy Clifton, Ph.D.
Senior Engineer


Return to “NWTC Wind Data”

Who is online

Users browsing this forum: No registered users and 1 guest