20 Hz sonic anemometer behavior

Discussion of meteorological data.

Moderators: Bonnie.Jonkman, Andy.Clifton

Jennifer.Rinker
Posts: 21
Joined: Tue Jun 25, 2013 11:34 am
Organization: Duke University
Location: NC, USA

20 Hz sonic anemometer behavior

Postby Jennifer.Rinker » Fri Nov 01, 2013 10:23 am

Hi everyone,

I have a few questions about the 20 Hz sonic anemometer data that I thought I'd bring up on the forums. Please feel free to direct me where to find the answers, if applicable, rather than answer them yourself.

1) Some records still have "spikes" in the data after Andy's denoising (e.g., 2013/02/11 at 30m), I assume because these spikes last for more than 1 sample and so are not caught by the denoising method laid out in the data guides. I'm working on a few methods for removing/interpolating these spikes, but how are other people dealing with this issue? The QC codes for that particular record are 1002, 1003, and 2023-2024, which I believe are all ignored.
2) What does QC 1002 mean? I can't find it in the unofficial guide or the official guide to the data. Additionally, what is the "range rate" (QC 1003) for the anemometers?
3) I see that the raw sonic data generally has lots of spikes before denoising, oftentimes over a hundred in a given record. Is there any concern of this affecting the quality of the sonic data?

Lastly, I've been seeing some records that have been quantized. I don't have any examples right now, nor have I checked to see if any QC codes will flag such an occurrence in the sonic data, but I wanted to make a note of it so people know it could be an issue.

All the best,

Jenni
Jenni Rinker, Ph.D.
Mechanical Engineering & Materials Science
Duke University/NWTC

Andy.Clifton
Posts: 83
Joined: Wed Feb 29, 2012 3:13 pm
Organization: NREL
Location: Boulder, CO
Contact:

Re: 20 Hz sonic anemometer behavior

Postby Andy.Clifton » Mon Nov 18, 2013 12:08 pm

Jenni - good questions. I've tried to answer them one at a time below.

Ramps
I think the events in the sonic data that you mean are these 10-20 second "ramps", where the 3 components and the temperature simultaneously depart from the trend (see the events at ~200 and ~ 400 seconds in the image below). Can you confirm these are what you are referring to?

10_Sonic_x_100.png
M5 100-m sonic x-velocity
10_Sonic_x_100.png (18.54 KiB) Viewed 9242 times


We've seen these before. When they occur we usually flag the data and swap out the sonic as soon as possible. I was thinking about how to detect these, and wondered if I could do something like cusum(difference) in 10-20 second windows; where the cusum is high, then we know something is going on.

We may also have found the hardware problem that is causing this - I'll update later if this is the case.

QC 1002
From the unofficial guide, p. 20:
... if there are insufficient data in the wind speed time series, the mean wind speed will also be flagged as having too few data points with the QC code 1002 or 1003 ...


The unofficial guide PDF should be searchable, which might help you in cases like this in the future.

Spike removal
Generally these single-point spikes are only expected to impact high-frequencies in the data. They should not impact the mean values too much. You may want to try taking a complete 10minute, 20-Hz data stream and randomly adding / subtracting 100-200 points to see the effect on the statistics.

Quantizing
I'm not sure what you mean by this?
Andy Clifton, Ph.D.
Senior Engineer

Jennifer.Rinker
Posts: 21
Joined: Tue Jun 25, 2013 11:34 am
Organization: Duke University
Location: NC, USA

Re: 20 Hz sonic anemometer behavior

Postby Jennifer.Rinker » Mon Nov 18, 2013 1:44 pm

Dear Andy,

The spikes that I am concerned about range from 2 samples in width (0.1 seconds) to ~6 samples (0.3 seconds) and are presented in the Sonic_u data, so it's already been cleaned and rotated. For example, the first plot in the below image is the sonic-u data from Feb. 11, 2013 at 10:10 and 30m; the spike is pretty obvious. I just re-downloaded this file from the MetData directory to make sure it wasn't an old version. I've been experimenting with a few detection methods and I think I now have one that will detect these spikes successfully, but I wanted to note it in case others weren't aware.

As for quantization, some records seem to have undergone some quantization in the data acquisition process, and because I'm looking at Fourier phase characteristics I have to be really sensitive about any quantization in the records. I defined a quantization "event" as 5 or more consecutive samples with essentially the same value, and flagged records that had more than 10 quantization events. Of the 27,000 records that I currently have on my desktop (up through 02/2013 and including some of 03/2013), about 1,000 have been flagged as quantized with this definition. The second subplot shows a record that has a bit of quantization; I couldn't find a more demonstrative record very quickly but I have seen records that were much more pronounced.

EDIT: I would have sworn I searched for the 1002 QC code, but apparently it was only in my dreams. Sorry about that.

Image
Jenni Rinker, Ph.D.
Mechanical Engineering & Materials Science
Duke University/NWTC

Andy.Clifton
Posts: 83
Joined: Wed Feb 29, 2012 3:13 pm
Organization: NREL
Location: Boulder, CO
Contact:

Re: 20 Hz sonic anemometer behavior

Postby Andy.Clifton » Mon Nov 18, 2013 1:53 pm

Thanks, Jenni. These are all fairly different questions, and I don't know that they belong together. For the sake of my / your sanity, maybe you could split these questions out into four new threads (one for each question)? I'll repost my answers.
Andy Clifton, Ph.D.
Senior Engineer

Jennifer.Rinker
Posts: 21
Joined: Tue Jun 25, 2013 11:34 am
Organization: Duke University
Location: NC, USA

Re: 20 Hz sonic anemometer behavior

Postby Jennifer.Rinker » Mon Nov 18, 2013 1:57 pm

Consider it done.
Jenni Rinker, Ph.D.
Mechanical Engineering & Materials Science
Duke University/NWTC

Everett.Perry
Posts: 33
Joined: Tue Jan 29, 2013 9:53 am
Organization: Texas Tech University
Location: Oregon

Re: 20 Hz sonic anemometer behavior

Postby Everett.Perry » Mon Nov 18, 2013 2:11 pm

Hi Jenni,

I have been following this thread and was wondering if you split this thread up, could you please include if you are seeing the behavior on M4, M5, or both?

Thank you,
Everett
Everett Perry
PhD Candidate, National Wind Institute
Texas Tech University

Jennifer.Rinker
Posts: 21
Joined: Tue Jun 25, 2013 11:34 am
Organization: Duke University
Location: NC, USA

Re: 20 Hz sonic anemometer behavior

Postby Jennifer.Rinker » Mon Nov 18, 2013 2:34 pm

Hi Everett,

My observations are only from the M4 tower data; I've added notes in the other two threads about which data I am analyzing. Thanks for the note!

- Jenni
Jenni Rinker, Ph.D.
Mechanical Engineering & Materials Science
Duke University/NWTC


Return to “NWTC Wind Data”

Who is online

Users browsing this forum: No registered users and 1 guest