8:42AM July 27, 2014: Making pretty pictures with the CLI-junky's waterfall
When most radio amateurs think of a waterfall display, there is a very set fixed idea. A chunk of spectrum scrolls by down the screen. There is a panel of knobs and buttons to one side. It looks flashy and needs to run on a fairly powerful computer.
For most, the waterfall is the pinnacle of radio software. But it has a number of issues that have bugged me.
- Limited frequency display. Usually can't do more than your SDR's native bandwidth.
- Limited time display. If you didn't notice something, it just scrolls off the edge into the void.
- Limited FFT bins. Usually doesn't work so well when you want substantially more bins than your screen has pixels.
- Qualitative rendering. It is not easy to go from colors to dB.
Rtl_power is a unix-hacker's approach to the waterfall. Its unique features include:
- Unlimited frequency range. You can do the whole 1.7GHz of a dongle.
- Unlimited time. At least until you run out of disk for logging.
- Unlimited FFT bins. But in practice I don't think I've taken it above 100k bins.
- Quantitative rendering. Exact power levels are logged.
- Runs on anything. A slower computer will use less samples to keep up.
Enough background. First, you should install rtl_power. But you probably already have it because it comes with the rtl-sdr driver. Next, what can you do with rtl_power?
Enough background. What can you do with rtl_power?
Rtl_power is not constrained by bandwidth or time. A survey is basically a summary of an entire band. If anything happens, it will appear in the survey. Let's use airband-voice as an example. This band occupies the range between 118MHz and 137MHz. I built an antenna just for airband, and then started looking for signals with a traditional waterfall. I found nothing, and came to the conclusion that either my antenna was bad or I was in a dead zone. In fact, it was the waterfall's fault. Take a look at the image now, if you haven't. It was made with the following command:
rtl_power -f 118M:137M:8k -g 50 -i 10 -e 1h airband.csv
It represents one hour of the entire 19MHz airband. Chatter and active frequencies are quite visible. But you could never see this in a normal waterfall. A normal waterfall restricts the view to a narrow slit. Compared with the survey, a waterfall displays a chunk that is 300 pixels wide and one pixel tall. No wonder I couldn't find any traffic.
When you are considering getting into a new branch of radio, performing surveys is a smart idea. First check if your location has good reception. This is a few images of LEO satellite passes, provided by x-f. Reception looks pretty good. Using rtl_power is much lower-stress than monitoring a waterfall. Timing does not really matter and you don't have to stay glued to the screen. Go eat dinner and check the plot afterwards.
Really wide band surveys
Taken to an extreme, the entire tunable range can be surveyed.
Superkuh has probably done the most with this, and was instrumental in the development of this feature. Each pixel on this diagram represents around five screen-fulls of traditional waterfall. You can make a similar plot with
rtl_power -f 24M:1700M:1M -i 100 -g 50 -e 24h data.csv
One popular topic is how to mod your dongle to minimize noise. Rtl_power gives a moderately qualitative way of measuring this. The setup is as follows: put a 75 ohm terminator on your dongle. Crank up the gain and measure the power levels across the entire range with the following command:
rtl_power -f 24M:1.7G:1M -g 50 -i 15m -1 noise.csv
This dongle is pretty clean. Laptop running on battery, 1 meter USB extension, ferrites on extension, dc-block on the USB shield, 75 ohm terminator, all inside a metal box. The 28.8MHz overtones from the on-board crystal are prominent and a little bump at 480MHz is from USB.
Unfortunately the dB values are mostly meaningless in an absolute sense. But they do have relative value, when compared between multiple runs. Make a change to the hardware, run the scan again and overlap the two sets of data. Did the change actually cause an improvement? Little experiments like this will help you separate the snake oil from the science.
Radar or meteor detection
The previous examples were all about how wide-band scanning can be used. But rtl_power can also perform high resolution narrowband scans as well. With a fine enough resolution, Doppler shift is visible and velocity can be calculated.
This is a sample of passive radar. First I poked around on TV Fool until I found a station with the following requirements:
- high output power
- only station on the channel
- several hundred kilometers away
Finally, use wikipedia to get the pilot frequency. Tune to that with a fairly high resolution FFT (bin size between 1Hz and 4Hz, usually) and planes will provide a Doppler-shifted reflection. Note this is very sensitive to ppm (because of the small range) and thermal drift. The thermal drift is evident because the pilot line is not perfectly vertical. For that run I used
rtl_power -f 674.230M:674.233M:1 -g 50 -i 1 -e 1h radar.csv
It is possible to calculate velocity from the Doppler shift, using the equation
Δv = c Δf / f
Because all the FFT bins are equal width, you can instead calculate the Δv of a single pixel and count pixels on the x axis to get the total velocity. In the above example, each pixel is 0.4 m/s wide, after subtracting out the thermal drift.
The same principle can be applied for meteor bounce, but I personally have not gotten that to work.
There is one benefit to how often the thermal drift effects appear. If you are doing modifications to minimize thermal drift, rtl_power provides a simple way to observe the effects as the dongle warms up and stabilizes.
Radio astronomy (RA)
Much of RA is based on power measurements, and rtl_power was originally developed with RA in mind. However rtl_power is not yet entirely suited for RA yet. Typically an astronomer will perform their own corrections/calibrations to the data and convert it to degrees Kelvin, without ever touching decibels. So rtl_power needs that added to it, and this should be fixed soon. (The example picture here was generated with an rtl-sdr, but not using rtl_power.) Otherwise rtl_power is excellent for RA, because its easy scriptability makes it simple to automatically record transits and other events.
Rtl_power logs data to a flat CSV text file. This is not easy on the eyes. There are a couple of scripts, utilities and frontends to convert the output nto graphics.
- heatmap.py - Written by myself, this was created because most software took too long or ran out of memory easily. Most of the images on this page were generated with heatmap.py
- flatten.py - Turn a 2D table into a 1D line. Useful for generating noise and RA plots.
- RTLSDR Scanner - Cross platform spectrum analyzer.
- ViewRF - Spectrum analyzer for the Beaglebone Black.
- RTL_SDR_Wide_Spectrum_Analyzer - A frontend written in Gambas.
- GUI for rtl_power - Windows only. Saves beavers.
- RTL-SDR Panoramic Spectrum Analyzer - Windows only.
Q: How do I measure a single frequency?
You don't. You measure power within a band. If you are really insistent about measuring the power of a single frequency, ask yourself what happens when the thermal drift of the crystal moves the tuner away from that infinitesimal frequency. You can only measure a frequency and bandwidth.
If you want to stream directly to a file, provide the file name as the final argument.
-f lower:upper:bin_size [Hz]
Set a frequency range. Values can be specified as an integer (89100000), a float (89.1e6) or as a metric suffix (89.1M). The bin size may be adjusted to make the math easier. Valid bin sizes are between 0.1Hz and 2.8MHz. Ranges may be any size.
Collect data for this amount of time, report it and repeat. Supports 's/m/h' as a units suffix. Default is 10 seconds. Minimum time is 1 second, but for extremely large ranges it may take more than 1 second to perform the entire sweep. Undefined behavior there.
Run for at least this length of time and exit. Default is forever. Like the other times, this supports 's/m/h' units.
Enable single-shot mode, default disabled. Perform a single integration interval, report and exit. It is not necessary to use
-e with this option.
When using multiple dongles, this indicated which. You can also identify dongles by the text in the serial number field of the EEPROM.
A floating point gain value. The dongle will use the closest gain setting available.
Correct for the parts-per-million error in the crystal. This will override a ppm value retrieved from eeprom.
The window is a shaping function applied to the data before the FFT. Each will emphasize or deemphasize certain aspects. The default is none (aka boxcar, rectangular). Options include: hamming, blackman, blackman-harris, hann-poisson, bartlett, and youssef.
The crop sets how much of the bandwidth should be discarded. 0% discards nothing, 100% discards everything. The edges of the spectrum are lower quality than the middle. There is less sensitivity, gain roll-off and out-of-band aliasing. Higher values of crop will produce a better spectrum, but do so more slowly. Values may be a decimal (-c 0.1) or a percent (-c 10%). Default crop is 0%, suggested crop is between 20% and 50%.
This setting has no effect on bins larger than 1MHz.
-F 0 | 9
Not exactly the best named option, this configures the downsampler and the downsample filters. Downsampling is only used when the total bandwidth range is under 1MHz. (Like in the radar example above.) Omitting the
-F option uses the default downsampler, rectangular. This downsampler is very fast but has bad spectral leakage.
Filters with minimal leakage are
-F 0 and
-F 9. 0 is a plain filter, but has bad droop at the edges of the spectrum. 9 uses the same filter as 0, but has a 9-point FIR filter to correct the droop. 0 needs more cpu than rectangular, 9 needs more cpu than 0. Rectangular needs the least cpu, 0 needs more, and 9 most of all. It is suggested to use 0 with
Enables peak hold. The default behavior is to average across time. Peak hold uses the maximum value across time. Note that averaging improves the SNR, and peak hold will tend to make a spectrum look much worse.
Enable direct sampling. Requires that you have first modified the dongle for direct sampling.
Enable offset tuning. Only applies to E4000 tuners.
Rtl_power produces a compact CSV file with minimal redundancy. The columns are:
date, time, Hz low, Hz high, Hz step, samples, dB, dB, dB, ...
Date and time apply across the entire row. The exact frequency of a dB value can be found by (hz_low + N * hz_step). The samples column indicated how many points went into each average.