Project Overview

GNU Radio is a free software development toolkit that provides the signal processing runtime and processing blocks to implement software radios using readily-available, low-cost external RF hardware and commodity processors. It is widely used in hobbyist, academic and commercial environments to support wireless communications research as well as to implement real-world radio systems.

Saturday
Dec152012

Metadata File Format

One thing that I think GNU Radio is great for is science and experimentation. But one of the main drawbacks that we haven't addressed until now is the storage of information about data that we've collected. When making a collection, later on, we'd like to know a lot about the collection setup, properties of the radio, information related to the samples, and possibly many other things (like the date of collection, location, the weather, etc.). Part of the last GR Hackfest was spent figuring out the proper format for including and managing this kind of metadata in our data files.

Well, I'm happy to announce that we now have this capability natively in GNU Radio. We have a file sink to write metadata files and a file source to read these files back again. We have a few simple utilities that can read the metadata out of a file, too. Now that the mechanics for metadata are there, we'll be interested in using this concept more and more. For example, we'd like to update our utility uhd_rx_cfile that just captures raw samples from a UHD device to use metadata. This would very simply put in header data that includes the time stamp from the UHD device and the sample rate, frequency, and gain of the receiver. And then there are some plotting tools that we have to display files in different domains. We need to update these so that they understand the metadata and can adjust themselves accordingly (like setting the right time or frequency axis based on the metadata as well as knowing the right data type of the samples).

We'd also like help in knowing what other tools people would like to have for managing metadata files. Even more, we'd like help in building them! One tool that I can think of is a way to read a metadata file and split it up into multiple files where there's a single metadata header and the data segment for each file. We could then have smaller, more manageable files which we are capable of reconstructing due to the metadata. And I'm sure that there are plenty of other tools that we can create to make the handling of this kind of information more useful.

To start explaining the metadata concept, I'll summarize here by (mostly) copying what I wrote on the Hackfests wiki page (http://gnuradio.org/redmine/projects/gnuradio/wiki/Hackfest1211).

We now have Version 0 of the metadata files in 'master' and 'next', so they will make there way into the 3.6.3 release. There is a blocks::file_meta_sink and blocks::file_meta_source. The file sink creates a file and writes all data to it, along with headers of metadata. The file source is resourceful in that you can just give it the metadata file itself and it's capable of reading out everything it needs in order to pull the data and source it into a flowgraph. Meanwhile, other metadata gets transferred as stream tags so blocks down the line can also know what's happening with the data. Pretty good stuff!

For a more complete discussion of the metadata header format and how to use it, we've added a page to the Doxygen manual that explains it all. This manual page is not on the gnuradio.org's website version of the documentation since we have not made a release containing this capability (but it will be in 3.6.3, which will be released soon). For now, you can build the documentation locally (in either 'master' or 'next') and you'll have access to the page, which is located under "Related Pages" and "Metadata Information."

Saturday
Dec152012

ControlPort now in 'next' branch

Part of the work at the latest GR Hackfest was to move ControlPort into gnuradio-core and get it ready to be merged into our 'next' branch (which is the upcoming 3.7 release). One of the main reasons why we wanted to put ControlPort into the core of the project was so that anything and everything in GNU Radio could make use of it. The start was to access parts of gr_top_block, the main component of the flowgraph. With (hopefully) a few tweaks, we expect to be able to remotely access the start/stop functionality of the top block, which gives us a whole new layer of control over GNU Radio applications.

The main reason that we didn't put ControlPort into gnuradio-core originally was that it was much easier to build as a conditional and optional component if it was its own separate gr-ctrlport.  But that meant that gr-ctrlport depended on gnuradio-core. We couldn't then also make gnuradio-core use gr-ctrlport, so nothing in gnuradio-core would have been able to use ControlPort functionality. Most of this would have changed come the complete switch over to the 3.7 structure and API where gr-ctrlport would only depend on gnuradio-runtime. But we discussed it and figured that allowing ControlPort options in things like gr_top_block is important, so we had no choice but to move it into gnuradio-core (and in 3.7 it will be in gnuradio-runtime).

The downside of this move was purely personal. I now had to work through the build system to have parts of gnuradio-core built if gr-ctrlport was enabled. But we did it, and now ControlPort is usable on all blocks and in any component. For anyone working off 'next', you now have complete access to ControlPort. Documentation, though, is still forthcoming...

Sunday
Nov182012

Public Release of ControlPort

http://www.trondeau.com/gnu-radio-conference-2012/I've (finally) pushed the ControlPort work out to my public repo on Github. The repo is:

git://github.com/trondeau/gnuradio.git

And the branch with ControlPort is, amazingly enough, 'controlport.'

For those reading this who are unfamiliar with ControlPort, we announced it and I demo'd it at GRCon12. You can find my presentation here. ControlPort is a new set of hooks into GNU Radio to allow a remote client to attach to a running GNU Radio program and get and set parameters. It's quite a powerful concept with an SDR application, and we think that there are tons of ways to use this to improve on current work and to think of entirely new ways of using SDR.

When you look at the source code of the branch, and right now that's by far the best way to learn anything since we haven't spent any time on documenting its use, you'll see a few new things. First, there is a new top-level component called gr-ctrlport. This is where all of the code that goes into mechanics of ControlPort live. Second, you can look at a couple of other components for how ControlPort is used and tied into the build system, specifically gr-digital, gr-filter, and gr-channels.

One of the driving concepts behind ControlPort is that it is only an add-on to GNU Radio and not a requirement. We do not want to force people to build this and the required dependencies (we are using ZeroC's ICE for the communications backend) to continue doing their other GNU Radio work. So everything had to be made as separate as possible with the ability to test for ControlPort before being able to use it. So when you look at the directories like gr-digital, you will see that the CMakeLists.txt files check for ENABLE_GR_CTRLPORT before linking against the library or using ControlPort as a dependency. There are other checks that are internal to each block that uses ControlPort for the same purpose.

ControlPort's main focus is to create an external interface to a running GNU Radio application that will advertise parameters that you can set and get remotely. Much of the structure of gr-ctrlport is designed to make this interface simple and generic. Each block is responsible for exporting the variables as either gettable, settable, or both. The two blocks that showcase these capabilities are gr::digital::pfb_clock_sync_ccf and gr::channels::channel_model. You can look at these files to see how we've implemented the exposure of the various block's parameters.

But keep in mind that the interface will likely change. We are even now talking about ways to simplify the declaration of the parameters to be accessible over ControlPort. Hopefully, when we're done, the code will be even more readable.

There are still lots of details to be discussed and documentation of the use and development of ControlPort to come. This is just a general announcement of it's public availability and a brief taste of what we're doing and where we're going.

As a way to get started, the 'controlport' branch includes an example that uses ControlPort (there is a WXGUI and a QTGUI based version of this, actually). In gr-ctrlport/examples you can open the pfb_sync_test[-qt].grc in GNU Radio Companion. This simply simulates a QPSK signal going through a channel model and begin recovered using the PFB clock sync and a Costas Loop. Since the channel model and clock sync blocks both export parameters over control port, we can dynamically alter the simulation from a remote client.

When looking at the GRC canvas, you'll see a couple of ControlPort-specific items. As part of the graph, we have a 'Complex Probe (fixed len)'. This is one of the few blocks in gr-ctrlport. It exposes a vector of samples so that we can access them remotely, such as to plot. The other one is 'CtrlPort Monitor', which is up in the top left near the 'Options' block. This monitor is an application that we wrote to provide access to the items available over ControlPort. First, it looks for exposed endpoints from the running flowgraph, and if found, it will attach itself. Below is a screenshot of this application. You can see that it's a table of parameters with some properties. The key is the parameter name, defined as the name of the block and its unique ID number (unique within the running graph). You then get the current value, any units if they were specified, and a more detailed description.

When you double click on one of the rows, it will pop up with a graph to display that parameter. This is true if a) the block has set a 'get' function for this parameter (which is true here since the application only displays those parameters that are gettable) and b) a default display type was declared when the variable was exported. If no display type was given, nothing will happen. However, right-clicking will bring up a menu where you can select the type of plot to use.

Below is a screenshot of double-clicking on the 'probe2_c1::constellation row. This shows the IQ plot of the samples gathered by the probe in the flowgraph.

If a variable is settable like 'channel_model14::noise, you can right-click and bring up the properties window. This will give you an edit box and a slider to set the value.

Since ControlPort is really meant to be used across a network, this GRC example isn't any more exciting than just hinting at what's possible. But we can easily transform this example into such a remote operation. First, in the GRC graph, bring up the properties of the 'CtrlPort Monitor' block and turn 'Enabled' from True to False. Now, when we run the program, it will not automatically start the monitor application.

In the GRC console window, notice that the application now announces that it has created an endpoint that should be something like "Ice Radio Endpoint: gnuradio -t:tcp -h <local IP> -p <Port #>". This is the Ice endpoint that we connect to. Using a terminal on the local machine or another machine where this GNU Radio branch has been built and installed, you can use the gr-ctrlport-monitor application to connect to the running flowgraph:

gr-ctrlport-monitor 192.168.1.1 12345

Just make sure you're firewall settings allow you to use that particular port.

There will be a lot more said about ControlPort as we continue to develop it. I'll post more here, but we will be producing more structured documentation for how to use and develop with ControlPort.

 

Saturday
Nov102012

Version 3.7: The Work in Progress

We've been hard at work over the past few months on getting GNU Radio completely switched over the the new 3.7 API. It's a hard path to go down with some many files that need to be touched, edited, fixed up, reworked, etc. But we're making progress, and I wanted to update everyone on where we are.

Recently, we finished up the new gr-analog component that takes a lot of blocks from gnuradio-core and moves them into the new analog namespace. These blocks include:

  • agc_cc
  • agc_ff
  • agc2_cc
  • agc2_ff
  • cpfsk_bc
  • ctcss_squelch_ff
  • dpll_bb
  • feedforward_agc_cc
  • fmdet_cf
  • frequency_modulator_fc
  • noise_source_X
  • phase_modulator_fc
  • pll_carriertracking_cc
  • pll_freqdet_cf
  • pll_refout_cc
  • probe_avg_mag_sqrd_c
  • probe_avg_mag_sqrd_cf
  • probe_avg_mag_sqrd_f
  • pwr_squelch_cc
  • pwr_squelch_ff
  • quadrature_demod_cf
  • rail_ff
  • sig_source_X
  • simple_squelch_cc
  • squelch_base_cc
  • squelch_base_ff
The current master branch reflects all of these blocks in gr-analog, but we have not removed and blocks from gnuradio-core, yet. This is part of our contract with our users that we don't change the API on the master branch until we are ready for a new API release. So all code built against master or the 3.6 versions will continue to work. We have simply added gr-analog as another set of blocks. We are encouraging people to start moving over to these new components since when 3.7 is released, anything using blocks in gnuradio-core (or the 'gr.' module in Python) will no longer work.
In the meantime, by duplicating blocks, one thing we are running into is a duplication of blocks in GNU Radio companion. You might notice that if you are working on the master branch that you'll have two sig_source or noise_source or any other blocks in gr-analog. This is just a consequence of using the same block and component names as previously but in two components. This is just something we're going to have to live with for the next couple of months.
You can tell which component you are using from the ID name in the block's properties window. We give them prefixes from the component they are from. So the old blocks will have a "gr_" in front while new blocks from gr-analog will have "analog_". They should be functionally equivalent, but the new "analog_" block is safer to use as it will not go away in the future. It's also, as far as I can tell, true that the top-most block will be the newer block, but it'd still be best to make sure.
When you look at the next branch, though, all of the components moved into places like gr-analog are being removed. Due to certain constraints and dependency issues, we cannot remove all duplicated code out of gnuradio-core until some things have progressed further.
One thing that I wanted to point out in this update is that there are bound to be a few problems that arise during this transition. Us and you will probably get a bit confused here and there. We've made some mistakes, like how gr_rotator is part of gr-analog on master, but we've removed it from there on next as it more appropriately belongs in gr-blocks.
And while we've been trying to make sure all Python, C++, and GRC example codes reflect the changes happening, we're going to miss some of those, too. I only as for both some patience with this process as well as help. Point out where we haven't properly updated some code, or if you'd really like to help, send us a patch!

 

Wednesday
Oct312012

Community Manager

We are so very pleased to announce that Martin Braun will be taking the position of Community Manager for the GNU Radio project. Martin was the obvious choice for this role since he has already been fulfilling a large portion of the role informally over the past few years.

The Community Manager role is a new one for our project, and so Martin will be establishing its full responsibilities over the next few months. The basics of what we imagine this position will entail includes:

  • Managing, editing, and developing gnuradio.org.
  • Growing the community of developers and other contributors
  • Organizing and spear heading our efforts with the Google Summer of Code.
  • Managing/moderating the mailing list.
  • Helping with the planning, promotion, and execution of the GRCon.

I would personally like to thank Martin for all of his help in the past and his continuing support in his new role.