<Musing>
It became obvious something had to give, something had to go or change. It just wasn't clear what it was. This started because I was completely unhappy with the persistence arrangements. The use of DETS (essentially just erlang tuple storage on disk) for the erlang side of things which was managed by erlink-sm, the Gen FSM behaviour that runs the whole caboodle. On the GUI side, or more specifically the java side it was JSON serialisation. Now, the java side needs to consume many of the structures defined in DETS and also needed something better than JSON for its own stuff. In fact they both needed to come together in technology terms but remain separate in terms of their function. The split of function was quite logical, things the application as a whole needs are stored as a central resource. Things a particular GUI or other plug-in needed are private resources.
I went through the usual; should it be a conventional RDBMS, which RDBMS, should it be XML. XML won for a short while, but I've never been keen on using XML on data that has relationships, although it can deal with it it is hard work to maintain. So it should be some form of RDBMS. I decided it was sensible to do this in Mnesia (the erlang distributed database) . I started out and pretty much got everything in the erlang side moved over (although I've not switched to using it yet). Then had another crisis of confidence and almost threw erlang out of the equation as it was clear I could implement most of what I had done in erlang in java and retain almost intact the pieces that were already java. The plug-in model would work just as well using what's referred to as an 'inversion of control' or 'dependency injection' pattern. I even chose the framework (PicoContainer). The database would be the pure java database (HSQLDB). JNI was working well for my shared memory so I had no qualms about hooking in control and dsp the same way. However, I would be throwing away quite a lot and potentially a lot of things I had not even though of doing yet, and that was a concern.
One message on the erlang list changed the shape again. It looked like erlang might be viable for a native GUI with the wxErlang bindings. Now I've used wx before in Python and I know it's limitations but the idea of completely removing this impedence mismatch between erlang and java was very appealing. I've done some tests and it looks viable although I'm expecting some terbulent waters ahead.
Finally, what does this mean. With the GUI as a native erlang node the mnesia database looks like a fantasic choice because all the data resides here and can be seamlessly distributed so my GUI and state machine and whatever other nodes need it can query the database (through the API I've already written of course). The most important thing however is no impedence mismatch. Mnesia is not just a relational database but also a distributed database and also an object database in that it can natively store any erlang term no matter how complex. This is ideal for arbitrary data that a plug-in wants to persist - I don't need to know or even care what it is. This opens the way for all nodes to be erlang as it only leaves the DSP and streaming nodes which I have not yet converted to the port driver model. When that is so I can use the Supervisor behaviour to manage my nodes and processes rather than my home grown supervisor.
Due to the way the system is architected I can move small pieces of GUI into wxErlang and they will interact with the rest of the system. That is my plan, to experiment with moving functional blocks, somewhat like I have done with the Jython script in the RadioAPI section.
O, Yes and what did I think of F# and D. F# on paper looks very good, I played a while and thought it's probably worth playing some more, later. I don't know what it was but it didn't feel right and wasn't grabbing my attention. I spent quite a bit longer on D. This is an interesting language and I think it will be important . It was tempting to have a fully compiled language which could do everything I needed and still be cross-platform and with a sprinkling of dynamic features. As I played it became obvious it was too early in the life cycle. Dynamic loading was a fundamental requirement for a plug-in architecture. On the surface the DDL project looked just the ticket except it turned out to be Windows only (as did quite a few other things). It was also very version dependent of libraries. Talking of which there are two competing libraries. A lot of the efforts will die over the next year with survival of the fittest . I think I will go back in a year and see where it's got to. D however is a potential replacement for the Java DSP that I have started and also for the control and streaming nodes glue but that's way down the list at the moment.
</Musing>
This is the home page of the ErLink-SR project. This project has been running for the best part of a 18 months and is starting to show real promise. The objectives are quite simply a platform which will support the rapid building of software radios across multiple operating systems and hardware implementations.
The key ideas behind this are....
|
|
A platform built from a number of components. Components are strongly functional, that is they perform a single major function and the architecture of a component is optimised for the function it performs. |
|
|
Major components of the system are easily extended and customised. |
|
|
Components are connected together by a common infrastructure. |
|
|
Components are predominently cross-platform (although there may be exceptions where a platform is not capable of providing the function). |
|
|
The infrastructure allows components to be created, connected and orchestrated across the same or multiple machines running the same or different operating systems. Distribution is transparent to the components. |
What's it all about
It's probably worth explaining a bit more about the what and the why. This is not just a web site, it's my note pad and design area. It's where I let ideas develop, that's why things here change from time to time because it's not just documenting what exists but where my thinking is going. This little write up is about the vision. Much of that vision is actually implemented, but sometimes I don't realise what I have until I look at it from a different angle. That angle was simply, where am I going with this API and from that sprung the diagram below which is just a different way to look at the architecture but may be somewhat easier to swallow than the stuff on the architecture page.
It's hard to give a name to this development, the terms platform, framework, workbench and even breadboard which I've used in the past don't convey any real information. In effect this is just another software radio system. The word system being used in it's proper sense (a much overworked word today) as being a full end to end implementation; batteries included as they say.The difference is that this system is very extensible and customisable. That's good for anyone in the future that might want to try it, but it's also good for me. If I see a new radio I like , provided it's interface is published I can slot it in without any unwanted side-effects to the existing system. It's this, no side-effect thing which is really important. As a software system gets more complex, most become high maintenance and small changes can cost a lot of time. That's why most systems get to a point of complexity where they have to be completely re-factored in order to move forward. I'm not saying this system somehow avoids needing re-factoring, I've done it once and there is a lesser one coming up as I move the existing components to use common API's but it's much less dramatic than completely changing an architecture.
The System Vision
The vision is a core that embodies much of the hard work that goes into building a radio system, especially those that span machines and operating systems. Most of the core has been around for quite a while. The Erlang message bus with the registration and switching system I've built around it. The core processes shown in mouve for DSP, Control, and managing the samples input and output streams are all there and have implementations that work well. These implementations to a greater or lesser extent are the plug-in blue blocks. I say greater or lesser because some require more work to make them truly loadable., but the pattern is there and implemented for both 'C' and Java type blocks and can be cloned to the nodes that were developed earlier.
The shared memory has proved to be invaluable as a way to get samples where they need to be and is going to make the API development much easier than it would have been.
The three API modules are now the most important pieces to develop. These will all offer exactly the same interface, except the Erlang API will not have shared memory access which will restrict its capability. The Java API is started and has a writeup on the API page.The 'C' API is coming together and will catch up with the Java one before they both move forward. The Erlang API is actually triavial to write so I will leave it to last. The Jython GUI will use the new Java API when it's finished as functionally it will do what the current UI interface does. This also means that the API will be fully capable of having a full-function UI written against it.
The major win on these API's is that they open up a whole world of possibilities. Remember that the API is not simply a way to talk to the DSP and Hardware as a single entity, it's much more than that. There is an application node out there, a profile and capability system and a messaging system that uses classes of node to route messages. What does that mean. In simple terms, you call an API function, say to set a frequency. If the application knows that setting a frequency means informing all nodes that have registered as a UI class, then they all get the frequency update. Think how easy that makes CAT and MIDI control.
A few possibilities:
- Take the existing PowerSDR console and strip it down to just the UI functions and plug it in. Any .NET language will be able to talk to this system and there are quite a few of those. Once stripped compiling under Mono should be straight forward and if the porting work has been done (see section at the end) then its a PowerSDR Linux system.
- Already demonstrated is the capability of any JVM based language that has a Java interface to talk to the system.
- Other favorite languages can be used. How about Squeak or Dolphin Smalltalk with a Seaside continuation based web interface.
- One of the very capable Forth implementations would make a great scripting environment, Win32Forth is a good example.
- Create a MIDI device and connect all manner of hardware gizmos and software clients
- A CAT interface.
- 3rd party software can be hooked in and sent samples from shared memory directly or down a VAC pipe without adding any special handing software into UI, DSP or streaming code.
What Needs to be Done
This list does not include the additional nodes that I want to create for MIDI, CAT and Scripting.- The API's needs to be completed.
- Modify the Jython GUI to use the Java API.
- Bring the DttSp node up to the same pluggable standard that the HW Controller node is (in tech-speak make it a port driver).
- Then there is some porting work (it's all Windows developed at the moment but because of the design porting does not mean adding or re-writing very much).
- Write a portable version of the shared memory library.
- Create a shared library for the Linux version of the SDR1000 HW Control.
- Create a shared library for the Linux version of the DttSp code.
- Write stream drivers for Jack/ALSA (actually, borrow the code).
- There are UI bugs to fix and further functional stuff to finish mostly around the option settings and TX path.
- New stream drivers for SDR5000 and HPSDR.
- Complete my jDSP (Java DSP implementation).