plotdat, a 4d plotting program for sdaq data

For PC type software that runs under some PC opsys.

plotdat, a 4d plotting program for sdaq data

Postby Doug Coulter » Sat Jul 21, 2012 2:00 pm

I've been working up a plotter program for sdaq - while I know hardly anyone has one yet - though everything you need is already up here to make your own, this is generic enough to be used with other things with only slight mods, or a little script to massage your data from wherever into the right format for this program. I'm doing scatter plots of X,Y,Z and color - there's your 4 dimensions. More are possible once I learn a little more about Gnuplot, which is what does the actual plotting, as the points can have a size, shape, color and transparency level as well (or so the gnuplot manual seems to imply).
To use this, you need to have Gnuplot, and of course, perl, on your system.

The idea behind the whole system design is - use everything for what it's good at. Thus, PC's stink at gathering data from a bunch of sources and keeping it all time-aligned - they just stink at deterministic timing. So, we used the sdaq hardware (which is the same uP board as the standard geiger counter with different code in it) to collect the data from various sources - time, a/d channels, a couple counters - and batch it into text lines for stuff that occurred all at the same time.

Then a PC reads this serial data stream and puts it into a file, while plotting it in 2d in real time so you can be sure you're acquiring the data correctly - it makes clipping on an a/d input or counters that either don't count or count ridiculously fast obvious during a run. So far, this is all with "raw" data. PC's are of course, great for writing things down and doing complex computer-intensive things to data, so that's what we use the PC For.

Finally we get to this program that works off log files made by sdaq, and we want "real" units - volts, amps, mbar, counts/second - not the raw numbers from the a/d. While acquiring, you want raw numbers, as it's easier to be sure you're getting the data acquired correctly in that case, but for later analysis, nope, you want the real deal. You also would like to be able to plot medium complex functions of the input data, as well as map anything to any axis on the plot. This lets the human part of the system - and we are part of any such system - do what humans do best - see patterns in the data, and learn what to try next, perhaps. The task of making scaling and mapping adjustable by the user in a gui had me stumped for awhile - but now I've at least got something that works, even if it does take a little bit of brainpower to work it.

Here's what the GUI looks like.
plotdatgui.png
screeenshot

For each (currently) possible axis, I simply allow you to type in perl code to map whatever onto whatever axis. Yes, this means you have to know perl syntax, and you have to know what the available input and output variables are. In this case, $x means the X axis data. $a0 is a/d channel zero, $c0 is counter zero - it's pretty obvious. And if you speak virtually any computer language, you can do this one pretty easily. You can even have multiple lines of code (perl doesn't need a <CR> between lines, just the semicolon terminator for each). Here, for the X axis, I have a signal from the power supply that says what voltage its putting out. Fine, but I have a 100k resistor between it and the fusor, and I want to plot the ACTUAL voltage on my grid, so I use a signal that tells me the current drain, so I can then calculate the drop.
Of course, for ohms law to work and this come out in kv, for example, I need to have the right scaling factors, which I got the good old hard way - set the supply to say 50kv, look at the raw a/d numbers and calculate a scale factor, manually. Ditto for things like supply current, and the pressure signal from the gas gage (not in these files, but I have it working now). You can then type various things in for axis labels, decide if you want log plots in various axes (haven't figured this out for color yet and may remove the checkbox) - then hit do plot and bingo. You now have a 4d plot you can mouse around in and twirl to any viewpoint to see your patterns. Since there is some setup involved in getting to a plot - I provide presets saved in a file to handle that more easily, and you can add as many of those as you want.

In the case of this plot (go run this, the log file is in the .zip so you can) I mapped time in the run onto color. You can see when in the run various events happened, and where the most neutrons came out. For fun, near the end of the run, I cranked up the power supply current limit - one gets tired of dumb kids (or adults) saying it's obviously going to work better with more power. I rest my case - go look. Obviously, the neutron production function is going almost straight up as I hit my 50kv limit. The L shape you can see is what happens if I don't control gas pressure finely - heating the tank causes more pressure (and takes D off the walls, which I want there, it turns out) and therefore more current - you can see the current limit of the supply in action - and also that the neutron production goes DOWN with current in limit, because it's a very sensitive function of actual grid voltage - we are space charge limited, and very severely so. In fact, here, running much less gas (which requires a separate ion source to make the main grid draw current) increased the Q quite a lot - less gas, less space charge, less defocusing, fewer energy-exchange (usually that's waste) collisions....

I'd surmised most of this by being compulsively observant of the data - but hey - not everyone can manage that - and it IS a pain - so here's the solution.


Here's the code and some sample log files so you can play with it all:
plotdat.zip
Source dir for plotdat, with log files etc
(36.66 KiB) Downloaded 331 times



There's a lot more to do here. One goal is to come up with a CD that will just install this stuff in windows (the code itself is platform independent) - as windows doesn't have perl, GTK, gnuplot, you name it, and finding and installing things based on error messages seems to be beyond all windows users other than professional developers. Linux has all this stuff already, except gnuplot, and it's as simple as clicking on "synaptic pakage manager" or "ubuntu software center" and typing in gnuplot - then click install - for us.

I'd like to be able to concatenate multiple logs and really fill in the scatter plot with more data - should be easy, even just using a text editor, but we might want that timestamp to be changed so we don't have points that "think" they are at the same time - from different runs. Heck, I'd like to add MySQL database, but then we are getting into things that are fairly challenging to deploy to non-sys-administrators.

I want those other dimensions of shape, size, transparency -
It would be nice, but probably won't happen - if twirling the plot could rotate it in more than the usual 3 dimensions. Troy once figured out how to do that for me, but I don't have the code anymore.

I expect this will be an ongoing project, so any requests for features should be directed to me - or any perl programmers who add them, I'd ask them to submit their code back here so we can all benefit -GPL2 is the word.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Return to PC

Who is online

Users browsing this forum: No registered users and 3 guests

cron