Science/Engineering/Technical forums

Is coming along, for linux first, then windows if there's demand for it from people who don't mind installing perl, cygwin, gnuplot etc on windows. Right now, I'm just doing the basics of getting the Standard Counter discovered (out of all possible USB devices) automatically, set up, and data brought to the PC for a rather simple plot (at the moment). I will be adding a GUI to handle things like what to log in a logfile, plot scalings, and optional things to plot, like smoothed data, the aux input for neutron counters, and so on. Lots yet to do, but this is what things look like now. This shows background, a lamp mantle, then a CS-137 spark gap, then background again. This just starts plotting at time zero and compresses the X axis as required during a long count. Obviously there should be some other options, but I'm just getting the functionality chunked right now.

I'll add log scaling for the Y axis and some other stuff of course. From here, it's quite trivial to also shove this into log files (named by time etc) or a SQL database, though the database would have to be created with a known table definition and so forth first (so other programs can all be putting stuff in the same database, different tables, all timestamped together and all that happy crap).

Making a little progress on this on the weekend. As promised, I'm doing a GUI for the program shown before. I'm doing this as a separate thing, as I'm having to kinda learn GTK2, Glade, and well, isn't that enough? The gui is coming along fine so far, but this is like creating the universe in a sense. It's not taking much code at all -- all you have to do is specify the properties of the universe so it works out, stars from, life appears and evolves and so on - might be as simple as choosing the right Calibu-Yau space for the string theory, or saying "let there be light" as in "let there be a space where Maxwell's equations are true" but boy, the homework is killer....

There's a fairly cool advantage here. Perl doesn't natively know how to "yield" to the opsys when it's in a forever loop polling the serial input (with time resolution better than a second). This isn't normally much of a problem, but it makes for nasty looking plots on the system resources, since Linux is cool enough to let you eat an entire core or thread doing that until things get busy, and it makes the program look like a real hog. With GTK and its pals, I can schedule polling for say 1/4 second intervals, and have the serial input code just do nothing if there's nothing to do right then (no complete lines), and that suspends the entire process until either the next tick, or someone clicks on it. For the moment, I'm also using STDIO on the terminal to emit debug and "I got here" type messages as I debug the code. Once you learn this, it doesn't seem so hard, but it's definitely another world from using DevStudio on Windows if you were used to that (and I was, more than a little). Neither worse nor better, maybe better -- I like the events auto-magically getting hooked up, and can tolerate the way Glade works to design a UI, so far. But the learning curve is quite steep -- Glade doesn't do it all for you and create the routines that happen on events. The old version did that, but generated code that changed everytime you edited the UI stuff, which wiped out any event-specific code you'd added, yuck. Devstudio would orphan code in that case, which would cause a different type of problem later...toss a coin, this is going to work in the end, and work well. This code should also play in Windows too, the way I'm writing it so far, with zero changes needed and not even a re compile -- take that, DevStudio

Here's what it looks like so far. The window on the right is going to get the output of the standard counter (like a terminal program or text editor), the left pane will fill with option check and radio buttons etc, and the gnu plot window will be elsewhere on the screen - its own process, which BTW is a good way to make things easy on the machine, we just let the opsys decide who gets cycles when. In a real situaion, there might after all be several other programs running for logging, and we might even add database connection to this one to shove this into a DB instead of "just a log file". Which also implies that at some point, I'll need to add a feature to put gnuplot on any of these logfiles, which will be simply named as something like StdCYYYY_MM_DD_HH_MM_SS.log so you don't have to type names and they'll be sortable easily. When I get this a little (or a lot) closer, I'll be able to cut and paste the real code that does things into this framework easily. As I said, in theory it should even work in windows. Right now, there are a collection of files that go with this. The glade file that is loaded to generate the GUI, and a file for initializing the serial port. Then there will be log files created. I am hoping to find a way to bundle the first two inside the perl program itself for simplicity...I know there's a way, even if I have more than one file...

This might not be the best way to do "cross platform" -- we'll have to see. It might be more trouble (for the user) to get windows installed with perl and all the other stuff than it would be to get them to install virtual box, install linux (which has all this junk already, or nearly all) and run it that way. It's kind of more fun in some ways to have your own little "cloud". I go the other way here, and run Windows in a window -- seems more reliable and solid and hey, that's where that stuff belongs - in a sandbox.

Edit: here's what the gui code looks like. Not usable as is (doesn't do much) but establishes a simple framework and "how to get rolling with this junk" kind of boilerplate. And here's the code from the previous post -- the rest of it. Merging these won't be hard I think. I'll be able to get rid of the "poll forever" as I do the merge, and have a sub called at intervals from the GUI to do that sort of thing, and only do it while we are running. I'll probably need a little code to toss out a line from the serial port after we tell the hardware to reset as we're probably already in the middle of one, or will get a borked one right then - better to start on second #1 or 2, and have it right than garble as the first line in a log file.

Anyone who wants to look this code over, a syntax highlighting editor is the tool of choice. Perl looks like gibberish enough even with that. Gedit, Padre, Geany all are free in linux, there might be some free option on one of the latter for windows, dunno -- I run perl there, but don't write it there.

Edit: I forgot to add the .glade file to this upload, and since I feel like this is useful for backup and example as is, here is that file too. It describes the UI for the project. I don't see or edit this, I use Glade to edit it graphically. If it's like most XML, you'd go blind reading it anyway. note that in general I have to rename all these with a .txt extension or the board won't allow them to upload. That's because some might be evil code that would hurt users. So any of these might need a name change to be used.

Ok, SC now has the more or less final GUI, and is now useful as a template for "doing this kind of thing" so I'm backing it up here. It will now write a log file if requested, for now of fake data we created for testing before I add the USB/serial stuff -- so it's wide open to "put your code here". I'll add the gnuplot stuff next (also will test with fake data) and do the actual serial stuff last. This is just the order I'm doing this all in to make it into a bunch of simple tasks that can be well tested at each step. I find this works well with the way I think, and results in code I can trust. If you can trust all the pieces, and some weird bug pops up, you can avoid having to re-test them all in frustration, because you already did and know they work right, piece by piece.

Someone once said "If architects built buildings like most programmers write code, the first woodpecker to come along would destroy civilization". It's not far off. So I used lots of tiny, unit tested subroutines to encapsulate each little bit of functionality and organize this. Perl is not wordy unless you make it so (I did), but this gives me more chances at reuse out of the "bag o' tricks" later, and the chance to put in more comments for later understanding after I've forgot what I had in mind...

Here it is. Rename the .txt extension off and look at it in a syntax highlighting editor, or it's real hard to read. Though the standard counter hardware/software does have provision to display the second counter or not, I'm not going to use it as it makes the UI more confusing. If asked, I can fix this to not display it in the text window if you're not going to plot it either, but...for now, simple is best. I'm sure most users can ignore a few columns of zeros if they're not using the second input, or maybe it will encourage them to hook something up to it!

Here's a vid of a test run of the PC software. In this case, I have it reading from a logfile to make the plot at about 40hz, which does use some cpu as you can see on the system monitor. Sorry about the lousy focus -- need to put a reading glass lens in front of that camera, I forgot. I'll do a moving screen capture when I have this finished, but it's showing promise now. It can also make a separate plot window for the alternate input data.
http://www.youtube.com/watch?v=D2F9O1s1o7A


This seemed to be using about half of two threads (cores) to run 40 times faster than real life. Not so bad. A few more details, now I add in the serial (USB) port stuff from the command line version instead of the dummy file read, and we're about good to go. I did it this way to have predictable data to test with -- at faster than realtime. Makes writing code better, and now I know if anything is weird from the hardware where to look for the trouble. Seems rock solid so far. This is for linux, but I wrote it such that anyone willing to install all the stuff on windows can use it there too.

Edit: Here's a much better one, on another slower machine, that happens to have an audio/video screen scraper software on it (kind of cool) so you can actually see and read the stuff.
http://www.youtube.com/watch?v=RQPy33Y0q2g

For completeness, here's the code for the PC side as it sits now. I put this dir under /bin and just run it from command prompt for the moment. When it's done I'll make a "launcher" for the taskbar. I'm only using the command prompt to put out debug stuff while working on it. SC is the program run above, and it's reading a data file to go fast while testing. All I need to add now is the serial port/USB stuff from the other code and tie it all together - probably finish this weekend. This code uses a number of perl modules that don't come in the standard distribution, and I'm making a list for install purposes for people who don't already have them. You'll need to install gnuplot as well, which it uses for plotting. All this is in theory, cross platform but there's a lot more missing from windows than linux, so it will be more work to get it going there. Some of the perl libraries are in C, which compile to binary (for speed, hey perl is for glue, C is for cranking out the number crunching) which means that to install them, you have to have a C compiler, adding that whole install to a windows setup (and learning how to work that too).

This is GPL v2 license -- share and enjoy, and there's a lotta comments as this is full of bits of "steal me, I know how to do this" code. If you know C, you can probably kind of squint and understand perl. I code perl as if it was C (it's pretty flexible that way) anyway, so this would be really wordy for a perl monk. But that makes reuse easier for me, and the code a little more clear I hope. Use something that knows how to syntax highlight or this will be hard going. I use the lowly Gedit on linux myself for all this kind of thing -- it's a much smarter version of windows notepad for coders. There are other things that work too - but gedit has near zero learning curve and it's free too. Probably the easiest way for someone running ubutntu linux (I run 10.04) to get this going is to get the ubuntu software center up and one by one type these into the search window, and install the matching names when they appear:

gnuplot (the program itself)
libgraphics-gnuplotif-perl
libgtk2-perl
libgtk2-gladexml-perl
libdevice-serialport-perl
libglib-perl

This uses a couple other modules for file and path munging, but I believe they are in almost all perl distributions already. I used Glade to lay out the GUI, but you don't need that to run the program, just to change the GUI (which means you have to change the perl code to match if you do more than move things around with Glade).

Here is the current latest PC software for the standard counter. It's tested in linux, but not windows yet -- you'll need some more work to get it going in windows, as in installing perl for windows, and getting all the same modules, or their windows equivalents. I've not done this yet, as the only way I run windows is in virtual box under linux anyway, but if I get requests, I'll work that out.

I have not yet found out the trick in windows for two things. One is since we don't tell lies about who we are to the host, windows can't tell we're a USB serial port (more or less) so it gets in a tizzy finding a driver which of course it can't find. I need to create an "install" that tells windows it can use the regular USB-serial driver for this, which I've not done yet (any experts on .inf files, pipe up!). Also, Linux has a slick trick that lets you open a USB port by name, using the name the device itself specified when hooked up. This makes the "which port is it on" problem go completely away in linux, and I'm sure there's a windows registry hack I could use to do the same, there, but I don't know it (so, any windows junkies -- fill me in). As it is, in either opsys, if you try to go by com ports, plugging and unplugging any USB device scrambles all the port assignments, leading to programs locking up or just not working...so that trick is very worthwhile.

The current software displays a log text window, and will plot whatever data is coming out of the unit as you specify in the gui checkboxes and radiobuttons. It will also create a log file in the program directory, with the name as the current time/date.log, which is basically in the same format you'd get if you just hooked to the counter with a gtkterm or hyperterm and saved the data. This way, any program that can plot or make use of this always has the same format to deal with. The logfile checkbox is only tested at startup -- changing it during a run has no effect, which is pretty much the case for the other gui elements. Anytime you hit the start, or restart button, all the gui settings are checked and the appropriate actions are taken. Starting a run automatically resets the standard counter clocks and data, so the first count comes out internally timestamped at second "one".

We will be selling standard counters (they are nice!) but we are also doing this completely open, so anyone who wants to can use the information here to duplicate this if they want, no problems. We figure enough people will find the price good enough, and just want to get on with their work that we'll get paid for doing the product development, which is all we want out of it anyway. There is a limited supply of these NOS Russian beta-enhanced pancake detectors at any rate, and those are the main thing required to get a standard, so we can compare readings across labs. (any more turn up, someone ping me, we'll buy them, they are really nice for this -- perfect for the job at hand as they are much more beta-sensitive than anything else).

At any rate, here's the source code in a zip file. SC is the perl program. There's a small config file for the USB/serial port that wants to be in the same directory as SC is, but the program will create one if it can't find it. The .glade file is the glade GUI project info. I've incorporated it into the main perl program already, but if for some reason you want to edit this, you need it as its own file, so here it is (the other way to get it would be to cut/paste all after the _END_ tag in the perl program and save that to open with glade). After any changes, you'd just paste it all back in under the _END_ tag in the perl program. Not that many people will do this successfully -- all the names have to match what the code expects for things to work, but in the spirit of "here's all the source code you need to roll your own" it's included too.

Linux hacks will know that if you don't have a bin directory under your home, you should make one. That way it's always on your path for the CLI. Of course, what I did here was create a custom application launcher and put a little icon on the menubar. I set it to "run in terminal" as that's where any error or warning messages would come out (STDERR), but it doesn't need that - it's a debugging thing for my convenience.

So, here it is! Enjoy!

And it's sweet. I've added some features, but the main new thing is a standalone plotter for the log files produced by SC. Here's the code. I included the latest/greatest SC, which is the code that talks to the hardware and makes logs and plots. But the new one, SCPlot, is used to plot and replot from the log files SC can create, so you can really dig down and analyze. It lets you choose which of the 4 possible data streams to plot (or any combination), what the plot style is - lines, points, or both, log or linear, and now you can set the stop and start times so you can blow up some section of the overall log for close inspection. As usual, gnuplot will let you put your cursor on a point, and report the X and Y values for that point, scaled in the same units as the plot. All you do is select what file to plot, your options from the GUI, and hit the doplot button. This runs very very fast, so it encourages you to mine around in your data.
Here's a more interesting plot, from the log file taken on one of Richard Hull's runs at HEAS 2011. Contrary to his humble statements he's reached 1.7 million neuts/second at least at times, on a ten second average, and got silver to a higher peak than he claimed as well - you can see the dip in the geiger counts when he turns off the fusor, then puts the silver on the counter. On this log plot an exponential decay is a straight line. I show silver peak at 690 cpm (one second) or 371 (ten second average peak value) on this run. Not too shabby at all!
And this next run hits 2 m neuts/second near the end for awhile. Richard, like most good scientists, doesn't over-claim on his results. But the numbers don't fib here.
Some of the variations on the X rays are people moving between the counter and the fusor, but the dropouts are where the fusor stopped for whatever reason. Happens here if I take out too much gas for it to stay "lit", or of course, when I turn it off.

Nice job Doug! I like the new options for replotting the results. This is probably the first time that there's been a direct comparison between two labs without a huge amount of uncertainty from everyone using different equipment. Even as hectic and disorganized as things were trying to get fixed timed runs, not to mention "Richard's run", the timing is all logged and the rest can be extrapolated and still produce good data. Awesome!