Remote viewing of fusor in realtime with raspi/cam

This is somewhat of an admission of failure. You can't easily pigeon-hole everything, and most real projects use commercial software, homebrew, and hardware all at once. So, for you makers out there (including me) - this is where to put whole projects that don't fit well in the other forums.

Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Thu Nov 06, 2014 7:43 pm

I need to be able to remotely view the fusor without getting the rads I was getting. Especially now that we've made a major breakthrough - I don't want to die for a 30 second look in there. So I've been exploring various ways to get far away and still see what's going on in there (along with the rest of the remote control junk required).
The inverse square law is your friend...when you're up into the milliwatts of output, as we are now. It's easier to move me to another building on campus than the fusor, so that's what's about to happen.

http://youtu.be/IkEud_IuBLY


For this, I used a pi B+ (they are really nicer...) built into a custom box with a real power supply and short wires (low 5v is the bane of pi's and I'm using a lot for things here), Adafruit's 320x240 LCD (it's cool but...needs a little tweaking), wifi, a wireless keyboard and mouse (another trick, they all use 2.4 ghz), the standard Pi camera with extra IR block filter I added that still isn't quite enough, silvmanchior's web streamer for the camera...and a few custom tweaks and mods to make these normally incompatible pieces of code work together. I can even SSH into the pi now, use it's basic login terminal, run X windows and LXDE if I like (with tweaks so all the windows don't go outside the tiny display), use the touchscreen as a rather crummy mouse, and so forth. This is pretty high resolution, almost 3k pixels/dimension, and has BY FAR the lowest latency over the ethernet of anything I've tried - for example, using VLC to stream a webcam over the LAN and then using mplayer (which for whatever reason plays back with less latency than VLC for this) does use less bandwidth than the Pi solution. But I don't care on gigE, I do care about the extra second or two delay, since I am using this to monitor something fairly dangerous to life and budget. I don't need a solution that shows me it blew up 2 seconds ago...I need NOW.

I'll be adding posts to this thread as I remember and dupe this (it's too cool to only have one, after all). For one thing, raspi-update, required for the web streaming, breaks the LCD display code...and so on - LX terminal is far too large for the little screen and I found the trick to make it right, all that kind of detail you'll need when you make one - and you want to - these are truly cool.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Thu Nov 06, 2014 8:00 pm

Here are some related posts on this board that show some of the problems with near IR - even with cameras that claim to stop it...nope.
viewtopic.php?f=45&t=882 from above, IR-bad, logitech
viewtopic.php?f=11&t=884 metrology post pi cam with and without welding filter
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Fri Nov 07, 2014 2:55 pm

Some quick notes that will save you some time I hope, if you dupe this.
First, the webcam streamer works fine, no sweat. But it calls for using raspi-update, which breaks the adafruit image for the touchscreen. Oops. Her script has to be run after the firmware update for it all to work again. Repeat: DO NOT use the "image", use the script. Using the image will wipe out the cam streamer you installed first if you're doing this the right way (eg, least work and it just works, I tried all the ways so you don't have to).
It will probably save you some time to get everything else going first - the camera, streaming, anything else you want to add with a gui - on a regular monitor, then do the touchscreen last. I switched partway through, and it made for a pain for some things I forgot to put in...

More dox on the basic camera streamer here: http://elinux.org/RPi-Cam-Web-Interface
I used this touchscreen. It is indeed cute, but I need reading glasses to use it much. http://www.adafruit.com/products/1601 At least the price was right, and now this is a nice standalone box.
I used this wireless dongle - it kicks butt - but does draw real power, so make sure you do as I did - I built a killer power supply right into the box and, gasp, soldered it right to the Pi board. It seems almost everything that isn't right about the Pi comes down to the idiotic choice of microusb for the power connector. Really? Here's the dongle: http://www.adafruit.com/products/1030 Gimme range or go home. This delivers.

Especially before setting up the touchscreen as default (as her script does) get rid of anything but a couple of icons on the LXDE desktop. I have LXTerminal, Shutdown (probably going to lose that one too) and Leafpad. There's just no space.
Even LXTerminal needs some help here or parts of it go off screen (not that you can't simply not use X in the first place, eg don't say startx - that terminal works fine as is).
That thing for LXterminal that shows on the desktop is actually a file, and if you edit it, you can make it behave. You want to add:
--geometry 37x12 after Exec=lxterminal near the bottom of the file. This file has a lotta lines because every language on earth is in there just in case...At any rate, this mod keeps the terminal from going off screen on the bottom with no way to raise it up - it has about 5 hidden lines down there, really a pain. you might go one column wider or one line longer, this is where I stopped because it "looked right".

Note the two negatives - this is pounding the crap out of your SD card, by taking images one by one, then converting them to a streaming mjpeg. The other one is that surfing to this on your LAN is gonna eat 2.2 MiB/sec. Not a big worry here with gigait E, but...in some cases more than perhaps your WiFi could do.

More as I have time and more to tell. This is really a pretty nifty thing. I'm right now using it with a wireless (yeah, 2.4 ghz but it works even with that powerful wireless dongle right next to it) Logitech keyboard and mouse, obtained from Amazon, that get this - they finally figured out that they needed to add an on/off mechanical switch to both mouse and keyboard so you aren't changing batteries because the mouse laser turns on every time a kitty walks by or you leave something sitting on the keyboard. This soft-off stuff is generally done by amateurs who don't get that the microwatts add up...it's really hard to get right (I did tactile hearing aids for infants and had to deal with things like that, it's not trivial).
This one: http://www.amazon.com/gp/product/B003VA ... UTF8&psc=1

Yes, I know there are ardent and perhaps justifiable "religious wars" about keyboards. I'm a very fast typist and somewhat picky myself. The really old, huge logitech remains my fave but I've worn it almost totally out - my thumb has worn a dip in the space bar, half the keys have deep cuts in them from my guitar-picking-hand-fingernails, and so on. The AULA boards I bought (backlit) for typing in the dark in the lab are just "that tiny bit different" in key shape and spacing they're a nightmare for a touch typist if you have to adapt all the time. This Dell I have on here at this instant (because this is a brand new NUC I built an hour ago) I hate with a passion for much the same reasons. This machine is getting the wireless board, 'nuff said.
The only thing nice about the dell is the hub in it (for raspi) so that's where it will go. Oh, there's one other nice thing about it - the price for these is zero at the local computer store...that's about the right price.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Sun Nov 09, 2014 12:11 pm

Since for whatever reason, this code beats the snot out of the SD card in the pi, I first thought I'd simply use a huge SD card so as blocks wore out - it would last a long time anyway.
I went crazy - I used a 64gb microSD card (this is a pi B+ that likes the micro size). On reflection, that was dumb. Most of the work was setting all this up in the first place, and copying 64 gb with dd like so:
doug@BigNuc ~/Public $ sudo dd if=/dev/sdd of=picam.img bs=4M
takes quite awhile....calender time to be sure, not my time so much if I've got other things to do, but it seems more reasonable to me to just use 8gb cards (cheap), make a couple copies and tape one into the box for instant recovery when one wears out (while also keeping a copy of the image on various big-computer backups just in case). So, what I'm doing now is making an image file (see above quote), which I plan to copy to a far faster USB3 stick, do gparted games with to shrink it down below 8gb, make another smaller image, and use that to burn a couple of smaller SD cards for this. By doing it this way, if it doesn't work, I still have my huge SD card that does work, and I can try again - for as long as it takes and the original image file on my BigNuc hangs on (it's on a seagate 2tb drive I use for beating on instead of the Crucial 480 gb mSata I use for read-mostly software and stuff).

So, now that the SD image has been copied bitwise (that's what dd does, it's super dumb and literal) - I'm now copying it back to a far faster USB3 64gb stick to play games on, hopefully shrinking the largely unused partition down to where I can burn a resulting shrunken image back to 8gb (or even smaller) SD cards. To do this part of the copy, the original big SD card was removed from the machine and the big USB stick substituted, and the if and of parameters to DD switched. In this case, it's the spinning rust drive that is the speed limit...still a lot faster.
doug@BigNuc ~/Public $ sudo dd if=picam.img of=/dev/sdd bs=4M
is what that looks like on my box. OK, that step is now done, and this time it happened at 104 mb/s....nice.

I'll now use gparted on the big USB stick to try and shrink that comparatively empty partition down to where it will fit on a smaller SD card - might even get it down below 4gb (though my bet is that like other things that seemed big at the time, 4gb cards will slowly disappear from the market. Just try getting 1-2 gb ones for old uP projects that don't support big filesystems...).

Gparted is now telling me it can't keep everything in sync without a reboot, so that I will do - I want this to work (it probably would anyway), and finish this post after I do.

Well, I tried this first, but it still resulted in a 64 gb file after resizing the partitions on the USB stick. It would probably work (we'll see) but I sure am glad I have huge storage on a new and fast machine...
doug@BigNuc ~/Public $ sudo dd if=/dev/sdd of=picams.img bs=4M

So I also tried this:
doug@BigNuc ~/Public $ sudo dd if=/dev/sdd of=picamss.img bs=4M count=1024

Which resulted in the file manager showing a 4.3 gb file (there's all this Gib vs GB junk, even in linux...GiB are the real power of two kind, the others use 1 billion = 1 gb, so this might actually still fit on a 4 gb card if it's really 4 GiB...which I doubt. The whole reason for this crazy unit stuff is marketing so they can sell you a smaller drive and claim it's larger.
Now, we'll see which one works with the Pi, the object of the excercise (and then delete all the big junk off my main machine just for being parsimonious).
No guts, no glory, so I'm trying this first. I should then be able to run raspi-config and take this 4.3 gb image and expand it back out to the full size of the 8gb card, assuming it all works...
doug@BigNuc ~/Public $ sudo dd if=picamss.img of=/dev/sdd bs=4M

It should run out of file before it runs out of SD card. Fine. It seems to be going a little slower than with the larger card, which is no surprise. The way they make these big is to use more flash chips, and the controller in the card uses the more chips in parallel to speed things up. Hopefully, it'll still be fast enough in use (if it works). After all, most use the 4gb flash that most buy along with the Pi...and this is twice that size.

Well, that took awhile...
doug@BigNuc ~/Public $ sudo dd if=picamss.img of=/dev/sdd bs=4M
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB) copied, 796.326 s, 5.4 MB/s

And things kept blinking for quite awhile after that. I ran sync just to be sure and waited for it to return, un and replugged in the SD card, and now it looks "right" as far as I can see.
Here's what the smaller SD card now looks like in gparted:
Screenshot--dev-sdd - GParted.png
Pre expansion of the last partition, which I may just do here instead of on the Pi, as raspi-config is one of those that doesn't fit on the LCD screen on the pi.

So, it probably worked. I'll test it, but then put it back in the big machine to expand the partition, I think. OK, it works unmodified. Now to expand that partition....and get at least the 8gb on the Pi.
So, I put it back in the NUC and did gparted to expand the main partition and now it looks like this:
GParted_afterResize.png
After expanding to the full SD card size as seen on linux on the NUC.

And now for a test of the thing, and if it works, another copy to tape (carefully avoiding putting adhesive on the contact pins) in the box for quick repairs later. I could have saved myself all this by just starting with the right size SD card, I suppose, but we'll see how things do in practice.
Ok, it works! Here's a test image. Now to make a spare and be medium-bulletproof in real life.
piTestScaled.jpg
Test pic, but this does huge videos too...neato


So, I'm going to declare this one complete at this point, barring the minor tweak here and there. I'm still working with IR block filters for use with the fusor, but that's it's own topic - in fact, the pic above was taken with one in place, since it was a bother to remove it. I now have a standalone box, with wireless, that serves video live, can record that or pictures (also timelapse) on its own card, then allow you to download them from the same webpage. It will also let you change almost everything about the camera - exposture, white balance, frame rate...it won't fit on a single page in my web browser without scrolling there's so many options. And it doesn't need a login or keyboard, or even that fancy LCD screen except to see if there's some error on boot. You don't even have to login for apache to serve all this stuff - but you can either at the pi, or remotely with ssh. Someone asked me just now why I'm not a gamer with so many hot machines. That's because this IS my game...I get more value I believe by creating cool toys than just killing imaginary foes.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Sun Nov 09, 2014 8:57 pm

http://youtu.be/tVcqoyxiMUA
Here we go with a very quick test of whether this will work at all and upload to youtube. Smells like victory.


The extra IR stop that makes it a little green, and the low light are on purpose - this is a test under close-to-real conditions. No audio, but I show up about in the middle. I started this from a computer upstairs, then went down to wave at the pi, then came back up to turn off the recording...I hope to do this for real with the fusor real soon now. 8.7 megabytes in 29 seconds at full resolution (1296x976, mjpeg). That's a lot of video if I have some few gigabytes of space on the pi to spare, which I do. It will also do time-lapse and get better quality pix by averaging a few, I may try that also for real life. Fun is part of my life mantra after all.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Mon Nov 10, 2014 4:12 pm

Just a teaser here, I'm working on the main show - I took videos with two cameras and under a variety of conditions to show myself the difference between the cameras, the mark 1 eyeball, and so on. Sorry about the hum in this video, it seems to be injected by linux "recordmyscreen" as a "feature" you can't get rid of easily unless you're better with editing mixed video and audio than I currently am.
http://youtu.be/iN9ZF9h0HX8


More to come as I learn how to edit mixed-format videos on linux in kdenlive so I can split-screen or at least insert one inside another for the two cameras I used.
This wasn't a full out fusor run by any means, this is just to test the equipment.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Mon Nov 10, 2014 7:21 pm

http://youtu.be/z1o42wGwHpc

One final one on this thread. A real-life test - the only sort that actually matters, showing both the handheld cam (high dollar) and the pi cam (better perspective) on a bog-normal low output fusor run.
The sparkles in the pi cam are because it's not behind any sort of X ray shielding, and the lens doesn't focus those. We can make a pinhole camera with phosphor later for that (and with a magnet so we can select out electrons, perhaps ions, and only see X rays or ions, our choice). That's a project for another day. This one is important because it can allow me to run remotely from the next building over on my campus and be out of the radiation field this creates, which is pretty serious close in (which is the point if what you want to do eventually is boil that water and turn turbines). But it isn't cool if I can't live though getting it to that point, so I need to create and test remote control tools, including this one that lets me see what's going on in there from far away with low latency.

Heck, even the X ray sparkles are data for this job.


Soory about the lousy vid editing, I'm a noob at this. Anyone here expert in kdenlive video editor? I was frankly astonished once I got just one step above "can't even" that it could mix mjpeg at one frame rate with mp4 HD from the other cam no problems. There are of course, some artifacts of the transcoding, surprisingly, mostly in what started out as the high quality stuff...but I can see what I need to see in this.

Now for the rest of the remoting. Most of it will be less technically complex, just boring hooking up pushbuttons that were local to the fusor to already-buried wires to the other building, and making a control panel over there...that kind of thing.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Donovan Ready » Mon Nov 10, 2014 9:39 pm

I don't use KDE, but I think maybe your audio hum could be from pulseaudio. I remove it from my systems as one of the first things to do out of the box.
Donovan Ready
 
Posts: 239
Joined: Thu Apr 17, 2014 1:22 pm
Location: Austin, Texas

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Doug Coulter » Tue Nov 11, 2014 3:02 pm

Hmm...I use the Mate desktop (fork of gnome that works far better), and there is pulseaudio, but zero other apps have hum. I think it's the app. As you know, you can run KDE apps over gnome if you get the libraries, which kdenlive for example does automatically when installed. Record my screen (the freeware) just uses whatever the default audio input is selected in the audio controls built-in.
Which I use for at least 10 other things. None of them have hum. And hum around here isn't that perfect sine wave either - I have inverter noise harmonics here that aren't in this (just touch a guitar amp input). It's a generated *pure* sine...very weird. It's like I'd have to find an app that could extract that audio from the stream, then notch that out, then put it back with the video. PITA, it's a good program otherwise, very useful for making tutorials etc. BTW, you get the hum whether there's any audio input selected or not...I'd say the bulk of evidence says it's a sick joke by the app writer. Maybe he had plans for a paid version that didn't have that issue or something (and then lost interest).
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Remote viewing of fusor in realtime with raspi/cam

Postby Donovan Ready » Tue Nov 11, 2014 10:07 pm

I stand corrected on the hum.

Somewhere I have a script that uses ffmpeg to grab the screen as an .mkv, then converts that to a high-quality .mp4 when the capture is finished. I'll see if I can find that, since a cursory search gave me nothing...
Donovan Ready
 
Posts: 239
Joined: Thu Apr 17, 2014 1:22 pm
Location: Austin, Texas

Next

Return to Combined projects

Who is online

Users browsing this forum: No registered users and 2 guests