Barometer and temperature on Raspberry Pi -perl

For stand-alone microprocessors

Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Thu Feb 12, 2015 4:32 pm

I thought I'd drop a few tips for those following along on the "LAN of things" project, about how I'm doing the development and some of the tricks. As many are no doubt aware, I kind of like that old perl language, so I try to do as much as I can in it.
When programmer speed is required, it's very hard to beat. When CPU speed is required, well, there's www://search.cpan.org - any compute-heavy perl module addon is likely to have been written in C for speed...so you get the best of both worlds, mostly things are already-written and at most you adapt or use perl to duct-tape them together, and it's still fast despite perl being an interpreted language (with interfaces to darn near everything). In this case, the Raspberry Pi hardware, among others.

After looking at quite a few alternatives, I've chosen the HiPi modules. There is also for example, Device::SMBus, but that one is "heavy" and takes literally several hours to install, as it pulls in tons of dependencies (test harnesses, moose...about 26 other modules, gheesh, what was anyone thinking for a Pi?). The HiPi stuff is lightweight and uses the linux kernel driver for the Broadcom SOC peripherals for speed, and compatibility.

Early on when doing a new Pi, I always do "$sudo apt-get install synaptic". At least that has a gui and you can search for things. About the next thing I do is install cpanminus (using synaptic), which makes getting and installing perl modules a breeze - and for cpanm (the executable, works kinda like apt-get), they use the same names as you see in the perl program, for example: $sudo cpanm Time::HiRes will install that module. The fact that if it's also in the distro's repository under a different name is now irrelevant, and it's easier to get perl programs running when they come from someone else (like me). You just try to run them, if they fail, there will be "can't find "some module, aborted" and you then just install it with cpanm until things work (there might be several needed for any given program, don't worry, they aren't large in general). Any linux will already have perl and the core modules, Raspian being no exception.

For installing hardware from AdaFruit, I generally use their directions and methods, and nearly all the time, that works fine. I'm for example using a 10" HDMI LED display on mine (this is going to be inserted in a wall), and there's a few tweaks for that which work as they say.

So, I'm busy designing and building my LAN of things, and I wanted to offload some things off the arduino UNOs I'm using to conserve on the number of UNOs I need, and do some non-real-time kinds of things on the pi itself (for the moment, I'm using a pi B+, but have a B2 on the way). I want barometric pressure, and got a BMP-180 breakout board for that and had it working fine on arduino, but wanted again to just have it on the pi itself - there's room in the box, and the BMP temperature sensor is useless to me elsewhere, as when I really want to know that, I want relative humidity along with it (DHT22 sensor). So why not put it right in the box, and let it tell me the box temperature? Well, that's what I did, anyway.

All this data from everything is going into a mysql database, which will then be read to generate a web page of system info and status using mod_perl and apache (which also supports that very nice camera, which I just found works fine with the 2 meter cable...nice security cam...with NOIR version it can see in the dark too...). Perl is once again ideal for talking to mysql, there's a simple DBI module for that and it's quick and compact.

This example program doesn't do that - I'll have the full suite later on, and my details will no doubt be different from yours anyway, so I'm hoping that these little piece-part examples will help you out doing your own thing. This version simply reads the BMP180 and prints it out. At present, you have to be root to run it, but you can chmod it to have root privileges later easily enough. I plan to have it start automatically at boot and just feed mysql every so often in real use. Most of the time, it will be asleep (no point in high time resolution for this data) and using no CPU, which is good, considering that's the weak point of a pi B+.

I found an example program in perl, but it used the aforementioned Device::SMBus, which took hours to put in, needed root anyway, and is a waste of space on the pi for what I'm doing, so it's going out of there. I modified it to use the much lighter weight HiPi::Device::I2C module instead, and will likely modify it somewhat further as it's bulkier and more general purpose, as well as "politically correct" than I actually require here - I'll do the "tighten it up" at some point before putting it in service, but it works fine as is. Example, every subroutine in this uses my declarations for variables it already has, wasting CPU and ram when it doesn't change them anyway, so it's safe to use the ones as passed in - or globals for even better speed, since most of us will only ever have one BMP180 - the barometric pressure had better be more or less the same anywhere on your campus as it is on mine...we are not using this as a differential wind detector, there are better things for that, anyway.

So, here's the code, and also what it looks like running...
readBMP.bz2
perl for I2C access to BMP180
(2.36 KiB) Downloaded 249 times

And it looks like this running:
readBMPScreenshot.png
screenshot from Pi, made with "Scrot" program


One less bit of data to move over wires, ah.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Thu Feb 12, 2015 5:54 pm

FWIW, and yes, I do use this board to help me remember things and back stuff up - if you want to know the actual CPU internal temperature of a Pi, it's in a file. /sys/class/thermal/thermal_zone0/temp is the path. It contains a single number that is degrees C times 1000 (it's in ASCII, human-readable). I may add this to the database table for the pi box temperature and barometer....why not? It's warm in this room right now, around 80F. The software above is showing a temperature inside the box of 91f, and the pi cpu temperature is 119.6F if I did the calculation right. Could be worse. The Pi isn't the only thing in the box that makes some heat (and yes, I designed the box for convecting away any excess heat). There is a moderately efficient switcher to generate the +5.1v for the Pi. There is also the driver for the HDMI display, which draws power comparable to the pi itself. I'm feeding in roughly 12v for that, and switching down from that with a National "simple switcher" I found on amazon - 10 for ~$15, shipped from China. Since the price of the IC alone is around 5 bucks at digi-key, yeah, I got ten. They are neither super good or super bad, but cheap as can be, and seemingly reliable and stable. Nice 10 turn pot for adjusting the output voltage. I have this one heatsinked to the box itself via a 1/4" thick by .4" by .8" aluminum block, which is glued to the box, with heatsink grease between the board (on 1/4" spacers) and the block. Stays nicely cool under load.

Another tip - to make this runnable as "not root" you have to add your user to the i2c group in linux (and in the case of Pi, reboot for it to take effect). I couldn't get this to run not-sudo, even though I changed the file owner to root and made it accessible to everyone, and executable by everyone, until I did this: $usermod -a -G i2c pi
Note: I'm using $ to mean "whatever your prompt is in the terminal" - you don't type that.

Now the regular pi user can run this. I'm not at this point sure if you also have to do the other things I did with chown and chmod. You do have to do this.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Fri Feb 13, 2015 8:52 am

Optimizations!

Worked late last night, but a super cold morning got me up early anyway, to rekindle the fire...
What I was doing was optimizing the code above, for a smaller memory footprint and general efficiency. The original authors had made some serious beginner mistakes, from originally pulling in "perl for mainframes" modules for a mere Raspberry Pi (I already fixed that one in the previous version), to over-generalizing the code. For example, there can be only one BMP180 on an I2C bus (fixed address), yet they passed an object to every subroutine rather than just accepting the global nature of it.
Which was doubly-silly as they used the same name, expecting you to understand that this wasn't necessarily the global one in each routine in some la-la land. Further, they used an excess of "my" variable declarations in subroutines, and evidently didn't understand you don't have to create a return variable, or even a my variable if you're only going to read it. All this spurious parameter passing and allocation adds up, and it simply didn't make any sense, or make anything actually clearer in the code. They also imported tons of unused functions from Time::HiRes, which is often a bad idea on several counts. It's not like you can't add them back later if you should need them.

While I was in there, I added the code to get the pi's actual CPU temperature, from it's internal sensor, and made some conversions to English units (because it's not metric land where I live). The result has a much smaller memory footprint, and more importantly, starts up a lot faster as there's less for the perl runtime to look over at start. That's important as this will be put in a module (and I might make more global variables once I have a namespace for them to make that safer), and along with other code for database stuffing - it will become part of the boot process to load and daemonize the resulting code. Speeding it up by a factor of around 3 as I did (for startup time, runtime was probably better yet) - will matter down the road.

After it's time to build a railroad, it's time to build a beautiful, efficient railroad, eh? I tend to do this even for much more capable hardware, I love producing the reaction "I didn't know my computer could DO that!".
So, a short course on optimizing perl (really, coding in general), some slick tricks etc, can be had by comparing the version above (which works fine) with this one. I named it a little different so you can have them both open easily in a syntax highlighting editor.
readBMPm.bz2
More-optimal code version
(2.15 KiB) Downloaded 232 times


I will be combining this with some other data aq code for data from arduinos, and making that all a linux daemon run from the boot stuff. It will just run and stuff data into a homestead database in the background. It's important that this be efficient, and it's nice that it starts quicker now. (3-4 seconds became 1-2). And not only is it faster, it's smaller, and it does more at the same time.
readBMPm.png
Screenshot - no longer need sudo as well.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Barometer and temperature on Raspberry Pi -perl

Postby solar_dave » Fri Feb 13, 2015 2:03 pm

Doug when you daemonize the code will you just let it listen to a port and when "tickled" with a connection it will then return the data you desire?
Dave Shiels

My TED 5000 power monitoring
http://phx-solar.no-ip.info:8081/Footprints.html
solar_dave
 
Posts: 108
Joined: Tue Aug 21, 2012 12:33 pm
Location: Phoenix, AZ

Re: Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Fri Feb 13, 2015 2:57 pm

I plan to have this run at startup, and continuously fill tables in a MySQL database. I have a huge USB drive mounted over /var to hold that (I can also have it sync to one of the larger machines, whenever one of them is on). Anything (anywhere in the network) that then needs data can simply query that, for instance, I plan a mod_perl status webpage (served from this machine) among other things. This won't be the only thing putting stuff in the DB - it will also be grabbing data from arduino(s) to put in there, all nicely timestamped etc. The system is designed to grow, and to eventually incorporate everything on the homestead (including scientific data acq), control what can reasonably be automated (and write down what it did as well), and generally be of service and ease my load around the place. The point here is that this will actually be my first "always on" computer here off-grid, so it can do the sorts of things one needs that for. I have an always-on router that also shares some significant storage, but I can't run programs on it. I'm trying to squeeze this into a Pi B+ (and it looks like I'm going to make it) - but of course, the same tricks work on any linux, and I understand (and will know Monday) that the Pi2 is more or less a drop-in replacement, with the same power footprint, but 4 faster cores...should do. A regular Pi is a bit stretched once you are also streaming video...this one should make it, barely, and the quad should loaf doing it all, assuming it's not too difficult to port over.

Down the road a little, once I have some good data, I plan to use it to do things like train neural networks for better predictive control of things. That can be done on a bigger machine that's on when the sun is shining. Nets are easy to run, just hard to train - you need data, and you need lots of megaflops to train them. But not that many to run them. Think of what I have in mind as a decent weather predictor, a decent predictor of when I should start or replenish a fire - or charge the Volt - a fancy PLC, if you will. I will be adding all manner of sensors to this rig, as things go along. Full weather station, including building-internal. Insolation. Solar system ins and outs. Water in and out. Everything I can reasonably put a sensor on, or control electronically will be part of this.

Which means, I'm spending a lot of time figuring out how to make it super reliable, responsive to "correction" when it's doing the wrong thing - fail into a safe state, have manual backup (as now, I have the electrical valves in the water system, but this isn't hooked to them yet, so there are switches...). I'm thinking of one or more old Star Trek episodes, where some alien gets the ship's computer and suddenly the elevators won't work and the food replicators are spewing nasty smelling brown stuff, and the life support fails. OK, I've been warned. So I'm planning on manual backups too. Fliipping a switch is still less work than opening a door (winch is screwed shut with several 3" wood screws in winter, else things freeze), walking 1/8 mile to flip a valve, walking back to monitor the process, then walking back to flop the valve and so on - while it's raining, after all.

To make some of the piece parts, software and hardware more worthy of sharing, what I'm sharing at this point is the building blocks, like the code above - it's not yet connected to a DB in a specific table format or anything - that would be too daunting and too limiting for many of the viewers at first. After all, you have to have the data before storing it becomes important. As this goes along, I'll also be sharing the integration phase stuff. But that, except for example use, is going to be far too user-specific to help most people, other than showing what can be done. Not many people have the exact same problem-set as I.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Barometer and temperature on Raspberry Pi -perl

Postby solar_dave » Fri Feb 13, 2015 7:30 pm

SO you are just going to sample at some frequency and load a DB accordingly.
Dave Shiels

My TED 5000 power monitoring
http://phx-solar.no-ip.info:8081/Footprints.html
solar_dave
 
Posts: 108
Joined: Tue Aug 21, 2012 12:33 pm
Location: Phoenix, AZ

Re: Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Fri Feb 13, 2015 11:08 pm

Yup, but that's only part of the story (the sample rates will of course vary with different types of data). To make a (possibly crummy) analogy, I have a lot of ways to get energy into the batteries of my alt-energy system - right now, solar is the main one, but there are various backups and supplements too - at that point it's fungible, kWh are kWh. Going to Angels 35 - that's the DB. Or financially, being a semi-conservative fiscally, you gotta make it before spending it (Yes, I know the current regime likes to punish savers and those who think like me, but I'm still winning compared to debt-nation). There will also be quite a lot on the other end of the DB, it's just in the middle of things, a handy way to make a lot of info available and persistent. There will then be a lot of uses for it - just like with the energy in my batteries.

Being "that way" - I tend to concentrate on income before I decide how best to spend it, though I might have some goals already. I do have some ideas and plans, of course, but for now, they are meaningless without the data for them to operate on, so I'm doing that part first. Preparing the soil, if you will, to make the planting and harvesting easier and more fun later.

I've already had some interesting surprises on a trial run of a little of this...I put a prototype in for the last serious cold snap (we've been breaking a lot of records of late, and will break one tomorrow night again - and it's a very old record, especially for this time of year). I found that with merely temperature and humidity and barometric sensors I could tell every time I went into that building, every time I used the plumbing, and with a good guess what for...when I was there, when I opened a window or stoked a fire, and that it was also possible to mis-interpret said data in some interesting ways as well. This is one reason I'm going for a LAN of things vs an internet of things. I just don't want my own data to be easily available outside the campus for (mis)interpretation.

For example, to save me those slogs through the cold I had set this thing up to monitor temperatures in various places, and relative humidity came along for the ride (temperature of various parts of the plumbing was also far more revealing than you'd think). At one point during the single digit F temperatures, I saw temperature inside dropping - while relative humidity also dropped. Since those don't generally go together, I immediately assumed that a propane or kerosene heater had run out (they make water, after all, and yes, I was running everything I had), or that the wood fire had (or the pot of water on it had evaporated to dryness) and it needed attention right now - at "0 dark thirty". PITA. Dead wrong analysis - what was going on was all the heaters etc were just fine, thanks - I had done my homework just fine. The windows were freezing and condensing all the water out of the air in fact, once I drug myself over there to check on things, I'd done the other parts right. Duh, I got dressed up and took a hike in the middle of the night for that? That sort of thing is very much a learning experience for when it may matter more - what should I be instrumenting on the fusor, for which this is a trial run, that I haven't thought of yet, so as not to miss some important observation, as I did on that much-simpler situation? Obviously, a camera would have saved a trip - if it were pointed correctly. In hindsight. Doh!

In theory, practice is the same as theory. In practice...well, we've been around long enough to know how that works out. I've just avoided talking about some of the more "blue sky" possibilities till I have something that gets off the ground at all.
This data and control system will be driving data and control in more than one direction here...and some things are more important than mere plumbing or HVAC; I far prefer to be a physicist vs a plumber. Saving me from excess of the latter labor will free up more time for the former - and teach me how to be more effective at it as well. That's the hope, anyhow.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Barometer and temperature on Raspberry Pi -perl

Postby Philipp Windischhofer » Sat Feb 14, 2015 6:56 am

I'm very interested in the neural network part for prediction. What kind of training strategy are you going to use?
Please keep us posted!

Philipp

EDIT: you might find this interesting: a couple years ago I worked on a similar (commercial) project, which was a network for steel bridge monitoring. Pretty similar to what you're doing, but with different sensors (strain, wind, ...) and on a much larger scale (couple 100m bus-length). I ended up doing my own protocol (RS-485 for the lowest layer), also (for various reasons) including ACKs and checksums.
The real power of the network was the following: the entire infrastructure for data acquisition (each node on the bridge had a MSP430 controller) was already existing and working well on a lab scale when I took the project. Except that the nodes were connected to the central computer (storing data and making it available over UMTS) via USB and about 5m of cable (FT232s on the nodes) - completely useless for a real-life deployment. Each node showed up on the computer as a virtual COM port and various applications would then connect to those.
Now, rewriting all those high-level applications was completely out of question. What I did was design a small add-on board that would fit into the boxes housing the nodes. On it was a small PIC, the RS485 interface and lots of lightning protection circuitry (all this was going outdoors). The PIC would connect to the single RS485 bus replacing all those USB-connections, handle all the protocol stuff and connect to the MSP430 (the main node controller doing the sensor readout) via UART, just as the USB link did before.
On the PC side I wrote a daemon that controlled the bus (by means of a single USB-RS485 converter), handled the master side of the protocol and for each node it found on the network, created a virtual serial interface for the high-level applications to connect to, forwarding data to the corresponding node.
So the entire network became completely transparent to the user programs on the PC and the firmware on the nodes, one could even reprogram the nodes remotely over the bus (without a bootloader on the MSPs).

This may not be really necessary for your application, since you (I assume) can get to all your sensor nodes easily should you want to update the firmware. But on a large-scale bridge you don't want to crawl up there to do the update if it can be avoided...
Philipp Windischhofer
 
Posts: 21
Joined: Wed Aug 13, 2014 2:12 pm

Re: Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Sat Feb 14, 2015 11:40 am

Guys, when I get to it, I'm going to move these "vision statement level" parts of this over to where I think they fit better - under the original lan of things thread/overview. We're getting quite a bit of interest on this, likely due to the hype over the "internet of things" out there, from various social media and such (the board stats exploded when I shared this on G+, then it got on FB and so forth). For people who just want ot know how to hang stuff on a Pi that is usually on an arduino - that's this thread, and I have more stuff to put here that is more on topic here. For example, the thermocouple breakout on SPI on the pi for woodstove flue temperatures. We're getting lots of lurkers using some of this for a How-to - and I approve of that, so lets not bury them in stuff that isn't that right here, since there is more how-to coming on this thread. We are collectively "writing the book" - let's make it a good one re organization.
This isn't bad stuff we're talking about - and it will be helpful for beginners to see how we more-accomplished people think and discuss big picture issues - so lets just put this kind of thing all together so it's easy to find too.

Phillip, do you know of Timothy Masters? I've been using the tricks from his neural net books for years now. They work. His work is highly recommended. A math guy who can write (well) in English, and C both, as well? Rare!
Gradient descent followed by simulated annealing for MLFF nets. I doubt any of this will want his extension of NN tech to complex numbers, but you never know.
I thought of using our old RS 485 tech/protocol, which was named DALOP for "dynamically addressed length-oriented protocol". We did some cool stuff back in the day, big buildings for telephone, background music, paging, HVAC etc, and all over the same pair of wires - we used a diode and 100 ohm resistor in one leg of each driver, since the 485 drivers could drive skinny and long wire so hard, we couldn't always detect a collision and do the ethernet-like back-off a random length of time thing.
So we fixed it so zero always "won" which would then create a detectable error everywhere along the wire. That one was hot-pluggable and dynamically discovered and addressed. I didn't want the complexity or the power draw for this, since I don't need all those features. This one's pure master-slave, no need for dynamic unit addresses and so on. We'll talk more about that "over there" if anyone wants to.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA

Re: Barometer and temperature on Raspberry Pi -perl

Postby Doug Coulter » Sat Feb 14, 2015 11:50 am

Back on topic (mostly). Here's the Pi box with some peripherals in it. I haven't yet drilled and tapped the holes for the thermocouple breakout board, but will, and get that working right off the Pi as well, also using HiPi modules. The barometer/temp board is in the upper left, the Pi is below that. To the right is the switcher which creates 5.1v for the Pi (most of the issues with Pi have to do with crummy power, I skipped the inadequate micro-USB connector here for that reason) - the box itself run on ~12v, the HDMI DVI display driver is on the right and will take the usual "car battery" range of voltages, so that's what we input from a temperature compensated deep cycle 60 Ah UPS system I built already, which will run this for a few days in the event of a power down situation - I'm likely to want that data even more than when things are normal, after all. In the center is the serial line driver on perfboard to adapt the Pi TX output to 5v - with drive up to 2 amps(!). I could even power the slaves from it, but I won't. It actually does that now, via the protection diodes in the arduinos. That's not a good thing, they could get fried if there are other big loads on them.
PiNPeriphs.JPG
The Pi chassis, to be mounted in a wall in my remote location - I'm moving from this man-cave to there.


The white cable is for the Pi camera - this unit can stream HD video with lower latency and higher rez than even a normal desktop computer/tower and a webcam - it's about twice the rez, and half the latency of typcial PC streaming tools. And I can get a camera (have one) that also sees in IR and does motion detection...(this will be a link to that part of the project at some point).
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 3515
Joined: Wed Jul 14, 2010 7:05 pm
Location: Floyd county, VA, USA


Return to Embedded software

Who is online

Users browsing this forum: No registered users and 2 guests

cron