Don't just code till it works!

Software related topics

Don't just code till it works!

Postby Doug Coulter » Tue Jun 03, 2014 1:27 pm

Ok, a little philosophy here - with justification, as there's all too much crap code out there - I'm fixing some now, written supposedly by pros. Not, and it's quite a mess in there - not only organized poorly, but plenty of sins of omission that "seemed to work fine at the time". And fixing sins of ommision requires that you actually know your stuff.

Example, I'm working on some data acquisition code for a Diligent Chipkit Uno32 with WiFi shield, and using their example web server - the idea being that I'll be able to suck down log files from the data aq part from any browser on my network.
Yes, I put tellem in this code (same protocol, but obviously not written in perl for this - it's their strange mod of C++ over there).

It sometimes works when it shouldn't. Not always, but enough that had I not known what was going on, fixing it would have been a very intense bear of a thrash. Why does it work when it shouldn't?

Well, WiFi and other routers often keep track of MAC addresses of things hooked up to them - at least until the next power failure - YMMV depending on the router involved. This means that my routher, my laptop, and perhaps my browser (on which I'm writing this) "already knows" a numeric IP for say, my Digilent Uno321...browser saves things across sessions, even.

That is, until something forgets how to map, say, http://uno321 to it's real address - which at the moment is 192.168.1.73, but which could change on the next boot of my router(s). That's why I wrote a lightweight network mapper - tellem and tellme, so that hosts files could be updated by machines, and go around any "remembering" that turned out to be wrong later.

I could have completely left this out as it turns out, and it would have worked for quite some time...and trying to find out why it stopped would have been a nightmare when browsers couldn't find the web server...which thing failed, and why?
Had I not known how things work - I'd have left all this extra stuff out that makes it work every time.

It's obvious that many so-called coders/programmers don't live by this rule. It's been made more obvious in hangouts on G+ where I meet wannabe programmers/script kiddies/hackers (who want to be uber hats, black or white, but it seems most of those who even want to be white hats are thinking "Walter White"). Many of these guys won't buy books and read them, and think we who do this right are merely keeping secret from them some small, simple, magic incantation that if they had it, they'd rule. It's simply not that simple - you really do have to know how all the layers work, and at all the levels appropriate for your particular project.

You can learn, say, C, from K&R's rather thin book - maybe in one day. Doesn't get you squat - if you don't know the system libraries, at times down to the source code level and how/why each thing is done, you can't do diddly. You want to program windows and just guess what MFC or .NET are doing? That's how you write code that crashes with errors that are a bear to find. FWIW, you can do the same with perl from "Learning Perl" - the concept I'm trying to get across here makes the actual language more or less not-relevant. Add some nifty drag-drop "a monkey can do this" IDE and you're good to go out and make horrible quality code.

The same of course holds for linux or embedded programming - if you don't know the totality of the language, you don't know squat. Yeah, you might make a monkey program that sometimes works, but that's not pro-grade stuff, and standards, from what I experience with using others' code show that in spades.

Do you really understand the interactions within your own code, much less with a multitasking opsys? And if you use threads, how about that too? Most don't, and then wonder why things crash at seemingly random, with of course, bogus info given by a debugger - for example, if you hit a breakpoint - of course a serial uart has over (or under) flowed by the time you see the info in the debugger - you have to know things like that.

In this case, I'm even having a struggle with the java-language IDE, MPIDE (it's written in Java, but it's for arduino's C++-like language development), as it's obvious that someone wanted something that looked like a real IDE, but didn't know WTF they were doing. Random crashes due to misuse of threads...loss of editor data...can't click on an error in the compile warnings and go to the file? So, not even the basic features on top of it not being basically reliable.
And it matters...boy, you sure don't want to click "serial monitor" near the end of an upload to the hardware so as to catch the first things it will print - it'll crash, and in the process, lose some of your more-recently edited source code if you didn't save it manually first (all REAL IDEs already do that when you compile).

There are a jillion examples of this, but a list of the ones I run into all the time would be more than this board's disks can hold. Do us all a favor - don't be one of them.

OK, rant-off.

But this is why I believe that some pretty smart guys don't trust embedded-cpu devices to act right, if not PC's, because the people who program them are often guilty of this error, along with others. It used to be so hard to program embedded chips that this wasn't much of a problem, and often as not, the guy who programmed the thing was also either the hardware designer or in the same room.

Dumb example of that - PIC chips in general come up with all pins floating. With a pullup or pulldown, you can ensure that in the "I've just booted or just crashed" state (remember, they tend to have a watchdog timer - use it, and use it right), things can be setup so that even in a fail mode, nothing connected to it goes up in smoke. Simple...but apparently all too complex for the latest generation of hackers (which I don't use in the new perjorative sense - I do make a distinction between an honest kludger and a cracker, which these days is what is seemingly meant by the word hacker, despite my protests for years). Funny how language changes due to ignorance - and lawyers. For example, FUD used to mean Fear, Uncertainty, and Deception. But you can't accuse someone of deception without a possible libel suit - so now it's taken to mean the redundant "Fear, Uncertainty, and Doubt" which change I think the second term already covers.

How can you tell the truth if A: you don't know it, and B: the language changes to mean something different, anyway?

OK, rant really off. Please, all you coders out there take this to heart - it's not one example, it's a philosophy.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 2944
Joined: Wed Jul 14, 2010 8:05 pm
Location: Floyd county, VA, USA

Re: Don't just code till it works!

Postby solar_dave » Tue Jun 03, 2014 10:53 pm

Big differences between Software Engineer, Programmer and coders.
Dave Shiels

My TED 5000 power monitoring
http://phx-solar.no-ip.info:8081/Footprints.html
solar_dave
 
Posts: 108
Joined: Tue Aug 21, 2012 1:33 pm
Location: Phoenix, AZ

Re: Don't just code till it works!

Postby Doug Coulter » Sun Jun 15, 2014 3:38 pm

Dave, I believe you've brought up the correct nail to hammer on. Yes, there's a difference, and you even left out a few - customer, systems engineer...

When you code for "yourself" but are an open-source type, the likelyhood of someone else having to maintain your code is high. Heck, that's true even if it never leaves my campus here - I have to figure out how to patch things as other stuff changes (or a lab assistent might), and don't always remember what I was thinking when I did the project. Thus, what some might deem an excessive amount of comments in my code, along with (well, when I'm a good boy), some paragraphs that say what things do, why I did them that way instead of another, and so on.

And as hard as I try to avoid it sometimes, change is inevitable. Yes, as "barney stringdup" (Bjarne Stroustrup of C++ fame) says, software doesn't rust or need fixed when it breaks. Well, in the real world, when standards change, hardware changes, opsys' change, that's just too simplistic a view to work out here.

Even if you're your own customer - you shouldn't take advantage of the customer and "release" code/projects that are likely to be flakey - it's your hair turning gray (if it isn't already, and you still have any, and as distinguished from grey codes, which were kinda cool). When you really think about this, doing a full project from having the vision ( a reasonably complete and detailed one is a lot harder than just "having a cool idea"), through hardware design or choice, software design (think of what might want to be extended along the way but don't get too crazy making everything too super-general), and then coding...well, that's actually harder than most average cube-slave jobs - or even project manager jobs (in their case, someone else generally has the vision and there's another customer who may or may not be reasonable and may or may not change the spec partway through in a way that makes things harder).

I found some inspiration early in life from "Farmer in the Sky" by Robert Heinlein. The protagonist's Dad is a real engineer, and reminds us that a real engineer thinks of it all, keeps a sense of perspective, has no "magic" in any project - knows how it all works inside even if he didn't do the inside, and keeps a sense of scale and perspective at all times. I took that to heart. And with essentially no paper qualifications found myself considered "highly" or even "over" qualified at every job I went for in the biz, till I started my own outfit - no one can pre-qualify for that one IMO - it adds leader, marketing guy, manager, financier, interface to regulationis, and a few other things it seems impossible for any one person to be competant all by themselves. So add quick learning to the above...

It's even bad enough around here. I'd clung to 32 bit (linux most often) opsys for as long as possible, as many apps didn't make the cut to 64 or don't work quite the same (or as correctly) in the new land of more wasted bits (well, we've thought of some cool things to do with some of those - reference counting pointers in the high bits and such can actually speed things up rather than slow them down, in some languages - you get something for fetching twice as many bits, sometimes - but not always, often it just slows you down).

OK, well, I'm now doing this series of uP projects for the homestead, to monitor and control a bunch of things on my 4 building campus. I'm using mainly Arduino Uno's and ChipKit (Digilent) WF32's, as those fit my needs and are "cheap enough" that designing and building the basic boards aren't worth it. The big boys - the WF32s (32 bit core, built in wifi, lots of room, extra USB OTG, tons of I/O, you name it) have both a 32 bit IDE which works, kind of well enough, but their 64 bit version (all this in Java - write once, fix everywhere) is horrid and simply not usable at least with the linux JRE. Consistency throughout the rest of my life is desired insofar as I can make it so.

You know what I wound up with? Heck, I only have a couple 32 bit machines left here (out of 20's machines) and they aren't the ones I like working on, they're in uncomfortable places, limited in other ways (display, speed, storage and so on). I really like this machine I'm posting from - a brandy new maxed out i5 Intel NUC, which runs like the wind - and any solar guy would understand - it draws 9 whole watts + monitor while doing realtime vid chats...less at idle, up to 40 total when gaming (which I never do). I don't even have to remember to turn it off. But it's got a 64 bit opsys on it...so guess what -
I built a 32 bit LTS linux just for uP dev for this project, in Virtual box, the oldest one I have on the oldest machine I had (slow laptop, core2 1.4 ghz), and put in just the stuff to do this job, knowing I can put that virtual machine anywhere I want, and it's not even that huge a file (1.6 gb ain't bad considering it's a full linux, all the tools and so on). And while it's slow as heck on the laptop - it runs like the wind here on th3e i5 and all my even-faster (in theory) machines - which don't turn out to be actually noticeably faster even though they're Haswell too, and have things like 16g of 2.4 ghz memory (not a typo, and that ram needs its own fan) along with SSD's. The NUC seems overpriced till you use it, but you get what you paid for, even maxing it out as I did. While I'm no fan of Oracle (it proably works because they just bought it, they didn't write it), I do like Virtual box around here - backup of an entire machine is just a file (and it's compressed), and I've found over the years that later versions eat machine files from earlier versions without so much as a burp - things just work perfectly.

FWIW, I'll be posting a file of that virtual machine real soon, or I should say, a link to it here on our extra space. It's uploading now and telling me it should all be there in 7-8 hours...crummy upload speed on ADSL, as if no one created anything and so no one needs speed in that direction.

If nothing else, and if others download and use this one - we can share a stable base for development on this kind of thing, and share our upgrades. I used 32 bit Mint 17-mate LTS (based on ubuntu 14.04 LTS but far nicer for us old guys who like discoverability and menus and detest unity). It just works...and for a bonus, I put a few of my more hair-pulling projects on it that have some general interest. I'll post again with a link when it's uploaded.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 2944
Joined: Wed Jul 14, 2010 8:05 pm
Location: Floyd county, VA, USA

Re: Don't just code till it works!

Postby solar_dave » Sun Jun 15, 2014 6:14 pm

Ah glad I read the whole post, I too have used the Virtual Box goodies when I needed a variety of operating systems to test against. You can isolate the network and do so many other cool things in each environment. My last use was for a DNS system that had global views and local views and the replication was very complex to work out. Views were not really designed to do what I got it to do! ;) It was a mix of Ubuntu, Centos and Winders.
Dave Shiels

My TED 5000 power monitoring
http://phx-solar.no-ip.info:8081/Footprints.html
solar_dave
 
Posts: 108
Joined: Tue Aug 21, 2012 1:33 pm
Location: Phoenix, AZ

Re: Don't just code till it works!

Postby Doug Coulter » Sun Jun 15, 2014 8:19 pm

Oops, did I say 1.6 gb? It's showing 2.7gb and still (4 hours after starting) telling me 7% done and 7 more hours to go (which is what it said 4 hours ago, must be those computer minutes). I'll post the link tomorrow AM (I hope!) I suppose. Sure is neat to have a fast machine that draws less than a nightlight I can leave on over night for things like this! Even the monitor draws under a watt once blanked. My adsl speed is all over the map - meaning always less than the 1 megabit promised, never more, but sometimes zero for seconds at a time. Someone must actually be using this ISP (which is oversubscribed like all the others - why they want to triple-dip netflix etc, but that's another rant). Oh, it's dropped down to 3% now, I think gftp has an issue with counting things over a gb or something, though I've put full DVD's up there with it. I'll have to check later and see. The original estimate was about 4 hours from now.

Good to know someone else uses VB. This should slide in easy and work great - it does on a couple boxen here, no problems. You'll of course want to customze this and that (and Mint-mate makes that easy as it used to be in gnome2, except it works right) - but I didn't want to put on anything to make it bloated. It's really just a dev enviornment with the fiddly bits already done for ya - the results run on your other stuff just fine as well as there or better. Not even a good calculator like qalculate. Boifh fhe arduino and mpide are set up, libraries downloaded and installed for all the cool stuff from X vendors (adafruit, sparkfun, marlin jones, digilent etc)...gnuplot and my full arduino data aq (which is tested on this config and gives realtime plots of all 6 a/d channels and 2 counters at 10 hz). You just add a programmed Uno (sketch provided) and type "adaq" at the prompt. More to come, but we gotta start someplace. Also, there is a web server that will run on either an Uno32+wifi shield or a WF-32, has CGI's (in C++) and some content to test with and templates to add more to. The regular UNO32 kinda ran outa room for anymore, and that combo costs more than the WF32 which has 4x the ram and rom, so....You can kinda see where I'm heading with this - a mesh of remote wifi connected computers on my campus where I can monitor, log, and control real physical things via just going to their webpages (they also log on SD cards internally, just in case).

Funny how with each new release from any distro, the calculator, which I actually use a lot - gets worse. They all used to have a 1/X button, X^2 and so on, so you could calculate the (usually more complex) denominator first, then 1/X it and go on with the (usually simpler) numerator without having to use memory or parens. WTF was anyone thinking? Hey, the calculator works great, let's give it to an intern to mess it up? Most of the new ones don't even do high precision past double float - you try to calculate either atomic or astronomic stuff and get NAN or zero when you know better. That never used to happen...progress?

People have asked why I don't just use the Raspberry Pi. While I admire how clean they made Raspian (and wish there was a version for intel) it's just gasping for air all the time - it's just not enough to run even a stripped down linux (a minute to open a web page, are you kidding me?) - no way it's going to fly on stuff that matters like hard real time deadlines. And it gets hot. While these Chipkit parts are less powerful in theory, not being burdened with a "real" opsys means you control it all and can make them fly - and 80 actual mips in a 32 bit machine with efficient code and no overhead is plenty quick. And oh yeah, no swapping to ruin the solid state storage and make it die young.
Posting as just me, not as the forum owner. Everything I say is "in my opinion" and YMMV -- which should go for everyone without saying.
User avatar
Doug Coulter
 
Posts: 2944
Joined: Wed Jul 14, 2010 8:05 pm
Location: Floyd county, VA, USA


Return to Software

Who is online

Users browsing this forum: No registered users and 1 guest