Configuring postfix is well-known to be the hardest, least intellectually stimulating task any IT person will ever pass off to a hapless subordinate. I, unfortunately, would seem to be that hapless subordinate. Eric recommended (and, in fact, convinced me) that I write a perl script to act as a delivery agent for postfix. This would offer grate flexibility, customizability, blah, blah, blah. Not to drag out the story, I got to a point where I realized I had a 200 line mess that didn’t work half of the time, wasn’t properly interacting with postfix, and needed another 1000 or so lines to do what was really required of it. So I dropped the project, and fell back to using mysql aliases, which looked to me like the easiest option when I need relatively dynamic mailing lists integrating with custom software that uses mysql.
Read the rest of this entry »Posts Tagged Programming
FIRSTSearch Released
Oct 16
After various delays of the server-is-down sort, I’ve finally finished fixing the indexing software, and FIRSTSearch is ready for (beta) release. It’s by no means an incredible piece of software – it has some bugs, it wants some features, the display of results is less than perfect, and I have my doubts that it’s really finding the most relevant results. But it’s working, and it’s ready forr other people to start to use.
I expect to have a non-beta release within a few weeks.
In the third after-school robotics meeting of the year, I ventured into that frightening and mysterious land of operating system development. It’s the hardest thing you’ll ever fail to do – and I started it while listening suggestions for the name of this OS. I think Freddy ultimately decided on “Sloth”. Great day. Interestingly enough, the only hard-core programmer who I told about this project who did not inform me that it would be an utter failure was also the only person I know who has ever worked on writing his own OS (although it wasn’t very successfull). So I’m hopeful – probably more so than I have a right to be.
Briefly, I can now start up the computer and print an ‘A’ to the screen. That means that I can claim to be 3 months ahead of schedule, since I wasn’t planning on starting this project until early January (I got bored). Of course, when I tried to print an ‘A’ to the second line, I got a blue rectangle instead. But hey, that’s pretty cool too!
Now for the long story. Walking in today, I had made progress: I had acquired several computers to be used for this project and some other (all the meantime fighting back the greed of my bosses, who actually wanted to use them for things that were – god forbid – practical *shudder*), I had acquired a floppy disk on which to write the OS, and… well yeah, that’s about it. I also knew nothing about how to write an OS.
First, I set up grub. It turned out to be pretty simple. Next, I started figuring out how to write the OS to the disk (no, I didn’t have an OS yet – did I mention that I’ve never done this before?). At this point, Freddy pointed out that I could just store the file on the hard drive, and have grub read it. Oh. Right. Thanks. And I spent 45 minutes acquiring that floppy disk why?
Next I went to my favorite wiki, and looked up the boilerplate OS code. Pretty simple, so I copied it onto qweffor and ran it. Well, actually, I told qweffor to reboot, but I was logged on via SSH, so I had to walk over to the room with qweffor in order to watch what was happening. Freddy had already started up the right OS. So, yeah, it had printed out an ‘A’. Grub had also printed out some stuff. I specified that grub should be ‘quiet’, and retried, but that didn’t fix it. So I got a message like:
Aooting from (hd0,0)...
I decided to let that message appear, and print out the rest of the stuff to the following lines. So instead of just setting the first character to be ‘A’, I set it to be a newline, and set the second character to be ‘A’. Result: a blue square with a white circle in the middle. Then I had to catch my bus.
Result: its time to actually read the documentation. And for those of you who were planning to look on, you didn’t miss much.
Oh, and today was the first day of training for freshmen. Not that that’s unimportant, but it was somewhat less than exciting…
I recently completed a manual migration of my data for CTDA (an analysis system for team data collected from thebluealliance.net and other sources) to PostgreSQL from MySQL (see my comparison for why). By ‘manual’, I do not mean that I typed the data in by hand – I mean that for each table, I wrote a separate script to transfer the data, instead of writing (or finding online) a single script to copy a whole database. My advice to you, if you are planning to migrate data to PostgreSQL from MySQL, is this: don’t do it by hand.
Let’s assume I would have been unable to find a workable script online to transfer the database for me. This is unlikely at best, but if I couldn’t, I could probably have written one that would cover all the cases needed for my database in about 3 or 4 hours (I’m rather new to postgres). So I sit down one day, write the script, test it a bit, and set it to run overnight, and I’m done.
But that’s not what I did. I decided that it would be ‘easier’ to write a script for each table. Perhaps it saved me some thinking – but the process ended up taking me two weeks, and much more than 4 hours. I would estimate that, because of this delay, CTDA will be released 3 weeks later than it would have been if I had written a good script to do this for me.
Oh, and in case you can’t guess. My advice to those who are planning to migrate from PostgreSQL to MySQL: don’t.
After looking into migrating to PostgreSQL, which seems to be a popular pastime among database people (migrating, not looking into), I decided to do my own benchmarks. Here are the results of the simple ones (I have yet to code the complex ones). I wrote all the code in perl, and ran it on quentin. I use InnoDB for mysql, with defaults, and everything default on postgres.
Read the rest of this entry »“If this explodes, we’re going to have video of your epic demise.”
So commented one team member on my attempt to run a CIM motor off an 18v drill battery using 22 guage wire.
Such was the atmosphere at our third summer meeting, hosted at my house, in preparation for the Battle o’ Baltimore. Our main goal was to replace the robot’s sub-par shooter with a simple hopper and roller with which to “dump” orbit balls.
The dumper design was considered during competition season and mostly built (as our witholding allowance) but was scrapped in favor of a catapult. We thought it would be a good idea to give it try at the Battle. After negotiating our way into the backroom at school, we collected the dumper scraps so the dumper could have a second chance at life. Disassembling the shooter was done quickly, and resurrecting the dumper was not difficult. 80% of the work was completed the first day.
The four people who decided they didn’t want to go home for the night slept with the robot in the basement. We were back at work 10 AM Sunday morning. A few bent pieces of flat stock and more than a few zipties later, the dumper was in working condition. For the rest of the day, the hum of tools and robot work was accompanied by the clatter of pool balls and sharp pings of air hockey pucks. A few mechanical bugs were worked out, which entailed a new gearbox and mounting plate. Some minor adjustments ensured that our rear wheel chain would stop ejecting itself.
Most students floated between game-playing and robot-building, but some chose to work on other projects. The fate of the programming subteam was discussed at length, and design work was done on the rumored “arm” that will be the pinnacle of our training program. Some people even found time to cut the pieces for a second modular battery charger.
While the bulk of the work is over, we will be meeting here again this Saturday. This time, the focus will be on the robotic arm and offseason projects, as well as some administrative work for the upcoming school year.
And just so you stop wondering, my ad-hoc wiring job didn’t explode.
Ferocious Zip Ties
Aug 4
WARNING: Never hold a zip tie by its tail (the long, flat end) and use it like a whip to hit someone with the head (the square end that you can feed the tail through).
This happened to me on Sunday where, upon arriving, I was attacked by a bored freshman sophomore. It kind of stung, but I didn’t think anything of it… until I started trying to work on planning pre-season lessons. That’s when I noticed my hand was bleeding in several places (5, to be precise). I had to go upstairs to have Eric’s mom put some antiseptic on my hand (I don’t trust sophomore boys to keep their hands clean. Heck, I don’t trust me to keep my hands clean).
Needless to say, I was kind of pissed.
On the plus side, I managed to work with Daniel to get most of the training schedule for next year set. We have 12 days this year, up from 9 last year (just how school scheduling works out) (hooray!).
Training this year is going to be a LOT more hands on than last year. Half the sessions are reserved for hands on practice, including wiring and programming a robotic arm and building a set of drive trains. Hopefully everyone will come out of training ready to head straight into Build Season (or at the very least, know the difference between a bolt and a nut).
Risky Plans
Jul 15
It’s been a strange week – it seems that I decided, once I was almost done with the search engine, that it was time to start some more crazy projects. The disease is catching – Eric has his own project, not yet approved, that even I consider to be extreme. We won’t go into it here.
My three projects are, in order of dubiousness of completion: an in-house listserv to be hosted on ogodei, an IRC server, and an analysis system for FIRST team data (not yet online). The first two will both be linked to the Eric’s “Team Management System” – which is what makes them especially challenging. The third is designed to find patterns (defined by me as a lack of randomness) in team data, including sponsors, team size, geographic location, and performance at competition. Obviously there are some trivial patterns with which I can test the system: “hey! most teams in the D.C. region go to the D.C. regional!” My hope is to find something slightly more useful than that: “hey! everytime ARL decides to sponsor a team, Microsoft joins them the next year!”
Read the rest of this entry »Note: this article is a continuation of previous articles on search engines (part 1, part 2).
After testing out an alpha version of my search engine for a while, I realized that its greatest flaw (other than printing out the results in a downright ugly format) was that it couldn’t recognize “programs” as a variant of the word “program”. I briefly considered programming it to automatically check for a limited set of common variants, but I decided that this was probably too much effort for what would be a decidedly low-quality result. I needed to find a list of, for every word, all of its variants.
What I was looking for (I discovered after about 45 minutes of IRC chat, google, and man pages) was the ispell english dictionary. ISpell dictionaries have a list of ‘roots’, and then, for every root, they have a list of flags that describe how that root can be transformed to form valid words. I could enter this information, via perl script, into a mysql table, and then retrieve it quickly both while indexing and while searching.
Read the rest of this entry »Note: this article is a continuation of a previous article on search engines, and has been continued with part 3.
After a bit over a week coding (and learning various Perl libraries), I have completed stages 1 and 2 of the search, although stage 2, the indexer, could do with a little improvement. Both are written in perl, and as usual, the complete code listings are below. I decided to write the entire spider and indexer in perl and optimize as necessary later on, so that I could get done with the thing and not get bogged down in C code. If the perl turns out not to be fast enough as the site grows, then I plan to port to C. Likewise, the actual search part (stage 3) will be written in PHP to save time. If the PHP is not fast enough, I’ll rewrite it in C – but I expect there to be no problems.
Read the rest of this entry »