I mentioned in one of my latest entries that I’ve done a bunch of fixed point stuff in Java lately. I work for a company that does many financial calculations, and much of the legacy code is in other languages. There’s always been a problem with rounding and consistency in these calculations. They’ve all been addressed, but there’s always pain to get it through testing when a change to calculation code arises.
Martin Fowler has discussed in severalvenues the idea of having a class that represents Money. This ensures semantic consistency, rounding precision, and especially it provides an allocate method to distribute Money among various individuals or accounts without losing any fractions. Everything is accounted for correctly.
We not only deal with Money, but other values which we must treat as always having exactly n number of decimals. Martin Fowler’s examples have often included code, and this guy wrote a Money class in the same spirit. But I didn’t want to rely on my own fixed point caculations to do the job right (you know, losing precision, increasing scale based on the arithmetic function performed, and all that stuff that’s real easy to ignore — even though you know it’s gonna come back and bite). I poked around and found out that IBM has done a great job of it already. They submitted a JSR long ago, and you can see that their work has been incorporated into J2SE 1.5 (or 5.0 — silly Sun marketing droids).
If you need to do reliable fixed point or decimal floating point calculations in Java, and you aren’t moving to J2SE 5.0 immediately, check out the small decimal.jar in this ZIP file (there’s even decimal1.jar for the 0.004% of Java users stuck on JDK 1.1) from IBM. It just may save the day.
My wife has been doing some work for a friend, which has involved some mailing lists and merging. This friend isn’t very computer savvy, and has obtained these lists from other people lacking in computer knowledge. We have received files in various formats, all with poor field design for this type of work, but tonight was the one that topped them all.
We received a disk with a file named something like mailing_list.dat. All attempts to open the file failed. Opening the file in Notepad showed that the file began with this information, including non-printable chars not listed here.
€ IPM.Microsoft Mail.Note
When I got back from work I did some Google digging, and came up with a few potential sites, including one where Eudora claimed:
This TNEF information [winmail.dat] is unopenable and irrelevant outside of Outlook.
But I persisted, annoyed again at Microsoft’s snub of standards, this time causing my wife’s confused friend to be unable to open her list file.
Then I hit the jackpot. A post at Google Answers pointed out that while the file was an annoying side effect of the MS Outlook, it could be decoded using a shareware time-limited program available for free download.
I ran the program and out popped a Microsoft Excel spreadsheet, which I happily opened with OpenOffice.
Google really does have all the answers to the messes Microsoft creates.
I just finished converting our Money class to extend a FixedPointValue class based on IBM’s BigDecimal. Hopefully this will prove a solid basis for fixed-point calculations involving financial data. Green bar on the former Money unit tests, decent JavaDoc for those who will extend FixedPointValue in the future. Extensions and heavy unit testing to come!
My open source work over the next little while is going to involve providing better integration between Hackystat and Maven. I’m excited, since the two tools seem to naturally fit together. Hackystat collects and analyzes data over time. Maven provides an extensible build framework and really good HTML reports of certain build metrics. Bringing the best of the two worlds together will be quite interesting to watch.
Well, so far it’s been a bit of a slow process to bring over my old entries from blosxom to Pebble. Both use files to track blog entries, but blosxom uses plain text files (with implicit meaning attached to certain items) and Pebble uses XML.
Creating XML templates
I started by taking an existing entry and building a header and footer to wrap the contents of the blosxom files. Pebble uses `` sections to wrap the entry bodies — great for what I had to do. I copied each entry to it’s Pebble date folder. Then I wrote a script to run in each folder and add the XML template around the blosxom entry. This little shell script seemed to do a decent job.
With that done, the next steps I had to do included moving the title line from the blosxom entry to the <title/> tag in the Pebble entry file, update the date to reflect the entry date, and rename the file using Pebble standards.
The Big Date Mess
Perhaps the biggest challenge has been to get the entry dates correct. One of the biggest reasons I left blosxom was the way their dates were stored. They simply used the last modified time of the file. This didn’t provide me any flexibility to (easilty) make a punctuation change later and have the entry still hold the original date. I used some tools that preserved the date, but I didn’t like them. Pebble stores the date directly in the XML file.
I didn’t want to write anything complicated, I just wanted to get my 15 or so entries converted, and was (unusually) willing to do it somewhat by hand. So I needed a way to easily get the file times. I dug around and found this:
# ls --full-time
It shows a bit more than I needed, but gives me what I need to know.
Next I needed to know how Pebble chose the filename. I thought it was related to the date, and I found the code that was responsible for setting the BlogEntry id field.
// blank out the milliseconds
Calendar cal = dailyBlog.getRootBlog().getCalendar();
this.date = cal.getTime();
this.id = "" + this.date.getTime();
With this information, I was ready to start editing the hastily templated files I created earlier. In one sweep I would find the file’s creation date, sweep the title to its rightful element, and then added the date into the required element. Afterwards, I’d rename the file using the same date format I’d copied into the XML file.
mv FileName.xml <code>date --date "23 Dec 2003 10:32:48 MST" +%s</code>000.xml
Because Simon eliminated the milliseconds from the filename, it works well with the date utility which can print seconds from 1970 to the given date. I simply added three zeros to the end to make a compatible filename.
After that point, I simply load up Pebble, and make sure that the entries appear correctly, and that all the permalinks work — the behavior makes me think that the permalink is calculated from the date in the entry file, and not the location that the file was loaded from for processing. This means that if your date in the entry file doesn’t match the location and name then you’ll probably have a nice looking 404 File Not Found message.
This created an unusual question for me in one case. I had an entry that was created in the wrong time zone. When I figured out what time the entry was created in my timezone, I switched the time and the time zone recorded for that entry. The UTC time was the same, so the filename did not change. However, Pebble couldn’t locate that particular file. When I inspected the link, though, I noticed that the day had changed and that I needed to move the file from one day folder to the other for Pebble to find it. In Pebble, entries are stored in the “day” folder according to the day indicated by their <date/> element, and not the UTC day.
Hopefully this quick description gives rise to efforts by people much smarter than me on using either tool. I had glimpses of using Pebble’s objects from a standalone application to construct the BlogEntry objects (and eliminate the tedious hand mangling of dates) and even use the Pebble objects to write the XML files. Perhaps you could also write something in Java that would know how to read special tags in blosxom. I never used many, so I’ll do the one or two by hand. I also chose not to deal with categories. I can reconstruct those by hand as well, but a major conversion would benefit from automatically populating them into the new entry files. A tool like this would definitely be needed to convert a sizeable history of a blog to the Pebble data format. If you know of a tool that does, drop me a line or comment here. Thanks!