Scattercode

Working in the financial industry, I have become rather strict about avoiding doubles in Java. The trouble is that they are a floating point representation of a number, which is just an approximation of the real value. This can lead to some unusual results.

For instance, according to this, 0.34 + 0.01 is not equal to 0.35.

double x = 0.35;
double y = 0.34 + 0.01;
System.out.println(x + " : " + y + " : " + (x == y));

0.35 : 0.35000000000000003 : false

Those inaccuracies might seem very small, but it’s surprisingly easy for them to start impacting a real world application. Imagine you wanted to sell dollars and buy Iranian Rial. You would be getting almost 20,000 Rial for every dollar. At that rate, imprecise floating point values could easily impact the final amount being sent. Although with the current US trade sanctions against Iran, that could be the least of your problems.

If you are running reports on a history of transactions, then a large number of smaller transactions can add up to large enough values that the imprecise doubles start affecting your totals. Even if you’re not dealing in huge numbers, things can go wrong easily enough. If your process takes a number through a sequence of multiplications or divisions, errors can be magnified. If you have a business rule to always round up, then according to our calculation above, 0.34 + 0.01 = 0.36.

However, that’s enough about why doubles are bad for financial calculations. What caught me by surprise was running accumulate functions in Drools. I was writing code to react to currency exposures being at particular limits, which were all being added up from nice healthy BigDecimal values. However, the numbers I was getting from the ‘sum’ accumulate function were not equal to the numbers my unit tests were expecting. A little investigation showed that the sum accumulator was converting all my nice fixed precision BigDecimal numbers into doubles. Oh dear…

So after a little further investigation, I established that there are no BigDecimal accumulators for Drools, and a bug has been open since 2008. The only workaround mentioned was to write your own ‘sumBigDecimal’ accumulator function.

This didn’t seem like great progress, but I thought it seemed like a good opportunity to learn a new corner of Drools, so I knocked together this BigDecimalAccumulator implementation:

I am a bit puzzled that I’m not finding examples of this all over the place, as Drools has seen a lot of uptake in the financial industry, and it seems like an obvious thing that anybody using Drools for financial rules and calculations would need. Maybe there are loads of private repositories out there, each with their own implementations?

Anyway, in the absence of BigDecimal accumulator functionality in core Drools, feel free to grab this for your own applications.

Sonatype Nexus is a repository for build artifacts, which is particularly handy if you have a Maven project. Once you have your Maven project configured, every time you run mvn deploy Maven will do a bit of building and then upload the resulting artifacts (.jar, .war, …) to the repository. If you browse Nexus you will then be able to find those artifacts with a unique name and download them.

This is all great, but if your project is running on a snapshot version, then every time you deploy to Nexus, the artifact file name will be appended with date and an ever-incrementing number. For my purposes, I wanted to be able to go on to a Linux test server where I have Apache Tomcat installed and grab the latest .war file. Maybe I need to relax more, but I was getting a bit irritated with having to manually find the snapshot in Nexus, copy the link and then fire off a curl -O -L http://... command every time I updated the project.

Fortunately it turns out that Nexus provides a REST API for searching. Unfortunately, it only returns the name of an artifact without the time-stamp. I think that this is intended to be a ‘good thing’ with Nexus automatically resolving the latest snapshot based on requesting the snapshot with no time-stamp. Unfortunately, requesting that artifact from the location indicated by the API results in a ‘not found’ response.

Therefore I knocked up a little Ruby script, which will go to the URI at which a full artifact list can be found, which includes resources such as poms, jars, sha1 and md5 hashes. It then parses the response XML to narrow down the results and selects the most recent artifact that matches the search criteria. Finally it will download the artifact.

The latest version of the script can be found as a gist on GitHub:
https://gist.github.com/1852106

Originally posted @ gratiartis.org on April 5th 2008.

When at work, I’m generally in the worlds of Windows and Solaris. But these days I’m doing my home development work on a MacBook Pro. But I’m keen to make sure that I can do everything that I do at work so that I can experiment in my own time. So here’s a list of the software I use to develop on the Mac and where to get it.

Java Development

Eclipse (http://www.eclipse.org/)

Still my favourite IDE for Java development with a whole host of useful plug-ins.

NetBeans (http://www.netbeans.org/)

A decent IDE for Java development, but not up there with Eclipse yet. Version 6 is good though if you’re happy to pick up the development builds. Once that’s complete t will be a real challenger.

Maven (http://maven.apache.org/)

You want to make sure your builds work no matter what platform you’re on? This is a very good way of making sure that happens.

BEA WebLogic

I’m generally a Java EE developer, and the platform that I target at work is WebLogic. Take a look at my WebLogic on Mac OS X page for more information about how to get the server running on OS X.

TeX Development

I do quite a bit of writing using LaTeX for my typesetting. You’ll find a few examples on this site where I have used it to generate PDFs. In the past I have worked on Windows with TeXnicCenter which made working with a large project easy and I used the MikTeX distribution which contains the useful utilities that allowed me to script generating the work products. But what do I do now that I’m trying to do everything on the Mac? As you might expect with something as well established as LaTeX, there is plenty of software that can be used on all sorts of platforms.

MacTex (http://www.tug.org/mactex/)

MacTeX is a collection of front-ends, documentation and utilities for TeX development. I should warn you that it’s over 700MB, so it will take some time to download!

Since I’m now playing around on a MacBook Pro I thought I’d see whether it was possible to get BEA WebLogic running on it. As it turns out it’s quite simple. These instructions seem to apply to anything from 9.0 upwards, but there’s a link to instructions for 8.1 below.

Here are some sites that I have drawn on for these instructions:

• How to install BEA WebLogic 9.0 on MacOS X
• How to install BEA WebLogic 8.1 on MacOS X

Originally posted @ gratiartis.org on Septenber 8th 2007.

Assuming that you’re familiar with WebLogic on other platforms you probaby already know you can get hold of it here:

http://commerce.bea.com/products/weblogicplatform/weblogic_prod_fam.jsp

Select the download page for the version you want. When selecting platform, choose the version for AIX. Others are designed for specific platforms, but this one is their ‘generic’ install that will work on pretty much anything with a JVM.

Start a console and navigate to where the downloaded JAR was saved and run:

java -Dos.name=unix -jar server100_generic.jar

Replace the JAR filename appropriately according to the version you got. The trick here is that it overrides the os.name environment variable so that the installer doesn’t make a mess of things.

And that’s it… the installer runs and WebLogic is installed.

This is as far as I have got, so I will update this as and when I come across any peculiarities.

Originally posted @ gratiartis.org on April 6 2007

As a developer of financial systems, I’m constantly disappointed by how difficult it is to perform precise decimal arithmetic in Java. It’s fairly common knowledge that the use of double is not much use for financial calculations. As a quick example, take a look at the following piece of code.

public static void main(String arg[]) {
    BigDecimal bd = new BigDecimal("58.99").add(new BigDecimal("0.99"));
    System.out.println("BigDecimal result: " + bd);

    double d = 58.99 + 0.99;
    System.out.println("Double result: " + d);
}

You would have thought both results should be the same. However, you end up with:

BigDecimal result: 59.98
Double result: 59.980000000000004

This is caused by the fact that in Java, operations on double primitive uses floating point arithmetic. The use of floating point arithmetic does make it relatively fast to perform the operations. The approximate nature of them may not matter in all situations, but for financial calculations it can cause all sorts of problems.

As shown above, this issue is resolved by using the BigDecimal. This makes use of precise decimal arithmetic. However, if you’re working on a relatively complex operation, the code bloat can be immense. I’m sure most folks would agree that

BigDecimal bd = new BigDecimal("58.99").add(new BigDecimal("0.99"));

is a lot harder to read than

double d = 58.99 + 0.99;

COBOL has maintained a lifespan beyond most poeple’s expections largely due to the fact that calculations use precise decimal arithmetic. This has meant that in banks and presumably elsewhere, there continue to be COBOL teams working on various systems. With the recent drive towards encouraging the use of domain-specific languages and the ease with which languages other than Java can be run on the JVM, I have been digging around to find out what people are using in these situations. Unfortunately, so far it looks as though back-end systems continute to use Java and C/C++ with additional (often third patry) precise math libraries. These allow precise math to be used, but as they are not integral to the language, they just cannot be as concise and readable as I would like.

This is where I should probably introduce Groovy. It’s billed as “An agile dynamic language for the Java platform”, with many of the features of other dynamic languages such as Ruby and Python. The syntax is deliberately similar to that of Java, as it is designed to be run on the JVM and interact easily with Java APIs. The reason I bring it up is that as a result of the philosophy of “no surprises” the Groovy Math model assumes exact decimal math as the default for calculations. Groovy literals with decimal points are instantiated as BigDecimal. Although it’s worth noting that if a calculation involves a double, the calculation will proceed using floating point math. You can read more about it at http://groovy.codehaus.org/Groovy+Math but my reason for interest is that for instance, 58.99 + 0.99 == 59.98 actually evaulates to ‘true’. This means that you can actually start working with exact math on the Java platform in a manner that is easy to read.

I’m not an expert on the internals of these things, so I would be keen to hear other people’s thoughts on the use of Groovy as a domain-specific language for financial calculations on the JVM. Are there better alternatives that I haven’t heard of yet?