Strange Loops

Simple but Useful Script

Over the course of my career, I've written a variety of scripts to do the same thing over and over again. This particular script here takes a list of files and combines the files together for use in a CLASSPATH environment variable. It can be used for other things but I use it a lot for constructing CLASSPATH.

bash$ export CLASSPATH=`join.pl ~/.swank-clojure/*`

bash$ echo $CLASSPATH

/Users/dc/.swank-clojure/clojure-1.1.0-master-20091202.150145-1.jar:/Users/dc/.swank-clojure/clojure-contrib-1.1.0-master-20091212.205045-1.jar:/Users/dc/.swank-clojure/swank-clojure-1.1.0.jar

This script gave me to the chance to try out my perl 5.10.1 distribution and a new perl 5.10 features e.g. say and Modern::Perl

Controlling the Mac with Java (and a little AppleScript)

A couple of weeks ago, a friend forwarded me this presentation about Scripting in the Java Platform. I didn't get the chance to read through and try out some ideas from the presentation until this weekend. 

My main takeaway from the presentation is developing applications on the Java platform with the right language for the problem. For example, my core business logic (representing the "hard" parts of my architecture) might be built with Java / Scala to provide the additional guarantee of type safety and then utilizing a dynamic language like JRuby, Jython, Groovy et al for scripting, writing unit tests. 

From the presentation I was introduced to a new CLI tool available in Java 6: jrunscript

The -q option shows all the scripting engines available. When I run it on my Mac, 

dc@feynman:jrunscript$ jrunscript -q

Language ECMAScript 1.6 implemention "Mozilla Rhino" 1.6 release 2

Language AppleScript 2.0.1 implemention "AppleScriptEngine" 1.0

ECMAScript was not surprising. I had known that a Javascript interpreter had been embedded in Java 6 for awhile but I found the AppleScript engine interesting. I've always thought that AppleScript was a weird language. Weird but useful on the Mac. If I could use AppleScript to extract data out of applications running on my Mac, I could process the data with Java (or another JVM based language like clojure).

For a first pass, I wanted to make a Java + AppleScript program put up a Growl message. After a quick crash course with AppleScript and Script Editor, I had this:

and on the Java side:

There were a couple of gotchas which I'll summarize.

1) You can't appear to set arbitrary variables with the various Binding/ScriptContext methods. 

2) To pass values, you have to define a "main" function for the script. That's what this line was doing

applescript.put("javax_script_function", "growlNotification");

It set growlNotification as the "main" function for the script.

3) growlNotification has 2 parameters. To pass arguments, I needed to define a List interface and add my two arguments there. This sort of made sense since the key we use to set the binding says "ARGV". When I initially passed my 2 arguments via an array, I received an error message. After a little bit of trial and error, I found that using a List interface worked. The documentation around this is spotty but the use of a List is somewhat intuitive. 

What's next? Possibly doing this with Clojure and learning more about the various application APIs (aka dictionaries in AppleScript parlance). 

Moving to Clojure 1.1

I'm not using the 1.1 features yet but I was planning on it. During the upgrade to clojure 1.1, I broke my emacs, slime and clojure setup. That's a good thing since the process has simplified since the last time I did it.

Here's what I did on my Mac (running Leopard). Git is a requirement. I installed mine through MacPorts.

First, I got myself an emacs starter kit. Since I'm using aquamacs, I went with an aquamacs variant of the starter kit. Aquamacs Emacs Starter Kit vs Emacs Starter Kit. Follow the instructions for installing the appropriate emacs starter kit. It should look something like this:

 

In this process I learnt a little about Aquamacs. The dot emacs file aka ~/.emacs are read after the preferences that are under ~/Library/Preferences/Aquamacs Emacs. Since I mostly used Aquamacs for hacking clojure, I didn't care and nuked all my emacs preferences. Your mileage may vary. Afterward your Preferences.el file should look like this:

Restart Aquamacs to ensure the new preferences get taken.

One of the important things the Emacs Starter Kit installs is the package manager for emacs (aka elpa). I used elpa to install the swank-clojure package. The starter kit also defaults Aquamacs to green text on a black background (changing that is left as an exercise to the reader).

You don't need to but I wanted to check out the packages available to be installed. The M key below is the meta key which on my Mac is bound to the Option and Esc key. Your key bindings might be set up differently but I would try the Option or Esc key to start with.

 

This is what the package list looks like.

To install a package, I used the package-install command.

You'll be prompted for the package to install.

You'll see some text scrolling by in a buffer. It looks like various emacs lisp files are being compiled. I got some errors. I ignored them. It seemed alright.

Once swank-clojure is installed, it's time to install clojure. To do this, I started up slime.

 

You'll get prompted about clojure not being installed. You want to say yes here.

 

You'll see some status messages about clojure getting installed. For reference purposes, this process installs clojure, contrib and swank clojure jars in ~/.swank-clojure 

Once that's completed, restart Aquamacs and type  'M-x slime'. You should now be ready to hack clojure code.

If you need a custom classpath, you'll need to set the swank-clojure-classpath variable. I created a directory containing all my jar files (in ~/.emacs.d/clojure) and have all the jar files there available in my slime session. To do this, my ~/.emacs file contains the following lines:

(setq swank-clojure-classpath 

      (append 

       (directory-files "~/.swank-clojure" t ".jar$")

       (directory-files "~/.emacs.d/clojure" t ".jar$")))

This approach of globbing jar files together is simple and pretty effective. Alternatively, leiningren appears to be the new hotness in terms of managing clojure dependancies. It's something I'll look at in the future.

ActiveMQ Gotcha!

We were doing a trial deploy of our brand new java service to the production host yesterday. The service had started up everywhere else so far

Of course it didn't start up on this production host. In the logs was a stack trace that included this error message:

cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'amq:broker'.

What caused this problem? This production host was firewalling off connections to activemq.apache.org. This host was serving a XSD file which was required to validate the XML namespace we have used.

I'm thinking WTF! Seriously? Why wasn't the XSD file deployed with jars?

The simplest solution/workaround was to take  http://activemq.apache.org/schema/core/activemq-core-5.2.0.xsd and make it http://activemq.apache.org/schema/core/activemq-core.xsd in our XML file. When validating the XML file, it'll use a version of the file available within the activemq jars.

Dropping the version causes Eclipse to complain but that's another problem (and another google search).

Compiling Jython Scripts to Java Class Files

Had this little snippet of code lying around my directory.

Didn't know what to do with it. It was somewhat useful. We were trying to compile unit tests written in jython for our java code. The jythonc tool had gone away in jython 2.5 but an ant task (JythoncAntTask) in 2.5 still did that. I did a little bit of code spelunking and extracted the code snippet seen above. I don't know the reasons for removal of jythonc but I suspect this usage is not generally supported by the jython maintainers. Please don't complain if this doesn't work for you. We went with another approach for getting our jython unit test to run.

One should be able to compile it from the command line. You'll need the jython.jar to compile it. To run this against a jython script, I threw together a little shell script.

This post was also a little experiment in embedding code samples into my blog posts. The trick is the script tags provided by gist do not work with the preview/rich text mode provided by typepad. They only work after the blog entry has been published. Other than this little gotcha,  Gist is a nice tool for collecting code snippets.

Building ØMQ on my Mac

In my plans for world domination, I need messaging software. I wanted to try out ZeroMQ (trying out RabbitMQ is for another weekend). I'm skipping ActiveMQ since we use it at work and I mostly understand what it provides.

Downloaded the tarball and it compiled right out of the box. However when I tried to get it to run with a simple java program I've written, I kept getting this error (this is on Leopard).

dc@feynman:dist$ java -Djava.library.path=/opt/personal/pkg/zmq/lib -classpath lib/Zmq.jar -jar ZmqTest.jar 
Creating ZMQ
Exception in thread "main" java.lang.UnsatisfiedLinkError: /opt/personal/pkg/zmq/lib/libjzmq.1.0.0.dylib: no suitable image found. Did find: /opt/personal/pkg/zmq/lib/libjzmq.1.0.0.dylib: mach-o, but wrong architecture /opt/personal/pkg/zmq/lib/libjzmq.1.0.0.dylib: mach-o, but wrong architecture
 at java.lang.ClassLoader$NativeLibrary.load(Native Method)
 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1881)
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1805)
 at java.lang.Runtime.loadLibrary0(Runtime.java:823)
 at java.lang.System.loadLibrary(System.java:1045)
 at org.zmq.Zmq.(Zmq.java:25)
 at zmqtest.Main.main(Main.java:15)

As far as I can guess, the JVM I'm using is 64bit so the libraries it loads also need to be compiled as 64bits. To fix this, I had to set the CXXFLAGS environment variable to build 64 bits. This set of steps worked for me.

export CXXFLAGS="-arch x86_64" 
./configure --prefix=/opt/personal/pkg/zmq -with-java
make clean
make
make install

Here's my small java program

package zmqtest;
import org.zmq.Zmq;

public class Main {
 public static void main(String[] args) {
  System.out.println("Creating ZMQ");
  Zmq zmq = new Zmq("localhost");
  System.out.println("Done creating ZMQ");
 }
}

Oracle to Mysql Gotcha

At the new job, we use MySQL. I've mostly worked with Oracle before this and while not expecting everything to be the same, the transition has been smooth.

The one gotcha that threw me for a loop is MySQL's default transaction isolation level for. With Oracle, once data is committed, it's available to be read by all other connections. With MySQL, your view of data is determined by the first query made after the connection is open.

One rationale (I was given) is MySQL has fast connection setups and connections are setup and torn down between queries. I'm not sure I believe that though....

The solution was to change the isolation level for my session via the following incantation:

SET TRANSACTION ISOLATION LEVEL READ COMMITTED

(Taken from the mysql reference manual page)

virtualenv

One recent change to my life has been more python. Lots of nice/bad things have been said about the language that I won't repeat here. I'm generally enjoying my time with it. One useful tool I've found is virtualenv and its bash wrapper functions.

virtualenv allows me to create an environment (python + versioned set of packages) for me to work in. For example I have two virtualenvs right now (temp and django). "temp" contains a myriad number of different libraries I want to experiment with and "django" contains (well) django and a more limited set of libraries. To use I do "activate temp" or "activate django" from my shell depending on what I want to use.

I have a similar problem with perl. I have 2 versions 5.8.9 and 5.10 that I want to use. To take care of this, I have both perls compiled into two different locations. To switch working between the both of, I borrowed some inspiration from virtualenv.

#!/bin/sh
# use-perl5.10.sh
path-to/cpanp 's program perlwrapper path-to/perl5.10/bin/cpanp-run-perl; s save'
PATH=path-to/perl5.10/bin:${PATH} PS1="(5.10) \h:\W \u\$ " /bin/bash

#!/bin/sh
# use-perl5.8.sh
cpanp 's program perlwrapper path-to/bin/cpanp-run-perl; s save'
PATH=path-to/bin:${PATH} PS1="(5.8) \h:\W \u\$ " /bin/bash

I call use-perl5.8.sh and use-perl5.10.sh as appropriate from my shell to setup a working environment from perl. This doesn't isolate the libraries like virtualenv but switching between the two perls is all I needed for now.

Down the Language Rabbit Hole

Once more into the breach.... Been going down the functional languages rabbit hole again, this time I've been more patient with it. The thing about self learning is that you can take your time with it. I rushed through the Little Schemer the first time, this time I'm taking my time going through it and trying to understand the various nuances to programming with recursion. Here we go again.