Wednesday, August 4, 2010

Should consensus affect the Supreme Court?

I received an email from the ACLU today that said, in part:
We must demonstrate that there is a national consensus in support of marriage for same-sex couples. Every step forward will make it harder for the Supreme Court is consistent with the Constitution.
Consensus shouldn't affect the Supreme Court, should it?  The inherent flaw in democracy is that the majority can oppress the minority.  The Constitution is supposed to keep that oppression from happening.

I don't care if 99% of people are against gay marriage or black education or .  The Supreme Court should use the 14th amendment to strike down any law that restricts the privileges of citizens.  A privilege is an entitlement granted by the state.  Marriage licenses are granted by the state and entitle citizens to certain benefits.  Seems pretty clear to me that that's a privilege.

Or is the ACLU just being pragmatic?  In an ideal world, perhaps, Supreme Court justices shouldn't be affected by popular opinion, but they are only human, so they probably are affected.

Thursday, May 6, 2010

Jaclyn: The Jarc Compiler

The Fandle R & D team has been busy writing a compiler for Jarc. Today we have released Jarc 12, which includes Jaclyn, the compiler.  Jaclyn compiles to JVM byte code.  Actually, Jaclyn generates a Jasmin assembler source code file and then calls Jasmin to create a .class file.


  Jarc> (compile "foo.arc")
  Generated: ./foo.class
  Jarc> (iload "foo.class")

Performance Improvements

This had a 5x improvement on the startup time for Jarc. From 2.5 seconds to 0.4 seconds.

The run-time improvement is interesting. Initially the compiled code runs more slowly, but then improves after the first two runs.  I ran the Jarc unit tests 10 times and recorded the time for each run.

This may be partially due to the JIT compiler.  Ignoring the initial runs, the compiled code runs about 1/3 faster.

This should shave a few seconds off the cold-startup time for Fandle on Google App Engine and allow slightly more processing per page hit or task.  Page hits (and tasks) have a 30 second limit on App Engine so applications have to be designed to do their processing in 30 second increments.

Tail Recursion

Jaclyn optimizes tail recursive functions to avoid using unnecessary stack space.  It's not generic tail call elimination, which is not possible in the JVM, but when a method calls itself, the compiler generates a jump to the top of the function instead of invoking the function.  This makes recursion feasible for iteration and avoids StackOverflowError exceptions.


These optimizations are needed to optimize tail calls.  If certain macros (like let) are not optimized then the recursive call winds up in a separate function and can't be optimized.  But by optimizing:

  • afn
  • rfn
  • do
  • let

Jaclyn can optimize many instances of tail recursion.

You can download Jarc 12 from SourceForge and you can also browse the source code.

Thursday, March 11, 2010

Using Jarc with App Engine

I've written a sample Jarc web app for Google App Engine.

Overview of the code


  id Long
  description String
  rating String
  created Date
This uses SML format to define the JDO object to persist in App Engine. The script src/mkjdo.arc converts this into a Java class.


(use 'jtml)

(def /home (req res)
      (title "WineRecord - What wines did I like?")
      (link rel "stylesheet" src "winerecord.css")
      (script src "")
      (script "$(document).ready(function() { $(\"input:text:first\").focus(); });"))
        (with-open db (getPersistenceManager pmf*)
          (form action "/add" method "POST"
              (let wines (execute (newQuery db "select from wine.Wine order by created desc range 0,500"))
                (tr (th "Buy Again?") (th "Wine") (th "Date Added") (th))
                (each wine wines
                  (tr (td wine!description)
                      (td wine!rating)
                      (td wine!created)
                      (td (a href (string "/edit?id=" wine!id) "Edit")))))
              (tr (td (text name "description" size 56))
                  (td (select name "rating"
                              (option "yes")
                              (option "no")
                              (option "maybe")))
                  (td (submit value "Add Wine"))
        (a href "/export" "Export Wines") " "
        (a href "/import" "Import Wines")))))

The above is just an excerpt. You can also view the complete winerecord.arc

No, it doesn't use the standard arc html package. I think it would be difficult to make that work since it would require serializing closures to make them available in multiple web servers.

The jtml package uses SML format which has a direct mapping to XHTML. So the tag names should be familiar to anyone familiar with HTML. The only additions to HTML in jtml are abbreviations for input tags based on the type, for instance you can do:

(text name "name" value "foo")
instead of requiring
(input type "text" name "name" value "foo")
Although the latter works also.

Given my rant about the advantages of saving one token you might wonder why I didn't abbreviate the html tags to within and inch of their lives. Well, in this case, I think leveraging compatibility with HTML is more valuable. And you can create your own macros to abbreviate it however you like. It doesn't have to be part of the Jarc language, unlike the Java access syntax which does have to be part of the language.


Jarc has it's own HttpRequest class which supports lookup using apply, so the code can do:

   ... req!rating ...
Instead of
   ... (getParameterValue req "rating") ...


The JDO class created from src/wine.sml is in package "wine". See build.xml where mkjdo is called (around like 32). The second argument to mkjdo is the Java package name to use. I couldn't get App Engine to work if the JDO object was in the default package.

Features of the JDO Wine class

  • Supports lookup using apply - wine!created
  • Supports sref - (= wine!created ...)
  • Has a constructor that takes a map to init the object - (new wine.Wine (getParameterMap req))
  • Has a putAll method to update the object - (putAll wine (getParameterMap req))
So if the HTML form has the same names as the JDO object it is easy to create or update the JDO object from a form.

Source code and Running App

You can download the entire source from and you can play with the running app at

Wednesday, March 10, 2010

Java access syntax from Jarc - less dots, more filling

Have you ever wondered if the ease of calling C libraries could be responsible for a lot of Python's popularity? C function calls look just like native Python calls.

import george;

You can't tell from the Python code whether the module, george, is written in C or Python. It doesn't matter to the calling program. That's a simple foreign function interface.

Jarc brings this same simple foreign function interface to Arc. Unlike Clojure and JScheme, the syntax for calling a Java method is the same as for calling any Lisp function.

Jarc> (getTime (new java.util.Date))
Jarc> (getTime "foo")
Error: Symbol 'getTime' has no value

Even though there is no function getTime defined, that function can still be called on a Date instance.

Jarc uses dispatch on first arg to figure out how to evaluate the method call. This was suggested by Paul Graham in Arc at 3 Weeks. Although Arc doesn't currently have dispatch on first arg it is ideal for Jarc to access Java methods.

If you've defined classes in Python (or Perl), this may seem intuitive.

class HelloClass:
    def f(self):
        return 'hello world'
That self there is the first argument to the function. Even though you call the function as x.f() what's happening under the covers is that x is passed as the first argument. This same thing is happening under the covers in Perl and C++.


1. You can treat Arc calls and Java calls exactly the same

Polymorphism, anyone? Here's the Jarc macro with-open, which is just like let except that it also calls close on variable. It is slightly more complicated then that because is always calls close even if there is an error. And it ignores any errors that might happen when calling close.

(mac with-open (var init . exprs)
   `(let ,var ,init
      (protect (fn () ,@exprs)
        (fn () (errsafe (close ,var))))))
This is quite handy and ensures that your "stream" gets closed, both when it is an arc type:
(with-open f (outfile "what.ever")
Where Jarc calls the Arc function close, and when the "stream" is a Java object:
(with-open db (java.sql.DriverManager.getConnection ...)
Where Jarc calls the close method on the java.sql.Connection instance.

2. Java objects work with map

No helper function (like memfn in Clojure) is needed to use Java instance methods with map.

(map 'getTime '(list (new java.util.Date)))
Both map and apply accept a symbol (in addition to a function, of course) and interpret that as a Java method call.

3. One less character

And of course, since succinctness is power, saving one whole character is an advantage as well. Clojure requires you to type an additional period.

(.getTime (new java.util.Date))
Astute readers will know that succinctness is defined by the number of nodes in the parse tree. And the Clojure example above still has the same number of nodes as the Jarc version. But the number of nodes is also a proxy for how hard it is to read the code. Our brains have to process the code too. And I think parsing .getTime requires parsing the dot separately. And it's not useful information. Just like in Python, I don't want to be distracted with extra syntax to indicate that this is a Java method call. It's just a function call and should be just as simple.

Thursday, February 25, 2010

Simplified s-expression XML (SML)

Have you ever finished something and released it to the world and then almost immediately afterward realized a way to improve it? Well, that happened with my last blog post. I woke up in the middle of the night and realized that I don't need the damn @ sign in SML.

This will work just fine:
 (tagname attr "value" attr2 "value2"
   (tagname3 "data"))
When parsing the SML if there's a symbol, it must be an attribute name, because the only other things allowed are sublists and strings. That makes the format a lot more readable. I've updated sml.arc to use this new format.

It will also still accept the old format with the @ signs. Might as well keep it backward compatible. I hope I won't wake up in the middle of the night again tonight.

Tuesday, February 23, 2010

Using s-expressions instead of XML

Last time I needed to manipulate a large XML document I remembered Paul Graham's comment in What Made Lisp Different that programs communicating with s-expressions is an idea recently reinvented as XML. I began to wonder if I could just use s-expressions instead of having to deal with XML.

Step 0: Define an s-expression representation for XML.

 (tagname (@ attr "value" attr2 "value2")
   (tagname3 "data"))
If the attributes are optional, then that requires an extra token (@) to distinguish between attributes and the first nested tag.

If the attributes are not optional, then that requires an extra token (nil) when there are no attributes specified.

Most XML documents I've used have more tags without attributes, so I opted for using @.

Since @ can't be a tag name, if the first thing in the list (after the tag name) is a list whose car is @ then it is the XML attributes for that tag. I dubbed this representation SML (S-expression Meta Language).

UPDATE: I came up with a simpler representation.

Step 1: convert XML to s-expressions.
This seems like a job for Perl. It's great at manipulating data formats. So I wrote which works great.

But it seems like a hack because there might be some XML syntax that it doesn't handle. XSLT was designed for transforming XML so it's a good choice for this also. So of course, I did some Googling and found this xml2sexp.xsl, but it's not complete. It can't even convert itself. So I decided to write my own. Yikes! Now I'm back to writing XML, which I was trying to avoid! I can't think of a programming language that is more unpleasant than XML. But it was a chance to learn XSLT, so I wrote xml2sexp.xsl too.

Step 2: Convert SML back to XML.
Now I'm in the Lisp world, so I can use my Lisp of choice, which happens to be Arc at the moment. So I wrote an Arc library, sml.arc, to convert SML back to XML. There's also a function to pretty-print the SML, since the SML created by the conversion from XML is pretty ugly SML.

Adios, XML! I'll never need to deal with you again. I can just use SML whenever I need to work with XML files.

Monday, February 22, 2010

Arc (Jarc) or anything as a scripting language

I'm testing the hypothesis that Arc could be used as a scripting language. I'm using Jarc, my own Arc interpreter, of course. This is easy on OS X because the execve() system call accepts any number of arguments. So I can start a script with
#!/usr/bin/java -cp /usr/local/lib/jarc.jar jarc.Jarc
But it doesn't work on Linux since the execve() system call only accepts one argument. :-( So I wrote a little C program, which was fun since I haven't written any C in over a decade at least. So now I can do
And voila! I can write scripts for Linux too. You can read the whopping 26 lines of jarc.c if you are interested in the not so fascinating details. Yeah, it'll probably need to be enhanced so I can pass JVM args also. But I haven't needed that yet, and I'm on a write-it-when-you-need-it regimen.

The other ugly bit is that I actually had to change the Jarc parser. Of course, this is the great thing about writing your own language implementation---you can change whatever you want! Jarc has to ignore the first line of the file. So it treats # in line 1 column 1 as a comment character. Yes, I could have had jarc.c make a temporary file without the first line, but that seems inelegant, though much more general purpose. So this requires the latest Jarc (version 2) which I released last week on Jarc SourceForge download page. Now whatever will I do with it?

Thursday, February 11, 2010

Working for myself is a pain in the back

My first challenge as president and factotum was a pain in the back. After only a few hours at my desk I was hurting. Day after day! Yow! I've worked a desk job for decades spending half a day (12 hours) at my desk. But suddenly my new desk was leaving me twinging after a few hours.

The chair height is right, the desk height is right. I finally realized the problem was that the chair doesn't fit under the desk far enough. So I was leaning in to type, hence not sitting up straight, hence pain. Simple fix—remove the middle drawer. Ah. Now I can really work. Yea!

Monday, February 1, 2010

Working for Equity

In 1992 I was hired by a small (5 person) startup. Shortly after it went public in 1999, my equity in the company was worth more than all the salary I had earned there. Wow! Unfortunately, most of that equity disappeared in the dot com bust. But I did learn about dollar cost averaging and diversification.

In 2001 I joined another (10 person) startup company. After 8 years there, my equity stake was also greater than my years of salary.

So now I'm emboldened to forgo salary for a while and just work for equity at my own (1 person) company. Incorporation (e)paperwork was submitted today. I now have no salary. In this blog I'll share with you all my adventures in working for equity as I build a web site to make it easy to find all the live performances of your favorite bands.