Planet Ruby

After years of resisting phone ownership followed by a few years of owning a 2003-era Nokia dumbphone, I finally decided to make the jump when the Nexus One was announced. I've got a strong distaste for systems that place arbitrary restrictions on their users, and while the Android OS itself doesn't have any, many Android phones before the Nexus One have had the carriers interfere with the user's control over their phone, though not to the same offensive degree as Apple. The Nexus One is sold directly through Google without giving the carriers a chance to sully it.

Daily Usage

The screen is just brilliant, and the 800x480 resolution means everything is sharp. The OS is very smooth and responsive. Having spent so long on a system where the keyboard is king and the mouse is only used in exceptional cases, switching to the inverse situation on the phone is a bit odd, but not as disorienting as you'd expect. Like any handheld keyboard, the Nexus's is bad for writing anything longer than a tweet, but it's certainly no worse than the hardware keyboard on the old Zaurus I toy around with occasionally. The built-in apps work great, and if you take the plunge to fully switch to GMail, it pretty much makes syncing your mailbox a solved problem.

There are a few nit picks like the color balance being a bit off on the camera, the way the face buttons don't trigger unless you push the upper half, and the fact that the built-in jabber client only supports a single account. But these are all pretty minor or easy to work around. The only thing that really bugs me about it is the fact that there's no ZeroConf implementation yet for the platform. But there are people working on this, so it's just a matter of time.

Oh, and using it to Talk?

It turns out you can also use the Nexus One to interface with the global legacy telephone system and make calls on that. Supposedly it has a very nice dual-mic noise suppression system for when you do this, but I've only made a handful of test calls so far. I got a data-only plan for half of what the regular voice+data plans go for and had planned to use Sipdroid to make VoIP calls with it, but then I realized I just don't make voice calls any more. So while there's a barely-noticeable delay with SIP calls over the 3G network, it really doesn't bother me. I also have used the Wired Tether app to hook up my laptop on the go and can confirm that calls via Skype sound fine too. So it's nice that T-Mobile isn't blocking that on a network level. They do seem to be the least-user-hostile of all the US carriers.

Hacking It

Of course once you get past the formalities, the question that matters to a hacker is how it feels to hack. I've only really gotten started with this, but my initial report is fairly positive. The official toolsets are either Eclipse or Ant, neither of which give me warm fuzzies, but luckily you can use Ant out of the box without getting exposed to the XML-editing ickiness.

Getting programs onto the device is pretty simple. Once your source is ready, you run ant debug, which produces a .apk package. You can use the adb (android debugger) program to load it up over USB, but since I keep leaving my USB cable various places, I prefer just scping it to my server and pointing my device's browser directly at the .apk. You can also use this to install dev builds of various apps before they have been uploaded to the Market.

The API seems pretty sane. A lot of thought has gone into the notion of supporting a single front-and-center application while allowing others to run in the background without impacting battery life and performance too severely. I've played a bit with the graphics tools, and they remind me a fair bit of Processing, which is a good thing. I haven't done much intricate UI work with a lot of buttons or menus, but that kind of stuff can be tedious even in the nicest environments.

Language of Choice

Since the Android VM is based on the JVM, there's a whole host of languages that can run on it. Unfortunately, Dalvik is no Hotspot—it currently lacks JIT, and the GC is merely serviceable rather than astoundingly good like Hotspot's. The lack of a good GC makes using persistent data structures a real drag since they generate a lot of ephemeral garbage, so Clojure is not a good choice. The lack of JIT coupled with CPUs that are comparatively low-powered means that while JRuby runs, it's not altogether pleasant, especially considering the blitz with which regular apps perform. I've been told there is some low-hanging fruit for improving performance on Android, so this is likely to improve to a degree. Rhino, Python, Lua, Scala, and others work, (including, I'm told, even some legacy languages like Java, if you can imagine that) but I decided to try the less-traveled route with something called Duby.

Duby is a language created by Charles Nutter, the head of the JRuby project. JRuby is an amazing feat in part because Ruby's object model is vastly different from what's natively available on the Java platform. By an astounding effort they've managed to put together a first-class Ruby implementation, but it does beg the question: what would a modern language look like that went with the grain of its host instead of violently against it? Duby is an attempt to answer that question.

The syntax of Duby is nearly identical to that of Ruby; it only adds type declarations to method definitions. Yes, that means it's statically-typed. While it has type inference, it's not Hindley-Milner-style, it's closer to Scala's. Locals get their types inferred, it's only arguments in method definitions that need hints. So far I keep forgetting this nearly every time I write a new method since it looks so close to Ruby otherwise, but I'm sure I'll get the hang of it. Closures are implemented, so you can iterate over collections with blocks. Duby is also unique in that it literally has no runtime— its literals translate directly to ArrayLists and HashMaps, so once you've compiled, the code is more or less identical to what the Java compiler would have output.

Progress

So far I've only put together a couple toy apps: Hello World, and a graphics demo with a bouncing ball. Unfortunately, Duby is a very immature language, and it shows. Starting out I had to go to Charlie at nearly every turn with stack traces. Half the time it would be my fault, and half the time it would be something as-yet-unsupported by the compiler. But so far he's been able to turn around and bring in all the features I need, which has been quite amazing. I'm hoping to get a chance to dive into the compiler source myself and get to the point where I can add features I need with minimal guidance.

Adapting the build process to Duby was surprisingly easy. You redeclare the compile task to call the Duby compiler instead of javac, tell it to output its bytecode in the right directory, and the rest of it just falls into place.

My next plans are to add more interactivity to my graphics demo; I'd like to play with creating objects and applying motion rules to them; I hope to come up with something my two-year-old would get a kick out of. So far it's been a lot of fun and a great way to explore the capabilities of this remarkable device.

system()

The fix

$ ruby -e 'system "\"\"fail"'
sh: fail: command not found

$ ruby1.9 -e 'system "\"\"fail"'
sh: fail: command not found
$ perl -e 'system "\"\"fail"'
sh: fail: command not found

Capistrano lets you enumerate your Rails application's dependencies so you can check them at deploy time. Mislav Marohnić did a good description of it a while back; here are some example depend entries:

depend :remote, :gem, "tzinfo", ">=0.3.3"
depend :local, :command, "svn"
depend :remote, :directory, "/u/depot/files"

The problem with depend :remote, :gem, though, is that it duplicates the config.gem entries that you already have in config/environment.rb. It'd be much nicer if you could just reuse those.

So, here's some code that you can paste in your deploy.rb to do just that:

# Add dependencies on gems listed in config/environment.rb
class Collecter
 attr_accessor :dependencies
 def initialize
  @dependencies = {}
  File.read(File.join("config", "environment.rb")).split("\n").select {|line| line.match(/^(\s)*config.gem/) }.each do |line|
   self.instance_eval(line)
  end
 end
 def gem(name, args)
  @dependencies[name] = args
  args[:version] = ">=0.0.1" unless args.include?(:version)
 end
 def config
  self
 end
end
Collecter.new.dependencies.each do |name, args|
 depend :remote, :gem, name, args[:version]
end
after "deploy:setup", "deploy:check

This parses your config/environment.rb, extracts the config.gem calls, and evaluates them in the context of an object that gathers up the dependency arguments. Then it declares an after hook for deploy:setup that runs deploy:check, so when you set up the application on a new server it'll ensure that the right stuff is in place.

Here's a sample run from my military reading list app:

$ cap deploy:setup
 * executing `deploy:setup'
 * executing "mkdir -p /var/www/militaryprofessionalreadinglists.com/"
[blah blah blah]
 command finished
 triggering after callbacks for `deploy:setup'
 * executing `deploy:check'
 * executing "test -d /var/www/militaryprofessionalreadinglists.com/releases"
 servers: ["militaryprofessionalreadinglists.com"]
 [militaryprofessionalreadinglists.com] executing command
 command finished
[blah blah blah]
 * executing "gem specification --version '>=0.0.1' mislav-will_paginate 2>&1 | awk 'BEGIN { s = 0 } /^name:/ { s = 1; exit }; END { if(s == 0) exit 1 }'"
 servers: ["militaryprofessionalreadinglists.com"]
 [militaryprofessionalreadinglists.com] executing command
 command finished
You appear to have all necessary dependencies installed

Good times. The nice thing here is that the developer only has to list the dependencies in one place, and the hook ensures that failures are loudly proclaimed during initial setup.

I'm not sure how to integrate this into Capistrano itself; if Lee Hambley or one of the other Capistrano gurus sees this perhaps they can weigh in... thanks!

You thought we were never going to get to this day, didn’t you? Ye of little faith. Because here is the first real, public release of Rails 3.0 in the form of a beta package that we’ve toiled long and hard over.

It’s surely not perfect yet, but we were out of blockers on the list, so here we go. Please give it a run around the block, try to update some old applications, try to start some new ones, and report back all the issues you find.

I’m really proud of this moment, actually. We’ve had more than 250 people help with the release and we’ve been through almost 4,000 commits since 2.3 to get here. Yet still the new version feels lighter, more agile, and easier to understand. It’s a great day to be a Rails developer.

There’s plenty to get excited about here. A few of the headliner features are:

• Brand new router with an emphasis on RESTful declarations
• New Action Mailer API modelled after Action Controller (now without the agonizing pain of sending multipart messages!)
• New Active Record chainable query language built on top of relational algebra
• Unobtrusive JavaScript helpers with drivers for Prototype, jQuery, and more coming (end of inline JS)
• Explicit dependency management with Bundler

But please take a look at the full release notes and enjoy the latest!

To install:

gem install tzinfo builder memcache-client rack rack-test rack-mount erubis mail text-format thor bundler i18n
gem install rails --pre

Notes: The first line is required because RubyGems currently can’t mix prerelease and regular release gems (someone please fix that!).

This is the fourth (and final) article in a series titled “There is no magic, there is only awesome.” The first article introduced the four cardinal rules of awesomeness, the second was about knowing thy tools, and the third encouraged you to know thy languages .

First off, I apologize for dragging this out. It’s really become a weight on my shoulders. I’ve been fretting and fretting about writing the last two or three posts in this series, and I just couldn’t find the inspiration to make them come out like I wanted…and they’ve been holding up other posts I’ve been wanting to write.

So I’m going to cheat. You’re going to get a braindump, more or less, of the last two rules of awesomeness. Yes, I am entirely cognizant of the irony here. Nonetheless, here goes.

Rule #3: Know thy libraries!

If you want to be awesome, know your dependencies. At the very least, understand what each plugin, library, and technique you use, does. If called on to explain why you are using a particular design pattern, could you justify it? If someone asked you to compare plugin X and plugin Y, could you explain what their different strengths and weaknesses are? If you discover a particular library behaving erratically, would you be able to dig into the library’s guts to tell the author roughly where the problem is? Or would you be one of those who has to tug on the author’s sleeve and say, metaphorically, “it hurts when I do this”?

By libraries, I mean (essentially) any bit of code that you depend on, that has a life of its own outside your application. I’m including design patterns in this, although they aren’t really code, but they do represent a “library” of possibly ways to architect code. Basically, if you’re using someone else’s code, algorithms, or techniques, know why you’re using them. Make sure you can answer the four questions:

1. What does this do best?
2. What does this do worst?
3. Why should I use this in particular?
4. When was the last time I learned something new about this?

Rule #4: Know thy communities!

The last rule says that if you want to be awesome, you can’t do it in a vacuum. By writing code, you automatically join an ecosystem, whether it be a network of coworkers who will help you write, debug, deploy, and maintain that code, or a community of hobbyists who might wish to use your code in applications of their own (or who are writing code that you want to use). Even if all you do is pop up out of your cave long enough to toss some code into the public domain, you’re still contributing to a community, and if you’re using other people’s code, then you’re a consumer as well.

Don’t be blind to those communities. Stop for a minute and think about the communities you belong to, whether explicitly (where you’re a card-carrying, due-paying member) or implicitly (where you’re passively consuming someone else’s code), or somewhere in-between.

Why is this important? Mostly because communities are resources. Each community will be good for different kinds of help, collaboration, or criticism, and if you want to make the most of those resources (which, if you’re really awesome, you will), you need to know where each shines. Again, apply the four questions. What does this community do best? What is it worst at? Why should I participate in this particular community, over all the possible alternatives? And when was the last time I learned from this community?

That last is important, and surprisingly deep. If you are no longer learning from a community, what are you still doing there? Some of you are no doubt saying “but, but, but” and eagerly pointing out that maybe someone is part of a community so that they can lift others up, but note: if you’re doing that right, you’re still learning from those you teach. You learn new ways that your craft can be confusing to people. You learn new ways of looking at things that you’ve been taking from granted. You learn new ways to rephrase concepts that have grown stale in your own mind. It keeps you fresh. If you’re doing it wrong, though, then the only reason you remain in a community that teaches you nothing is complacency. And complacency is on the opposite end of the excellence spectrum from awesomeness.

Words of caution

So, those are the four rules. There is one overriding caveat to all of this, though, and that is: use moderation. You’re not going to master all of this in a day. Or even a week, or month, or year. I don’t know anyone that is perfect at all four things (though I know many who are much, much better than I am). To sound all zen and mysterious: awesomeness is a journey, not a destination. Enjoy the trip, appreciate the view, but don’t overdo it.

When you overdo it, you risk burning out. I’ve been riding that line for a couple years now, which was frightening. When writing code is your bread and butter, paying the bills and putting food on the table for you and your family, it is scary to contemplate “what do I do when I suddenly don’t want to do what I’ve been doing”. This is a big reason why I had to let go of Capistrano and other projects, and why I’ve discovered other hobbies (like woodcarving, string figuring, and cooking) that do not involve computers. I’m trying to find the middle ground, so that I can recapture the joy of writing code.

So: be awesome. But only in moderation.

Anyone who follows my exploits will have noticed I'm a tireless proponent of ELPA, the Emacs Lisp Package Archive. As a maintainer of several Elisp libraries, ELPA makes my life easier by helping me sidestep the boring problems of distribution and installation. You may not know that package.el, the software behind ELPA, has been submitted for inclusion in the next version of Emacs. I've taken up the task of getting it ready.

Including something like package.el into Emacs is a big job, and it's something that can only happen gradually. Emacs comes with a number of applications such as Org Mode and Gnus that are developed externally to Emacs and merged periodically into the main Emacs source tree. If they were to be redone as packages they could still be distributed with Emacs builds but kept out of the source tree. They could also be upgraded and installed/removed independently of Emacs' historically long release cycles.

If you've submitted packages to ELPA before, you know it's a process that could use some streamlining. Currently it's all done by email, and packages must be manually uploaded by a single maintainer before they appear to users. This has long been the biggest shortcoming of ELPA. I've written some additions (package-maint.el) that allow you to automate the maintenance of a package source. Basically you provide it with a list of git URLs, and it will check out each tagged version and create a package from it. Of course, that wouldn't be useful without giving clients the ability to get packages from multiple sources at once, which I also added to package.el.

If you maintain any Emacs packages of your own, please try out my changes to package.el. If you use any of my packages, try upgrading and adding my package source to your list.

(add-to-list 'package-archives
             '("technomancy" . "http://repo.technomancy.us/emacs/") t)

That way you'll get access to my updates as soon as they're tagged rather than waiting for them to be manually uploaded, though currently the latest versions of all my packages are in ELPA. Next steps are closer integration with Emacs in order to have packages installable on a system-wide level as well as a per-user level, prerelease version number support, and extraction of some built-in Emacs libraries as packages. Suggestions, bug reports, and patches welcome!

I got a note from a community member about an annoying problem that a few people have run into when installing activerecord-jdbc-adapter (AR-JDBC) into a C Ruby implementation:

NameError: uninitialized constant
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::PGconn

Turns out it’s pretty easy to momentarily forget to use jruby -S gem or jgem and just gem install activerecord-jdbc-adapter and suddenly your pg Postgres gems are not working properly. I thought it was worth documenting here in case others run into this problem.

I apologize for the clash. I had to provide a stub pg.rb in AR-JDBC inside of JRuby so that I could get active_record/connection_adapters/postgresql_adapter.rb to load with a database adapter type of postgresql. Because of load path order issues, I couldn’t get AR-JDBC’s code to load before ActiveRecord’s. At the time I was thinking this wouldn’t be a problem because the pg library won’t work on JRuby anyway, right? Wrong.

I can think of a couple options going forward:

1. Submit a patch to ActiveRecord so that active_record/connection_adapters/postgresql_adapter.rb can load without requiring pg up front and then AR-JDBC won’t have to stub it out.
2. Display a big fat warning message when AR-JDBC is installed into anything other than JRuby.

Any other thoughts?

I’ve decided to travel less this year. Since my job involves a lot of travel, this mostly translates into going to less conferences.

So for 2010 I so far have only one conference on the agenda (other than those I’m co-organizing): RubyConf India.

I’m speaking at the event and also helping out on the proposal committee along with Pratik Naik and Ola Bini. So far the program is shaping up well, and I’m excited about the conference.

If you’re looking for a combined technical and cultural education experience, I highly recommend going to RubyConf India. Kelly and I lived in Bangalore several years ago and absolutely loved it there. We can’t wait to get back to our second home. Bangalore was a culture shock at first, but that’s another strong reason to go. For us, it went from culture shock to comfort. Bangalore certainly didn’t change to make that happen—we did.

So, while it’s hectic, noisy, and sometimes shocking, Bangalore is a beautiful place which had a profound effect on me. When I was there, I couldn’t find any other Ruby programmers. It will be really exciting to go back to an entire conference devoted to Rubyists.

Registration will be opening in the next couple of weeks. Follow RubyConfIndia on twitter for announcements. Bangalore men milenge?

(cross-posted from Yehuda’s Blog)

So people have been attempting to get a Rails app up and running recently. I also have some apps in development on Rails 3, so I’ve been experiencing some of the same problems many others have.

The other night, I worked with sferik to start porting merb-admin over to Rails. Because this process involved being on edge Rails, we got the process honed to a very simple, small, repeatable process.

The Steps

Step 1: Install bundler (version 0.8.1 required)

$ sudo gem install bundler

Step 2: Check out Rails

$ git clone git://github.com/rails/rails.git
$ cd rails

Step 3: Bundle Rails dependencies

$ gem bundle --only default

Step 4: Generate a new app

$ ruby railties/bin/rails ../new_app --dev
$ cd ../new_app

Done

Everything should now work: script/server, script/console, etc.

When you execute rails APP_NAME --dev, it will create a new Rails application with a Gemfile pointing to your Rails checkout and bundle it right after.

Also notice that in Step 3 we pass --only default to the bundle command. This will skip bundling of both mysql and pg (for postgresql) gems.

Enjoy!

Updated on 01/15/2010 – Rewrote steps to include gem install bundler and use rails APP_NAME --dev.

RailsBridge has organized a Rails 3 Bugmash on January 16th and 17th. The idea is to try and upgrade your apps and favourite plugins/gems to work with Rails 3 and make the upgrade path as smooth as possible for everyone else by documenting the process and fixing the bugs you encounter. Rails core team and others will be around in #railsbridge to help out the participants during the bugmash. Check the RailsBridge announcement for more details.

~/projects/jruby ➔ gem install com.lowagie.itext-rtf
Successfully installed bouncycastle.bcmail-jdk14-138-java
Successfully installed bouncycastle.bcprov-jdk14-138-java
Successfully installed bouncycastle.bctsp-jdk14-138-java
Successfully installed com.lowagie.itext-rtf-2.1.7-java
4 gems installed
Installing ri documentation for bouncycastle.bcmail-jdk14-138-java...
Installing ri documentation for bouncycastle.bcprov-jdk14-138-java...
Installing ri documentation for bouncycastle.bctsp-jdk14-138-java...
Installing ri documentation for com.lowagie.itext-rtf-2.1.7-java...
Installing RDoc documentation for bouncycastle.bcmail-jdk14-138-java...
Installing RDoc documentation for bouncycastle.bcprov-jdk14-138-java...
Installing RDoc documentation for bouncycastle.bctsp-jdk14-138-java...
Installing RDoc documentation for com.lowagie.itext-rtf-2.1.7-java...

• Using Rake tasks from Ant and Ant tasks from Rake
• Calling Rake targets from Ant and Ant targets from Rake
• Mixed build systems with part in Rake and part in Ant

require 'ant'

ant.load 'build.xml' # defines tasks :mkdir + :setup in ant!

task :compile => [:mkdir, :setup] do
  ant.javac(:srcdir => src_dir, :destdir => "./build/classes") do
    classpath :refid => "project.class.path" 
  end
end

#include <stdio.h>
#include <ruby.h>
 
VALUE HelloModule = Qnil;
VALUE HelloClass = Qnil;
 
VALUE say_hello(VALUE self, VALUE hello);
VALUE get_hello(VALUE self);
 
void
Init_defineclass()
{
    HelloClass = rb_define_class("Hello", rb_cObject);
    rb_define_method(HelloClass, "get_hello", get_hello, 0);
    rb_define_method(HelloClass, "say_hello", say_hello, 1);
}
 
VALUE
say_hello(VALUE self, VALUE hello)
{
    return Qnil;
}
VALUE
get_hello(VALUE self)
{
    return rb_str_new2("Hello, World");
}

require 'java'
 
m = org.jruby.cext.ModuleLoader.new
m.load(self, "defineclass")
 
h = Hello.new
puts "Hello.new returned #{h.inspect}"
puts "h.get_hello returns #{h.get_hello}"

When you read about functional languages, one of the things that frequently comes up is the value of persistent data structures. If you come from an OOP background where persistent means "it gets saved to disk", this is a little confusing until you do a little digging to find out it's a different meaning of the word persistent; you discover that it just means the data structures are immutable. This is technically true; all persistent data structures are immutable. But this understanding is a little bit lacking—it doesn't really get at the meaning of persistent.

The point of persistence in this case is that future versions of the object in question can be created without changing either the value or the performance characteristics of the existing instance. So when you've got a vector that you want to work with, you can create another vector based on the original, but with a few new items added to it. In languages that provide persistent data structures this is done without copying; internally the portions of the vector that are the same use a shared structure. But there are some pseudo-persistent implementations that cheat; as you create more and more versions based on the original vector, the performance of the original degrades even though the value is preserved. This is avoided in true persistent implementations such as Clojure's.

The other important thing about understanding persistence is understanding what it's not. A new feature in Clojure 1.1 is the addition of transient data structures. Transients provide speed boosts in cases where you decide performance is more important than persistence by using mutable data structures in a controlled, thread-safe way. If you don't understand what persistence means then you might see the fact that they are mutable and use them as you would in an imperative language—but that's not what they're meant for! The key to understanding transients is not that they're mutable but that they're not persistent. The fact that they are mutable is an implementation detail; you should treat them like regular immutable data structures, you just shouldn't rely on their persistent qualities.

Focusing on immutability is focusing on the negative: what you can't do. Thinking in terms of persistence is focusing on the positive: there are a certain set of guarantees that we may rely on. If you decide in some cases to give up those guarantees for speed benefits, transients allow you to do that, but you shouldn't think of them as your old imperative friends you can alter as you wish.

A few weeks ago I came across François Lamontagne's "Ruby and C : Part 1" post. He talks about the VALUE type in Ruby and how that's typedef'd to an unsigned long. You can see this in ruby.h; in ruby 1.8.6 p369 it's line 86:

typedef unsigned long VALUE;

One interesting thing that François' article notes is that VALUE can contain either a pointer to the data or the data itself. You can see this in Aman Gupta's latest post to Time to Bleed. In one example he creates an Array containing a few Fixnums and a String and then dumps the heap using memprof. This yields:

  {
    "type": "array",
    "length": 4,
    "data": [
      1,
      2,
      3,
      "0x12aa0c0"
    ]
  }

So the three Fixnums are stored as immediate values, and the last value is the pointer to the String.

What other Ruby types are stored as immediate values? From ruby.h around line 697 in the rb_type function:

    if (FIXNUM_P(obj)) return T_FIXNUM;
    if (obj == Qnil) return T_NIL;
    if (obj == Qfalse) return T_FALSE;
    if (obj == Qtrue) return T_TRUE;
    if (obj == Qundef) return T_UNDEF;
    if (SYMBOL_P(obj)) return T_SYMBOL;

Looks like Fixnums, Nil, False, True, and Symbols. Everything else gets identified using the BUILTIN_TYPE macro. The Ruby 1.9.1 code looks kind of similar, although it uses different macros (like RTEST) and does the comparisons in a different order.

Rails 2.3.5 was released over the weekend which provides several bug-fixes and one security fix. It should be fully compatible with all prior 2.3.x releases and can be easily upgraded to with “gem update rails”. The most interesting bits can be summarized in three points.

Improved compatibility with Ruby 1.9

There were a few small bugs preventing full compatibility with Ruby 1.9. However, we wouldn’t be surprised you were already running Rails 2.3.X successfully before these bugs were fixed (they were small).

RailsXss plugin availability

As you may have heard, in Rails 3 we are now automatically escaping all string content in erb (where as before you needed to use “h()” to escape). If you want to have this functionality today you can install Koz’s RailsXss plugin in Rails 2.3.5.

Fixes for the Nokogiri backend for XmlMini

With Rails 2.3 we were given the ability to switch out the default XML parser from REXML to other faster parsers like Nokogiri. There were a few issues with using Nokogiri which are now resolved, so if your application is parsing lots of xml you may want to switch to this faster XML parser.

And that’s the gist of it

Feel free to browse through the commit history if you’d like to see what else has been fixed (but it’s mostly small stuff).

I’m switching from self-hosting Redmine to using Pivotal Tracker for issue tracking on my Open Source projects.

Switching to Pivotal Tracker

After running Redmine locally for a while, I’ve decided to switch to a hosted issue tracking service. I’ve moved all the open tickets on the onestepback.org Redmine app to my account on Pivotal Tracker.

Current Project Links

Here are the links to my current projects:

• Rake
• FlexMock
• Builder
• TExp
• Given

All the projects are marked public so you should be able to view the projects (and subscribe to an RSS feed) without actually signing up for anything.

Did I miss anything?

All the open tickets should be migrated to Pivotal Tracker. If you notice anything missing, let me know. Thanks.

So build tools have been a long-standing pain point when working on Clojure projects. Most projects have used JVM-centric tools like ant or maven, which are far more complicated than what you need to build Clojure, and on top of that must be configured with XML. It could be made to work, and in the case of projects with significant dependencies it was much better than doing it by hand, but it was a pain.

Last week I launched Leiningen, which is a project that brings a native build tool to Clojure. It seems to have people pretty excited about it, which leads me to believe I'm far from the only one who's been feeling the pain here. It's configured using a Clojure file called project.clj like the sample below.

(defproject leiningen "1.0.0-SNAPSHOT"
  :description "A build tool designed not to set your hair on fire."
  :main leiningen.core
  :dependencies [[org.clojure/clojure "1.1.0-alpha-SNAPSHOT"]
                 [org.clojure/clojure-contrib "1.0-SNAPSHOT"]
                 [ant/ant-launcher "1.6.2"]
                 [org.apache.maven/maven-ant-tasks "2.0.10"]]
  :dev-dependencies [[org.clojure/swank-clojure "1.0"]])

Testing

This one's pretty straightforward: lein test searches the test/ directory for namespaces and runs the tests in each.

Building

Running lein jar will compile your code and package it up as a .jar file. If you specify a :main class in your project config, this will be an executable jar with the class you specified. If you just want to compile your code, there's a compile task for that too. Alternatively if you want to build a standalone jar that includes all your project's dependencies for easier distribution, you can do that with lein uberjar.

Dependency Management

This is the fun part. Rather than tracking down and downloading all your dependency jars manually (or worse: checking them in to your source repository), Leiningen provides a deps task to automatically download them and place them in the lib/ directory. This uses some Maven code under the covers, but don't panic—you won't have to see a bit of XML unless you want to. It pulls from the central Maven repositories by default as well as from Clojure's nightly builds and http://clojars.org. Clojars has built-in search, but if you want to pull a library from the central repository, you can search for it on Jarvana. You can also specify development dependencies with the :dev-dependencies key for things that shouldn't be pulled in at production or included in the standalone jar; the syntax is the same.

Publishing with Clojars

By a stroke of luck I dropped hints about Leiningen the week before I made a public announcement only to find out that Alex Osborne was approaching the same problem from the other direction. He'd been cooking up a web site and service for hosting Clojure libraries independently of my work on Leiningen, and we started talking about ways to integrate the two. So now Leiningen makes it easy to publish jars publicly via Clojars so other projects can depend on them. The day after announcing Clojars, Alex was interviewed on InfoQ about the site and how it interacts with Leiningen, so I was thrilled to see both our projects getting exposure so quickly.

Coming Soon...

Soon after the 0.5.0 release, Alex along with Dan Larkin (of the wonderful Clojure JSON library fame) started diving in and adding features. We've got a few more tasks on the plate targeted for a 1.0 release, but should be right around the corner. Update: Leiningen 1.0 released. If you'd like to say hi or drop a line, there's a #leiningen channel on freenode as well as a mailing list. If you found this useful, you should take a look at the Clojure PeepCode screencast.

Logo by Loren Broach.

JRuby is pretty fast these days, in most cases it is faster than MRI. JRuby is especially good at handling long-lasting repeating tasks (on server side). But one thing where JRuby is not blazingly fast is during the startup. Well, actually, JRuby starts up within 0.5 second, which is really nice for the JVM-based implementation. But as soon as rubygems comes into play, the situation gets worse.

So, I spent some time trying to understand why just loading the rubygems package eats up a second or two more. Well, it turned out that rubygems loads almost entire standard library, which is crazy! Sure, this is a bit of exaggeration, but still, let’s see, shall we?

First, fileutils are loaded, 100+ms passed, then rubygems tries to figure out where is the system-level config file, and for that on Windows it uses Win32API library (!!!), which in turn requires FFI (the native interface). Ka-ching, 300+ ms for that. Then, we need to read the configs and stuff, this requires YAML parser, and in JRuby that means loading Yecht, 350+ ms more. Next, JRuby provides some extension of rubygems functionality, like possibility to load gems from the JAR files, and for that we need some classpath magic and the ‘uri’ library. To obtain the JRuby’s classloader, we need to require ‘jruby’, which in turn requires ‘java’, and the java library loads the whole bunch of extra functionality (which we are not using here, btw), that adds up to 400 ms. So, we barely loaded the rubygems library, but already consumed 1200ms of time, or more.

Take a look at the following picture that shows the times to load all those libraries:

As you see, loading rubygems themselves is not that time consuming, most of the time is spent in loading the dependencies. There are some low-hanging fruits here, like lazily loading the ‘uri’ library. I also don’t like that rubygems loads the Win32API and the whole FFI stuff, removing that would also eliminate quite a lot of startup time. Loading the whole bunch of java integration magic in order to only be able to obtain the JRuby classpath also seems to be overkill. All those tweaks, if implemented, might actually cut the rubygems startup time in half. We’ll see.

UPDATE: Wayne Meissner has just tweaked the loading of Win32API, and I adjusted JRuby-specific rubygems tweaks to lazily load the ‘uri’. So, the loading of rubygems is already about 200ms faster than it was yesterday!

UPDATE#2: Here’s the log of rubygems loading timings in JRuby, in easy to read text format: http://gist.github.com/240322.