Jabberwocky

Ruby Garbage Collection

2010-09-03T00:00:00Z

Last week, I came across a very interesting tidbit of information: The Ruby Garbage Collector was not meant to run

Garbage collection is one of the Achilles'heels of MRI ruby performance. It combines a fairly naive algorithm with a 'stop the world' assumption, as in it's not a background process, and any other ruby activity can bloody well wait until it is done.

I found this thread in the ruby-lang mailing list very interesting. Next quotes are enlightening:

In particular, the GC algorithms in MRI and YARV are specifically designed with the assumption that they will never actually run in 99.999% of all cases. They are designed for scripting, where a script doesn't even allocate enough memory to trigger a collection, runs for a couple of seconds and then exits, after which the OS simply reclaims the memory: no GC needed.
That's why YARV and especially MRI are so exceptionally bad for server loads. It's also why REE can never be merged into mainline. (Jorg W. Mittag Aug 21 2010)

99.999% is a bit over-exaggerated, but it is true that garbage collection algorithm of YARV and MRI focus for throughput on non-memory extensive short-running programs, and GC of REE is not suitable for those programs. (Matz Aug 21 2010)

Scripting was the original use case for ruby, so that's understandable. It explains a lot.

In other related news, I asked on Twitter whether there was an Enterprise Edition for Ruby 1.9 in the works. Enterprise edition Ruby fixes some of the worst evils by, amongst other things:

exposing some garbage collection parameters, and tuning it a little out of the box
doing Copy on Write, allowing several threads to use common resources in a read-only way.

Unfortunately, Phusion people told me there are no immediate plans to make an enterprise edition of ruby 1.9, since they are now going full throttle on an overhaul of Passenger. Additionally MRI ruby 1.9 needs some patching to make it Copy on Write friendly, and that hasn't happened yet.

I'd like to have a look at Rubinius, where garbage collection is done differently. And of course JRuby benefits from the JVM memory handling, which has been optimized to death by many PhDs and all the developer budgets Sun and IBM could spend on it.

Selenium

2010-07-24T00:00:00Z

Selenium is a tool for integration testing - you run tests directly in your browser - the browser of your choice - and test the whole stack, from client javascript to database access.

Cucumber can integrate with selenium. It's one of the ways to factor in javascript in your tests.

Installing

Installing Selenium itself is a piece of cake. Download Selenium Core from here. Unzip the archive, and copy the content of the core directory into a publically accessible directory on the same location (virtualhost/server) as your application. In rails, this means in a subdirectory of public.

IDE

In our project, we plan to use Selenium for User Acceptance testing. The testers don't want to redo every test scenario. Selenium IDE is the answer: before you start to run a scenario you start recording, and you stop after you're done.

Selenium IDE is a Firefox extension. Download it from the same location. Load it using File -> Open File.

Integrating selenium tests in test sets

Selenium IDE allows you to export to Ruby, .Net, Java. They might need to be edited a little - the generated class for test cases for instance, or the configuration under which to run to run the tests (url, browser).

To run those tests an extra tool is needed: Selenium Remote Control, and corresponding driver for your language. Download Selenium RC from the same site again. The only component you're going to use is selenium-server.jar (Under Debian, you might need to install the packages sun-java6-jdk update-java-alternatives -s java-6-sun)

java -jar selenium-server.jar

Which starts jetty, a light java server, by default on port 4444

In Ruby

gem install selenium-client

Then you can run your tests, as you would normally, maybe with their own rake task.

Selenium does Firefox, IE, Safari - not yet Chrome for as far as I know. These have to be installed on the machines you run the test on.

It might make more sense to run these tests as part of continuous integration, as they are known to be slower. But it's really handy to have them, as they do reflect the users' experience.

RPCFN #11

2010-07-01T00:00:00Z

The Ruby Programming Challenge For Newbies is an initiative on the Rubylearning.org blog started by Satish Talim of Josh Software. It's usually a Ruby programming problem that aims to combine fun and a difficulty level that is just hard enough to be a challenge for Ruby newbies.

I was contacted to provide the challenge for this month, and I selected the Game of Life.

It's a not very difficult problem by itself, but can be approached creatively, especially in the initialization (seeds of the game), and in the dimension variables of the chosen universe (edge handling, n-dimensional space).

If you feel tickled, but you are not a newbie, you can have a go in the 'just for fun' category (there's a few interesting submissions in that category already, like an unreadable perl implementation by Ian Dekkers and movie generation by James Edward Gray II).

Additions to my Ruby Gallery

2010-06-29T00:00:00Z

I got both pictures in the space of two weeks, in different timezones. Go on, laugh, but (if you know at all who they are) admit you're jealous.

Railsconf 2010

2010-06-27T00:00:00Z

Highlights of Railsconf 2010 in Baltimore for me:

the keynote by Yehuda Katz, where everyone left wide-eyed and energized - great speaking. basically, most keynotes were interesting (Gary Vee gets my vote in the impro category). The Youtube channel is here
Async & Cooperative Web Servers with Ruby 1.9 by Ilya Grigorik, encouraging Rails to go in the evented direction, pointing to the success of node.js
Ruby on Rails: Tasty Burgers talk by Aaron Patterson, with some clever ruby examples
Garbage Collection and the Ruby Heap, where Joe Damato explained how they implemented memprof
Bohconf, the unconference organized by the Baltimore Ruby User Group.
The evenings at the pub with many people :)

And I regret missing many other great talk.

My talk went well. I feared to have an audience of 2, because of competition with some great names on other tracks, but I still had a decent size of group in front of me. Enjoyable experience, and a good opportunity to meet the US rubyists.

bikeshedding for fun

2010-06-07T00:00:00Z

Note: If you don't know the reference to the the color of the bikeshed, you should definitely check it out.

My bikeshed of the month was my blog. A blog is nice and simple, so thinking about its implementation is really, really easy.

I wanted a blog where I could just commit and push a markdown file to a repository, and voila: it is published.

The first and obvious candidates are static sites generators like nanoc and jekyll. But I wasn't really convinced, as I was thinking of adding more dynamic features, like, say, comments, or rack middleware shininess.

I started making my own blog engine with sinatra, like everyone and his neighbour, I even had a nice name for it, but then I stumbled onto toto. Toto is a thin layer of blog icing on top of rack.

So I took toto, played a bit to be able to use haml with it and have a decent error page, and here's my blog, such as it is. I'll probably add a fork and a comment gem to toto at some point (haven't decided how to call it, probably something Oz-related).

The amendments to config.ru for haml:

require 'haml'
...
set :to_html   do |path, page, ctx|
  Haml::Engine.new(File.read("#{path}/#{page}.haml")).render(ctx)
end

and the error page with haml and layout:

require 'haml'
set :error do |code|
  # fake the bindings for the layout haml
  class LayoutCtx
    def title; 'title' end
    def archives; "" end
  end
 
   Haml::Engine.new(File.read("templates/layout.haml")).render(LayoutCtx.new) do |content|
      Haml::Engine.new(File.read("templates/pages/error.haml")).render(Object.new,:code => code)
   end
end

And then, the cherry on the cake: the git hook required to make it work on your web server: here

Note: I still have an issue with markdown and code, as you can notice. Still have to look that up.

So there. git push.

Euruko 2010

2010-06-01T00:00:00Z

I finally attended my first Euruko. It had, for me, the perfect size (250 people) and the perfect length (2 days). And it was an opportunity to see Krakow, a lovely town in the south of Poland. The castle, the historical buildings, the churches, the lively nightlife in well hidden locations, you name it, Krakow has it. Very green too, with lots of parks and trees. (note to self: go back there for a proper short holiday)

I did a talk, which is a condensed and enhanced version of the talk I did at ScotRuby. While I was again extremely nervous (which did influence my perception of the event), it went well and people seemed interested.

Other talks I recommend to check out: Ruby Profiling by Piotr Szotkowski, DSL or NoDSL by Jose Valim, Choice architectures by Elliot Crosby-McCullough, Florian Gilcher's talk about encodings. Unfortunately, I missed some interesting talks on saturday and also on sunday since I had to clear out to catch my plane.

I also enjoyed Matz's talk, in which he said 2 things: that the Ruby community should stay nice, open and tolerant, even through its growth, and that standing still in open source is death, so we should always move forward.

I got to take a picture with to Matz (I look like I felt, terribly self-conscious), I got to talk to the developer of rbdbgr (Rocky Bernstein), which I wanted to do, I got to sit next to Jose Valim and Yehuda Katz (secretly trying to soak up some of their 'fu). Medium-sized conferences are great.

Thanks Paul Klipp, Ela Madej, Agatha and team for the fantastic conference !

12 hours to rate a rails application

2010-03-29T00:00:00Z

Here's the presentation I did this weekend.

12 Hours To Rate A Rails Application

Of course, just as I was about to start, my laptop died and refused to start again. Lost me some precious minutes, but fortunately I had a USB key with a backup, and my coworker Alain Ravet lent me his laptop (Julian Fischer also kindly offered, but his computer didn't like my presentation). Murphy's law in all its glory. After that, things went more or less smoothly.

If you had the opportunity to attend, I would really appreciate your feedback on speakerrate, as I'm going to give this talk another couple of times. I sensed when it worked, and when it didn't, but the more input, the better I can fine-tune it.

Fscking up with git and how to solve it

2010-03-24T00:00:00Z

I've been using git for nearly two years now, but this job is the first time I'm using it with more than 2 people on the same project. And it's different, I can tell you.

In this case you need a workflow: we adopted this one, also described here.

It's a good workflow. But human error means that you can go wrong.

Forgetting to branch

The workflow demands that you do all your work on branches - that way you avoid merges on master, and can maintain a nice straight and clear master branch. But, as you start working on a new feature, you might forget to do that git checkout -b boldly_go And start coding and committing on your local master. What now ? Well, git offers an easy solution. Tag your latest commit (give it a label)

git tag boldly_go_tag

reset the master to its remote position (do not use --hard or you will lose your work)

git reset --soft HEAD

create a branch out of your tagged commit, which is not on any branch just now

git checkout -b  boldly_go boldly_go_tag

Better:

git checkout -b boldly_go

to create the branch

git checkout master
git reset --hard origin/master

to reset the master to its original position.

My last commit message lacks poetry

Or I forgot to add this one file. Solution:

git commit --amend

Forget I ever did that last commit

Careful, this will remove any changes you added in that commit. This will only work locally, of course:

git reset --hard HEAD^

if you want your commit to go away, but your code to still be around:

git reset --soft HEAD^

Going off the map

As you work on your branch, you want to do regular rebase with the remote master, to minimize any later merges and avoid surprises. After every commit:

git checkout master
git pull
git checkout boldly_go
git rebase master

Here's where you can go very, very wrong: as long as you're in rebase state, you're in a situation where you're NOT on a branch. So if after a coffee break you forgot you're in rebase, and you continue working, you're working into the void !!

$> git branch
*(no branch)
boldly_go
master

If you then try to do a rebase again to get up to date with master, you won't find your commits where you expected them. Oh noes ! Where did my work go ! No panic (to be honest, i did panic a little): git keeps all your commits. You can find them in the .git/objects directory, if you care to have a look. What you just did is create a 'dangling commit', that is a commit unlinked to any branch. Fortunately, there's the aptly named git fsck command.

git fsck | grep commit

(the grep commit helps separating the commits from the other dangling stuff) then use the SHA associated with your commit to create a tag:

git tag sos 9a4e6286aa2e2bd97334ad35b555169c2d3033b4
git checkout -b not_so_bold_now sos

this creates a branch based on the tag, and you can continue working.

another good one to know in that case is

git reflog

this shows all commits, dangling ones included, with their message, making it easier to find the commit you're looking for. Same procedure for the rest.

This and many other tips, and sources, can be found on git-ready. Thanks to Alain Ravet for some of the tips :)

Scottish Ruby Conference

2010-03-20T00:00:00Z

Time flies, and next week already,

in case of acquisitions (I've been asked to look over an application as an outside expert)
when you're going to take over legacy codebase

I'm going the extra mile on metrics (or quantitative code analysis), which are excellent tools to judge a large-ish codebase. Whoo ! Large and interesting subject, all compressed in 45 minutes. A day might come when I'll be totally relaxed about giving talks, but I'm not there yet. Cross fingers.