Little Functional Programming Lexicon

| Comments

cross-posted from Ruby Learning, intended for people new at functional programming

With Clojure, Scala and Haskell on the scene, functional programming is getting a lot of attention. I’m going explain some terms that are related to the functional programming, to help you understand, and – who knows – nod intelligently random discussions you read or overhear.

This is meant to be a little “Don’t panic” lexicon, not going incredibly in-depth but trying to describe the terms in as simple and friendly a way as possible. To know more, I invite you to read up on the concepts, but I hope this’ll get you started.

Closure

A closure is a function that “stores” the surrounding scope. An example in javascript to make this clearer:

1
2
3
4
5
  function multiplier(factor) {
    return function(otherFactor) {
      return factor*otherFactor;
    }
  }

The function multiplier returns another function, which will multiply any given number with the argument factor.

The function that is returned by this function “closes” over factor – that is it will retain the factor variable information even though it is no longer in the scope of the multiplier function. Every number that is fed to the returned function will be multiplied by factor.

In this example we used an argument of the surrounding scope, but it could also be any other variable in the function scope.

Currying

You have a function that takes several arguments – currying allows you to apply one or more of these arguments and return a new function which takes the remaining arguments. Applying one of the argument is called partial application.

In some programming languages, this is a very easy operation

haskell currying
1
2
plusThree = (+3)
plusThree 5 -- 8

in some others, it’s a little more work syntactically, but it’s possible

javascript currying
1
2
3
4
5
function plusThree(x) {
  return function(x) {
    return x+3;
  }
 }

Higher Order Functions

In functional languages functions are first-class citizens. You use them more or less as you would use any other type of value (I say more or less, because establishing equality of two functions is not possible, you cannot compare 2 functions as you would some other types).

Higher order functions act on this concept, and either:

Hindley-Milner Type system

The Hindley-Milner type system is the name of a type system for the lambda calculus, which comes with a fast type inference algorithm. It’s called Hindley-Milner because it was independently described by first Roger Hindley, then Robin Milner.

Type inference means you don’t have to specify the type of every single variable or function (as is the case in java or C), because the compiler will infer the type for you, which will save a lot of typing and makes it nicer to read.

The H-M type system is used in Haskell and ML type languages (like OCaml).

Homoiconicity

Homoiconicity means that the abstract syntax tree has the same structure as the program. Going from one to the other is a straightforward conversion.

Another property of homoiconic languages is that the program representation is also a data structure in the language. Example below, for Clojure: every expression is also a list (which is why it’s a Lisp-like language, LISP = LISt Processing)

1
2
  (defn addTwo [x] (+ x 2))
  (addTwo 5)

The advantage of homoiconicity is that code = data. You can manipulate your program as if it were data, since it’s effectively a data structure already. Homoiconicity is used in lisp-like languages to allow powerful macros – anything that lisp can do to data structures, lisp macros can do to lisp code. Move over ruby DSL metaprogramming!

Idempotence

This is a property of pure functions – if you apply a function on a set of input, then it will always return the same output. This is due to the lack of side-effects (see side-effects for explanation), which means no hidden parameters will change anything about the execution.

As an example of a function that wouldn’t be idempotent, consider a function that would use a random number in its result. The random number (a side-effect) will change the result every time the function is run, so the function is not idempotent.

why is this useful?

Well, idempotent part of the program are dependable and easy to test. You only have to test on the sets of expected arguments without setting up any other state that could influence it, and it will reliably crank out the same output every time you run it.

Immutability

Strictly speaking not a property of functional programming, though it is a corrollary of purity. If your function can not change the state of the program, variables will be immutable. Say adding an element of a list will take a list as an argument and return another list, which is the same but with the element added.

This may seem like a dreadful waste of memory (especially when the data structures/objects are large), but in functional programming languages there are often optimizations under the hood which will re-use the existing data structures.

Immutability is considered an advantage when working in concurrent programs. There is no danger that the data you’re currently working on will be changed while you’re working on it, since it’s immutable. Then there are strategies of reconciliation to work out which function output will win.

Impure

Impure programming languages allow side-effects in the code without pointing them out with loud syntactic claxons.

The difference between a pure and an impure programming language is that in a pure programming language, it’s made very explicit when a function has side effects, and which kind, and that it’s impossible to confuse functions doing side-effecting with pure functions.

Popular impure functional programming languages are Clojure and Lisps, OCaml, with Scala and Javascript in their own category since they implement both functional and object-oriented paradigm.

Lambda

greek letter λ, used (in the functional programming context) to refer to:

  1. lambda calculus
  2. an anonymous function – a function which is used immediately and doesn’t need naming for further reference, for instance being passed in as an argument to a higher order function (like filter, map, etc).

Lambda calculus

λ-calculus, Wikipedia says, is a formal system for expressing computation based on function abstraction and application using variable binding and substitution. Lambda calculus is a universal model of computation (one can express anything a Turing machine can do in lambda calculus).

Should you know the details of lambda calculus to do functional programming?

No, unless you’re really interested in the mathematical underpinnings of functional programming, have some time and aren’t afraid to spend some time reading with pen and paper scribbling mathematical formulas.

Lazy

Lazy evaluation (so not lazy like lying on the couch) is strictly speaking independent of functional programming. Lazy evaluation means that an expression is only evaluated when the resulting value is used or displayed. So you could have lazy evaluation in imperative languages.

I mention it in this little lexicon because lazy evaluation was actually introduced for the lambda calculus, and Clojure and Haskell (especially the latter) have plenty of lazily evaluated functions in their standard library.

Advantages:

The opposite of lazy in this context is eager.

Monads (et Al.)

There are numerous text and blog posts about what monads are, some of them crystal clear and some of them slightly obfuscating the concepts. Here’s the thing though: unless you’re doing Haskell or similar statically typed pure functional languages, you don’t really need to know what they are.

In short:

  1. monads allow people to bundle in side-effects in a pure typed language (IO monad, state monad, etc). They have a type, which indicates which kind of side-effect they’re used for
  2. monads also have a number of mathematical properties and associated functions. Those functions are designed to let you daisy chain monad-handling functions or to change ordinary functions to handle monads.

Other terms you might hear: Monoids, Functors, Applicative, … these are also only significant in the context of Haskell and company. Using them other types of languages is a purely academic exercise.

Purity

Purity can be used in two contexts: a pure programming language, and a pure function.

Purity for a functional programming language means: side-effecting has to be explicitly indicated, as is the case for Haskell (both in the function signature, usually returning monads, and in the code).

In pure functions, there are no ‘leaks’. A pure function’s only input are its arguments, and its only output is its return value(s). This brings us to Idempotence

Side Effects

Side effects are everything that can change the state of the world – that means the state of your program (outside of the scope of current function), or the standard output of your terminal, or a file, or database content.

Let’s be clear, a program has to have side effects (if only displaying a result in the terminal), otherwise it has very little point. Let’s put it more strongly: the program’s sole reason of existence is to have some desired side-effects, like migrating a database, showing a web page, calculating some statistics and showing them to you

I hope this was helpful and will give you some terms to be going on with! Welcome to the wonderful world of functional programming, I wish you all a pleasant journey!

Full Names Only

| Comments

If there’s one thing I’ve learned over my years of development, through reading my own and other peoples’ code:

NEVER try to save on keystrokes for identifiers

Please, please don’t write p instead of product, don’t write fl instead of fileLocation. I may be preaching to the choir, but a random look at code shows that it bears repeating.

There are many possible reasons to use abbreviations instead of the full name of things:

In my experience attempts at typing less (in this instance) will result in blood and tears later. It’s an implicit part of the imperative Code for the maintainer

Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live.

On the practical side, you can fix your workflow so you won’t be tempted. All sensible editors have autocomplete facilities. In vanilla vim Ctrl-N will get you autocomplete. The snippet below pasted in .vimrc will change Ctrl-N to tab (one keystroke less!) and will give you the dictionary to boot (adapt the location of the dictionary to your system).

1
2
3
4
5
6
7
8
9
function! Tab_Or_Complete()
  if col('.')>1 && strpart( getline('.'), col('.')-2, 3 ) =~ '^\w'
    return "\<C-N>"
  else
    return "\<Tab>"
  endif
endfunction
:inoremap <Tab> <C-R>=Tab_Or_Complete()<CR>
:set dictionary="/usr/share/dict/words"

I’m reasonably sure emacs, sublime text and certainly IDE will have equivalent features. So laziness is not an acceptable excuse not to use the full name of things.

Do you want to be exposed to code that looks like this?

Mystery code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
if (a_neg ^ b->neg)
  {
  /* only one is negative */
  if (a_neg)
      { tmp=a; a=b; b=tmp; }

  /* we are now a - b */

  if (BN_ucmp(a,b) < 0)
      {
      if (!BN_usub(r,b,a)) return(0);
      r->neg=1;
      }
  else
      {
      if (!BN_usub(r,a,b)) return(0);
      r->neg=0;
      }
  return(1);
  }

(bonus points if you guess which project this is from :) ) (no cheating)

Coders of the world, type it like it is.

Internationalization With Yesod

| Comments

Internationalization with Yesod

Internationalization can be a pain, but it’s often a necessary one. Especially in Europe, composed of a patchwork language communities which correspond only marginally to national borders, it’s often mandatory.

Fortunately Yesod offers good tools for that, allowing you to work through your app and hand off a friendly YAML file to your translators.

There’s good guides in the Yesod book, but they often explain the why of a feature more than a how (also interesting, but less immediately practical). This is a small “Getting started” guide for internationalisation with Yesod.

We need a language switcher

Yesod lets you change the language with

1
setLanguage :: String -> GHandler sub master ()

Often the first step is to allow users to set the language from the web interface so that you can see immediately if what you’re doing is working. Language switch handler:

1
2
3
4
5
6
postLangR :: Handler ()
postLangR = do
    setUltDestReferer
    lang <- runInputPost $ ireq textField "lang"
    setLanguage lang
    redirectUltDest HomeR

setUltDestReferer and redirectUltDest will make sure that if you decide to switch languages, you’ll still be on the page you were on before.

Where do the translations go

The translations are all stored in a YAML formatted file.

1
messages/<2-letter language code>.msg

OR

1
messages/<2-letter language code>-<2-letter locale code>.msg

Simple vanilla translation

1
Home: Startseite

Use in the template:

1
_{MsgHome}

Or using a variable or more (with a type, of course)

1
Price price@Int: "This will cost you #{show price} "

Use in the template:

1
_{MsgPrice priceVar}

For your own types you can try and define a better show or even define custom functions to display it in the appropriate language. From Yesod docs:

1
2
3
4
5
6
7
8
9
renderEn Hello = "Hello"
renderEn (Greet name) = "Welcome, " <> name <> "!"
renderHe Hello = "שלום"
renderHe (Greet name) = "ברוכים הבאים, " <> name <> "!"
instance RenderMessage MyApp MyMessage where
    renderMessage _ ("en":_) = renderEn
    renderMessage _ ("he":_) = renderHe
    renderMessage master (_:langs) = renderMessage master langs
    renderMessage _ [] = renderEn

How to render templates

Your normal

1
$(widgetFile"templateName")

fitted in a layout: works out of the box.

hamlet quasiquotation:

1
2
3
giveUrlRenderer [hamlet|
  ...
|]

becomes

1
2
3
4
import Text.Hamlet (ihamlet)
ihamletToHtml [ihamlet|
  ...
|]

If you’re using hamletFile (which I’m told is not the approved way)

1
giveUrlRenderer $(hamletFile "templates/patient-details.hamlet")

becomes

1
ihamletToHtml $(ihamletFile templates/patient-details.hamlet")

page title

1
setTitle "Home"

becomes

1
setTitleI MsgHomeTitle

Custom bits

Sometimes, you may want to create custom helpers which don’t fit into the above framework. In that case you need to retrieve the message renderer from the Handler monadic context, and work with that.

1
2
render <- getMessageRender
render MsgModuleManagement  -- -> Text

That should help you do what you need to get your Yesod web app internationalized. Have fun :)

How to Start a New Haskell Project

| Comments

I’m working on a Haskell project at the moment, I guess my first Haskell client work. It’s a lot of fun, and a little bit different from the more loosely typed languages I’ve used before.

So, Haskell. Not as hard as you’d think to write something in it, though like with any language, I suspect it will take one a while to write it really well. Even as a beginner, it’s actually surprisingly easy to write expressive and friendly code.

I was intending to write a “getting started” for haskell projects (not including basic knowledge of the language’s syntax and philosophy), and then found that other people had done the same job perfectly well, so I’ll refer to them first.

I’ll still add points that weren’t covered much in other posts and I had to cobble together for myself, looking at other people’s code and various blog posts and stack overflow questions.

Useful references

Cabal project

Cabal leaves you a fair bit of freedom on how to organize your code, as long as you edit the cabal file to let them know where to find it. Repeating slightly what the blog posts pointed to above said:

1
➜  new-project  cabal init

will produce a cabal file and some bits and bobs, but not much in the way of directory structure.

Browsing around and looking at other open source project, the cleanest minimum seems to me to be as follows:

1
2
3
4
5
6
7
➜  new-project  tree
.
|-- LICENSE
|-- Setup.hs
|-- new-project.cabal
|-- src
`-- test

With a code directory (src) and a test directory (test). Now you need to give cabal this information: under “library” or “executable”:

1
2
library
  hs-source-dirs: src

and for the tests:

1
2
test-suite Tests
  hs-source-dirs: test

Duh, you’ll say. Well, a few OS projects I looked at dumpled the code in the root folder, which is cabal’s default, but I find this a bit messy. YMMV.

Edit:: Daisuke Fujimura pointed me to his project template generator here. You can also go quite a bit further by using this structure, which to me is slightly over the top.

Cabal Sandboxex

(Edit) Cabal sandboxes allow you to shield yourself from dependency hell to a certain extent, by having a sandbox in which you’ll install only the dependencies of your project: read more here

Namespaces

Those of you who have done java before will groan: basically same principle. Maybe something in the air of the 90s. Namespaces will be placed in a directory structure that corresponds to the name.

Say your namespace is Carbs.Pasta.Linguine (notice the capitalized names), the code will need to be under src/Carbs/Pasta/Linguine.hs

Tests

I’ve heard some people talk about the fact that statically typed and compiled languages like Haskell don’t need tests. I tend to view such statements with suspicion, since it seems to me that type checking will eliminate some, but not all possible forms of human error. The fact that there is good tooling for testing in Haskell tends to support my view.

Like any language, Haskell has a range of test libraries to offer. You can do the simple assertion thing, with HUnit, or a more BDD like style, with HSpec. Both will have familiar syntax.

Cabal allows you to specify how to run your tests in the cabal file. For a test suite (you can have more than one) running HUnit:

1
2
3
4
5
6
7
8
9
10
11
test-suite Tests
  hs-source-dirs: test
  main-is: tests.hs
  Type: exitcode-stdio-1.0
  build-depends:       base ==4.6.*, Cabal >= 1.16.0
                       , your-project
                       , QuickCheck
                       , HUnit
                       , test-framework
                       , test-framework-hunit
                       , test-framework-quickcheck2

The dependencies include your own project, and any test libraries you choose to use. Here I’m using test-framework, which adds several good features to basic hunit and quickcheck tests. The type specification exists for backwards compatibility purposes, it’s recommended to use the detailed-0.9 or detailed-1.0 interface, but exitstdio is the one that works most painlessly for me at the moment. To be revisited in future projects.

Small gotcha for tests: you need to expose the necessary namespaces you want your tests to use in your cabal file. Otherwise the linker (ld) will complain with scary-looking errors. Under your library or executable spec in the cabal file:

1
2
  Exposed-modules:      Carbs.Pasta.Linguine
                        Carbs.Pasta.Vongole

This will allow you to use the task ‘cabal test’.

Then you can define several suites for your different namespaces:

1
2
3
4
5
6
7
8
9
module Main where

import Test.Framework (defaultMain)

import Carbs.Pasta.Linguine.Test
import Carbs.Pasta.Vongole.Test

main :: IO ()
main = defaultMain [linguineSuite, vongoleSuite]

Tests themselves, using HUnit, are structured same as namespaces (test/Carbs/Pasta/Linguine/Test.hs). It often makes sense to define custom test functions to make the test statements concise (using Assertion producing functions specified in HUnit).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
module Carbs.Pasta.Linguine.Test where

import Test.Framework (testGroup, Test)
import Test.Framework.Providers.HUnit
import Test.HUnit hiding (Test)

import Carbs.Pasta.Linguine

linguineSuite :: Test
linguineSuite = testGroup "Linguine cooking"
   [testCase "cooking linguine takes 10 minutes" (testCook Linguine 10)]

testCook :: Pasta -> Int -> Assertion
testCook pasta expected = expected @=? cook pasta

(HSpec will be familiar for those who know RSpec, if you prefer the style).

QuickCheck

QuickCheck is one of Haskell’s flagship tools. QuickCheck will allow you to test a function with a number of randomly generated input to prove properties your function should always have (invariants), effectively producing combinatorial numbers of test for free – including edge cases.

Again, I believe that nothing is foolproof: the quality of your QuickCheck tests will depend on the quality of your assumptions as related to the problem you want to solve. But then QuickCheck allows you to test those assumptions to the hilt.

Let’s go with a toy example. Say we have a function that counts the kilocalories in your vongole dish.

1
2
3
4
type IngredientCaloryCount = Int

calories :: [IngredientCaloryCount] -> Int
calories ingredients = foldr (+) 100 ingredients

The property I want to prove is that every calory count will be higher than the basic 100, which is the assumed number for a portion of the pasta.

Our property, and test, will look as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
module Carbs.Pasta.Vongole.Test where

import Test.Framework (testGroup, Test)
import Test.Framework.Providers.QuickCheck2 (testProperty)

import Carbs.Pasta.Vongole

vongoleSuite :: Test
vongoleSuite = testGroup "Vongole"
   [ testProperty "calory count >= 100" prop_minimumCalories ]

prop_minimumCalories :: [Int] -> Bool
prop_minimumCalories list = calories list >= 100

This test fails very quickly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Running 1 test suites...
Test suite cooking: RUNNING...
Linguine cooking:
  cooking linguine takes 10 minutes: [OK]
Vongole:
  calory count >= 100: [Failed]
*** Failed! Falsifiable (after 4 tests):
[-1]
(used seed 4962194017620629261)

         Properties  Test Cases  Total
 Passed  0           1           1
 Failed  1           0           1
 Total   1           1           2
Test suite cooking: FAIL
Test suite logged to: dist/test/pasta-0.1.0.0-cooking.log
0 of 1 test suites (0 of 1 test cases) passed.
‚ûú

why? Of course, we’re assuming that all calory counts passed into the function will be positive!

So we have two ways to solve this:

Spoiler: once this is fixed, it fails again when the calory counts are insanely high, because then Int doubles back upon itself. You get the idea: the random approach, in sufficient numbers, will cover cases you wouldn’t normally have thought of yourself.

I will neatly sidestep the rabbit hole that is making generators for your own types for now (QuickCheck comes with a set of data generators for all the basic Haskell data types).

QuickCheck and HUnit tests can be grouped in one file without any problems in test-framework. (FWIW our carb-loaded example is on github)

Haddock

Haskell docs, of course. Again, not unlike java, it allows you to document library from annotated Haskell source code. You can annotate functions, various type declarations (data, newtype, type), typeclasses, modules.

Basically you annotate just above the entity you wish to describe, and you use comment (single-line — or multi-line {–) followed by the pipe symbol, or if you’re documenting in-line — followed by ^.

1
2
3
4
5
data T a b
  -- | This is the documentation for the 'C1' constructor
  = C1 a b
  -- | This is the documentation for the 'C2' constructor
  | C2 a b
1
2
3
f  :: Int      -- ^ The 'Int' argument
   -> Float    -- ^ The 'Float' argument
   -> IO ()    -- ^ The return value

(examples from the haddock documentation)

Run it

Tests are great, even better is to shorten the feedback loop by running them at every little change. For now I’m using a ruby solution to do that: using the ruby gem guard. guard will run cabal build and cabal test every time one of the source or test files change.

I found bindings for libevent in Haskell, so in theory there is nothing preventing such a tool to be written in Haskell. For all I know it exists, I just haven’t found it yet.

Edit: for OS X there is the tool hobbes. Would be interesting to make it cross-platform (thanks to Nick Partridge for the pointer).

That’s it for now. Comments welcome, as always.

Promises in Javascript With Q

| Comments

The async programming style promoted by vanilla node.js development is called continuation passing. If you’ve ever used node.js you’re no doubt familiar with the style:

1
2
3
4
5
6
validateGame(db, game, function(err, result) {
  if (err) {
    return handleError(err);
  }
  return actualCreate(db, game)
});

The continuation being passed (how the program should continue) is the function(err, result) that needs to be executed when the current function completes. This is not massively difficult to work with, and with judicious refactoring, it’s even possible, to avoid what is known as ‘callback hell’, by grouping functions into coherent building blocks.

However, visually, it’s not very intuitive. You’re indicating something that should happen after current function as an argument of the function, almost inside the function call.

What really happens is that the async function completes, and then you go on to the continuation. And this essential visual change is exactly what promises allow you to do.

What are promises

I was first made aware of the possibility of using promises in this context by James Coglan, who has vocally advocated this style over standard continuation passing. He’s compared promises to monads of the asynchronous world which makes sense if you consider the piping behavior I’ll explain a little bit lower.

Promises are concurrency constructs

that acts as a proxy for a result that is initially unknown, usually because the computation of its value is yet incomplete.

Which fits very well with async programming, where the control flow is waiting (and potentially yielded to other request) while mostly non-blocking IO actions complete.

I’ve been using the Q library for promises, and the code equivalent to the one I gave above looks like this:

1
2
3
4
5
6
validateGame(rc, game)
.then(function(result) {
   return actualCreate(rc, game);
}, function(error) {
   return handleError(err);
});

As you can see, there is a very clear sequence of events – “do this, and THEN do the next thing”.

The Q library essentially plans for 2 outcomes for every action, one is error and one is success. When a function has failed, the promise is ‘rejected’, when a function has succeeded, the promise is ‘resolved’ (fulfilled). The ‘then’ can take two functions for every one of those outcomes (success first, error after).

Concatenating ‘then’ allows you to do ‘promise pipelining’, having promises that take the result of previous functions and carry on. The error is also bubbled down the promise pipeline, and the first error callback to be provided will catch it.

1
2
3
4
5
6
7
8
9
10
action1(args)
.then(function(result) {
  return action2(result);
})
.then(function(result2) {
  return action3(result2);
},
function(error) {
  return errorHandling(error);
})

In this example, an error occurring in action1 or action2 will be handled in the errorHandling function in the last ‘then’.

It takes some getting used to working with promises, since every function you want to chain in this manner has to return a promise itself. Another change in the signature of functions is that we no longer expect a callback as argument. Fortunately, the Q library also gives a convenient helper function, which will transform any standard node async function (expecting a callback that takes as arguments errors and results) into a promise-resolving function. example: go from

1
2
3
4
5
6
var findById = function(db, id, callback) {
  db.get("gamesessions", id, function(err, result) {
    if (err) { return handleError(err) };
    callback(result);
  });
}

To the equivalent promise-returning version:

1
2
3
4
5
var findById = function(db, id) {
  var defer = q.defer();
  db.get("gamesessions", id, defer.makeNodeResolver());
  return defer.promise;
}

defer.makeNodeResolver() rejects the promise if there is an error, and resolves the promise if the function succeeds.

More complex flow with qx

qx is a library that adds some familiar constructs to the promise pipeline, allowing you to execute arrays of promises in various ways. map:

1
2
3
4
5
6
7
8
qx.map([
  function() { return action1(args) },
  function() { return action2(args2) },
  function() { return action3(args3) }
])
.then(function(allResults) {
 ...
})

The ‘then’ is only executed when all promises are fulfilled, and allResults is an array containing the results of all the promises, in the order given in the program. q also offers a convenience function ‘spread’, so you can name the results individually.

1
2
3
4
5
6
7
8
qx.map([
  function() { return action1(args) },
  function() { return action2(args2) },
  function() { return action3(args3) }
])
.spread(function(resultAction1, resultAction2, resultAction3) {
 ...
})

Another convenient use of map, is when you have an unknown number of arguments you want to all process the same way:

1
2
3
4
Game.allIds()
.then(function(ids) {
  return qx.map(ids, function(id) { return Game.findById(id) })
})

something which is usually a bit harder to carry out in continuation-passing style.

qx has other interesting functions, like every() or any(), which will execute the next step only if all or any of the promises resolves to true. It doesn’t have reduce (yet), but by digging around in the code, I found a reduce in the q library itself. In short, you can combine promises in ways that should be familiar from standard map-reduce-… type operations.

This allows the promise pipeline to take on a more complex flow. Combining parallel (as in for non-blocking IO – node.js remains single-threaded) and sequential actions becomes easy.

Gotchas

The most important gotcha in working with q is to always, always make sure there is error handling at the end of the pipe. I would urge you to use tests to make sure of that. What happens when there is no error handling? Well, that’s the problem: nothing. If you are writing a web application, like I was, the request just hangs, no explanation given.

On my wishlist for q would be a solution for this, a default fallback error handler, throwing a neat stack trace, if the execution has no explicit error handler.

The Q documentation advises to add a ‘done’ function at the end of your chain as a stopgap.

1
2
3
4
5
foo()
.then(function () {
    return "bar";
})
.done();

In short

After developing a fairly complex node.js application using Q and Qx, I must say that I’m hooked. I find it much clearer to have a sequential path of events, with two clear possible outcomes, than to have to wrangle the usual passing around of callbacks. I’m not completely sure why continuation passing was chosen over promises by node.js core.

There are other promise libraries around (see the Promises/A page), however, after a bit of initial wrestling, I’ve found Q to be a good fit.

Wheelcrowd

| Comments

Edit: Please do use wheelmap.org instead, they are awesome and have recently added Android and iPhone apps too. On top of that they use open data from openstreetmap.

My better half, Joseph Wilk, is a wheelchair user. Time and again, we have the same issue when going out. We hear about a nice restaurant, we travel there, and when we get there, the place isn’t wheelchair accessible. Same story when friends are organizing a party in the private room of a pub.

This kind of ‘ah, well sod that then’ experience is part and parcel for wheelchair users and their families.

The last few weeks, we’ve been organizing our move to Berlin, and so we’ve had some time to spare in between flat visits and organizing things. I’ve decided to adress this issue and start work on Wheelcrowd.

What does it do

Wheelcrowd is a mobile web application to search for accessible places around you. It is a layer on top of Foursquare data, uses Foursquare venues, categories and tips. It records the fact that the places are accessible or not.

home list venue

If this app is useful to you, I recommend that you bookmark it on the home page of your smartphone. This is relatively straightforward on iPhone and Android (for the latter: bookmark the app and use the bookmark widget, that you can find in the widgets).

What does wheelchair accessible mean?

Step-free or with only one step that is not too high, so that the person can pull themselves up (murky waters there, because that would be difficult for a motorized wheelchair users).

You can help by using Foursquare: tag it

The main challenge of such an application is always getting enough data to be useful to people. Two ways for now:

The obvious one: adding data directly in the application.

Joseph had a great idea, which is to let people tag their tips on Foursquare.

If you use Foursquare, add a tip containing the tags #accesspass or #ap (for accessible) and #accessfail or #af (or inaccessible). I would recommend to add this to a normal tip, as in “Slow service, but the coffee is worth it #accesspass”. Foursquare tends to classify adding the same tip over and over as spam, so #accesspass alone might get rejected. Note: that is an actual tip, not a check-in message. Wheelcrowd will automatically pick up on the tips.

foursquare tips

Which places are relevant?

Everywhere!

If you can’t find a venue in the app, add it on foursquare (I’ll add a button in Wheelcrowd to link to the relevant place in Foursquare soon).

Technical notes

This is my first clojure app, and it’s hosted on Heroku. Code is here. If you’re a designer, and you would like to contribute, I can use your help. Joseph has been helping me on the UX, but the design is basically the jQuery Mobile default theme.

Road map

Ultimately

We want to raise awareness of accessibility issues, not only with the public but also with business owners. Sometimes a small effort can make a difference, like asking a carpenter to add a bit of ramp.

I would also consider it a victory if Foursquare and other apps added accessibility to their venue attributes, and allowed people to filter on wheelchair accessibility as a standard. Making this app obsolete would be a good thing!

Let everyone know, especially people who’ll find it useful!

Networking Is a Dirty Word

| Comments

At a recent event somebody asked me how I used to find work as a freelancer. My answer to them was basically “networking”. This drew some startled, even dubious looks from my fellow devs – too polite to call me on it, though. This made me realize the subject might require some explanation.

How-to

The first step of anyone’s version of networking is to attend lots of events and conferences – not limited to just your favourite technologies, but also – gasp – non-tech events. Startup, early adopter meets, pecha kucha, UX, it sounds interesting, then show up.

The second thing is to (in my case) overcome a shy, introverted nature and avoid doing the wallflower thing. Mingle, boldly join random groups of conversations – what’s the worst that could happen? By being genuinely curious, wanting to hear people’s story, you go a long way without having supply too much of the dialogue yourself. Most people are happy to talk to a friendly, attentive face. Most people don’t know that many people either, so you might be doing them a favour by approaching them for a chat.

Be yourself

That’s where most people think it goes wrong: they equate networking to schmoozing shamelessly, sucking up to people who might provide you with whatever you need. Your mileage may vary, but I couldn’t really stomach doing that. Besides, life is too short to spend too much time with unpleasant individuals.

My theory is simple: you meet many people, there’s bound to be a percentage of people there you like, have common interests with, and enjoy hanging out with. I go for those people – one never has too many friends. If I meet people I could have an enjoyable lunch with, or go to a photo exhibition with, I consider my goal achieved.

I do know that the number of “real friends” one has in life can usually be counted on the fingers of one hand – but there are a lot of shades between perfect stranger and friend for life.

It’s give and take

Friendship goes both ways. Naturally, you’ll want the people you like to do well in life, and they’ll feel the same about you. This means you’ll share information and opportunities as they arise. This is taking networking again a step away from self-centered connection gathering to something that works for all parties involved. Being genuine pays off all the way.

So

I hope this convinces you that my version of “networking” is not dirty. It’s more about expanding the number of people you know and like, to the advantage of everyone involved. Thoughts welcome.

Monads for Dummies

| Comments

A while ago I gave a talk at jsconf.eu about functional programming. I wasn’t happy with the explanation I gave on monads. Since I feel I can do better than that, I’m going to add my blog post to the hundreds of posts already floating around on the subject, hoping that it will help some people to see things more clearly

Should you know about monads?

Strictly speaking, despite what the gurus may tell you, no. I read some blog posts from Haskell users stating that you did not need to know about monads to use them – and for Haskell monads are bread and butter (one source contradicting this a little bit). Some libraries (say IO) will take care of business for you, and you can use them without getting into intricacies of how they work. And that’s fine.

However, if, like me, you are curious and like to know what’s under the hood, you should read on.

Also, reading blog posts and listening in to conversations you get the feeling the world is split between those who understand monads, and those who don’t. I’d like to say I give a clear view of what monads are, but you still need to get your head around them – I encourage you to try some of the code examples.

What are monads?

The term monad covers both a mathematical concept from category theory and a functional programming concept. The two are losely related, I’m told, but I won’t lie: I haven’t looked at the mathematical definition. I’ll be talking about monads – the functional programming concept.

Basics: You have a data structure, and two functions, wrap (return in Haskell) and bind. I’ll explain about the 2 functions first, and then show some examples of kinds of monads.

Getting started with the Identity Monad

wrap (I like this name better than ‘return’ or ‘unit’, because it’s much more expressive), creates a monadic value from any value you feed it. What is a monadic value? It’s a function, that closes over the value you gave it.

The simplest wrap function in javascript:

1
2
3
  var wrap = function(value) {
    return function() { return value }
  };

in clojure:

1
(defn wrap[value] (fn[] value))

If you call wrap, it will return a function (the monadic value), which when called will return the original value passed in.

1
  wrap('candy')(); // => 'candy'

or

1
  ((wrap "candy")) ; => "candy"

bind will allow you to ‘bind’ a function that takes a monadic value to the monadic value. The idea is that the bind operation will take a monadic value as argument, and a function that returns a monadic value, and combine the monadic value with the function to return another monadic value. in javascript

1
2
3
  var bind = function(mv, f) {
    return f(mv());
  };

in clojure

1
  (defn bind [mv f] (f (mv)))

demonstration in JS

1
2
3
  var more = function(value) {
      return wrap("more " + value);
  }

does:

1
2
  bind(wrap('candy'), more) // function which returns a monadic value
  bind(wrap('candy'), more)() // => "more candy"

in clojure

1
2
3
(defn more[value]
    (wrap (str "more " value)))
(bind (wrap "candy") more) ; function which returns a monadic value

bind and wrap are dual, two faces of the same coin – if your wrap creates a more elaborate monadic value, then your bind must be able to handle that and pass all all the corresponding parameters (see State monad a bit lower).

To chain functions working in this way, we need functions that integrate bind and return a monadic value. A monadic version of more, integrating bind:

1
2
3
4
5
  var mMore = function(mv) {
    return bind(mv, function(value) {
      return wrap("more " + value);
    })
  }

So we could transform a ‘normal’ more function:

1
2
3
  var normalMore = function(value) {
    return "more " + value;
  }

which is what you’d use in the first place, in a monadic version using a ‘lift’ function

1
2
3
4
5
6
7
  var lift = function(f) {
    return function(mv) {
      return bind(mv, function() {
        return wrap(f(mv()));
      })
    }
  }

so that lift(normalMore) is basically equivalent to mMore. clojure version:

1
2
3
4
5
6
7
8
(defn mMore[mv]
  (bind mv (fn[value]
    (wrap (str "more " value)))))

(defn lift[f]
  (fn[mv]
    (fn[] (bind mv f))
   ))

Having ‘lifted’ functions allows us to daisy-chain operations seemlessly, because they take a monadic value and output a monadic value.

The examples so far are all built around the simplest monad of all, the Identity monad. I’ll show you a couple of other kinds of monads. The data structure of the monad basically comes down to what is enclosed by the monadic value, and whether the monadic value takes arguments – this will become clearer in the examples.

Maybe Monad

The second simplest monad is basically a variant on the identity monad, but a significant one: the bind function will not execute the function if the value is null (or equivalent), but just pass the value on. This difference is significant, because it means that a null value will not cause functions to error. Obviously, you’d only use the maybe monad if a null value was acceptable in your context. So: the wrap function is the same The bind function changes a little: in js

1
2
3
4
5
6
7
8
9
  var bind = function(mv, f) {
    return function() {
      if (mv()) {
        return f(mv())();
      } else {
        return mv();
      }
    }
  };

in clojure

1
2
3
4
5
(defn bind[mv f]
  (let [value (mv)]
    (if (nil? value)
      (wrap nil)
      (f (mv)))))

State Monad

This wiki entry: the State monad is quite different from the Maybe and the list monads, in that it doesn’t represent the result of a computation, but rather a certain property of the computation itself. What we do is model computations that depend on some internal state as functions which take a state parameter.

The State monad allows you to handle computations that depend on an internal state. The state is actually an argument of the monadic value. This means changing the data structure returned by the monad, because the state needs to be passed on, and bind function needs reflect this too.

The wrap function: note that the output is a function that takes state.

1
2
3
4
5
6
7
  var wrap = function(drink) {
    return function(sugars) {
      return {drink: drink,
              sugars: sugars};
    }
  };
  wrap('coffee')(2); // => {drink: 'coffee', sugars: 2}
1
2
3
(defn wrap[drink]
  (fn[sugars]
    {'drink drink 'sugars sugars}))

The bind function needs to handle the more complex situation.

1
2
3
4
5
6
7
8
  var bind = function(mv, f) {
    return function(sugars) {
      var value     = mv(sugars);
      var drink     = value.drink,
          newSugars = value.sugars;
      return f(drink)(newSugars);
    }
  };
1
2
3
4
5
6
  (defn bind[mv f]
    (fn[sugars]
      (let [realized  (mv sugars)
            drink     (realized 'drink)
            newSugars (realized 'sugars)]
        ((f drink) newSugars))))

We can modify the given monadic value:

1
2
3
4
  var more = function(value) {
      return wrap("more " + value);
  }
  bind(wrap('coffee'), more)(2); // => {drink: 'more coffee', sugars: 2}
1
2
3
  (defn wrap[drink]
    (fn[sugars]
      {'drink drink 'sugars sugars}))

clj

And we can also modify state, like so:

1
2
3
4
5
6
  var addSugar = function(drink) {
    return function(sugars) {
      return { drink: drink, sugars: sugars + 1 }
    }
  }
  bind(wrap('coffee'), addSugar)(2); // => {drink: 'coffee', sugars: 3}
1
2
3
4
5
6
(defn bind[mv f]
  (fn[sugars]
    (let [realized  (mv sugars)
          drink     (realized 'drink)
          newSugars (realized 'sugars)]
      ((f drink) newSugars))))

With these, you can chain operations to create a sequence of things to happen, and then feed it state (avoiding global state).

1
2
3
4
5
6
7
8
var mMore = function(mv) {
  return bind(mv, more);
}
var mAddSugar = function(mv) {
  return bind(mv, addSugar);
}

var realized = mMore(mAddSugar(mMore(mAddSugar(wrap("coffee")))))(0); // => {drink: "more more coffee", sugars: 2}
1
  ((mMore (mAddSugar (mMore (mAddSugar (wrap "coffee"))))) 0) ; => {'drink "more more coffee" 'sugars 2}

A more elegant syntax can be obtained by working with ‘pipe’ operators (see J Coglan’s post for an example for the identity case), or do-monad using the clojure monads library.

As you can see bind, wrap and the output format are all related, which is why in statically typed languages like Haskell, kinds of monads are fully fledged types of their own.

This post is already becoming a little too long, otherwise I’d like to introduce you to another interesting monad, the list monad, which allows us to do the usual list operations, but in monad style.

Monadic Laws

It’s possible that you stumbled onto monads without knowing it yourself, while working on some of your code – some blog posts seem to imply that this happens.

You can easily check whether that’s the case by checking whether your functions answer the formal definition of monads, codified in the 3 monadic laws:

1
2
bind(wrap("candy"), more) // is equivalent to
more("candy")
1
2
(bind (wrap value) f) ; is equivalent to
(f value)
1
2
bind(wrap("candy"), wrap) // is equivalent to
wrap("candy")
1
2
(bind (wrap value) wrap) ; is equivalent to
(wrap value)
1
2
bind(bind(mv,f),g) // is equivalent to
bind(mv,function(x) { return bind(f(x),g) })
1
2
(bind (bind mv f) g) ; is equivalent to
(bind mv (fn[x] (bind (f x) g)))

These basically constitute a series of fire-proof tests to see whether you have a monad or not. With this small caveat: unless you can reduce one to the other by using refactoring or logic, equality between two functions is not that straightforward (and programming languages usually don’t let you anyway). So when you’re talking about 2 functions being equal, it means the function yields the same results for all possible values. So technically, the tests I have in my code base are woefully insufficient.

Before you give up, usually your language contains a library that will hide much of the boilerplate shown above! I showed all that code to make the basics understandable, but chances are we can bypass most of this in practice. clojure javascript and probably a few others In Haskell the syntactic sugar is just part of the language.

Good talk explaining Monads by Carin Meyer: http://www.infoq.com/presentations/Why-is-a-Monad-Like-a-Writing-Desk

James Coglan wrote some very interesting posts playing with monads in javascript: http://blog.jcoglan.com/2011/03/05/translation-from-haskell-to-javascript-of-selected-portions-of-the-best-introduction-to-monads-ive-ever-read/ http://blog.jcoglan.com/2011/03/06/monad-syntax-for-javascript/

Mentalguy was exploring the subject in Ruby in 2005 (way ahead of the curve, mind = blown, as usual): http://moonbase.rydia.net/mental/writings/programming/monads-in-ruby/00introduction.html

More importantly, what are monads for?

This was for me the hardest part to figure out. To be perfectly honest, you don’t need monads for an impure functional language. Haskell is the only pure functional language I have any acquaintance with, and there, monads are unavoidable, because they’re the only way to do IO without violating referential transparency.

However, it’s unlikely you’d ever be forced to use monads in Clojure or javascript.

Monads essentially mean you’re working with ‘boxed’ (i.e. closed over) values, which are unpacked only at the very last moment, when they’re needed.

Monads’ benefits:

According to this blog post, monads are appropriate when you’re composing functions and you want the input to be fed into the output directly, no matter the order in which the functions are composed. Another very good blog post about their use in Haskell.

Monads are but one of the functional patterns to explore. Others on the way down the rabbit hole:

So there’s food for thought and learning – I’ve only started on my journey into FP land, but it’s already proving to be rewarding.

This blog post’s code is available on github – with tests included to show that I’m not bamboozling you.

Holiday Reading

| Comments

I’m about to go on holiday. It’s been ages, it’s my first proper break in a good while, so I’m really looking forward to it. As always, I’m going to go overboard and plan for a whole wash list of books to read – I mean, I’ll have time, right? We’re doing a road trip, so I’ll be driving a lot of the time, but surely I’ll have hours and hours to read?

Machine Learning for Hackers Nice practical overview of a number of machine learning use cases using R

Purely functional data structures I’m still bending my head round functional paradigms, so I’m hoping this provides more insights.

While we’re at it: I’m hoping Learn you some Haskell for Great Good will help rewiring my brain.

And I should finally finish Thinking, fast and slow which is an extraordinary book about cognition, which I pretty much recommend to everyone I meet.

And start on What I talk when I talk about running of Murakami, a long-time favourite writer. This is an essay, and I’m curious about what he has to say.

Speaking of which, I heartily recommend this video by Bradbury, who not only wrote great books but also sounds like a very likeable person: Talent is nothing without focus or endurance

Why I’m Not Using Clojurescript

| Comments

People all over the place have been promoting alternative syntaxes for javascript. The clojure community in particular has been working with clojurescript, the idea of using clojure in every part of the stack being very seductive. Here’s one reason why I’m not thrilled about using clojurescript:

The browser doesn’t speak clojure, it speaks javascript.

So switching to clojurescript adds a few constraints, which personally, I’m not thrilled about.

Admittedly, the third point is also true for any high-level language – why not work in C instead of Ruby, Clojure, or others? However, javascript is no C, not in difficulty level. It’s not very pretty, it’s got warts that come with a long and twisted history, but it’s possible to produce fairly friendly-looking code. javascript by itself isn’t excruciating enough to justify the potential difficulty caused by using clojurescript.

I admit I’m sometimes a curmudgeon and resistant to change, so as always, happy to hear any arguments as to why I might be glaringly wrong.

(Note: do read the comments by Stuart Sierra and David Nolen, for a different point of view)