F# vs OCaml vs Haskell: hash table performance
I just stumbled upon an interesting benchmark. Put mappings i -> i for i in 1..10M in a hash table, print the entry for 100. Don Stewart's optimized Haskell:
import Control.Monad
import qualified Data.HashTable as H
main = do
m <- H.new (==) H.hashInt
forM_ [1..10000000] $ \n -> H.insert m n n
v <- H.lookup m 100
print v
My OCaml translation:
let n = 10000000
let () =
let m = Hashtbl.create n in
for n=1 to n do
Hashtbl.add m n n
done;
Printf.printf "%d\n%!" (Hashtbl.find m 100)
My F# translation:
do
let m = System.Collections.Generic.Dictionary()
for i = 1 to 10000000 do
m.[i] <- i
printf "%d\n" m.[100]
Performance:
Haskell: 22.8s
OCaml: 2.82s
F#: 0.706s
I found it remarkable that the performance difference is so large: F# is 32× faster than the optimized Haskell here!
Thanks to everyone who has replied. Apparently this is a fundamental design flaw in GHC's garbage collector which is incapable of handling the mutation of arrays of boxed values (e.g. hash table buckets) efficiently. Unfortunately, this means that Haskell's defacto-standard compiler is incapable of handling any data structure that maps values onto values efficiently. The nearest alternative is a purely functional tree-based map but that is not only still 11× slower than F# but it culminates in a data structure that is then asymptotically slower to search as well:
import Data.IntMap
import Data.List
main = print $ m ! 100
where m = fromDistinctAscList [(i,i) | i <- [1..10000000]]
This program still takes 7.519s.
Perhaps the most remarkable find is not that OCaml and F# thrash Haskell on this benchmark (which is to be expected: OCaml and F# are not toys) but that even a Python one-liner thrashes the most heavily optimized Haskell code:
$ time python -c 'import psyco;psyco.full();print dict((i,i) for i in xrange(10000000))[100]'
100
real 0m5.738s
user 0m5.428s
sys 0m0.308s
If only Haskell programmers could take some time away from writing Fibonacci functions perhaps they could also build some kind of adequate compiler.
Comments
Post a Comment