In Learning Ruby by Extension I claimed that learning a programming by writing extensions was a worthwhile exercise to really get to grips with a language. This is because to write a natural-feel extension you are forced to learn a fair bit about the internals of the language. I was wrong. Well sort of, let me explain.
Last week I bemoaned the poor network programming support in my Common Lisp implementation and so this week I resolved to get my beloved multicast sockets working by breaking the abstraction a little and just implementing the missing socket option call as a foreign function call. I was looking forward to it because I had anticipated that it would teach me something about Lisp. It didn’t. Well at least not in the way I was expecting. Absolutely nothing, and there’s a reason for this.
The foreign function interface for Lisp is at a higher level of abstraction than what I was doing with Ruby or Python. With Python, especially, I was actually creating Python objects and classes within C++ modules that Python would allow me to load as if they were native modules. This meant I had to delve into topics about Python’s garbage collection, threading and more. With Lisp’s CFFI (and all of the other FFI’s I’ve seen so far) things don’t work quite like that because they don’t need to. The language allows for much of the mechanics to be completely abstracted away and become almost transparent to the user. Let’s take a look at this by looking at what I did:
(require 'cffi)
(use-package :cffi)
(defcstruct ip-mreq
(multiaddr :uint32)
(interface :uint32))
(defcfun "setsockopt" :int
(socket :int)
(level :int)
(option :int)
(option-value :pointer)
(option-length :int))
(defcfun ("inet_addr" inet-addr) :int
(cp :string))
(defun socket-add-mcast-membership (socket-fd mcast-addr iface-addr)
(with-foreign-object (mreq 'ip-mreq)
(setf (foreign-slot-value mreq 'ip-mreq 'multiaddr)
(inet-addr mcast-addr)
(foreign-slot-value mreq 'ip-mreq 'interface)
(inet-addr iface-addr))
;; level = 0 = IPPROTO_IP, option = 12 = IP_ADD_MEMBERSHIP
(setsockopt socket-fd 0 12 mreq
(foreign-type-size 'ip-mreq))))
That's it, no external C modules or makefiles it's all done inside Lisp. I'd make this a bit cleaner before I incorporated it into my application because I've intentionally cut some corners to show the relative simplicity. However, it goes to show how easy it is to do. For example if you look at the inet_addr C function definition it only requires a two-line symbolic-expression to have a callable version inside my program . Indeed this solution is on about the same level of complexity as defining an entry point in .NET which couldn't really be any simpler. And like .NET there's no C 'glue' libraries to write because CFFI is doing all the heavy lifting. All arguments and return values are being marshalled into and out of dynamically allocated intermediate values. Which is pretty neat because it means all I have to do is concentrate on the abstraction. Even better is that CFFI allows the partial definition of structures which is great when you have a large and complicated struct which you only care about some of the values for. But that's another story.
Now if we're going to be fair to Ruby, Ruby actually has two ways to write non-local extensions. The first is to use the supplied C programming interface, similar (but simpler) to how Python does it. The second goes a step further and has a gem that allows you to embed inline C directly into the Ruby code. Although I don't think I could have used it for the tibcorv project (because I needed to support callbacks) that's also pretty impressive. Having said that Ruby can only achieve that by having access to a C compiler.
At the risk of stating the obvious a simple method for the definition and invocation of foreign functions is a requirement, I think, for any language to become and remain successful. The easier it is to get the foreigners in the more possibilities you can exploit because there's a C library somewhere for just about any task you can imagine. However, I qualify my original statement that writing extensions can teach us a lot about a language. More specifically I now think that writing extensions will teach you a lot about a language only if the level of abstraction that you have to work at to get the job done forces you to. And as far as I'm concerned, the more abstract the better.