Making Nifs Portable

A few weeks ago, I saw that Mike Binns put together a project using the Xbox Kinect. Although I was stoked to see that, it had one huge glaring issue. There was no Linux support -and by extension, no Nerves support! Let’s take some time to fix that.

The Build System

Mike used his own Elixir compiler step to build his NIF. While this works, Elixir-Lang itself actually has a package to do this for you.

We'll start by adding `:elixir_make` to the deps:

```elixir
defp deps do
  [
    # ...
    {:elixir_make, "~> 0.6.0", runtime: false}
  ]
end
```

Then we replace the compiler step `:my_nifs` with `:elixir_make`. This means the entire `:my_nifs` module in the mix.exs can be deleted.

def project do
  [
    #... 
    compilers: [:elixir_make] ++ Mix.compilers(),
  ]
end

Now if we are on a Mac, the compilation will take place just like normal. However there are some more updates we can do. If we open up the Makefile in the project root, we see this line:

ERL_INCLUDE_PATH = $(shell erl -eval 'io:format("~s", [lists:concat([code:root_dir(), "/erts-", erlang:system_info(version), "/include"])])' -s init stop -noshell)

Since we moved to `:elixir_make`, we will change this to be:

# Set Erlang-specific compile and linker flags
ERL_CFLAGS ?= -I$(ERL_EI_INCLUDE_DIR)
ERL_LDFLAGS ?= -L$(ERL_EI_LIBDIR)

The next line:

all: native/kinext.so

This says that the `all` task should run the `native/kinext.so` task, which we can see below. Traditionally, nifs are stored in the `priv` dir of an OTP app. `:elixir_make` has us covered here as well.

Instead of outputting the shared object (NIF output) in the same folder as the source code, we can move it to the `priv` dir. The only weird thing we have to do here is create the `$(PREFIX)` directory if it does not exist. This is done by adding another task.

# MIX_APP_PATH is defined by Elixir Make
PREFIX = $(MIX_APP_PATH)/priv

all: $(PREFIX)/kinext.so

$(PREFIX)/kinext.so: $(PREFIX) native/nifs/kinext.c
	cc -fPIC -lfreenect $(ERL_CFLAGS) $(ERL_LDFLAGS) -dynamiclib -undefined dynamic_lookup -o $(PREFIX)/kinext.so ./native/nifs/kinext.c

$(PREFIX):
  mkdir -p $(PREFIX)

Great! Now the NIF object is placed in the correct spot. We just need to update the Elixir side code that uses the NIF.

In `lib/kinext/native.ex` there is the standard NIF load setup:

@on_load :load_nifs

def load_nifs do
  path =
    __DIR__
    |> Path.join("../../native/kinext")
    |> String.to_charlist()

  :erlang.load_nif(path, 0)
end

We need to change it to the much cleaner:

@on_load :load_nifs

def load_nifs do
  path = Application.app_dir(:kinext, ["priv", "kinext"])
  :erlang.load_nif(path, 0)
end

However, we are still faced with the glaring problem of the nif only working on MacOS. Back in the Makefile - let’s first change the `cc` line to be more cross compilation friendly:

# $(CC) is used by developers who want to 
# be able to "cross compile" target code from their host machines.
# gnu make will set this value for you if it's not set.

$(PREFIX)/kinext.so: $(PREFIX) native/nifs/kinext.c
	$(CC) -fPIC $(ERL_CFLAGS) $(ERL_LDFLAGS) -dynamiclib -undefined dynamic_lookup -o $(PREFIX)/kinext.so ./native/nifs/kinext.c

We can also extract the CFLAGS and LDFLAGS into variables to make them easier to set:

# default MacOS values
CFLAGS=-fPIC -I$(ERL_INCLUDE_PATH)
LDFLAGS=-lfreenect -dynamiclib -undefined dynamic_lookup

$(PREFIX)/kinext.so: $(PREFIX) native/nifs/kinext.c
	$(CC) $(CFLAGS) $(ERL_CFLAGS) $(LDFLAGS) $(ERL_LDFLAGS) -o $(PREFIX)/kinext.so ./native/nifs/kinext.c

And finally now we can conditionally set those values depending on which target is set.

# Check for the CROSSCOMPILE prefix. This is set by other build tools.
ifeq ($(CROSSCOMPILE),)
    # Not crosscompiling, so check that we're on Linux.
    ifneq ($(shell uname -s),Linux)
        # not linux, so use MacOS shared library ld flags
        LDFLAGS += -undefined dynamic_lookup -dynamiclib
    else
        # linux and other platforms use `shared` instead of `dynamiclib`
        LDFLAGS += -fPIC -shared
    endif
else
# Crosscompiled builds are the same as linux
LDFLAGS += -fPIC -shared
endif

$(PREFIX)/kinext.so: $(PREFIX) native/nifs/kinext.c
	$(CC) $(CFLAGS) $(ERL_CFLAGS) $(LDFLAGS) $(ERL_LDFLAGS) -o $(PREFIX)/kinext.so ./native/nifs/kinext.c

And that's all there is to it. It’s a tedious task, but once it's done - it's done forever (famous last words). This attention to detail will allow users from any environment to use a library.

Like what you read? Subscribe to our mailing list below to stay up to date with Binary Noggin!