Installing DynamoDB_local on FreeBSD 10.0 amd64

As part of setting up a development environment for an application that would later run in AWS and use Amazon's DynamoDB service, I set up a DynamoDB local installation on a 64bit FreeBSD 10.0 box. DynamoDB local requires some Linux shared libraries to run, but the compat layer only supports 32bit libraries. Since I couldn't find out how to install a 32bit capable OpenJDK on 64bit FreeBSD, I decided to set up the whole ecosystem using a 32bit Linux JDK instead.

Starting from a 64bit 10.0-RELEASE FreeBSD with no JDK installed and an up to date portstree:

Load the linux compatibilty kernel module. If it shows up with kldstat, it worked!

root@vm02:~ # kldload linux
root@vm02:~ # kldstat
Id Refs Address            Size     Name
1    6 0xffffffff80200000 15f0310  kernel
2    1 0xffffffff81a12000 432d4    linux.ko
root@vm02:~ #

To enable the linux compat mode at boot time, edit /etc/rc.conf and add:

linux_enable="YES"

Next, install the linux-f10-expat port, which will provide some libraries that are required by DynamoDB_local. Unfortunately, as of 2014-09-02 this requires some manual Makefile fixing as outlined in https://www.mail-archive.com/freebsd-emulation@freebsd.org/msg03988.html. Your Makefile should now look like this:

# Created by: bsam
# $FreeBSD: head/textproc/linux-f10-expat/Makefile 340722 2014-01-22 17:00:46Z mat $

PORTNAME=       expat
PORTVERSION=    2.0.1
PORTREVISION=   1
CATEGORIES=     textproc linux
MASTER_SITES=   http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/10/i386/ \
	http://archive.fedoraproject.org/pub/archive/fedora/linux/updates/10/SRPMS/
PKGNAMEPREFIX=  linux-f10-
DISTNAME=       ${PORTNAME}-${PORTVERSION}-${RPMVERSION}

MAINTAINER=     emulation@FreeBSD.org
COMMENT=        Linux/i386 binary port of Expat XML-parsing library (Linux Fedora 10)

CONFLICTS=      linux-expat-[0-9]* linux-f8-expat-[0-9]*

ONLY_FOR_ARCHS= i386 amd64
USE_LINUX_RPM=  yes
LINUX_DIST_VER= 10
RPMVERSION=     8.fc10
BRANDELF_FILES= usr/bin/xmlwf
USE_LDCONFIG=   yes

PLIST_FILES=    lib/libexpat.so.1 lib/libexpat.so.1.5.2 \
	usr/bin/xmlwf usr/share/man/man1/xmlwf.1.gz
DOCSDIR=        ${PREFIX}/usr/share/doc/${PORTNAME}-${PORTVERSION}
PORTDOCS=       COPYING README
DESCR=          ${.CURDIR}/../${PORTNAME}2/pkg-descr

.include 

and don't forget to edit the distinfo.i386 file as well, as outlined in the mailing list post linked above. Now install the port (I'm simply doing make install clean here, as this is a throw-away VM, but of course you can use portmaster etc.)

root@vm02:/usr/ports/textproc/linux-f10-expat # make install clean
===>   linux-f10-expat-2.0.1_1 depends on file: /usr/local/sbin/pkg - found
=> expat-2.0.1-8.fc10.i386.rpm doesn't seem to exist in /usr/ports/distfiles/rpm/i386/fedora/10.
=> Attempting to fetch http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/10/i386/expat-2.0.1-8.fc10.i386.rpm
expat-2.0.1-8.fc10.i386.rpm                   100% of   82 kB  154 kBps 00m01s
...
... this will download a bunch of patches ...
...

====> Compressing man pages (compress-man)
===>  Installing for linux-f10-expat-2.0.1_1
===>  Checking if textproc/linux-f10-expat already installed
===>   Registering installation for linux-f10-expat-2.0.1_1
Installing linux-f10-expat-2.0.1_1... done
===>  Cleaning for linux_base-f10-10_7
===>  Cleaning for linux-f10-expat-2.0.1_1
root@vm02:/usr/ports/textproc/linux-f10-expat #

Next, download the Linux JDK from http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html. To do that, you'll need to accept the license agreement on the page, then manually download jdk-7u67-linux-i586.tar.gz (it's in the section "Java SE development kit 7u67") and copy it to /usr/ports/distfiles. Be careful to download the x86 one, not the x64! And install it from /usr/ports/java/linux-sun-jdk17

root@vm02:/usr/ports/java/linux-sun-jdk17 # make install clean
....
===>  Cleaning for pkg-1.3.7
===>  Cleaning for javavmwrapper-2.5
===>  Cleaning for linux-f10-fontconfig-2.6.0
===>  Cleaning for linux-f10-xorg-libs-7.4_1
===>  Cleaning for linux-sun-jdk17-7.67
root@vm02:/usr/ports/java/linux-sun-jdk17 #

Next, set the library path so Java finds all the linux libs, and try running java -d32, which should give a helpful message:

root@vm02:~ # setenv LD_LIBRARY_PATH /usr/local/linux-sun-jdk1.7.0/jre/lib/i386/jli
root@vm02:~ # java -d32
Java HotSpot(TM) Client VM warning: Can't detect initial thread stack location - find_vma failed
Usage: java [-options] class [args...]
           (to execute a class)
   or  java [-options] -jar jarfile [args...]
           (to execute a jar file)

We're almost there now! Download the DynamoDB JAR from http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest and extract the contents.

root@vm02:~ # wget http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest

To run DynamoDB local, you will need to disable the linux epoll functions, as these are not implemented in the compat layer yet (see https://wiki.freebsd.org/linux-kernel for all the gory details). Luckily this can be done with a command line switch. Ignore the warning.

root@vm02:~ # java -d32 -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar
Java HotSpot(TM) Client VM warning: Can't detect initial thread stack location - find_vma failed
2014-09-19 18:29:46.922:INFO:oejs.Server:jetty-8.1.12.v20130726
2014-09-19 18:29:46.973:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:8000

Using the awscli tools, you should now be able to connect to DynamoDB local:

root@vm02:~ # aws dynamodb list-tables --endpoint-url http://localhost:8000
{
    "TableNames": []
}

This tutorial was written as part of my work at MyControl