Making Erlang package management easy with EPM

February 26, 2010

Categories: erlang, epm

I don't like complicated tools. When it comes to installing software, regardless of the OS or type of package I'm installing, I want the same interface. There should be commands for installing, removing, updating and searching. This is how APT, Yum, and RubyGems work. Now you can add EPM to that list. EPM is meant to help us Erlang folks setup and maintain our development environments with minimal effort and with a minimally invasive effect on our projects.

» http://github.com/JacobVorreuter/epm

A Quick Look

jvorreuter$ ./epm install dynamic_compile
epm v0.1.0, 2010

===============================
Install the following packages?
===============================
    + JacobVorreuter-dynamic_compile-master

[y/n] y

+ downloading http://github.com/JacobVorreuter/dynamic_compile/tarball/master
+ running dynamic_compile build command
+ running dynamic_compile install command


jvorreuter$ ./epm info dynamic_compile
epm v0.1.0, 2010

===============================
INSTALLED
===============================
  name: dynamic_compile
  owner: JacobVorreuter
  vsn: master
  install dir: /Users/jvorreuter/dev/dynamic_compile-0.1
  homepage: 
  description: compile and load erlang modules from string input


jvorreuter$ ./epm remove dynamic_compile
epm v0.1.0, 2010

===============================
Remove the following packages?
===============================
    + JacobVorreuter-dynamic_compile-master

[y/n] y

+ removing package JacobVorreuter-dynamic_compile-master from /Users/jvorreuter/dev/dynamic_compile-0.1

Download

EPM is an executable escript. It has no dependencies other than Erlang. All you need to do to use it is download it:

curl "http://github.com/JacobVorreuter/epm/raw/master/epm" > epm
chmod +x epm

EPM uses GitHub

Currently, EPM only supports packages hosted on GitHub. This will probably change in the near future. Fortunately, nearly all popular Erlang apps are already on GitHub or are mirrored there. The advantage of using GitHub is that EPM is not responsible for hosting its own repository that would require maintaining.

The EPM GitHub user is given precedence

Some projects have been forked under the EPM GitHub account. If you install a project that has been forked by EPM and do not specify a user, the EPM repo will take precedence. Projects that were forked by EPM were most likely done so because they lacked an app file or did not properly list their dependencies. If you encounter a project on GitHub or even google code that does not conform to the EPM standards, feel free to let me know and I'll fork it under the EPM account.

Sometimes GitHub sucks

EPM relies on the GitHub API, which is prone to timeouts or sometimes worse. When GitHub is having problems then EPM is having problems.

Occasionally something like this will happen:

./epm install mochiweb_server_behavior
epm v0.1.0, 2010

- failed to locate remote repo for mochiweb_server_behavior

This is most likely because the GitHub API call failed. Try it again and it will probably work.

EPM is non-invasive

You don't have to change the structure of your project for it to be accessible to EPM. By default EPM assumes that a given project has no dependencies and can be built by running make in the project's root directory.

Determining dependencies

If a project relies on other projects it must list them in its epm file. EPM will look for a file named ProjectName.epm in the project's root directory. This file has the following structure:

excavator.epm:

[{deps, [
    {"clones/mochiweb", []},
    {"mochixpath", []},
    {"dynamic_compile", []},
    {"epm/etap", []},
    {"mochiweb_server_behavior", []}
]}].

The argument list specifies project arguments, such as what version to install. Valid version arguments are tag, branch and sha. If the argument list is left empty the default value of {branch, "master"} is used.

example.epm:

[{deps, [
    {"mochiweb", [{tag, "0.01"}]},
    {"log_roller", [{branch, "stable"}]},
    {"excavator", [{sha, "040092db3caa221bd0322430b580cb9f3b6ef919"}]}
]}].

Building and installing projects

The default behavior of EPM is to execute make in a project's root directory. Then the entire project directory is copied to your install location. This can be seen more clearly by including the --verbose tag when installing a package:

jvorreuter$ ./epm install emongo --verbose
epm v0.1.0, 2010

===============================
Packages already installed:
===============================
    + epm-etap-master (0.3.4)
===============================
Install the following packages?
===============================
    + JacobVorreuter-emongo-master

[y/n] y

+ downloading http://github.com/JacobVorreuter/emongo/tarball/master
+ running emongo build command
    make
    sh ebin/emongo.app.in 0.0.1
    erl -make
    Recompile: src/emongo_packet
    Recompile: src/emongo_conn
    Recompile: src/emongo_bson
    Recompile: src/emongo_app
    Recompile: src/emongo
+ running emongo test command
    make test
    sh ebin/emongo.app.in 0.0.1
    erl -make
    prove t/*.t
    t/001-load..........ok
    t/002-bson..........ok
    t/003-find..........ok
    t/004-cond-exprs....ok
    All tests successful.
    Files=4, Tests=65,  1 wallclock secs ( 0.60 cusr +  0.15 csys =  0.75 CPU)
+ running emongo install command
    mkdir -p /Users/jvorreuter/dev/emongo-0.0.1; cp -R ./* /Users/jvorreuter/dev/emongo-0.0.1

emongo.epm looks like this:

[ 
  {deps, [
    {"etap", []}
  ]},
  {test_command, "make test"}
].

make test is specified in the epm config file as the test command to run as part of the build process. The default make command is used to compile the code. The install command copies the project files and directories into an app directory in my ~/dev directory. This happens because I've set a custom install location in my ~/.epm file.

Customizing EPM's behavior with a ~/.epm file

my ~./epm file looks like this:

[  {build_dir, "/tmp"},
   {install_dir, "/Users/jvorreuter/dev"} ].

This file is optional. If it is not present then the following values will be used by EPM:

{build_dir, "./"}
{install_dir, code:lib_dir()}

Override the default install_dir to avoid running sudo epm

The default install_dir is the value of code:lib_dir(), which on my machine is /usr/local/lib/erlang/lib and is owned by the root user. This means that epm would not have write access to that directory unless run as the super user.

Separate third-party apps from core libs and OTP apps

I recommend installing third-party apps in a location other than the value of code:lib_dir(). This will allow you to more easily manage these apps and keep your Erlang installation clean. On my machine I set the ERL_LIBS environmental variable to the same directory that I specify as my EPM install_dir:

export ERL_LIBS=/Users/jvorreuter/dev

This allows me to run epm not as the root user and it keeps the apps I install with EPM separate from system libs.

Specifying custom build commands on the command line

EPM allows you to override the prebuild, build and test commands as project args on the command line.

emongo doesn't have a configure script so this breaks:

jvorreuter$ ./epm install emongo --prebuild-command "./configure"
epm v0.1.0, 2010

...
+ downloading http://github.com/JacobVorreuter/emongo/tarball/master
+ running emongo prebuild command
- /bin/sh: line 1: ./configure: No such file or directory

now we try a non-existent make target as the build command:

jvorreuter$ ./epm install emongo --build-command "make bacon"
epm v0.1.0, 2010

...
+ downloading http://github.com/JacobVorreuter/emongo/tarball/master
+ running emongo build command
- make: *** No rule to make target `bacon'.  Stop.

EPM Usage

install

Several packages can be installed at once by chaining package names together:

./epm install log_roller ibrowse etap

Also project owners can be included in the project names:

./epm install JacobVorreuter/log_roller cmullaparthi/ibrowse ngerakines/etap

Arguments can be associated with each project by placing them after the project name:

./epm install log_roller --tag 0.3 ngerakines/etap --branch stable

Non package specific tags can be included at the end of the epm command:

./epm install log_roller --tag 0.3 --verbose

By default, dependencies will be installed with packages. The --without-deps tag will prevent depedencies from being installed:

./epm install log_roller --without-deps

remove

The remove command has similar options to install. All packages that require the package being removed will also be removed:

./epm remove log_roller etap
./epm remove JacobVorreuter/log_roller
./epm remove JacobVorreuter/log_roller --tag 0.3
./epm remove log_roller --verbose

By default package dependencies will not be removed with a package. However, the --with-deps tag will remove child dependencies at the same time:

./epm remove log_roller --with-deps

info

The info command displays info about either installed or available packages:

./epm info emongo etap


===============================
INSTALLED
===============================
  name: etap
  owner: epm
  vsn: master
  install dir: /Users/jvorreuter/dev/etap-0.3.4
  homepage: 
  description: etap is a simple erlang testing library that provides TAP compliant output.

===============================
AVAILABLE
===============================
  name: emongo
  owner: JacobVorreuter
  followers: 25
  homepage: 
  description: the most Emo of mongo drivers
  branches:
    master

list

The list command displays info about all installed packages

./epm list


===============================
INSTALLED
===============================
  name: excavator
  owner: JacobVorreuter
  vsn: master
  install dir: /Users/jvorreuter/dev/excavator-0.3
  homepage: 
  description: An Erlang application for ingesting data from various sources (APIs, data feeds, web content, etc)
  dependencies: 
    clones/mochiweb/master
    mochixpath/master
    dynamic_compile/master
    epm/etap/master
    mochiweb_server_behavior/master

  name: mochixpath
  owner: JacobVorreuter
  vsn: master
  install dir: /Users/jvorreuter/dev/mochixpath-0.1
  homepage: 
  description: Mochiweb html parser xpath extension

  name: dynamic_compile
  owner: JacobVorreuter
  vsn: master
  install dir: /Users/jvorreuter/dev/dynamic_compile-0.1
  homepage: 
  description: compile and load erlang modules from string input

  name: mochiweb_server_behavior
  owner: JacobVorreuter
  vsn: master
  install dir: /Users/jvorreuter/dev/mochiweb_server_behavior-0.1
  homepage: 
  description: Erlang behavior for a simple mochiweb web server
  dependencies: 
    clones/mochiweb/master

  name: etap
  owner: epm
  vsn: master
  install dir: /Users/jvorreuter/dev/etap-0.3.4
  homepage: 
  description: etap is a simple erlang testing library that provides TAP compliant output.

  name: mochiweb
  owner: clones
  vsn: master
  install dir: /Users/jvorreuter/dev/mochiweb-0.01
  homepage: http://code.google.com/p/mochiweb/
  description: mochiweb clone

search

The search command searches the remote repository for available packages:

./epm search ibrowse


===============================
AVAILABLE
===============================
  name: ibrowse
  owner: cmullaparthi
  followers: 47
  homepage: 
  description: Erlang HTTP client
  branches:
    master

  name: erl-couch
  owner: bdionne
  followers: 5
  homepage: 
  description: erlang couchdb client based on ibrowse
  branches:
    master

  name: ibrowse
  owner: cstar
  followers: 1
  homepage: 
  description: 
  tags:
    1.4.1-json
    1.4.1
    1.5.3-json
  branches:
    master

update

The update command re-installs the currently installed package

./epm update emongo

===============================
Update the following packages?
===============================
    + JacobVorreuter-emongo-master

([y]/n) y

+ downloading http://github.com/JacobVorreuter/emongo/tarball/master
+ running emongo build command
+ running emongo test command
+ running emongo install command

latest

The latest command downloads the latest version of epm and replaces the existing executable

jvorreuter$ ./epm latest

+ updated epm (./epm) to latest version

EPM is not a packaging and deployment solution

EPM is meant to be used for local development. It does not include functionality for packaging OTP releases or deploying releases.

EPM is beta software

Please give me input as to how to improve EPM and make it more useful.

» http://github.com/JacobVorreuter/epm

blog comments powered by Disqus