Let Gemnasium auto-update your dependencies

Starting from this week, Gemnasium can update the dependencies of your project automatically. It is now easier than ever to keep your project in shape by getting rid of the outdated dependencies!

The auto-update feature leverages both your CI server and our new Gemnasium client to ensure that the update is compatible with your project. Let Gemnasium do the job, and prepare new dependency files for you!

There are a few checks and a little setup involved before the auto-update is ready. Also, it is worth understanding the underlying process before firing it up. But before diving in, it is good to remember how it was when we had to do it by hand.

Updating by hand

As the auto-update feature is only compatible with Ruby projects so far, let’s go through the manual dependency update using Bundler.

Let’s say we contribute to a Rails application called manydeps. This project has many dependencies including resque for background processing. It also depends on the redis gem to connect to the Redis datastore. manydeps has a good test suite, including acceptance tests that fire both Resque and Redis, for real.

We got an email from Gemnasium telling that a few dependencies are out-of-date. So we go to the project directory and double check using Bundler:

$ bundle outdated
...
Outdated gems included in the bundle:
  ...
  * redis (3.1.0 > 3.0.7) Gemfile specifies "~> 3.0.1"
  * resque (1.25.2 > 1.25.1)
  ...

Updating resque dependency is straightforward: all we have to do is to run bundle update resque. There is one more step involved to update redis as the latest version is “out of range”. 3.1.0 does not satisfy the ~> 3.0.1 requirement. So we edit the Gemfile, set the requirement to ~> 3.1.0, and update the dependency:

# update "redis" requirement to ~> 3.1.0
$ vim Gemfile

# update the "redis" package
$ bundle update redis

That’s it! Well, not quite. Upgrading redis from version 3.0.7 to 3.1.0 looks harmless but we can’t tell for sure. It’s best to run the tests as we’ve got good ones!

# run all the tests
$ bundle exec rake
................................

All good! It took a while but that was both easier and safer than guessing about the possible issues the update may cause.

So Bundler does the job really. But the process we’ve just described is probably not the ideal one. We keep repeating the very same tasks everyday, and each iteration takes a significant amount of time. Some day, we may give up and go away from this tedious process. And months later, we realize that many dependencies are outdated… and that it’s hard to catch up.

Auto-update setup

So updating the dependencies by hand is tedious, time consuming, and even risky. Let’s see how Gemnasium can handle this daily task for you.

First of all, your project has to match a few requirements:

  • it is a Ruby project; support for more languages is on the way
  • it has a Gemfile.lock; this is not a Rubygem
  • the source code is hosted on GitHub
  • it has a good test suite!

Good? Then fire up a browser and get your project ready:

  • go to the “settings” page of your project
  • copy the “project slug” as the client will need it
  • open the “Auto-update” tab and enable the feature

Notice that the auto-update feature takes one slot per project. See the pricing page.

We then move to the Unix shell, install the Gemnasium toolbelt, and set all the environment variables it needs.

Here is the cheat sheet:

# unique project identifier
# see the "settings" page of your project
export GEMNASIUM_PROJECT_SLUG="tech-angels/npmhooks.org"

# user API key, from the user "settings" page
export GEMNASIUM_API_KEY="xxxxxxxxxxxxxxxxxxxx"

# the command to run the test suite
export GEMNASIUM_TESTSUITE="bundle exec rake"

You can also put these in your .gemnasium.yml configuration file.

Auto-update!

Once the setup is done, updating the dependencies is as simple as running gemnasium autoupdate.

Here is an example from a real Rails project we’ve got here at Gemnasium:

$ gemnasium autoupdate
....
Executing update commmand: bundle update sqlite3 omniauth-github redis resque resque-retry
  less-rails uglifier rvm-capistrano mocha resque_unit unicorn airbrake rest-client jquery-rails
Done
Executing test script.........done (9.008456s)
Pushing result (status='test_passed'): done
2 file(s) to be restored.
Restoring file Gemfile: done
Restoring file Gemfile.lock: done
Job done!

After some time, Gemnasium was able to update 16 dependencies of this Rails project.

The new dependency files can be downloaded from the project page, on gemnasium.com. You can either download a patch or download a zip file containing the updated files.

How it works?

Under the hood, the client connects to Gemnasium.com and asks for possible updates. It gets a list of update instructions, execute them, run the tests, and send the result back to the server (passed or failed). Once it gets the result, the server responds with new instructions, the client execute them, etc. The loop is over when the client finds a good “update set” or when there is no solution at all.

Let’s illustrate with the first and last iterations of a run:

$ gemnasium autoupdate
Executing test script: Executing test script.........done (9.007509s)

========= [UpdateSet #1176] =========
Patching Gemfile
Running bundle install
...
Executing test script....done (4.001817s)
Pushing result (status='test_failed'): done
2 file(s) to be restored.
...

========= [UpdateSet #1196] =========
Patching Gemfile
Running bundle install
...
Executing test script.........done (9.008456s)
Pushing result (status='test_passed'): done
2 file(s) to be restored.
Restoring file Gemfile: done
Restoring file Gemfile.lock: done
Job done!

Here the client tried about 20 possibilities before finding a good one.

Notice that the client restores the dependency files after each iteration. This is exactly what we would do if we were to do it manually.

What’s next?

Today the plan is to extend the auto-update feature to support as many projects as possible:

  • Nodejs, Python and PHP projects
  • GitLab hosted and off-line projects

And we’re also working on the auto-update strategy to make it faster, and detect incompatibilities between versions. The more results we get, the more smarter our algorithm will be.

Stay tuned!