Updating a homebrew formula

Below are a few notes regarding the best ways that I’ve found regarding how to go about updating homebrew formula. Now, there are many good resources out there including the official formula cookbook, formula documentation, and how to make a homebrew PR. By no means are these notes meant to replace them. Simply put, they provide an abridged approach to updating or tweaking an existing formula since homebrew docs are a bit of a jungle to navigate.

Case Study: mlpack

In this case, we’ll opt to use the mlpack formula in homebrew/science. In short, the objective of mlpack is to be:

a scalable machine learning library, written in C++, that aims to provide fast, extensible implementations of cutting-edge machine learning algorithms.

Since the formula is housed in a tap instead of the main brew repo, there is a slightly different contributing guide to be aware of.

Fork the repository

The main git repo is hosted by GitHub, so one will also need to setup an account there. Once there, fork the repository:

From there, let’s add a remote and create a branch to hold the experiment.

cd $(brew --repo homebrew/science) # cd $(brew --repo homebrew/core) # if in core

# GH_USERNAME is your github username
git remote add <GH_USERNAME> https://github.com/<GH_USERNAME>/homebrew-science.git # homebrew-core # if core

git checkout master                         # Checkout master branch

brew update                                 # Update brew
brew update                                 # Twice. 
brew upgrade                                # Then upgrade brew

git checkout -b <BRANCH_NAME> origin/master # Create a branch to house contribution

Update brew

Before continuing, make sure to update homebrew twice and then upgrade:

brew update
brew update
brew upgrade

Note: This was done once before in the prior

Check for formula changes

Before modifying the formula, it is highly advisable to see if there are any issues with the present formula. Issues can arise simply because of a style shift or an update to brew internals.

brew audit --strict --online <FORMULA>

Make a note of any errors that come to light.

Access the formula

To edit a brew formula use:

brew edit <FORMULA>

This will open the formula in your preferred editor. By default, brew will use vim.

The next step involves replacing versioning information. If the aim is to update the software version, e.g. v1.1 -> v2.0, update the url and delete both the sha256 and revision (if it exists). We will have to add the sha256 back in later.

Original:

url "http://www.mlpack.org/files/mlpack-2.0.1.tar.gz"
sha256 "87305f7003e060d3c93d60ce1365c4ec0fa7e827c356e857be316b0e54114f22"
revision 3

New:

url "http://www.mlpack.org/files/mlpack-2.1.1.tar.gz"

Save and close the formula. Under vim this is done by pressing Esc + :wq

Two ways to find the sha256 hash of the file. The first way uses brew’s built in method for obtain the SHA256 hash. The second assumes you already have the source archive from the above URL.

Option 1: Using brew to extract the sha256 hash

brew fetch <FORMULA> --build-from-source

Output:

==> Fetching mlpack from homebrew/science
==> Downloading http://www.mlpack.org/files/mlpack-2.1.1.tar.gz
######################################################################## 100.0%
Downloaded to: /Users/agentxyz/Library/Caches/Homebrew/mlpack-2.1.1.tar.gz
SHA256: c2249bbab5686bb8658300ebcf814b81ac7b8050a10f1a517ba5530c58dbac31
Warning: Cannot verify integrity of mlpack-2.1.1.tar.gz
A checksum was not provided for this resource
For your reference the SHA256 is: c2249bbab5686bb8658300ebcf814b81ac7b8050a10f1a517ba5530c58dbac31

Highlight and copy the SHA256 string, e.g. c2249bbab5686bb8658300ebcf814b81ac7b8050a10f1a517ba5530c58dbac31

Option 2: Extracting hash from downloaded file

Or, if you have previously downloaded the source, use:

shasum -a 256 <path/to/download/source>

e.g.

shasum -a 256 ~/Downloads/mlpack-2.1.1.tar.gz

Output

c2249bbab5686bb8658300ebcf814b81ac7b8050a10f1a517ba5530c58dbac31  ~/Downloads/mlpack-2.1.1.tar.gz

Highlight and copy the SHA256 string, e.g. the string on the left c2249bbab5686bb8658300ebcf814b81ac7b8050a10f1a517ba5530c58dbac31.

Combine

Open up the formula again and include the sha256 hash for the new version.

url "http://www.mlpack.org/files/mlpack-2.1.1.tar.gz"
sha256 "c2249bbab5686bb8658300ebcf814b81ac7b8050a10f1a517ba5530c58dbac31"

Note on revision

Now, only add revision back if you did not change the software version. The revision flag simply indicates that the formula updated but the software did not change. Hence, the clients should redownload it.

Fix errors or include new features

If there were any errors that initially came up with formula during the first, now is an ideal time to make the fixes. Alternatively, you could improve the formula by adding in additional options:

Adding Optional Steps

# Add new options
option "with-toad", "Enable toad"             # Opt-in build with, default is to build without.
option "without-mope", "Disable mope"         # Opt-in build without, default is to build with.

# Remove options
deprecated_option "with-check" => "with-test" # Redirect old option
option "with-test", "Run build-time tests"    # Must supply new option

# Use of options
if build.with? "ham"                          # No "with-" necessary
  # Do something if option was set
end

if build.without? "ham"                       # No "without-" needed
  # Do something if option was set
end

Specifying other formulae as dependencies

# New software requirements
depends_on "pkg-config" => :run         # Permenantly available
depends_on "pkg-config" => :build       # Available only during build not availabled if formula is installed by bottle! 
depends_on "pkg-config" => :recommended # Enabled by default, user can disable with --without-pkg-config
depends_on "pkg-config" => :optional    # Automatically generates --with-pkg-config option

Build the formula

Begin the process of installing the new version of the formula.

brew install --verbose --debug <FORMULA>
  • The --verbose flag gives you a copious amount of information regarding what is happening behind the scenes.
  • The --debug flag provides the ability to open a shell to debug where the error occurred.

There is one of flag that may be useful, --build-from-source, however by default when a new sha256 hash is present this should automatically be triggered.

Run built-in tests

Many pre-existing formulas have tests. As a result, its often a good idea to see if the tests still function after installing the new formula:

brew test <FORMULA>

Re-run audit

Re-run the strict audit with the new version information.

brew audit --strict --online <FORMULA>

It is very important that all is well. If a new issue arise, go back into the formula and try to correct it.

Commit change

Now, the world of brew is ready to be bombarded with the potential contribution.

To contribute, the change must go through version control system (VCS) named git.

git add <FORMULA>.rb               # Formula/<FORMULA>.rb            # if in core
git commit                         # Commit changes...

There is a specific formulation of how commit messages must be framed.

In particular, we have the following guidance:

  • the first line is a commit summary of 50 characters or less
  • two (2) newlines, then
  • explain the commit thoroughly

The first lines take the following statues:

  • Version upgrades are only titled by version:
    • mlpack 2.1.1
    • sphynx 0.5.0 (new formula)
  • For slight changes, we have:
    • foo: add test
    • bar: make X11 optional

Push

Next, push the local changes to the fork:

git push --set-upstream <GH_USERNAME> <BRANCH_NAME>

Then, open up a pull request:

Select the appropriate branch from the forked repo to compare to.

Fill out the template and make sure to tick the correct boxes.

Wait for a response. If changes are required, then make the changes. Otherwise, if the PR is closed, learn from the maintainers feedback and try again a later day if it was advised.

Cleanup or I’ve maded a terrible mistake

Once the PR is merged in or if everything has gone down the tube, revert back to the main build.

git checkout -f master
git reset --hard origin/master

Fin

In short, the above guide pulls out useful factoids from the official documentation. The objective behind this guide is to provide quick reference guide as to how to update homebrew. The guide is not sufficient for developing new formula or making substantial modifications.

comments powered by Disqus