Intro

Over the past weekend, I’ve been working on translating R code into Rcpp, a plugin C++ for R, to increase the speed of Markov Chain Monte Carlo (MCMC). There have been quite a few things I’ve learned. From the top: 1.) RStudio is fantastic since not only does it handle R, but it also works splendidly with Rcpp! 2.) Be prepared to write your own operations (i.e. matrix multiplication, solve, etc.) or use RcppArmadillo. 3.) Buzz Lightyear voice, “Compiler errors … everywhere!” 4.) Benchmark, benchmark, and then benchmark. I had such a positive experience with Rcpp, I most likely will be talking about it more in the future. The same cannot be said for initial experience I had with RcppArmadillo, a critical extension package of Rcpp that all Statisticians will want to use.

The Problem

The initial experience I had with RcppArmadillo was problematic since it didn’t work out of the box even with the slew of development tools (XCode) I have installed on macOS. Therefore, I would like to draw attention to a wee bit of a hiccup users can potential experience on macOS starting at OS X Mavericks (OS X 10.9) and going until macOS 10.12 Sierra. Chiefly, the show-stopping errors of “-lgfortran” and “-lquadmath.”

Examples of the error texts in R 3.0 - 3.3.0 are:

“-lgfortran” Error

ld: warning: directory not found for option

'-L/usr/local/lib/gcc/i686-apple-darwin8/4.2.3/x86_64'

ld: warning: directory not found for option '-L/usr/local/lib/x86_64'

ld: warning: directory not found for option
'-L/usr/local/lib/gcc/i686-apple-darwin8/4.2.3'

ld: library not found for -lgfortran

clang: error: linker command failed with exit code 1 (use -v to see
invocation)

“-lquadmath” Error

ld: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'

ld: library not found for -lquadmath 

clang: error: linker command failed with exit code 1 (use -v to see invocation) make: ***
[sourceCpp_93074.so] Error 1 clang++
 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include  -I"/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include"
-I"/Library/Frameworks/R.framework/Versions/3.1/Resources/library/RcppArmadillo/include"
-fPIC  -Wall -mtune=core2 -g -O2  -c rmvrnorm_arma.cpp -o rmvrnorm_arma.o clang++ -dynamiclib -Wl,-headerpad_max_install_names
-undefined dynamic_lookup -single_module -multiply_defined suppress -L/usr/local/lib -o sourceCpp_93074.so rmvrnorm_arma.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2 -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

Examples of the error text in R 3.4.0 would be:

ld: warning: directory not found for option

'-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'

ld: warning: directory not found for option '-L/usr/local/gfortran/lib'

ld: warning: directory not found for option
'-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'

ld: library not found for -lgfortran

clang: error: linker command failed with exit code 1 (use -v to see
invocation)

The good news is that both errors originate from the same problem. Mainly, R for macOS was compiled using either gfortran-4.8 (R 3.0.0 - R 3.3.0) or gfortran-6.1 (R 3.4.0). What does this mean?

For starters, the official cran mirror of R tools for OS X and R tools for OS X on r.research.att.com that lists the gfortran binary are out of date. However, obtaining the correct gfortran binary is slightly different depending on your version of R. The reason for this is R 3.4.0 began using OpenMP whereas prior versions omitted it.

In particular, for those users on R 3.0.0 - R 3.3.0, the official gfortran binaries break the compile chain because the binaries listed were not used to compile R. As a result, the using the gfortran will lead you to switch from receiving a “-lgfortran” error to a “-lquadmath” error since a few files are missing from the expected locations.

For R 3.4.0 and later versions of R, you must obtain the official gfortran binary.

The Solution

Please note, the solution has changed to be dependent on the R version being run.

R 3.0.0 - R 3.3.0

Go to the optional libraries, frameworks and applications for Mac OS X on r.research.att.com and download gfortran-4.8.2-darwin13.tar.bz2. Extract the package in /, which is root. The files should be unpacked into /usr/local/ (see the appendix for file structure).

Alternatively, open terminal and type:

curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
sudo tar fvxz gfortran-4.8.2-darwin13.tar.bz2 -C /

Removing R’s gfortran-4.8.2 binary

With the transition to R 3.4.0 requiring a different set of binaries, you may wish to remove the above binary to free up space. The best way to go about it is by running a bash script that first deletes files found in the archive and then attempts to remove the directories containing the files (assuming they are empty). The reason for this approach is this binary was designed at a time when code was integrated tightly with macOS routines.

# Download installer into working directory
curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2

# Remove _files_ associated with the binary
for file in $(tar tfz gfortran-4.8.2-darwin13.tar.bz2); do
   sudo rm -f /$file; 
done

# Remove empty _folders_ associated with the binary
for file in $(tar tfz gfortran-4.8.2-darwin13.tar.bz2); do 
   sudo rmdir -p /$file; 
done

# Delete the installer
rm -rf gfortran-4.8.2-darwin13.tar.bz2

R 3.4.0

Obtain the official gfortran 6.1.0 binary for OS X El Capitan (10.11) from https://gcc.gnu.org/wiki/GFortranBinaries. Run the downloaded installer and rejoice!

Note: This advice holds even if you are not on El Capitan (10.11) but are on the macOS Sierra (10.12) or later macOS versions. So, please only use gfortran 6.1.0 with R 3.4.0!

Later

Check back when new versions of R are released.

Fin

Happy Rcpp and RcppArmadillo ‘in on OS X Mavericks through macOS!

Appendix

To view the file structure of the gfortran binaries from http://r.research.att.com/libs/ we can extract it from the archive using:

## Assumes homebrew is installed
# brew install tree 
curl -O ~/Downloads/gfortran-4.8.2-darwin13.tar.bz2  http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
tar fvxz gfortran-4.8.2-darwin13.tar.bz2
tree usr

Extracted directory structure:

usr
└── local
    ├── bin
    │   ├── gfortran -> gfortran-4.8
    │   ├── gfortran-4.8 -> x86_64-apple-darwin13.0.0-gfortran
    │   └── x86_64-apple-darwin13.0.0-gfortran
    ├── lib
    │   ├── gcc
    │   │   └── x86_64-apple-darwin13.0.0
    │   │       └── 4.8.2
    │   │           ├── crt3.o
    │   │           ├── crtfastmath.o
    │   │           ├── crtprec32.o
    │   │           ├── crtprec64.o
    │   │           ├── crtprec80.o
    │   │           ├── crttme.o
    │   │           ├── crttms.o
    │   │           ├── finclude
    │   │           │   ├── omp_lib.f90
    │   │           │   ├── omp_lib.h
    │   │           │   ├── omp_lib.mod
    │   │           │   └── omp_lib_kinds.mod
    │   │           ├── i386
    │   │           │   ├── crt3.o
    │   │           │   ├── crtfastmath.o
    │   │           │   ├── crtprec32.o
    │   │           │   ├── crtprec64.o
    │   │           │   ├── crtprec80.o
    │   │           │   ├── crttme.o
    │   │           │   ├── crttms.o
    │   │           │   ├── libcaf_single.a
    │   │           │   ├── libcaf_single.la
    │   │           │   ├── libgcc.a
    │   │           │   ├── libgcc_eh.a
    │   │           │   ├── libgcov.a
    │   │           │   ├── libgfortranbegin.a
    │   │           │   └── libgfortranbegin.la
    │   │           ├── include
    │   │           │   ├── adxintrin.h
    │   │           │   ├── ammintrin.h
    │   │           │   ├── avx2intrin.h
    │   │           │   ├── avxintrin.h
    │   │           │   ├── bmi2intrin.h
    │   │           │   ├── bmiintrin.h
    │   │           │   ├── bmmintrin.h
    │   │           │   ├── cpuid.h
    │   │           │   ├── cross-stdarg.h
    │   │           │   ├── emmintrin.h
    │   │           │   ├── f16cintrin.h
    │   │           │   ├── float.h
    │   │           │   ├── fma4intrin.h
    │   │           │   ├── fmaintrin.h
    │   │           │   ├── fxsrintrin.h
    │   │           │   ├── ia32intrin.h
    │   │           │   ├── immintrin.h
    │   │           │   ├── iso646.h
    │   │           │   ├── lwpintrin.h
    │   │           │   ├── lzcntintrin.h
    │   │           │   ├── mm3dnow.h
    │   │           │   ├── mm_malloc.h
    │   │           │   ├── mmintrin.h
    │   │           │   ├── nmmintrin.h
    │   │           │   ├── omp.h
    │   │           │   ├── pmmintrin.h
    │   │           │   ├── popcntintrin.h
    │   │           │   ├── prfchwintrin.h
    │   │           │   ├── quadmath.h
    │   │           │   ├── quadmath_weak.h
    │   │           │   ├── rdseedintrin.h
    │   │           │   ├── rtmintrin.h
    │   │           │   ├── smmintrin.h
    │   │           │   ├── ssp
    │   │           │   │   ├── ssp.h
    │   │           │   │   ├── stdio.h
    │   │           │   │   ├── string.h
    │   │           │   │   └── unistd.h
    │   │           │   ├── stdalign.h
    │   │           │   ├── stdarg.h
    │   │           │   ├── stdbool.h
    │   │           │   ├── stddef.h
    │   │           │   ├── stdfix.h
    │   │           │   ├── stdint-gcc.h
    │   │           │   ├── stdint.h
    │   │           │   ├── stdnoreturn.h
    │   │           │   ├── tbmintrin.h
    │   │           │   ├── tgmath.h
    │   │           │   ├── tmmintrin.h
    │   │           │   ├── unwind.h
    │   │           │   ├── varargs.h
    │   │           │   ├── wmmintrin.h
    │   │           │   ├── x86intrin.h
    │   │           │   ├── xmmintrin.h
    │   │           │   ├── xopintrin.h
    │   │           │   ├── xsaveintrin.h
    │   │           │   ├── xsaveoptintrin.h
    │   │           │   └── xtestintrin.h
    │   │           ├── include-fixed
    │   │           │   ├── README
    │   │           │   ├── limits.h
    │   │           │   ├── math.h
    │   │           │   ├── stdint.h
    │   │           │   └── syslimits.h
    │   │           ├── install-tools
    │   │           │   ├── fixinc_list
    │   │           │   ├── gsyslimits.h
    │   │           │   ├── include
    │   │           │   │   ├── README
    │   │           │   │   └── limits.h
    │   │           │   ├── macro_list
    │   │           │   └── mkheaders.conf
    │   │           ├── libcaf_single.a
    │   │           ├── libcaf_single.la
    │   │           ├── libgcc.a
    │   │           ├── libgcc_eh.a
    │   │           ├── libgcov.a
    │   │           ├── libgfortranbegin.a
    │   │           ├── libgfortranbegin.la
    │   │           └── plugin
    │   │               ├── gtype.state
    │   │               └── include
    │   │                   ├── ada
    │   │                   │   └── gcc-interface
    │   │                   │       └── ada-tree.def
    │   │                   ├── alias.h
    │   │                   ├── all-tree.def
    │   │                   ├── alloc-pool.h
    │   │                   ├── ansidecl.h
    │   │                   ├── auto-host.h
    │   │                   ├── b-header-vars
    │   │                   ├── basic-block.h
    │   │                   ├── bitmap.h
    │   │                   ├── builtins.def
    │   │                   ├── bversion.h
    │   │                   ├── c-family
    │   │                   │   ├── c-common.def
    │   │                   │   ├── c-common.h
    │   │                   │   ├── c-objc.h
    │   │                   │   ├── c-pragma.h
    │   │                   │   └── c-pretty-print.h
    │   │                   ├── c-tree.h
    │   │                   ├── cfg-flags.def
    │   │                   ├── cfghooks.h
    │   │                   ├── cfgloop.h
    │   │                   ├── cgraph.h
    │   │                   ├── cif-code.def
    │   │                   ├── config
    │   │                   │   ├── darwin-protos.h
    │   │                   │   ├── darwin-sections.def
    │   │                   │   ├── darwin.h
    │   │                   │   ├── darwin10.h
    │   │                   │   ├── darwin9.h
    │   │                   │   ├── i386
    │   │                   │   │   ├── biarch64.h
    │   │                   │   │   ├── darwin.h
    │   │                   │   │   ├── darwin64.h
    │   │                   │   │   ├── i386-opts.h
    │   │                   │   │   ├── i386-protos.h
    │   │                   │   │   └── i386.h
    │   │                   │   ├── initfini-array.h
    │   │                   │   └── vxworks-dummy.h
    │   │                   ├── config.h
    │   │                   ├── configargs.h
    │   │                   ├── coretypes.h
    │   │                   ├── cp
    │   │                   │   ├── cp-tree.def
    │   │                   │   ├── cp-tree.h
    │   │                   │   ├── cxx-pretty-print.h
    │   │                   │   └── name-lookup.h
    │   │                   ├── cppdefault.h
    │   │                   ├── cpplib.h
    │   │                   ├── debug.h
    │   │                   ├── defaults.h
    │   │                   ├── diagnostic-core.h
    │   │                   ├── diagnostic.def
    │   │                   ├── diagnostic.h
    │   │                   ├── double-int.h
    │   │                   ├── dumpfile.h
    │   │                   ├── emit-rtl.h
    │   │                   ├── except.h
    │   │                   ├── filenames.h
    │   │                   ├── fixed-value.h
    │   │                   ├── flag-types.h
    │   │                   ├── flags.h
    │   │                   ├── function.h
    │   │                   ├── gcc-plugin.h
    │   │                   ├── genrtl.h
    │   │                   ├── ggc.h
    │   │                   ├── gimple-pretty-print.h
    │   │                   ├── gimple.def
    │   │                   ├── gimple.h
    │   │                   ├── gsstruct.def
    │   │                   ├── gtm-builtins.def
    │   │                   ├── gtype-desc.h
    │   │                   ├── hard-reg-set.h
    │   │                   ├── hashtab.h
    │   │                   ├── highlev-plugin-common.h
    │   │                   ├── hwint.h
    │   │                   ├── incpath.h
    │   │                   ├── input.h
    │   │                   ├── insn-constants.h
    │   │                   ├── insn-flags.h
    │   │                   ├── insn-modes.h
    │   │                   ├── insn-notes.def
    │   │                   ├── internal-fn.def
    │   │                   ├── internal-fn.h
    │   │                   ├── intl.h
    │   │                   ├── ipa-prop.h
    │   │                   ├── ipa-ref-inline.h
    │   │                   ├── ipa-ref.h
    │   │                   ├── ipa-reference.h
    │   │                   ├── ipa-utils.h
    │   │                   ├── is-a.h
    │   │                   ├── java
    │   │                   │   └── java-tree.def
    │   │                   ├── langhooks.h
    │   │                   ├── libiberty.h
    │   │                   ├── line-map.h
    │   │                   ├── machmode.h
    │   │                   ├── md5.h
    │   │                   ├── mode-classes.def
    │   │                   ├── objc
    │   │                   │   └── objc-tree.def
    │   │                   ├── obstack.h
    │   │                   ├── omp-builtins.def
    │   │                   ├── options.h
    │   │                   ├── opts.h
    │   │                   ├── output.h
    │   │                   ├── params.def
    │   │                   ├── params.h
    │   │                   ├── plugin-api.h
    │   │                   ├── plugin-version.h
    │   │                   ├── plugin.def
    │   │                   ├── plugin.h
    │   │                   ├── pointer-set.h
    │   │                   ├── predict.def
    │   │                   ├── predict.h
    │   │                   ├── prefix.h
    │   │                   ├── pretty-print.h
    │   │                   ├── real.h
    │   │                   ├── realmpfr.h
    │   │                   ├── reg-notes.def
    │   │                   ├── rtl.def
    │   │                   ├── rtl.h
    │   │                   ├── safe-ctype.h
    │   │                   ├── sanitizer.def
    │   │                   ├── sbitmap.h
    │   │                   ├── splay-tree.h
    │   │                   ├── statistics.h
    │   │                   ├── symtab.h
    │   │                   ├── sync-builtins.def
    │   │                   ├── system.h
    │   │                   ├── target-hooks-macros.h
    │   │                   ├── target.def
    │   │                   ├── target.h
    │   │                   ├── timevar.def
    │   │                   ├── timevar.h
    │   │                   ├── tm-preds.h
    │   │                   ├── tm.h
    │   │                   ├── tm_p.h
    │   │                   ├── toplev.h
    │   │                   ├── tree-check.h
    │   │                   ├── tree-dump.h
    │   │                   ├── tree-flow-inline.h
    │   │                   ├── tree-flow.h
    │   │                   ├── tree-inline.h
    │   │                   ├── tree-iterator.h
    │   │                   ├── tree-pass.h
    │   │                   ├── tree-pretty-print.h
    │   │                   ├── tree-ssa-alias.h
    │   │                   ├── tree-ssa-operands.h
    │   │                   ├── tree-ssa-sccvn.h
    │   │                   ├── tree.def
    │   │                   ├── tree.h
    │   │                   ├── treestruct.def
    │   │                   ├── vec.h
    │   │                   └── version.h
    │   ├── libgcc_ext.10.4.dylib
    │   ├── libgcc_ext.10.5.dylib
    │   ├── libgcc_s.1.dylib -> libgcc_s_x86_64.1.dylib
    │   ├── libgcc_s_x86_64.1.dylib
    │   ├── libgfortran.3.dylib
    │   ├── libgfortran.a
    │   ├── libgfortran.dylib -> libgfortran.3.dylib
    │   ├── libgfortran.la
    │   ├── libgfortran.spec
    │   ├── libgomp.1.dylib
    │   ├── libgomp.a
    │   ├── libgomp.dylib -> libgomp.1.dylib
    │   ├── libgomp.la
    │   ├── libgomp.spec
    │   ├── libquadmath.0.dylib
    │   ├── libquadmath.a
    │   ├── libquadmath.dylib -> libquadmath.0.dylib
    │   └── libquadmath.la
    ├── libexec
    │   └── gcc
    │       └── x86_64-apple-darwin13.0.0
    │           └── 4.8.2
    │               ├── cc1
    │               ├── collect2
    │               ├── f951
    │               ├── lto-wrapper
    │               └── plugin
    │                   └── gengtype
    └── share
        └── man
            ├── man1
            │   └── gfortran.1
            └── man7
                ├── fsf-funding.7
                ├── gfdl.7
                └── gpl.7