GNU Libs with Debug Information: Rebuilding the GNU ARM Libraries

with my diy tool chain (see “ constructing a classroom ide with eclipse for arm “) i get a complete tool chain. i do not need to build that tool chain from the sources for windows, as all the binaries are nicely pre-compiled and made available. but there is one issue i face from time to time: as the libraries provided by arm do not come with sources and debug information enabled, i end up with that “no source available for …” message in the debugger:

the solution is to grab the c/c++ library sources from the arm launchpad site and get it built locally the way i need it.

building the libraries on windows

i’m rebuilding the libraries on my windows 7 64bit machine. unfortunately, i have not found any good steps or tutorials how to do this anywhere (that’s why i’m documenting my findings here, otherwise i simply could point to an existing tutorial).while it would (or should?) be possible to re-build all the binaries arm provided, i’m only interested in building the newlib and newlib-nano libraries, so i can have debug and source information enabled.

actually it seems that doing things like this on a windows machine is even more challenging that on linux: the make files and scripts are all tuned for a linux system, so to have it built on my machine, some extra steps and tools are needed. oh well….

setup of tools and environments

the first step is to setup the environment and to download and install the tools and files needed.

setup of arm-none-eabi tools

because i want to build the libraries with the provided arm-none-eabi tools (compiler, linker, etc), i need to make sure they are found by the build process.

:idea: usually i do *not* want the gnu binaries in my global system path. setting global environment variables is a road to disaster in my view. with my eclipse tool environment, i reference the tool chain locally from my ide/tools folder. oh well, it looks like at least for building the gnu libraries i have to break with my principles yet again :-(

mingw and msys

the build process is assuming a linux environment and utilities. to run it on a windows host, the corresponding environment needs to be installed. this is mingw (minimalist gnu for windows) and msys (a set of shell utilities). the build process needs to run from the mingw shell, and cannot be run from a normal windows dos prompt.

go to http://www.mingw.org/ and download the installer (links on the left hand side):

in the mingw installation manager, make sure these tools are installed (marked with green box):

we need to use the mingw/msys shell later on, and it needs to be run from a shortcut. so from the msys.bat, create a short cut on the desktop or in the start menu:

later i will run the shell from that shortcut:

the documentation about msys can be found at http://www.mingw.org/wiki/msys

system path

i have the path where i have the compiler (arm-none-eabi-gcc.exe) added to the end of my path system variable. for my diy ide this isc:\tools\ide\gcc\bin

c:\tools\ide\gcc\bin

arm-none-eabi-cc

the next thing is a weird thing: somehow the library build process is looking for a arm-none-eabi-cc.exe. if not found, it will report an error later in the build process that arm-none-eabi.cc.exe has not been found.

others have reported this problem too (see http://stackoverflow.com/questions/3660672/getting-error-on-compiling-newlib ). to solve that problem, i have copied arm-none-eabi-gcc.exe and named it arm-none-eabi-cc.exe.

installing the library sources

from https://launchpad.net/gcc-arm-embedded , download the source package (at this time it is gcc-arm-none-eabi-4_8-2014q2-20140609-src.tar.bz2), then unpack it.

in this tutorial, i have unpacked the files into a folder named ‘lib_build’ inside my tool chain:

c:\tools\ide\gcc\lib_build\gcc-arm-none-eabi-4_8-2014q2-20140609

there are multiple zip files in it, but i only need the newlib-nano-2.1.tar.bz2 and the newlib.tar.bz2 unpacked. i have them unpacked into the ‘src’ folder (see above screenshot).

patching the library build

unfortunately, the build does not work out of the box (at least under windows). i have found two issues which need to be fixed to run the build successfully:

1. need to change line endings for makeinfo in a .texi file
2. fix the make file rules in makefile.in file

fix for makeinfo

on windows 64bit system there seems to be a long-standing issue that makeinfo fails with:

unknown index 'fn' and/or 'cp' in ...

makeinfo creates documentation, and seem failing to read library info (while it works on linux). see https://sourceware.org/bugzilla/show_bug.cgi?id=14678 for details on this bug. the problem is very likely because makeinfo has a problem with windows line endings. open the file

src\newlib-nano-2.1\etc\standards.texi

in eclipse and convert the file line delimiters to unix style, then save the file:

do the same thing for the newlib file

src\newlib\etc\standards.texi

patching the library makefile.in file

there is a problem with the make file rules in the libgloss part:

*** no rule to make target '../../../../../newlib-nano-2.1/libgloss/arm/../config/default.hm', needed by 'makefile'. stop.

the workaround (see this discussion ) is to patch the ‘makefile.in’ file:

src\newlib-nano-2.1\libgloss\arm\cpu-init\makefile.in

the patch details:

from: agustin henze <tin@debian.org>
date: fri, 3 jan 2014 11:29:55 -0300
subject: [patch] fix wrong path to libgloss/config/default.mh

---
 libgloss/arm/cpu-init/makefile.in | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libgloss/arm/cpu-init/makefile.in b/libgloss/arm/cpu-init/makefile.in
index 547c58d..d63149f 100644
--- a/libgloss/arm/cpu-init/makefile.in
+++ b/libgloss/arm/cpu-init/makefile.in
@@ -18,6 +18,7 @@ libdir = @libdir@
 tooldir = $(exec_prefix)/$(target_alias)
 
 objtype = @objtype@
+host_makefile_frag = $(srcdir)/../../config/default.mh
 
 install = @install@
 install_program = @install_program@
@@ -80,7 +81,7 @@ info:
 install-info:
 clean-info:
 
-makefile: makefile.in ../config.status @host_makefile_frag_path@
+makefile: makefile.in ../config.status ${host_makefile_frag_path}
     $(shell) ../config.status --file cpu-init/makefile
 
 ../config.status: ../configure
-- 
1.8.5.2

either apply the patch file, or open the makefile.in in an editor and add:

host_makefile_frag = $(srcdir)/../../config/default.mh

down further the file, change the makefile line to:

makefile: makefile.in ../config.status ${host_makefile_frag_path}

makefile.in patch part 2

do the same for this make file in the newlib library sources:

src\newlib\libgloss\arm\cpu-init\makefile.in

with the two patches applied to both the newlib-nano and newlib, i’m able to build the libraries in the next step.

building the libraries

building each of the libraries (newlib and newlib-nano) is a multi-step process:

1. configure the build
2. run the make file to build
3. run the make file to install/copy the files

the question is: what are the correct parameters to configure the build?

the answer is: have a look at the build-toolchain.sh provided in the source package, and locate the parameters for the newlib-nano and newlib packages:

i have created two shell scripts which will do the necessary calls for me, but you can execute the commands one-by-one on the shell too:

this is the build script for the newlib-nano:

# script to build newlib and newlib-nano
# important:
# - make sure the cross-tool chain is present in the path
# - untar newlib-nano-2.1.tar.tar.bz2
# - untar newlib.tar.bz2
 
target=arm-none-eabi
root=/c/tools/ide/gcc/lib_build/
builddir_native=$root/install-native.newlibnano
 
echo creating build directory...
mkdir newlib-nano.build
 
echo entering build directory...
cd newlib-nano.build
 
# run configure program to configure the build
echo configuring newlib-nano build...
../gcc-arm-none-eabi-4_8-2014q2-20140609/src/newlib-nano-2.1/configure \
  --target=$target \
  --prefix=$builddir_native/target-libs \
  --disable-newlib-supplied-syscalls \
  --enable-newlib-reent-small \
  --disable-newlib-fvwrite-in-streamio \
  --disable-newlib-fseek-optimization \
  --disable-newlib-wide-orient \
  --enable-newlib-nano-malloc \
  --disable-newlib-unbuf-stream-opt \
  --enable-lite-exit \
  --enable-newlib-global-atexit \
  --disable-nls
 
echo building newlib-nano...
make -j
 
echo installing...
make install

and the same for the newlib:

# script to build newlib
# important:
# - make sure the cross-tool chain is present in the path
# - untar newlib.tar.bz2
 
target=arm-none-eabi
root=/c/tools
installdir_native=$root/install-native.newlib
installdir_native_doc=$installdir_native/share/doc/gcc-arm-none-eabi
 
echo creating build directory...
mkdir newlib.build
 
echo entering build directory...
cd newlib.build
 
# run configure program to configure the build
echo configuring newlib build...
../gcc-arm-none-eabi-4_8-2014q2-20140609/src/newlib/configure \
  --target=$target \
  --prefix=$installdir_native \
  --infodir=$installdir_native_doc/info \
  --mandir=$installdir_native_doc/man \
  --htmldir=$installdir_native_doc/html \
  --pdfdir=$installdir_native_doc/pdf \
  --enable-newlib-io-long-long \
  --enable-newlib-register-fini \
  --disable-newlib-supplied-syscalls \
  --disable-nls
 
#echo building newlib-nano...
make -j
 
#echo installing...
make install

to build the libraries, launch the msys.bat with the shortcut previously created, and cd (change directory) into the folder we have unpacked the source package and run the above shell scripts, or execute the commands on the command line.

if doing it directly on the command line, i show it here for the newlib-nano.

create a folder ‘newlib-nano.build’

mkdir newlib-nano.build

cd (change directory) into that folder.

cd newlib-nano.build

run the configure command:

../newlib-nano-2.1/configure --target=arm-none-eabi --prefix=/c/tools/newlib-nano-arm-none-eabi --disable-newlib-supplied-syscalls --enable-newlib-reent-small --disable-newlib-fvwrite-in-streamio --disable-newlib-fseek-optimization --disable-newlib-wide-orient --enable-newlib-nano-malloc --disable-newlib-unbuf-stream-opt --enable-lite-exit --enable-newlib-global-atexit --disable-nls

then do a make witih

make -j

and finally install it with

make install

the needed configuration options can be taken from the ‘build-toolchain.sh’ present in the root of the source zip file.

the build is then started with

make -j

the build process takes about 30 minutes.

if finished successfully, use

make install

to ‘install’ the libraries into the folder specifed with –prefix in the configuration settings.

assembling the libraries

now i have the two libraries built:

• newlib-nano in lib_build\install-native.newlibnano\target-libs\arm-none-eabi\lib
• newlib in lib_build\install-native.newlib\arm-none-eabi\lib

the newlib-nano libs have a *_s.a extension, so i rename them with a shell script:

newlib_nano_lib=./install-native.newlibnano/target-libs/arm-none-eabi/lib
inst_lib=./build.install/arm-none-eabi/lib

echo creating directories
mkdir ./build.install
cd ./build.install

mkdir ./arm-none-eabi
cd ./arm-none-eabi

mkdir ./lib
cd ./lib

mkdir ./armv6-m
mkdir ./armv7-ar
mkdir ./armv7e-m
mkdir ./armv7-m
mkdir ./cpu-init
mkdir ./fpu
mkdir ./thumb

cd ../..

echo rename newlib-nano files
mv $newlib_nano_lib/libc.a $(newlib_nano_lib)/libc_s.a
mv $newlib_nano_lib/libg.a $newlib_nano_lib/libg_s.a

mv $newlib_nano_lib/armv6-m/libc.a $(newlib_nano_lib)/armv6-m/libc_s.a
mv $newlib_nano_lib/armv6-m/libg.a $newlib_nano_lib/armv6-m/libg_s.a

mv $newlib_nano_lib/armv7-ar/libc.a $newlib_nano_lib/armv7-ar/libc_s.a
mv $newlib_nano_lib/armv7-ar/libg.a $newlib_nano_lib/armv7-ar/libg_s.a

mv $newlib_nano_lib/armv7e-m/libc.a $newlib_nano_lib/armv7e-m/libc_s.a
mv $newlib_nano_lib/armv7e-m/libg.a $newlib_nano_lib/armv7e-m/libg_s.a

mv $newlib_nano_lib/armv7-m/libc.a $newlib_nano_lib/armv7-m/libc_s.a
mv $newlib_nano_lib/armv7-m/libg.a $newlib_nano_lib/armv7-m/libg_s.a

mv $newlib_nano_lib/fpu/libc.a $newlib_nano_lib/fpu/libc_s.a
mv $newlib_nano_lib/fpu/libg.a $newlib_nano_lib/fpu/libg_s.a

mv $newlib_nano_lib/thumb/libc.a $newlib_nano_lib/thumb/libc_s.a
mv $newlib_nano_lib/thumb/libg.a $newlib_nano_lib/thumb/libg_s.a

and then move the newlib libraries over the newlib-nano libraries (merge):

mv -r $new_lib/* $newlib_nano_lib/*

with this, i have a new ‘lib’ folder i can use. i rename my original gcc/arm-none-eabi/lib one to lib.orig and copy my new library folder:

with this, i have my new library in place, and can go back to the original state if necessary.

debugging the library with source code

the the benifit of all the work: i can now debug and step through the library code with source files displayed:

:-) :-) :-)

summary

it takes some time and knowledge to build the newlib and newlib-nano libraries from the sources. but doing so, i can build the libraries with debug information. what is missing now is a good way to provide my custom optimization options for the library build, and an automated way/script to build my set of library files.

having the ability to re-build the library sources the way i want is a huge advantage of using open source libraries and tools. yet another reason the swap out proprietary libraries as shown in this post :-).

happy libraring :-)