Factory Troubleshooting

Table of Contents

Quick tips

  • Save output by using "2>&1 | tee log"
  • Save working workorders for comparison later
  • Changing architecture options or libc is very invasive and is often better as a new workorder
  • Identify the failed package and its particular stage of building by the last line of output

Asking for help

Several things come in handy when you need help troubleshooting your factory builds. Providing the work order (.config) is probably the most important. You should also provide the version of the factory build engine you are using (see doc/VERSION).

Common Problems

My build failed, what went wrong?

Look at the last line of output:

make: *** [/var/tmp/builder/build_sh4-timesys-linux-uclibc/pango-1.22.0/.stamp_built] Error 2

Here we can see pango was the culprit. We can also tell that it failed during the build stage due to the .stamp_built target being the last run target. Diagnosing what went wrong during the pango build will require more context than the last line of output. But you now have a point from which to begin.

My download fails mysteriously, what can I do?

-- fetching ccache 2.4 -- 1228162192 [Mon, 01 Dec 2008 13:09:52 -0700]
make: *** [/builder/src/dl/ccache-2.4] Error 1

You may be behind a proxy server and the transfer resumption may be causing your problem. You can manually edit the TSWO_URL_CMD in the .config file and remove the "-c" option or visit the "Advanced Build Configuration" menu and remove the "-c" flag from the "URL command" option.

Build failure "No rule to make target `_config'." with u-boot

You may see something like the following:

-- configuring u-boot "" -- 1228704850 [Sun, 07 Dec 2008 21:54:10 -0500]
cd /tmp/builder/build_armv5l-timesys-linux-gnueabi/u-boot-1.3.2/u-boot-1.3.2 && \
        PATH=/tmp/builder/build_armv5l-timesys-linux-gnueabi/fake/bin:/tmp/builder/build_armv5l-timesys-linux-gnueabi/toolchain/ccache:/tmp/builder/build_armv5l-timesys-linux-gnueabi/toolchain/bin:/tmp/builder/build_armv5l-timesys-linux-gnueabi/toolchain/usr/bin:/tmp/builder/build_armv5l-timesys-linux-gnueabi/toolchain/sbin:/tmp/builder/build_armv5l-timesys-linux-gnueabi/toolchain/usr/sbin:/usr/bin:/bin \
        make \
        CONFIG_NOSOFTFLOAT=1 \
        ""_config
make[1]: Entering directory `/tmp/builder/build_armv5l-timesys-linux-gnueabi/u-boot-1.3.2/u-boot-1.3.2'
make[1]: *** No rule to make target `_config'.  Stop.
make[1]: Leaving directory `/tmp/builder/build_armv5l-timesys-linux-gnueabi/u-boot-1.3.2/u-boot-1.3.2'
make: *** [/tmp/builder/build_armv5l-timesys-linux-gnueabi/u-boot-1.3.2/.stamp_configured] Error 2

You have enabled the host tool mkimage but there is no known u-boot configuration target for your board. If you look in the .config file, you should see TSWO_UBOOT_CONFIGTARGET="". If you need u-boot, you must specify a valid u-boot config target, or disable the mkimage host tool.

Download and Patch failures

Sometimes your .config references a file that may not exist. This is problematic for patch files as they are fetched along with the source files during the package's -fetch target. If the patch fails but the original source succeeds to be retrieved, you may get a situation like the following scenario.

Running make begins fetching sources:

-- fetching kernel -- 1229443515 [Tue, 16 Dec 2008 11:05:15 -0500]
11:05:23 URL:http://engservices.timesys.com/mirrors/sources//k/kernel/kernel-2.6.27//linux-2.6.27.tar.bz2 [50355835/50355835] -> "linux-2.6.27.tar.bz2" [1]
11:05:23 URL:http://engservices.timesys.com/mirrors/sources//k/kernel/kernel-2.6.27//fix-cygwin-2.6.27.patch [10554/10554] -> "fix-cygwin-2.6.27.patch" [1]
http://engservices.timesys.com/mirrors/sources//k/kernel/kernel-2.6.27//foo.patch:
11:05:23 ERROR 404: Not Found.
make: *** [/tmp/builder/src/dl/k/kernel/kernel-2.6.27/linux-2.6.27.tar.bz2] Error 1

Then rerunning make produces the following:

-- extracting linux-2.6.27 -- 1229444074 [Tue, 16 Dec 2008 11:14:34 -0500]
cd /tmp/builder/build_sh4-timesys-linux-uclibc/linux-2.6.27 && patch  -p1 -i /tmp/builder/src/dl/k/kernel/kernel-2.6.27/fix-cygwin-"2.6.27".patch
patching file arch/powerpc/Makefile
patching file lib/gen_crc32table.c
patching file Makefile
patching file scripts/gcc-version.sh
patching file scripts/kallsyms.c
patching file scripts/Kbuild.include
patching file scripts/kconfig/lxdialog/check-lxdialog.sh
patching file scripts/kconfig/Makefile
patching file scripts/Makefile.build
patching file scripts/mkcompile_h
patching file scripts/mod/file2alias.c
patching file scripts/mod/modpost.h
patching file scripts/mod/sumversion.c
cd /tmp/builder/build_sh4-timesys-linux-uclibc/linux-2.6.27 && patch  -p1 -i /tmp/builder/src/dl/k/kernel/kernel-2.6.27/foo.patch
patch: **** Can't open patch file /tmp/builder/src/dl/k/kernel/kernel-2.6.27/foo.patch : No such file or directory
make: *** [/tmp/builder/build_sh4-timesys-linux-uclibc/linux-2.6.27/.stamp_patched] Error 2

In this situation you must remove the initial source to refetch the source and additional files for the package.

rm -rf src/dl/k/kernel/

Why am I getting "ERROR: I cannot handle the URL type for '/'!" ?

If you see something like the following:

$ make mycustompackage-package
-- Reading configuration and build instructions -- 1229453689 [Tue, 16 Dec 2008 13:54:49 -0500]
Please wait, this will take some time...
-- fetching mycustompackage -- 1229453806 [Tue, 16 Dec 2008 13:56:46 -0500]
ERROR: I cannot handle the URL type for '/'!
make: *** [/home/user/builder-217/src/dl/mycustompackage-] Error 1

You have added a package to the factory builder but do not have any configuration stored for it in your .config. You should rerun the configuration interface and then exit and save. You do not need to enable or make any configuration differences. The configuration system will pick up the new package and store the default values in your .config.

Make menuconfig fails

If menuconfig fails during with an error like the following:

make KBUILD_HAVE_NLS=no CC="/usr/lib/ccache/gcc" -C include/config conf mconf
make[1]: execvp: ././lxdialog/check-lxdialog.sh: Permission denied
make[1]: Entering directory `/home/Builder/include/config'
/usr/lib/ccache/gcc  -DLOCALE -DKBUILD_NO_NLS  -c lxdialog/checklist.c -o lxdialog/checklist.o
In file included from lxdialog/checklist.c:24:
lxdialog/dialog.h:32:10: error: #include expects "FILENAME" or <FILENAME>
In file included from lxdialog/checklist.c:24:
lxdialog/dialog.h:97: error: expected specifier-qualifier-list before âchtypeâ
lxdialog/dialog.h:187: error: expected â)â before â*â token
...
make[1]: *** [lxdialog/checklist.o] Error 1
make[1]: Leaving directory `/home/Builder/include/config'
make: *** [include/config/mconf] Error 2

The include line is: #include CURSES_LOC

Which should be set with -DCURSES_LOC="<ncurses.h>" when compiled. This is
filled in with:

$ cd include/config && ./lxdialog/check-lxdialog.sh -ccflags
-I/usr/include/ncurses -DCURSES_LOC="<ncurses.h>"

Make sure that the include/config/lxdialog/check-lxdialog.sh file is executable.

chmod +x include/config/lxdialog/check-lxdialog.sh

My build fails with "fakeroot: preload library not found, aborting"

The factory uses fakeroot for building packages as well as the RFS images. Fakeroot uses a hardcoded LD_LIBRARY_PATH to preload the fakeroot library. If the build directory is moved, this invalidates those paths. The easiest fix is to rebuild the temporary fakeroot that is built during the normal build process:

make fakeroot-distclean fakeroot-host-install

GDB startup error about libtinfo.so.5

When running gdb from the starting point toolchain the error "error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory" is printed and gdb won't start. This is caused by library structure changes in recent versions of ncurses. It can be fixed with one of the following commands:

Ubuntu 64-bit:
ln -sfn libncurses.so.5 /lib32/libtinfo.so.5

32-bit Linux and 64-bit Linux distros that don't have /lib32/ directory:
ln -sfn libncurses.so.5 /lib/libtinfo.so.5

This error can also manifest in TimeStorm. It will start gdbserver, but never start the debugger user interface. The error message about libtinfo will appear in the log. The solution is the same.

Gtk applications failed to build with: Cannot open pixbuf loader module file '/etc/gtk-2.0/gdk-pixbuf.loaders'

Run

gdk-pixbuf-query-loaders > /etc/gtk-2.0/gdk-pixbuf.loaders

Download failures due to the use of a proxy

There are two ways to make the factory build system use a proxy. Since the factory uses the wget command internally, the ~/.wgetrc file can be used to specify the proxy configuration:

$ cat ~/.wgetrc
http_proxy = http://ip:port/

Or the following could be added to the factory work order configuration:

Factory menuconfig -> Advanced Build Configuration -> URL Command -> wget -nv -c -e 'http_proxy = http://ip:port/'

Resulting in the following key/value pair in the .config:

TSWO_URL_CMD="wget -nv -c -e 'http_proxy = http://ip:port/'"

Packaging step fails with "No such file or directory"

/bin/sh: line 0: cd: ~/build_<target>/packages/ncurses_install: No such file or directory
make: *** [~build_<target>/ncurses-5.9/.stamp_post_packaged] Error 1

This is due to a new change in the framework that adds an extra step for packaging and root file system installation to register init scripts. Existing builds will be missing the stamp file for this step and the final package target will expire and rerun without the prior targets being run. To resolve this, run 'make packages-distclean' to reset the packaging targets. This will regenerate all packages.

RFS installation fails for a package, requesting relinking a libtool convenience library

libtool: install: error: relink `foo.la' with the above command before installing it

The C library RFS installation always occurs at the end of the build process due to the confusion of libtool when the final C library linker scripts are present. If the above error occurs after adding a new package to a previously built platform, use 'make rfs-distclean' to reset the RFS staging directory. The time required to perform an rfs-distclean and then repopulate the rfs is comparable to building an RFS from binary packages.