Hacking The Factory

Table of contents
Hacking The Factory

See also FactoryBestPractices, HackingTheFactoryResults

Navigating the Factory directory

When in its pristine state, the Factory directory contains the following directories:

toolchain/ directory contains the KConfig definition files (Config.in) for each toolchain component as well as the host utilities that are built with the toolchain. It also contains each components corresponding make rules file (*.mk).
target/ directory contains sub-directories for the kernel, bootloaders, rfs, and software that can be included within the target RFS.
target/kernel/ contains the kernel KConfig definition file and make rule file.
target/bootloader/ contains subdirectories for each supported bootloader.
target/software/ contains the package category subdirectories and their containing package directories.
target/configs contains configuration files for the kernel and applications.
target/early-userspace contains the build instructions for early userspace support.
target/installer contains the installer build instructions.
bin/ directory contains scripts that are useful in extending the factory builder.
doc/ directory contains the documentation for the factory builder.
include/ directory contains common make rules (common.mk, target.mk, and host.mk) along with the KConfig library.

The top level makefile includes all other make rule files within the factory builder tree. It also includes the .config file (the generated work order) that is used by the make rules in generating output.

The top level Config.in includes all other Kconfig Config.in definition files. It also sets global and default options.

Extending the Factory

The Factory supports a number of evaluation boards, but at some point, you'll want it to work for your custom hardware. It's easy to extend the Factory to support completely new boards and architectures. Simply modify a few configuration files, and you'll be on your way.

The typical path for extending the Factory is:

Add Toolchain Support
Add Board Support
Add Kernel Support
Add Bootloader Support
Add New Packages

Adding a New Toolchain

This procedure is only necessary if you wish to extend Factory to support additional cpu cores. If your core is already supported, you can skip these steps.

Edit target/Config.in.arch.<arch> (ARM, POWERPC, MIPS, SH, I386, X86_64, XTENSA):
- Add new flag at the top of the file for the cpu core variant.
  Example:
```
config TSWO_CPU_POWERPC_E500
    bool
```
- Add a selectable option to the choice section in the bottom of the file for your CPU. It should have the following parameters associated with it:
  - bool "<human readable name>"
  - Select TSWO_LIBC_PORTS (if the toolchain requires glibc or eglibc ports).
  - Select TSWO_NO_FPU (if the toolchain does not support floating point).
  - Select TSWO_BIG_ENDIAN (if the toolchain is big endian. Note that ALL powerpc boards are big endian, and this value is already selected on the architecture level).
    Example:
```
config TSWO_powerpc_e500
    bool "e500" 
    select TSWO_LIBC_PORTS
```
- Add a new default option for the choice that ties the flag and configuration option together.
  Example:
```
    default TSWO_powerpc_e500    if TSWO_CPU_POWERPC_E500
```
Edit toolchain/cc/gcc/Config.in
- In configure section TSWO_gcc_CONFIGURE_OPTIONS, add gcc configure options for your cpu type if necessary.
Edit toolchain/libc/*/Config.in
- Verify valid uClibc config location, patches
- Verify glibc patches
- Verify eglibc patches
Edit target/Config.in:
- Set TSWO_TARGET_CFLAGS and TSWO_TARGET_CXXFLAGS as required
Edit toolchain/Config.in:
- Set TSWO_HOST_BUILD_OPTIONS as required

You can use the generic target/configs/${app}/ ${board|arch} directories for uClibc, the kernel, perl, busybox, etc or define your own.

Some packages have arch specific concerns:

openssl
ltrace
strace
perl
wpa_supplicant

Adding a New Board

Edit target/Config.in.arch for your board-specific information:
- In the first "choice" section:
  - Add config TSWO_BOARD_<boardname> with the following properties:
    - bool "<Human Readable Name>"
    - Select TSWO_ARCH_<arch name> (ARM, POWERPC, MIPS, SH, I386, X86_64, XTENSA).
    - Select TSWO_CPU_<arch name>_<cpu name> (see target/Config.in.arch.<arch name> for canonical list of values).
    - Select TSWO_KERNEL_TARGET_<kernel target type> (UIMAGE, ZIMAGE, BZIMAGE, VMLINUX_SREC).
    - Select TSWO_KERNEL_REQUIRES_DEVICE_TREE (only if board requires device tree).
    - Select TSWO_BUILD_BOOTLOADER_<bootloader> (U-BOOT, YABOOT, AT91BOOTSTRAP, if needed).
      Example:
```
config TSWO_BOARD_mpc8548cds
    bool "Freescale MPC8548CDS" 
    select TSWO_ARCH_POWERPC
    select TSWO_CPU_POWERPC_E500
    select TSWO_KERNEL_TARGET_UIMAGE
    select TSWO_KERNEL_REQUIRES_DEVICE_TREE
    select TSWO_BUILD_BOOTLOADER_U-BOOT
```
- In the config TSWO_BOARD section:
  - Add default "<boardname>" if TSWO_BOARD_<boardname>
    Example:
```
    default "mpc8548cds" if TSWO_BOARD_mpc8548cds
```
Add Kernel patches and configuration information — See Adding a New Kernel.
Specify an External U-Boot (instructions here), or add Bootloader patches and configuration information to the target/bootloaders/<bootloader name>/Config.in file for your bootloader, if necessary.

(optional - only necessary if not using mdev/udev to generate device nodes on the hardware target) Add a customized device table to target/configs/device_table/ that defines the basic device files and directories required in the file system. This should define a ttyDefault device that maps the major/minor of the default tty device for the target.

Defaults to the generic device table as defined in target/rfs/Config.in.

Parsed by target/rfs/makedevs/makedevs.c originating from busybox.

makedevs: [-d device_table] rootdir
Creates a batch of special files as specified in a device table.
Device table entries take the form of:
type mode user group major minor start increment count
Where name is the file name,  type can be one of:
      f       A regular file
      d       Directory
      c       Character special device file
      b       Block special device file
      p       Fifo (named pipe)
uid is the user id for the target file, gid is the group id for the
target file.  The rest of the entries (major, minor, etc) apply to
to device special files.  A '-' may be used for blank entries.
For example:
<name>    <type> <mode> <uid> <gid> <major> <minor> <start> <inc> <count>
/dev         d    755    0    0     -       -       -       -     -
/dev/console c    666    0    0     5       1       -       -     -
/dev/null    c    666    0    0     1       3       0       0     -
/dev/zero    c    666    0    0     1       5       0       0     -
/dev/hda     b    640    0    0     3       0       0       0     -
/dev/hda     b    640    0    0     3       1       1       1     15
Will Produce:
/dev
/dev/console
/dev/null
/dev/zero
/dev/hda
/dev/hda[0-15]

Adding a New Kernel

To add a new kernel for an existing board permanently to your local copy of the factory, do the following:

Specify an External Kernel

Legacy method:

Add the patches and base kernel tarball to a directory. This directory can on your own machine, or on another machine which you can access from your machine via ssh or http.
Example:

# mkdir -p /home/user/sources/k/kernel/kernel-2.6.28/
# cd /home/user/sources/k/kernel/kernel-2.6.28/
# wget http://www.kernel.org/pub/linux/kernel/v2.6/patch-2.6.28.bz2
# cp /home/user/patches/2.6.28-board_support.patch ./

Add your configuration file to ./target/configs/kernel/<board name>/. If your kernel uses a device tree file, copy it there as well.
Example:

# cp /home/user/kernel/.config target/configs/kernel/<board name>/config-2.6.28
# cp /home/user/kernel/arch/powerpc/boot/dts/my.dts target/configs/kernel/<board name>/dts-2.6.28

Modify the file ./target/kernel/Config.in to add sane defaults for your new kernel.
- Add a default kernel version for your board for the configuration option TSWO_kernel_VERSION.
  Example:
```
    default TSWO_KERNEL_2_6_30 if TSWO_BOARD_mpc8548cds
```
- Add a default configuration file for your board, if it doesn't already exist, for the option TSWO_KERNEL_CONFIG.
  Example
```
    default "file://$(BASE_DIR)/target/configs/kernel/mpc8548cds/config-$(TSWO_kernel_VERSION)" if TSWO_BOARD_mpc8548cds
```
- Add a default value for your patch list to the configuration option TSWO_kernel_PATCHES.
  Example:
```
    default "2.6.28-board_support.patch" if TSWO_BOARD_board_name && TSWO_KERNEL_2_6_28
```
- Optionally, add a default value for your device tree file to the configuration option TSWO_KERNEL_DEVICE_TREE_FILE.
  Example:
```
    default "file://$(BASE_DIR)/target/configs/kernel/<board name>/dts-$(TSWO_kernel_VERSION)" if TSWO_BOARD_board_name
```
When configuring your workorder, make sure to modify TSWO_kernel_URL to point to your source location in step 1.

Additional Steps for RT kernels

If your new kernel is a realtime kernel, there are a few additional lines to add/modify in target/kernel/Config.in.

Add an additional choice for your RT kernel if it does not already exist, or add your board to the dependency list for that choice:
Example:
```
config TSWO_KERNEL_2_6_28_RT
    depends TSWO_BOARD_board_name
    bool "2.6.28 real-time" 
```
Add a default value for your RT kernel to TSWO_kernel_VERSION if it does not already exist.
Example:
```
    default "2.6.28"        if TSWO_KERNEL_2_6_28_RT
```

Add the RT kernel version to the default y options for TSWO_kernel_RT, if it is not already there.
Example:

config TSWO_kernel_RT
    bool
    default y if TSWO_KERNEL_2_6_24_RT || TSWO_KERNEL_2_6_23_RT || TSWO_KERNEL_2_6_26_RT || TSWO_KERNEL_2_6_28_RT
    default n

Add and additional line to TSWO_KERNEL_CONFIG, if it does not already exist. This line should precede any other default values for your board.
Example:
```
    default "file://$(BASE_DIR)/target/configs/kernel/<board name>/config-$(TSWO_kernel_VERSION)-rt" if TSWO_BOARD_board_name && TSWO_kernel_RT
```

Adding a New Package

Run:

./bin/new_pkg {pkg_name} {Category}

where category is in Desktop, Development, Graphics, Multimedia, Networking, Runtimes, System, or Utilities.

This will copy target/software/.template_{in,mk} into the target/software/{Category} directory, customize the package name in those files, create an entry in target/software/{Category}/Config.in, and open each in ${EDITOR}.

The template files contain commented sections. The Config.in should be customized for the version, source file name (.gz or .bz2 etc), dependencies (both build time and run time), etc. The mk template contains duplications of the default targets that are handled by the macros in include/common.mk. Please trim out any unused, commented out sections before committing. Refer to the list of common make variables below and the meanings associated with them when customizing the package make file.

Things to Remember

Enter a description in the Config.in (in the help section of menuconfig TSWO_SOFTWARE_package).
Edit target/software/<CATEGORY>/Config.in to place your package correctly in alphabetical order.
Use "select" kconfig directive for runtime dependencies.
Make sure to keep the alphabetical ordering in target/software/{Category}/Config.in within any subcategory submenus.
Create an empty host install make target or remove target device binaries in the post host install make target.
CONFIGURE_OPTIONS are for passing values to the configure step, typically autoconf or some other mechanism.
BUILD_OPTIONS are set in the environment prior to calling make or some other build mechanism.
Required build options or options that are dynamic (things that use a make variable value) are most often best placed within the .mk file itself instead of in the Config.in defaults.
Variables like foo_CONFIGURE_OPTIONS are stripped versions of TSWO_foo_CONFIGURE_OPTIONS with quoting removed.
Rely on variables that come from the .config instead of using external variables (defined dynamically or in another makefile). Make targets using these variables should always $(strip $(call UNQUOTE_VAR,$(TSWO_foo_OPT))). Otherwise for non make targets, paths and file names can use $(TSWO_foo_OPT) directly (quotes are ok for paths and file names as far as the shell is concerned).

Try to make every patch and option universal. If you must add a libc or kernel version specific configure option, append it in the mk file:

ifneq (y,$(TSWO_LIBC_UCLIBC))
    samba_CONFIGURE_OPTIONS+=linux_getgrouplist_ok=yes
else
    samba_CONFIGURE_OPTIONS+=linux_getgrouplist_ok=no
endif

If adding new files such as init scripts, prepend the command in the mk targets with a dash (-) so make ignores failures in case of older work orders that do not contain the new files:
```
-chmod +x $(TSWO_RFS_INSTALL_DIR)/etc/init.d/S75-start-xserver
```
init scripts should end in .initscript and have the LSB header included. The framework will handle how to use it with the various init systems that are supported.
Try not to hard code package relationships. If a package can use foo, then in the mk file add foo to the optional dependency declaration:
```
$(eval $(call OPTIONAL_DEPS,mypackage,foo bar baz))
```
For optional dependencies, optionally enable/disable configure options that rely upon the optional dependency being present:
```
ifneq (y,$(TSWO_SOFTWARE_esound))
    libao_CONFIGURE_OPTIONS+=--enable-esd=no --disable-esdtest
endif
```
If your package's source tarball does not follow an pattern easily expressed as variables, just hard-code the name (foo-src.tgz) in the SOURCE variable.
If your package does not have a configure step, remove the CONFIGURE_OPTIONS tags from the makefile. Remember, even packages that don't use autoconf can have their own concept of a configure step. In this case, map this variable into your packages concept of configure options.
If your package needs an architecture string that does not directly correspond to any of the pre-defined arch strings (TSWO_ARCH, TSWO_KERNEL_ARCH), then you may need to define your own. Usually you can start with the one that matches the closest, and then put in exceptions from there. A good example of this is openssl, which generally uses TSWO_KERNEL_ARCH but has a number of exceptions:
```
config TSWO_openssl_ARCH
    string
    default "i386-i486" if TSWO_x86_i486
    default "i386-i586" if TSWO_x86_i586
    default "i386-i686/cmov" if TSWO_x86_i686
    default "i386" if TSWO_i386
    default "sh4" if TSWO_sh4
    default TSWO_KERNEL_ARCH
```
Generally you do not need to include a prompt for this value if it can be determined automatically. You will then need to either refer to this variable in your TSWO_package_CONFIGURE_OPTIONS or TSWO_package_BUILD_OPTIONS, or within the recipe (package.mk) itself.

Common make variables

Various variables are defined by the factory builder for use within the unit make targets. include/common.mk defines a COMMON_SETUP macro that can be called from a unit rule file. This generates targets and variable aliases.

The following make variables are available for use when the unit rule file calls COMMON_SETUP:

$(package)_VERSION — package version
$(package)_URL — package URL
$(package)_SOURCE — package source file
$(package)_CONFIGURE_OPTIONS — configure options for this package
$(package)_BUILD_OPTIONS — environment variables to set when configuring and building
$(package)_PATCHES — space delimited list of patches to fetch from the package URL
$(package)_FILES — additional files fetched from package URL
$(package)_DEPENDS — build time dependencies
$(package)_RUNTIME_DEPENDS — runtime dependencies
$(package)_BASIC_MAKE — boolean to disable parallel make
$(package)_SOURCE_URL — URL + SOURCE
$(package)_DIR — directory within BUILD_DIR, typically {package}-{package_VERSION}
$(package)_WORKINGDIR — the working directory created when uncompressing the source archive
$(package)_DDIR — the working directory within the package directory.
$(package)_SRC_DL_DIR — the package directory within the source download location
$(package)_DL_SRC — downloaded source file location, SRC_DL_DIR + SOURCE
$(package)_PKG_INSTALL_DIR — temporary location to install into for packaging
$(package)_BUILDNUM — defaults to 1
$(package)_PKG_NAME — the name of the package to generate
$(package)_MAKE_CMD — the generated make command to use

The following variables are generally available even if the rule file does not call the COMMON_SETUP macro:

LIBC — uClibc, glibc,
BUILD_DIR — working directory created within the build tree, representing architecture and C library (example: armv5l-linux-timesys-gnueabi)
BASE_DIR — top-level directory of the build tree
SRC_DL_DIR — directory within which the per unit sources are saved: $(BASE_DIR)/src/dl
DATE — year,month, day format: 20080301
TSWO_FULL_TARGET_NAME — string representing arch-timesys-linux-libc_string
TSWO_HOST_INSTALL_DIR — directory containing the toolchain
TSWO_RFS_INSTALL_DIR — RFS working directory
TSWO_HOST_NAME — $(shell gcc -dumpmachine)
TSWO_BUILD_NAME — alias to TSWO_HOST_MACHINE
BUILD_PATH — contains sanitized path that includes host utility and toolchain locations
TARGET_PATH — path within the RFS working directory
TARGET_CROSS_PREFIX — prefix for toolchain and utilies (for example: $(TARGET_CROSS_PREFIX)gcc) that includes the full path along with full target name
PACKAGE_DIR — directories to copy packages
IMAGES_DIR — directories to copy images
TARGET_ALIASES — aliases for common utilities such as: CC, AR, LD, NM=, AS, RANLIB, STRIP, CXX, and PKG_CONFIG
TARGET_CFLAGS — CFLAGS for the target
TARGET_LDFLAGS — LDFLAGS for the target
TARGET_BUILD_OPTIONS — global environment variables to set when configuring and building.
TARGET_CONFIGURE_OPTIONS — global configure options to pass.
KERNEL_ARCH — the kernel compatible board architecture
HOST_CFLAGS — cflags to use when building host utilities
HOST_LDFLAGS — ldflags to use when building host utilities
HOST_BUILD_OPTIONS — environment to pass when building host utilities
HOST_CONFIGURE_OPTIONS — configure options to pass when building host utilities