\\
===== How to integrate NXP eIQ Software Stack into your BSP =====
=== Prerequisites ===
The default-BSP has already been built and the build-environment is set up, ''ci-meta-tq'' is the current directory.\\
Refer to [[en:arm:tqma8mpxl:linux:yocto:quickstart_yocto|Quickstart BSP]] for more information.
The goal of this guide is to make use of the i.MX Machine Learning features as described in [[https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf|i.MX Machine Learning User's Guide]].
Those features are cumulated inside the yocto layer ''meta-ml'', provided by ''meta-imx''.\\
It adds the i.MX Machine Learning features and includes the //eIQ Software Stack// which enables the use of specialized hardware accelerators (NPU/GPU).\\ \\
For the sake of simplicity, changes will be done in the local build directory (local.conf) for this example.\\
This is a valid approach for evaluation purposes, for proper implementation however, a new custom yocto layer should be created.
==== Cloning resources ====
At first, move into your sources directory and clone ''meta-imx'' aswell as its dependencys ''meta-freescale-distro'' and ''meta-python2''.\\
Each layer needs to be checked out with your specific branch, this example is referenced to BSP-revision 0085
cd sources
git clone https://github.com/nxp-imx/meta-imx.git
cd meta-imx/
git checkout kirkstone-5.15.71-2.2.0
cd ..
git clone https://github.com/Freescale/meta-freescale-distro.git
cd meta-freescale-distro/
git checkout kirkstone
cd ..
git clone git://git.openembedded.org/meta-python2
cd meta-python2/
git checkout hardknott
cd ..
cd ..
\\
==== Extending BBlayers ====
Next, move into the conf-Folder in your buildspace directory and edit the **bblayers.conf**
cd tqma8mpxl_build/conf/
nano bblayers.conf
now add the following layers to the **BBLAYERS** variable:
${BSPDIR}/sources/meta-imx/meta-ml
${BSPDIR}/sources/meta-imx/meta-bsp
${BSPDIR}/sources/meta-imx/meta-sdk
${BSPDIR}/sources/meta-freescale-distro
${BSPDIR}/sources/meta-python2
''meta-ml'' requires ''meta-bsp'' and ''meta-sdk'' (also provied by ''meta-imx'') to be present aswell
The contents of **BBLAYERS** should look something like this when you're done:
BBLAYERS = "\
${BSPDIR}/sources/poky/meta \
${BSPDIR}/sources/poky/meta-poky \
\
${BSPDIR}/sources/meta-openembedded/meta-oe \
${BSPDIR}/sources/meta-openembedded/meta-python \
${BSPDIR}/sources/meta-openembedded/meta-multimedia \
\
${BSPDIR}/sources/meta-freescale \
\
${BSPDIR}/sources/meta-qt5 \
\
${BSPDIR}/sources/meta-tq \
${BSPDIR}/sources/meta-dumpling \
\
${BSPDIR}/sources/meta-imx/meta-ml \
${BSPDIR}/sources/meta-imx/meta-bsp \
${BSPDIR}/sources/meta-imx/meta-sdk \
${BSPDIR}/sources/meta-freescale-distro \
${BSPDIR}/sources/meta-python2 \
"
\\
==== Adding the packages ====
Next, you have to append the required packages to your image, done in the **local.conf** here. You may simply add ''packagegroup-imx-ml'' to the ''IMAGE_INSTALL_append'' variable.
To find out which packages are included in ''packagegroup-imx-ml'', for example if you don't need all packages this packagegroup contains and you want to append only a few of them, refer to **sources/meta-imx/meta-ml/recipes-fsl/packagegroup/packagegroup-imx-ml.bb**
For running some of the included python demos, the python module ''pillow'' is required.\\
You may simply append the package ''python3-pillow'', however if you need additional python modules anyway for python developement later on, we recommend appending ''python3-pip''.\\
This will allow for easy installation of any python module directly on target (given an internet connection to [[https://pypi.org]] or a local python wheel).\\
The result, depending on the packages you want to append, should look something like this:
IMAGE_INSTALL_append = " packagegroup-imx-ml python3-pip python3-pillow"
Note: The ''packagegroup-imx-ml'' does apparently not include //OpenCV DNN// (described in [[https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf#_OPENTOPIC_TOC_PROCESSING_d514e1817|i.MX Machine Learning User Guide Chapter 7 OpenCV machine learning demos]]), however since Models from the //OpenCV DNN// Module can not run on the specialized NPU/GPU-accelerator hardware, you are most likely going to ignore it anyway. If you need OpenCV DNN, you should build a ''imx-image-full'' because the majority of OpenCV DNN Demos require a Qt GUI
You may also want to build NXP eIQ Packages into the SDK. For this purpose add this line to your local.conf:
TOOLCHAIN_TARGET_TASK_append += " tensorflow-lite-dev onnxruntime-dev"
\\
==== Activating hardware accelerators with kernel patch ====
The hardware accelerators are (as of BSP-revision 0085) disabled by default, therefore a kernelpatch has to be created to active them in the device tree.
Without the patch, a warning ''[query_hardware_caps:66]Unsupported evis version'' will be thrown, and execution will automatically fall back CPU using the standard C floating point libary operations, which results in even slower inference than with the "used by default" //XNNPACK delegate//, as it implements optimized float-operations for CPU
To learn how to create kernel patches and add them to the kernel recipe, you may refer to section [[en:arm:tqma8mpxl:linux:yocto:how_to:yocto_build_system#creating_and_adding_linux_kernel_patch|Creating and adding Linux kernel patch]] in our Yocto Build System Guide.
To activate the machine learning hardware accelerators, add the following lines in the device tree file **arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mpxl.dts**:
&ml_vipsi {
status = "okay";
};
\\
==== Build your image ====
Now repeat building your image like described in [[en:arm:tqma8mpxl:linux:yocto:quickstart_yocto|Quickstart BSP]] (''bitbake tq-image-weston-debug'').\\
=== BSP revision 0085 - Known issues ===
- Despite accelerators being activated in the device tree, when trying to force acceleration using the GPU instead of NPU (by changing environmentvariable ''USE_GPU_INFERENCE=1''), a ''[query_hardware_caps:66]Unsupported evis version'' will still be thrown with execution falling back to CPU.