Citms.OnnxRuntime.Gpu-arm64

Microsoft.ML.OnnxRuntime.Gpu ARM64

基于ARM服务器编译OnnxRuntime V1.16.3版本GPU依赖库，编译步骤如下

1.下载V1.16.3版本源码

git clone -b v1.16.3 --depth 1 https://github.com/microsoft/onnxruntime.git
git submodule update --init --recursive --progress

2.修改ARM64 cuda dockerfile文件

#进入到下载好的源码目录
cd onnxruntime

cd dockerfiles

#开始修改Dockerfile.cuda文件

• 原始dockerfile文件内容

# --------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
# --------------------------------------------------------------
# Dockerfile to run ONNXRuntime with CUDA, CUDNN integration

# nVidia cuda 11.4 Base Image
FROM nvcr.io/nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04
ENV         DEBIAN_FRONTEND=noninteractive
MAINTAINER Changming Sun "[email protected]"
ADD . /code

ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
RUN apt-get update && apt-get install -y --no-install-recommends python3-dev ca-certificates g++ python3-numpy gcc make git python3-setuptools python3-wheel python3-packaging python3-pip aria2 && aria2c -q -d /tmp -o cmake-3.26.3-linux-x86_64.tar.gz https://github.com/Kitware/CMake/releases/download/v3.26.3/cmake-3.26.3-linux-x86_64.tar.gz && tar -zxf /tmp/cmake-3.26.3-linux-x86_64.tar.gz --strip=1 -C /usr

RUN cd /code && python3 -m pip install -r tools/ci_build/github/linux/docker/inference/x64/python/cpu/scripts/requireme\
nts.txt && /bin/bash ./build.sh --allow_running_as_root --skip_submodule_sync --cuda_home /usr/local/cuda --cudnn_home /usr/lib/x86_64-linux-gnu/ --use_cuda --config Release --build_wheel --update --build --parallel --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) 'CMAKE_CUDA_ARCHITECTURES=52;60;61;70;75;86'

FROM nvcr.io/nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04
ENV         DEBIAN_FRONTEND=noninteractive
COPY --from=0 /code/build/Linux/Release/dist /root
COPY --from=0 /code/dockerfiles/LICENSE-IMAGE.txt /code/LICENSE-IMAGE.txt
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends libstdc++6 ca-certificates python3-setuptools python3-wheel python3-pip unattended-upgrades && unattended-upgrade && python3 -m pip install /root/*.whl && rm -rf /root/*.whl

• 修改后的文件内容

# --------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
# --------------------------------------------------------------
# Dockerfile to run ONNXRuntime with CUDA, CUDNN integration

# nVidia cuda 11.4 Base Image
FROM nvcr.io/nvidia/cuda:11.4.3-cudnn8-devel-ubuntu20.04
ENV         DEBIAN_FRONTEND=noninteractive
MAINTAINER Changming Sun "[email protected]"
ADD . /code

ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
RUN apt-get update && apt-get install -y --no-install-recommends python3-dev ca-certificates g++ python3-numpy gcc make git python3-setuptools python3-wheel python3-packaging python3-pip aria2 && aria2c -q -d /tmp -o cmake-3.26.3-linux-aarch64.tar.gz https://github.com/Kitware/CMake/releases/download/v3.26.3/cmake-3.26.3-linux-aarch64.tar.gz && tar -zxf /tmp/cmake-3.26.3-linux-aarch64.tar.gz --strip=1 -C /usr

RUN  /bin/bash ./build.sh --allow_running_as_root --skip_submodule_sync --tensorrt_home /usr/lib/aarch64-linux-gnu --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu/ --use_cuda --config Release --build_shared_lib --build_wheel --update --build --parallel --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) 'CMAKE_CUDA_ARCHITECTURES=52;60;61;70;75;86'

3.开始编译

docker build -t onnxruntime-cuda-build -f Dockerfile.cuda ..

整个执行过程预计2小时，镜像构建完成后，启动容器将文件拷备出来

docker run --rm -it onnxruntime-cuda-build     /bin/bash

# 新开一个shell窗口,其中f96cb33b6482为刚启动的容器id，可通过docker ps进行查看
docker cp f96cb33b6482:/code/build/Linux/Release /root/linux-onnx-cpu

• 编译后的文件，全都拷贝到宿主机 /root/linux-onnx-cpu目录中了 libonnxruntime.so libonnxruntime_providers_cuda.so libonnxruntime_providers_shared.so 这三个文件是我们需要的最终文件