Linux 新系统安装配置清单
主要以 debian 系统为示例
一、 bash 配置
/root/.bashrc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| PS1='\n${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h \a\]$PS1"
;;
*)
;;
esac
export LS_OPTIONS='--color=auto'
eval "`dircolors`"
alias ls='ls $LS_OPTIONS'
alias ll='ls $LS_OPTIONS -lh'
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
|
二、 公共组件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
| # 备份原来的 sources.list
DATE_TIME_NOW=`date +"%Y%m%d_%H%M"`
mv /etc/apt/sources.list /etc/apt/sources.list.bak_${DATE_TIME_NOW}
# 设置 sources.list ，使用清华的镜像源
cat << \EOF > /etc/apt/sources.list
deb https://mirrors.tuna.tsinghua.edu.cn/debian buster main contrib non-free
deb https://mirrors.tuna.tsinghua.edu.cn/debian buster-updates main contrib non-free
deb https://mirrors.tuna.tsinghua.edu.cn/debian buster-backports main contrib non-free
deb https://mirrors.tuna.tsinghua.edu.cn/debian-security buster/updates main contrib non-free
EOF
# libsm6, libxrender1 for opencv
apt install -y \
curl \
gnupg2 \
vim \
python3-pip \
libsm6 \
libxrender1 \
tmux \
nfs-common \
zip \
unzip
# vim 设置，写到 vimrc.local 中，避免更新时冲突。
cat << \EOF > /etc/vim/vimrc.local
# 显示行号
set nu
# 设置 yaml 文件缩进
autocmd FileType yaml,yml setlocal ts=2 sts=2 sw=2 expandtab indentkeys-=<:>
EOF
|
三、 私有证书集成
1
2
| cp ./my_ca.crt /usr/local/share/ca-certificates/
update-ca-certificates
|
四、 python 环境
1. pip 私服
/etc/pip.conf
1
2
3
4
5
6
7
| [global]
# 豆瓣源
# index-url = https://pypi.douban.com/simple
# 清华源
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
# 阿里云源
# index-url = https://mirrors.aliyun.com/pypi/simple
|
2. 常用包
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| # 科学计算
numpy
scipy
pandas
matplotlib
sklearn
# 图像
opencv-python
pillow
# jupyter notebook
jupyterlab
# 文件规范
pylint
autopep8
# 虚拟环境
pipenv
virtualenvwrapper
|
1
| pip3 install -r requirements.txt
|
3. 虚拟环境设置
/etc/bash.bashrc
1
2
3
4
5
6
7
8
9
| # 加载 virtualenvwrapper 脚本
if [ -f /usr/local/bin/virtualenvwrapper.sh ]; then
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export WORKON_HOME='~/.virtualenvs'
source /usr/local/bin/virtualenvwrapper.sh
fi
# 让 pip 使用系统证书
export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
|
五、 docker 环境
1. 安装
添加文件 /etc/apt/sources.list.d/docker.list
1
2
3
4
| # curl https://download.docker.com/linux/debian/gpg | sudo apt-key add -
# apt install docker-ce
deb [arch=amd64] https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/debian buster stable
|
运行以下命令进行安装
1
2
3
| curl https://download.docker.com/linux/debian/gpg | apt-key add -
apt update
apt install -y docker-ce
|
2. 下载 docker-compose
1
2
3
4
| # curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
curl -L "https://get.daocloud.io/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
|
3. 设置镜像源
/etc/docker/daemon.json
1
2
3
4
5
6
7
| {
"registry-mirrors": [
"https://docker.mirrors.ustc.edu.cn/",
"https://hub-mirror.c.163.com",
"https://registry.docker-cn.com"
]
}
|
六、 nodejs 环境
1. 安装
添加文件 /etc/apt/sources.list.d/nodesource.list
1
2
3
4
5
| # curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
# apt install nodejs
deb https://mirrors.tuna.tsinghua.edu.cn/nodesource/deb_12.x buster main
deb-src https://mirrors.tuna.tsinghua.edu.cn/nodesource/deb_12.x buster main
|
运行以下命令进行安装
1
2
3
| curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
apt update
apt install nodejs
|
2. 设置
代理设置
1
| npm config set registry https://registry.npm.taobao.org
|
私有证书设置
1
2
| # 先把私有证书集成到系统中
export NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt
|
七、 nvidia cuda 10.1 + cudnn 7
前提：先安装好 nvidia 显卡驱动，即 nvidia-smi 命令可用且有结果输出。（以后补教程）
1. 设置 apt 源
参考资料:
nvidia base dockerfile
nvidia runtime dockerfile
nvidia cudnn7 dockerfile
添加文件 /etc/apt/sources.list.d/cuda.list
1
2
3
4
| # curl -fsSL https://mirrors.cloud.tencent.com/nvidia-cuda/ubuntu1804/x86_64/7fa2af80.pub | apt-key add -
deb https://mirrors.cloud.tencent.com/nvidia-cuda/ubuntu1804/x86_64/ /
deb https://mirrors.cloud.tencent.com/nvidia-machine-learning/ubuntu1804/x86_64/ /
|
虽然是 ubuntu 18.04 的源，不过 debian 同样可以用。清华镜像站没有对 nvidia 进行收录，这里使用腾讯云的镜像。
2. 安装
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
| CUDA_VERSION=10.1.243
CUDA_PKG_VERSION=10-1=$CUDA_VERSION-1
NCCL_VERSION 2.4.8
CUDNN_VERSION 7.6.5.32
curl -fsSL https://mirrors.cloud.tencent.com/nvidia-cuda/ubuntu1804/x86_64/7fa2af80.pub | apt-key add -
apt update
apt install cuda-cudart-$CUDA_PKG_VERSION \
cuda-compat-10-1 \
cuda-libraries-$CUDA_PKG_VERSION \
cuda-nvtx-$CUDA_PKG_VERSION \
libcublas10=10.2.1.243-1 \
libnccl2=$NCCL_VERSION-1+cuda10.1 \
libcudnn7=$CUDNN_VERSION-1+cuda10.1
apt-mark hold libnccl2 libcudnn7
|
八、 docker 添加 nvidia 的运行环境
参考资料
1
2
3
4
5
6
7
8
9
| # 添加源
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | tee /etc/apt/sources.list.d/nvidia-docker.list
# 安装
apt update && apt install -y nvidia-docker2
# 重启 docker
systemctl restart docker
|
测试
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06 Driver Version: 450.51.06 CUDA Version: 11.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 On | 00000000:00:1E.0 Off | 0 |
| N/A 34C P8 9W / 70W | 0MiB / 15109MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
|
九、 安装 Kubernetes
1. 安装
1
2
3
4
5
6
7
8
9
10
| # curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
cat << EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.tuna.tsinghua.edu.cn/kubernetes/apt kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
|
十、 安装 clickhouse
1
2
3
4
5
6
7
8
| apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4
cat << EOF >/etc/apt/sources.list.d/clickhouse.list
deb https://mirrors.tuna.tsinghua.edu.cn/clickhouse/deb/stable/ main/
EOF
apt-get update
apt-get install -y clickhouse-server clickhouse-client
|
十一、 安装 gitlab-ee
1. gitlab-ee
1
2
3
4
5
6
7
8
| curl -L https://packages.gitlab.com/gitlab/gitlab-ee/gpgkey | apt-key add -
cat << EOF >/etc/apt/sources.list.d/gitlab-ee.list
deb http://mirrors.tuna.tsinghua.edu.cn/gitlab-ee/debian buster main
EOF
apt-get update
apt-get install -y gitlab-ee
|
2. gitlab-runner
1
2
3
4
5
6
7
8
| curl -L https://packages.gitlab.com/runner/gitlab-runner/gpgkey | apt-key add -
cat << EOF >/etc/apt/sources.list.d/gitlab-runner.list
deb http://mirrors.tuna.tsinghua.edu.cn/gitlab-runner/debian buster main
EOF
apt-get update
apt-get install -y gitlab-runner
|