0x01 前言

为减少公网压力,及提高安全性,建议在企业内部部署一个仓库镜像。因为Debian发行版更新较快,需要根据实际情况即使调整配置文件。我目前使用Debian10及11,这篇文章将同步这两个版本的一些仓库。

0x02 准备

首先准备一个大容量磁盘或分区,建议使用LVM,方便后续扩容:

root@debian10-mirror-p1:~# lvs
  LV     VG               Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root   debian10-02-1-vg -wi-ao----  48.56g
  swap_1 debian10-02-1-vg -wi-a----- 980.00m
  lv01   vg01             -wi-ao---- 999.99g

然后安装apt-mirror

root@debian10-mirror-p1:~# apt install apt-mirror -y

接着需要找到离你最近的镜像,这能极大地提升同步速度:

https://www.debian.org/mirror/list

最后找到必要的仓库,以下是我用到的:

Debian10 Buster
Debian11 Bullseye
Zabbix
Docker-ce
Mysql
Powerdns
Elastic

0x03 mirror.list

上面完成apt-mirror的安装后,会在以下路径生成一个配置文件:

/etc/apt/mirror.list

首先修改文件存储路径:

set base_path    /mnt/lv01/apt-mirror

然后手动创建以下目录:

mkdir -p /mnt/lv01/apt-mirror/{mirror,skel,var}

再创建以下文件:

touch /mnt/lv01/apt-mirror/var/clean.sh
touch /mnt/lv01/apt-mirror/var/postmirror.sh

线程默认为20,按需调整即可。我的虚拟机一般是1vCPU, 1G RAM,所以线程设置为3.

set nthreads     3

一般情况下,还需要根据网络及磁盘性能限制落盘速度:

set limit_rate 30m

0x03.1 Debian10 Buster

Debian10 Buster的源列表:

############# Debian 10 Buster ##############
# Debian 10 Buster
deb http://deb.debian.org/debian buster main contrib non-free
deb http://deb.debian.org/debian buster-backports main contrib non-free
deb http://deb.debian.org/debian buster-updates main contrib non-free

# zabbix
deb https://repo.zabbix.com/zabbix/5.4/debian buster main

# docker
deb [arch=amd64] https://download.docker.com/linux/debian buster stable

# mysql
deb http://repo.mysql.com/apt/debian/ buster mysql-apt-config
deb http://repo.mysql.com/apt/debian/ buster mysql-5.7
deb http://repo.mysql.com/apt/debian/ buster mysql-tools

# powerdns
deb [arch=amd64] http://repo.powerdns.com/debian buster-auth-47 main
deb [arch=amd64] http://repo.powerdns.com/debian buster-rec-47 main

0x03.2 Debian11 bullseye

Debian11 bullseye的源列表:

# Debian 11 bullseye
deb http://deb.debian.org/debian bullseye main contrib non-free
deb http://deb.debian.org/debian-security/ bullseye-security main contrib non-free
deb http://deb.debian.org/debian bullseye-updates main contrib non-free
deb http://deb.debian.org/debian bullseye-backports main contrib non-free

# zabbix
deb https://repo.zabbix.com/zabbix/5.4/debian bullseye main

# docker
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable

# mysql
deb http://repo.mysql.com/apt/debian/ bullseye mysql-apt-config
deb http://repo.mysql.com/apt/debian/ bullseye mysql-8.0
deb http://repo.mysql.com/apt/debian/ bullseye mysql-tools

# powerdns
deb [arch=amd64] http://repo.powerdns.com/debian bullseye-auth-47 main
deb [arch=amd64] http://repo.powerdns.com/debian bullseye-rec-47 main

0x03.3 其他及收尾

有些二进制包没有限制系统,只写一行即可:

# elasticsearch
deb https://artifacts.elastic.co/packages/7.x/apt stable main

最后还需要清理:

# clean
clean http://ftp.hk.debian.org/debian
clean https://repo.zabbix.com/zabbix/5.2/debian
clean https://download.docker.com/linux/debian
clean http://repo.mysql.com/apt/debian/
clean https://artifacts.elastic.co/packages/7.x/apt
clean http://repo.powerdns.com/debian
clean http://deb.debian.org/debian

完整的配置文件如下:

############# config ##################
#
set base_path    /mnt/lv01/apt-mirror
#
set mirror_path  $base_path/mirror
set skel_path    $base_path/skel
set var_path     $base_path/var
set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     3
set _tilde 0
set limit_rate 30m
#
############# end config ##############

############# Debian 10 Buster ##############
# Debian 10 Buster
deb http://deb.debian.org/debian buster main contrib non-free
deb http://deb.debian.org/debian buster-backports main contrib non-free
deb http://deb.debian.org/debian buster-updates main contrib non-free

# zabbix
deb https://repo.zabbix.com/zabbix/5.4/debian buster main

# docker
deb [arch=amd64] https://download.docker.com/linux/debian buster stable

# mysql
deb http://repo.mysql.com/apt/debian/ buster mysql-apt-config
deb http://repo.mysql.com/apt/debian/ buster mysql-5.7
deb http://repo.mysql.com/apt/debian/ buster mysql-tools

# powerdns
deb [arch=amd64] http://repo.powerdns.com/debian buster-auth-47 main
deb [arch=amd64] http://repo.powerdns.com/debian buster-rec-47 main

############# Debian 11 bullseye ##############
# Debian 11 bullseye
deb http://deb.debian.org/debian bullseye main contrib non-free
deb http://deb.debian.org/debian-security/ bullseye-security main contrib non-free
deb http://deb.debian.org/debian bullseye-updates main contrib non-free
deb http://deb.debian.org/debian bullseye-backports main contrib non-free

# zabbix
deb https://repo.zabbix.com/zabbix/5.4/debian bullseye main

# docker
deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable

# mysql
deb http://repo.mysql.com/apt/debian/ bullseye mysql-apt-config
deb http://repo.mysql.com/apt/debian/ bullseye mysql-8.0
deb http://repo.mysql.com/apt/debian/ bullseye mysql-tools

# powerdns
deb [arch=amd64] http://repo.powerdns.com/debian bullseye-auth-47 main
deb [arch=amd64] http://repo.powerdns.com/debian bullseye-rec-47 main

############## Common ##############
# elasticsearch
deb https://artifacts.elastic.co/packages/7.x/apt stable main

# clean
clean http://ftp.hk.debian.org/debian
clean https://repo.zabbix.com/zabbix/5.2/debian
clean https://download.docker.com/linux/debian
clean http://repo.mysql.com/apt/debian/
clean https://artifacts.elastic.co/packages/7.x/apt
clean http://repo.powerdns.com/debian
clean http://deb.debian.org/debian

写入计划任务:

0 2 * * * root apt-mirror >> /var/log/apt-mirror.log

最后执行同步命令:

apt-mirror

根据网络、磁盘及下载文件的大小,同步时长可能长达几小时,我同步上面的资源总共占用395G空间。

0x04 http服务

在这里我使用nginx,使用以下命令安装:

apt install nginx -y

配置文件如下:

server {
    listen                  80;
    server_name             debian-mirror.internal.ngx.hk debian10-mirror.internal.ngx.hk debian11-mirror.internal.ngx.hk;

    root                    /usr/local/html/debian-mirror.internal.ngx.hk/public_html;
    access_log              off;

    location / {
      autoindex on;
      autoindex_exact_size off;
      autoindex_localtime on;
   }
}

创建nginx文件夹:

mkdir -p /usr/local/html/debian-mirror.internal.ngx.hk/public_html

创建软链接:

ln -s /mnt/lv01/apt-mirror/mirror/deb.debian.org/debian/ /usr/local/html/debian-mirror.internal.ngx.hk/public_html/debian
ln -s /mnt/lv01/apt-mirror/mirror/deb.debian.org/debian-security/ /usr/local/html/debian-mirror.internal.ngx.hk/public_html/debian-security
ln -s /mnt/lv01/apt-mirror/mirror/download.docker.com/linux/debian/ /usr/local/html/debian-mirror.internal.ngx.hk/public_html/docker-ce
ln -s /mnt/lv01/apt-mirror/mirror/artifacts.elastic.co/packages/7.x/apt/ /usr/local/html/debian-mirror.internal.ngx.hk/public_html/elastic
ln -s /mnt/lv01/apt-mirror/mirror/repo.mysql.com/apt/debian/ /usr/local/html/debian-mirror.internal.ngx.hk/public_html/mysql
ln -s /mnt/lv01/apt-mirror/mirror/repo.powerdns.com /usr/local/html/debian-mirror.internal.ngx.hk/public_html/powerdns
ln -s /mnt/lv01/apt-mirror/mirror/repo.zabbix.com/zabbix/5.4/debian/ /usr/local/html/debian-mirror.internal.ngx.hk/public_html/zabbix

 

建议在http服务内创建一个pgp文件夹,将所有pgp key都保存到本地备用。创建系统模板的时候建议预先导入pgp key。

0x04.1 debian10 source.list

将以下内容写入:

/usr/local/html/debian-mirror.internal.ngx.hk/public_html/debian10-mirror.internal.ngx.hk.list

# Debian 10 Buster
deb http://debian-mirror.internal.ngx.hk/debian/ buster main contrib non-free
deb http://debian-mirror.internal.ngx.hk/debian/ buster-backports main contrib non-free
deb http://debian-mirror.internal.ngx.hk/debian/ buster-updates main contrib non-free

# zabbix
deb http://debian-mirror.internal.ngx.hk/zabbix buster main

# docker-ce
deb http://debian-mirror.internal.ngx.hk/docker-ce/ buster stable

# elasticsearch
deb http://debian-mirror.internal.ngx.hk/elastic/ stable main

# mysql
deb http://debian-mirror.internal.ngx.hk/mysql/ buster mysql-apt-config
deb http://debian-mirror.internal.ngx.hk/mysql/ buster mysql-5.7
deb http://debian-mirror.internal.ngx.hk/mysql/ buster mysql-tools

# powerdns
deb http://debian-mirror.internal.ngx.hk/powerdns/debian buster-auth-47 main
deb http://debian-mirror.internal.ngx.hk/powerdns/debian buster-rec-47 main

0x04.2 debian11 source.list

将以下内容写入:

/usr/local/html/debian-mirror.internal.ngx.hk/public_html/debian11-mirror.internal.ngx.hk.list

# Debian 11 bullseye
deb http://debian-mirror.internal.ngx.hk/debian/ bullseye main contrib non-free
deb http://debian-mirror.internal.ngx.hk/debian-security/ bullseye-security main contrib non-free
deb http://debian-mirror.internal.ngx.hk/debian/ bullseye-updates main contrib non-free
deb http://debian-mirror.internal.ngx.hk/debian/ bullseye-backports main contrib non-free

# zabbix
deb http://debian-mirror.internal.ngx.hk/zabbix bullseye main

# docker-ce
deb http://debian-mirror.internal.ngx.hk/docker-ce/ bullseye stable

# elasticsearch
deb http://debian-mirror.internal.ngx.hk/elastic/ stable main

# mysql
deb http://debian-mirror.internal.ngx.hk/mysql/ bullseye mysql-apt-config
deb http://debian-mirror.internal.ngx.hk/mysql/ bullseye mysql-5.7
deb http://debian-mirror.internal.ngx.hk/mysql/ bullseye mysql-tools

# powerdns
deb http://debian-mirror.internal.ngx.hk/powerdns/debian bullseye-auth-47 main
deb http://debian-mirror.internal.ngx.hk/powerdns/debian bullseye-rec-47 main

最后重启nginx即可。

0x05 结语

在内网有大量虚拟机的情况下,本地镜像是非常有用的,在禁止访问公网的环境中也是如此。

为进一步提升自动化,我会在创建系统模板时定时下载更新镜像中的source.list文件:

root@demo1-ngx-p1:~# cat /opt/ngx-shell/get-ngx-repo.sh
#!/bin/sh

REPO_DOMAIN="debian10-mirror.internal.ngx.hk"

HTTP_STATUS=$(curl -I -w '%{http_code}' -s -o /dev/null "$REPO_DOMAIN"/"$REPO_DOMAIN".list)

if [ "$HTTP_STATUS" -ne 200 ] ; then
        echo "error with http status code $HTTP_STATUS" && exit 1
else
        rm -f /etc/apt/*.list
        curl "$REPO_DOMAIN"/"$REPO_DOMAIN".list > /etc/apt/sources.list
fi

将其加入定时任务:

0 6 * * * root sh /opt/ngx-shell/get-ngx-repo.sh