local pipy mirror server

紀錄架設 python pip 用的 module server。一般不會真的把全部mirror進來,但是要也不是不行!

需要mirror pipy全部的modules 可以用 bandersnatch

環境,準備VM,準備Storage

twcc create VM or any linux ubunt host machine

current storage check:" https://p.datadoghq.com/sb/7dc8b3250-85dcf667bd"

2020-914, current storagr is 12TiB

We prepare 15TiB for this poc, considerate more storage, we enable LVM2.

after VM ready...

====

base requirment

需要空間可能比較大,所以我用了LVM,必要時可以擴展空間!

gdisk /dev/vdb // create vdb1 with 15TiB yourself pvcreate /dev/vdb1 gvcreate data /dev/sdb1 lvcreate ... mkfs mount xxx /srv/pypi/ edit fstab

install python and pip

apt-get install python3 apt-get install python3-pip

install bandersnatch

install bandersnatch

pip install bandersnatch
python3.6 -m venv bandersnatch

bandersnatch/bin/pip install bandersnatch
bandersnatch/bin/bandersnatch --help

config and run mirror

change to root foe config file geretation

create /etc/bandersnatch.conf

sudo su -
. bin/active
bandersnatch mirror

run

check configuration generated by bandersnatch

less /etc/bandersnatch.conf
bandersnatch mirror

crontab

*/2 * * * * root bandersnatch mirror |& logger -t bandersnatch[mirror]

apache2 and cert

prepare web server and cert

sudo apt-get install apache2
sudo add-apt-repository ppa:certbot/certbot
sudo apt install python-certbot-apache
sudo certbot --apache  -d pypi.libthomas.org
sudo ln -s /srv/pypi/web/ pypi

use local mirror

test local pipy mirror site

vi ~/.pip/pip.conf

[global]
index-url = https://pypi.libthomas.org/pypi/simple

pip install ckanapi

reference

https://pypi.org/project/bandersnatch/

Comments