python镜像环境搭建总结

2024-04-02 19:55

短信预约 -IT技能 免费直播动态提醒

取最小安装包

使用python-3.7.3-embed-win32，注意对：python37._pth的修改，修改方法：

# uncomment_pth.py
from pathlib import Path
def main(path_cur):
    for p in path_cur.glob('*._pth'):
        if not (p.suffix == '._pth' and p.stem.startswith('python')):
            continue
        print('found: ' + str(p))
        f_lines = []
        with p.open() as f:
            for line in f.readlines():
                if line.startswith('#import site'):
                    line = line.split('#', 2)[1]
                f_lines.append(line)
        with p.open('w') as f:
            f.writelines(f_lines)
        return True
    return False

if __name__ == '__main__':
    path_cur = Path('.')
    main(path_cur)

使用get-pip.py安装pip，并在有网终的情况下安装好环境

::first_install_with_net.bat
 
set old_dir=%cd%
 
cd /d "%~dp0"
 
python uncomment_pth.py
python get-pip.py
pip install devpi devpi-web
 
cd /d "%old_dir%"
pause

使用pip打包好环境，注意打包get-pip.py

::pkg_py_env.bat
 
set req=..\req_pkg_py_env.txt
set whs=..\wheels
pip freeze >%req%
pip wheel -r %req% -w %whs%
pip wheel pip -w %whs%
pip wheel setuptools -w %whs%
pip wheel wheel -w %whs%
pip uninstall wheel -y
pip uninstall setuptools -y
:: pip 不能用pip卸载自己，需要用下列语句
python -m pip uninstall pip -y
rd /s /q Lib
rd /s /q Scripts
pause

压缩清理后的环境，并传入有网络的环境后，解压基础环境

::unpkg_py_env.bat
 
set req=..\req_pkg_py_env.txt
set whs=..\wheels
set get-pip=%~dp0get-pip.py
set uncomment=%~dp0uncomment_pth.py
python %uncomment%
python %get-pip% --no-index --find-links=%whs%
set PATH=%PATHON_HOME%\Scripts
pip install  -r %req% --no-index --find-links=%whs%
 
pause

编写配置环境变量的脚本

:: py_env.bat
 
@echo off
set old_dir=%cd%
cd /d "%~dp0"
set PATHON_HOME=%cd%
set PATH=%PATHON_HOME%\;%PATHON_HOME%\Scripts;%PATH%
title Ptython Env:PATHON_HOME=%PATHON_HOME%
cd /d "%old_dir%"
 
if "%1" == "" call cmd 
 
REM @echo off
REM set PYTHONHOME=%cd%
REM set PATH=%PYTHONHOME%/Scripts;%PYTHONHOME%;%PATH%
 
REM set PYTHONPATH=%PYTHONHOME%/Lib
REM title %PYTHONHOME%\python.exe
REM call cmd

安装devpi-server服务，并初始化好环境，这里需要依赖nssm，需要自行下载，注意nssm需要管理员权限

::install_service.bat
 
set svr_name=devpi
set svr_dir=d:\devip_server_datas
cd /d "%~dp0"
::用于获取权限
nssm
call py_env.bat
devpi-init --serverdir %svr_dir%
devpi-gen-config --serverdir %svr_dir%
nssm install %svr_name% "%cd%\Scripts\devpi-server.exe" --serverdir %svr_dir% --host 0.0.0.0
pause
nssm start %svr_name%
devpi use root/pypi --always-set-cfg=yes
pause

如果不需要了，则卸载相关服务

::remove_service.bat
 
net stop devpi
nssm remove devpi confirm
pause

镜像更新问题

编写基础功能脚本文件，并放到pythong.exe同级目录

# tasks.py
 
import datetime
import io
import os
from invoke import task
from tqdm import tqdm
 
# @task
# def package(c):
#     dir_path = datetime.datetime.now().strftime('%Y_%m_%d-%H_%m_%S-devpi_server_datas')
#     c.run(f'devpi-export.exe --serverdir D:\\devip_server_datas --include-mirrored-files {dir_path}')
#     c.run(fr'"C:\Program Files\WinRAR\Rar.exe" a {dir_path}.zip {dir_path}')
#     # print('hello')

@task(iterable=['p'])
def down(c, p):
    from pathlib import Path
    Path(r'tmp_whl_dirs').mkdir(exist_ok=True)
    c.run(fr'cd tmp_whl_dirs & pip download {" ".join(p)}')
@task
def up(c):
    dir_path = datetime.datetime.now().strftime('%Y_%m_%d-%H_%m_%S-devpi_server_datas')
    c.run(f'devpi-export.exe --serverdir D:\\devip_server_datas --include-mirrored-files {dir_path}')
    c.run(fr'"C:\Program Files\WinRAR\Rar.exe" a {dir_path}.zip  {dir_path}')
 
    from ftplib import FTP
    ftp = FTP('xxx')
    ftp.login('xxx', 'xxxxx')
    # ftp.cwd('devip_server_datas')
 
    with open(dir_path + '.zip', 'rb') as f:
        f.seek(0, io.SEEK_END)
        file_size = f.tell()
        f.seek(0, io.SEEK_SET)
        with tqdm(unit='blocks', unit_scale=True, leave=True, miniters=1, desc='Uploading......',
                  total=file_size) as tqdm_instance:
            ftp.storbinary(f'STOR {dir_path}.zip', f, 2048, callback=lambda sent: tqdm_instance.update(len(sent)))
    c.run('net stop devpi')
    c.run('net start devpi')
        # ftp.storbinary(f'STOR {dir_path}.zip', f)
    # print('finished!!')
    # os.unlink(dir_path)

下载需要的包：

:: down.bat
@echo off
set old_dir=%cd%
cd /d "%~dp0"
set PATHON_HOME=%cd%
set PATH=%PATHON_HOME%\;%PATHON_HOME%\Scripts;%PATH%
title Ptython Env:PATHON_HOME=%PATHON_HOME%
 
:download
 
set /p down_pkgs=请输入需要下载的包名,用空格隔开:
inv down -p "%down_pkgs%"
 
goto :download
pause

导出需要镜像的包，并上传镜像包到无网络的环境里

:: up_load.bat
@echo off
set old_dir=%cd%
cd /d "%~dp0"
set PATHON_HOME=%cd%
set PATH=%PATHON_HOME%\;%PATHON_HOME%\Scripts;%PATH%
title Ptython Env:PATHON_HOME=%PATHON_HOME%
REM set /p down_pkgs=请输入需要下载的包名 用空格隔开
inv up
pause

导入镜像