xreference
安装
pip install "xinference[all]" -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install "xinference[transformers]" -i https://pypi.tuna.tsinghua.edu.cn/simple
大模型下载的环境变量
export HF_ENDPOINT=https://hf-mirror.com
export XINFERENCE_MODEL_SRC=modelscope
运行
xinference-local -H 0.0.0.0
