others-how to solve db-gpt startup error?

Oct 23, 2023

1. Purpose

In this post, I will show you how to solve db-gpt startup error as follows:

(dbgpt_env) root@my_ubuntu:/opt/dbgpt# python pilot/server/dbgpt_server.py
2023-10-12 06:08:40 | INFO | numexpr.utils | Note: NumExpr detected 64 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
2023-10-12 06:08:40 | INFO | numexpr.utils | NumExpr defaulting to 8 threads.


=========================== WebWerverParameters ===========================

host: 0.0.0.0
port: 5000
daemon: False
share: False
remote_embedding: False
log_level: INFO
light: False

======================================================================


/opt/dbgpt/pilot
2023-10-12 06:08:41 | INFO | pilot.component | Register component with name dbgpt_thread_pool_default and instance: <pilot.utils.executor_utils.DefaultExecutorFactory object at 0x7fdb12a21d50>
2023-10-12 06:08:41 | INFO | pilot.component | Register component with name dbgpt_model_controller and instance: <pilot.model.cluster.controller.controller.ModelControllerAdapter object at 0x7fdbdede6b30>
2023-10-12 06:08:41 | INFO | pilot.server.component_configs | Register local LocalEmbeddingFactory
2023-10-12 06:08:41 | INFO | pilot.model.cluster.worker.embedding_worker | [EmbeddingsModelWorker] Parameters of device is None, use cpu
2023-10-12 06:08:41 | INFO | pilot.server.component_configs |

=========================== EmbeddingModelParameters ===========================

model_name: text2vec
model_path: /opt/dbgpt/models/text2vec-large-chinese
device: cpu
normalize_embeddings: None

======================================================================


2023-10-12 06:08:42 | INFO | sentence_transformers.SentenceTransformer | Load pretrained SentenceTransformer: /opt/dbgpt/models/text2vec-large-chinese
2023-10-12 06:08:42 | WARNING | sentence_transformers.SentenceTransformer | No sentence-transformers model found with name /opt/dbgpt/models/text2vec-large-chinese. Creating a new one with MEAN pooling.
/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/torch/cuda/__init__.py:546: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
2023-10-12 06:08:44 | INFO | pilot.component | Register component with name embedding_factory and instance: <pilot.server.component_configs.LocalEmbeddingFactory object at 0x7fdb12a50040>
Model Unified Deployment Mode!
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Worker params:

=========================== ModelWorkerParameters ===========================

model_name: vicuna-13b-v1.5
model_path: /opt/dbgpt/models/vicuna-13b-v1.5
worker_type: None
worker_class: None
host: 0.0.0.0
port: 5000
daemon: False
limit_model_concurrency: 5
standalone: True
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20

======================================================================


2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Run WorkerManager with standalone mode, controller_addr: http://127.0.0.1:5000
  Found llm model adapter with model name: vicuna-13b-v1.5, <pilot.model.adapter.VicunaLLMAdapater object at 0x7fdc783cafb0>
2023-10-12 06:08:44 | INFO | LOGGER | Found llm model adapter with model name: vicuna-13b-v1.5, <pilot.model.adapter.VicunaLLMAdapater object at 0x7fdc783cafb0>
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.default_worker | model_name: vicuna-13b-v1.5, model_path: /opt/dbgpt/models/vicuna-13b-v1.5, model_param_class: <class 'pilot.model.parameter.ModelParameters'>
Get model chat adapter with model name vicuna-13b-v1.5, <pilot.server.chat_adapter.VicunaChatAdapter object at 0x7fdc783cbfd0>
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.default_worker | [DefaultModelWorker] Parameters of device is None, use cpu
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Init empty instances list for vicuna-13b-v1.5@llm
2023-10-12 06:08:44 | INFO | pilot.component | Register component with name dbgpt_worker_manager_factory and instance: <pilot.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x7fdd124abe20>
INFO:     Started server process [172117]
INFO:     Waiting for application startup.
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Begin start all worker, apply_req: None
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.<locals>._start_worker at 0x7fdd0fe7b520>
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Apply to all workers
INFO:     Application startup complete.
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.default_worker | Begin load model, model params:

=========================== ModelParameters ===========================

model_name: vicuna-13b-v1.5
model_path: /opt/dbgpt/models/vicuna-13b-v1.5
device: cpu
model_type: huggingface
prompt_template: None
max_context_size: 4096
num_gpus: None
max_gpu_memory: None
cpu_offloading: False
load_8bit: True
load_4bit: False
quant_type: nf4
use_double_quant: True
compute_dtype: None
trust_remote_code: True
verbose: False

======================================================================


INFO:     Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit)
  8-bit quantization and 4-bit quantization just supported by cuda
2023-10-12 06:08:44 | WARNING | LOGGER | 8-bit quantization and 4-bit quantization just supported by cuda
2023-10-12 06:08:44 | ERROR | pilot.model.cluster.worker.manager | Error starting worker manager: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/dbgpt/models/vicuna-13b-v1.5'. Use `repo_type` argument if needed.
2023-10-12 06:08:44 | ERROR | asyncio | Task exception was never retrieved
future: <Task finished name='Task-3' coro=<_setup_fastapi.<locals>.startup_event.<locals>.start_worker_manager() done, defined at /opt/dbgpt/pilot/model/cluster/worker/manager.py:656> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 658, in start_worker_manager
    await worker_manager.start()
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 487, in start
    return await self.worker_manager.start()
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 107, in start
    await self._start_all_worker(apply_req=None)
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 419, in _start_all_worker
    await self._apply_worker(apply_req, _start_worker)
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 387, in _apply_worker
    return await asyncio.gather(
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 398, in _start_worker
    await self.run_blocking_func(
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 103, in run_blocking_func
    return await loop.run_in_executor(self.executor, func, *args)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/dbgpt/pilot/model/cluster/worker/default_worker.py", line 80, in start
    self.model, self.tokenizer = self.ml.loader_with_params(model_params)
  File "/opt/dbgpt/pilot/model/loader.py", line 120, in loader_with_params
    return huggingface_loader(llm_adapter, model_params)
  File "/opt/dbgpt/pilot/model/loader.py", line 182, in huggingface_loader
    model, tokenizer = llm_adapter.loader(model_params.model_path, kwargs)
  File "/opt/dbgpt/pilot/model/adapter.py", line 145, in loader
    tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 686, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 519, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/utils/hub.py", line 429, in cached_file
    resolved_file = hf_hub_download(
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
    validate_repo_id(arg_value)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
    raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/dbgpt/models/vicuna-13b-v1.5'. Use `repo_type` argument if needed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle
  File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run
  File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 661, in start_worker_manager
    sys.exit(1)
SystemExit: 1
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Stop all workers
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.<locals>._stop_worker at 0x7fdd0fedc670>
2023-10-12 06:08:44 | INFO | pilot.model.cluster.worker.manager | Apply to all workers
2023-10-12 06:08:44 | WARNING | pilot.model.cluster.worker.default_worker | Model has been stopped!!
INFO:     127.0.0.1:37884 - "DELETE /api/controller/models?model_name=WorkerManager%40service&host=192.168.11.3&port=5000&weight=1.0&check_healthy=true&healthy=false&enabled=true&prompt_template=&last_heartbeat= HTTP/1.1" 200 OK
2023-10-12 06:08:44 | INFO | httpx | HTTP Request: DELETE http://127.0.0.1:5000/api/controller/models?model_name=WorkerManager%40service&host=192.168.11.3&port=5000&weight=1.0&check_healthy=true&healthy=false&enabled=true&prompt_template=&last_heartbeat= "HTTP/1.1 200 OK"
INFO:     127.0.0.1:37900 - "DELETE /api/controller/models?model_name=vicuna-13b-v1.5%40llm&host=192.168.11.3&port=5000&weight=1.0&check_healthy=true&healthy=false&enabled=true&prompt_template=&last_heartbeat= HTTP/1.1" 200 OK
2023-10-12 06:08:44 | INFO | httpx | HTTP Request: DELETE http://127.0.0.1:5000/api/controller/models?model_name=vicuna-13b-v1.5%40llm&host=192.168.11.3&port=5000&weight=1.0&check_healthy=true&healthy=false&enabled=true&prompt_template=&last_heartbeat= "HTTP/1.1 200 OK"
ERROR:    Traceback (most recent call last):
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 658, in start_worker_manager
    await worker_manager.start()
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 487, in start
    return await self.worker_manager.start()
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 107, in start
    await self._start_all_worker(apply_req=None)
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 419, in _start_all_worker
    await self._apply_worker(apply_req, _start_worker)
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 387, in _apply_worker
    return await asyncio.gather(
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 398, in _start_worker
    await self.run_blocking_func(
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 103, in run_blocking_func
    return await loop.run_in_executor(self.executor, func, *args)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/dbgpt/pilot/model/cluster/worker/default_worker.py", line 80, in start
    self.model, self.tokenizer = self.ml.loader_with_params(model_params)
  File "/opt/dbgpt/pilot/model/loader.py", line 120, in loader_with_params
    return huggingface_loader(llm_adapter, model_params)
  File "/opt/dbgpt/pilot/model/loader.py", line 182, in huggingface_loader
    model, tokenizer = llm_adapter.loader(model_params.model_path, kwargs)
  File "/opt/dbgpt/pilot/model/adapter.py", line 145, in loader
    tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 686, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 519, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/utils/hub.py", line 429, in cached_file
    resolved_file = hf_hub_download(
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
    validate_repo_id(arg_value)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
    raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/dbgpt/models/vicuna-13b-v1.5'. Use `repo_type` argument if needed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 555, in uvloop.loop.Loop._run
  File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle
  File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run
  File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 661, in start_worker_manager
    sys.exit(1)
SystemExit: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/starlette/routing.py", line 686, in lifespan
    await receive()
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/uvicorn/lifespan/on.py", line 137, in receive
    return await self.receive_queue.get()
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/asyncio/queues.py", line 159, in get
    await getter
asyncio.exceptions.CancelledError

The core error is huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/dbgpt/models/vicuna-13b-v1.5'. Use repo_type argument if needed.：

ERROR:    Traceback (most recent call last):
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 658, in start_worker_manager
    await worker_manager.start()
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 487, in start
    return await self.worker_manager.start()
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 107, in start
    await self._start_all_worker(apply_req=None)
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 419, in _start_all_worker
    await self._apply_worker(apply_req, _start_worker)
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 387, in _apply_worker
    return await asyncio.gather(
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 398, in _start_worker
    await self.run_blocking_func(
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 103, in run_blocking_func
    return await loop.run_in_executor(self.executor, func, *args)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/dbgpt/pilot/model/cluster/worker/default_worker.py", line 80, in start
    self.model, self.tokenizer = self.ml.loader_with_params(model_params)
  File "/opt/dbgpt/pilot/model/loader.py", line 120, in loader_with_params
    return huggingface_loader(llm_adapter, model_params)
  File "/opt/dbgpt/pilot/model/loader.py", line 182, in huggingface_loader
    model, tokenizer = llm_adapter.loader(model_params.model_path, kwargs)
  File "/opt/dbgpt/pilot/model/adapter.py", line 145, in loader
    tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 686, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 519, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/transformers/utils/hub.py", line 429, in cached_file
    resolved_file = hf_hub_download(
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
    validate_repo_id(arg_value)
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
    raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/dbgpt/models/vicuna-13b-v1.5'. Use `repo_type` argument if needed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda/envs/dbgpt_env/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 555, in uvloop.loop.Loop._run
  File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle
  File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run
  File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run
  File "/opt/dbgpt/pilot/model/cluster/worker/manager.py", line 661, in start_worker_manager
    sys.exit(1)
SystemExit: 1

The models used in db-gpt: there are two models in the directory:

ubuntu@xxx:/opt/dbgpt$ ll models/
total 16
drwxr-xr-x  4 root root 4096 Oct 12 06:16 ./
drwxr-xr-x 16 root root 4096 Oct 12 06:27 ../
drwxrwxr-x  2 1014 1015 4096 Jul 17 06:23 chatglm2-6b/
drwxrwxr-x  3 1014 1015 4096 Jul 17 07:05 text2vec-large-chinese/
ubuntu@xxxx:/opt/dbgpt$

2. Solution

The problem is caused by the misconfigured environment settings in db-gpt, we can solve it by doing the following:

Add .env file add the root directory of db-gpt as follows:

#*******************************************************************#
#**             DB-GPT  - GENERAL SETTINGS                        **#
#*******************************************************************#
## DISABLED_COMMAND_CATEGORIES - The list of categories of commands that are disabled. Each of the below are an option:
## pilot.commands.query_execute

## For example, to disable coding related features, uncomment the next line
# DISABLED_COMMAND_CATEGORIES=

#*******************************************************************#
#**                        Webserver Port                         **#
#*******************************************************************#
WEB_SERVER_PORT=7860

#*******************************************************************#
#***                       LLM PROVIDER                          ***#
#*******************************************************************#

# TEMPERATURE=0

#*******************************************************************#
#**                         LLM MODELS                            **#
#*******************************************************************#
# LLM_MODEL, see /pilot/configs/model_config.LLM_MODEL_CONFIG
#LLM_MODEL=vicuna-13b
LLM_MODEL=chatglm2-6b
MODEL_SERVER=http://127.0.0.1:8000
LIMIT_MODEL_CONCURRENCY=5
MAX_POSITION_EMBEDDINGS=4096
QUANTIZE_QLORA=True
## SMART_LLM_MODEL - Smart language model (Default: vicuna-13b)
## FAST_LLM_MODEL - Fast language model (Default: chatglm-6b)
# SMART_LLM_MODEL=vicuna-13b
# FAST_LLM_MODEL=chatglm-6b


#*******************************************************************#
#**                         EMBEDDING SETTINGS                    **#
#*******************************************************************#
EMBEDDING_MODEL=text2vec
KNOWLEDGE_CHUNK_SIZE=500
KNOWLEDGE_SEARCH_TOP_SIZE=5
## EMBEDDING_TOKENIZER   - Tokenizer to use for chunking large inputs
## EMBEDDING_TOKEN_LIMIT - Chunk size limit for large inputs
# EMBEDDING_MODEL=all-MiniLM-L6-v2
# EMBEDDING_TOKENIZER=all-MiniLM-L6-v2
# EMBEDDING_TOKEN_LIMIT=8191


#*******************************************************************#
#**                  DATABASE SETTINGS                            **#
#*******************************************************************#
#LOCAL_DB_USER=root
#LOCAL_DB_PASSWORD=xxxxxx
#LOCAL_DB_HOST=127.0.0.1
#LOCAL_DB_PORT=3306


### MILVUS
## MILVUS_ADDR - Milvus remote address (e.g. localhost:19530)
## MILVUS_USERNAME - username for your Milvus database
## MILVUS_PASSWORD - password for your Milvus database
## MILVUS_SECURE - True to enable TLS. (Default: False)
##   Setting MILVUS_ADDR to a `https://` URL will override this setting.
## MILVUS_COLLECTION - Milvus collection, change it if you want to start a new memory and retain the old memory.
# MILVUS_ADDR=localhost:19530
# MILVUS_USERNAME=
# MILVUS_PASSWORD=
# MILVUS_SECURE=
# MILVUS_COLLECTION=dbgpt

#*******************************************************************#
#**                         COMMANDS                              **#
#*******************************************************************#
EXECUTE_LOCAL_COMMANDS=False



#*******************************************************************#
#**                  ALLOWLISTED PLUGINS                          **#
#*******************************************************************#

#ALLOWLISTED_PLUGINS - Sets the listed plugins that are allowed (Example: plugin1,plugin2,plugin3)
#DENYLISTED_PLUGINS - Sets the listed plugins that are not allowed (Example: plugin1,plugin2,plugin3)
ALLOWLISTED_PLUGINS=
DENYLISTED_PLUGINS=


#*******************************************************************#
#**                 CHAT PLUGIN SETTINGS                          **#
#*******************************************************************#
# CHAT_MESSAGES_ENABLED - Enable chat messages (Default: False)
# CHAT_MESSAGES_ENABLED=False


#*******************************************************************#
#**                  VECTOR STORE SETTINGS                       **#
#*******************************************************************#
VECTOR_STORE_TYPE=Chroma
#MILVUS_URL=127.0.0.1
#MILVUS_PORT=19530
#MILVUS_USERNAME
#MILVUS_PASSWORD
#MILVUS_SECURE=


#*******************************************************************#
#**                  WebServer Language Support                   **#
#*******************************************************************#
#LANGUAGE=en
LANGUAGE=zh


#*******************************************************************#
# **    PROXY_SERVER (openai interface | chatGPT proxy service), use chatGPT as your LLM.
# ** if your server can visit openai, please set PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
# ** else if you have a chatgpt proxy server, you can set PROXY_SERVER_URL={your-proxy-serverip:port/xxx}
#*******************************************************************#
PROXY_API_KEY={your-openai-sk}
PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions


#*******************************************************************#
# **    SUMMARY_CONFIG
#*******************************************************************#
SUMMARY_CONFIG=FAST

Now start again:

NFO:     Started server process [173179]
INFO:     Waiting for application startup.
2023-10-12 06:27:27 | INFO | pilot.model.cluster.worker.manager | Begin start all worker, apply_req: None
2023-10-12 06:27:27 | INFO | pilot.model.cluster.worker.manager | Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.<locals>._start_worker at 0x7ff73d47f0a0>
2023-10-12 06:27:27 | INFO | pilot.model.cluster.worker.manager | Apply to all workers
INFO:     Application startup complete.
2023-10-12 06:27:27 | INFO | pilot.model.cluster.worker.default_worker | Begin load model, model params:

=========================== ModelParameters ===========================

model_name: chatglm2-6b
model_path: /opt/dbgpt/models/chatglm2-6b
device: cpu
model_type: huggingface
prompt_template: None
max_context_size: 4096
num_gpus: None
max_gpu_memory: None
cpu_offloading: False
load_8bit: True
load_4bit: False
quant_type: nf4
use_double_quant: True
compute_dtype: None
trust_remote_code: True
verbose: False

======================================================================


INFO:     Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit)
  8-bit quantization and 4-bit quantization just supported by cuda
2023-10-12 06:27:27 | WARNING | LOGGER | 8-bit quantization and 4-bit quantization just supported by cuda
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████| 7/7 [00:07<00:00,  1.14s/it]
INFO:     127.0.0.1:32876 - "POST /api/controller/models HTTP/1.1" 200 OK
2023-10-12 06:27:35 | INFO | httpx | HTTP Request: POST http://127.0.0.1:5000/api/controller/models "HTTP/1.1 200 OK"
INFO:     127.0.0.1:32878 - "POST /api/controller/heartbeat HTTP/1.1" 200 OK
INFO:     127.0.0.1:32892 - "POST /api/controller/models HTTP/1.1" 200 OK

It’s working.

3. Summary

In this post, I demonstrated how to solve the huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/dbgpt/models/vicuna-13b-v1.5'. Use repo_type argument if needed. error when trying to startup db-gpt on ubuntu. That’s it, thanks for your reading.