如果我想使用这个项目支持自己的模型模型，应该做什么调整？

[Hapluckyy]

可以出一个教程，或是通用的模板，让我可以根据自己的模型填写config.yaml文件吗

[ssbuild]

例如基础模型是llama:

1. copy 一份model_handler/llama/infer.py ，改成你想要的模块名字
2. 模版使用方法参考

   aigc_serving/serving/model_handler/llama/infer.py

   Line 160 in d93761a

   if self.is_openbuddy:

3. aigc_serving/serving/workers/llm_worker.py

   Line 15 in d93761a

   def get_worker_instance(model_name,config,group_name,worker_idx):

   
   根据 model_name 或者 model_type 导入自己的模块即可。

[Hapluckyy]

对不起，我还不是很明白

import torch
from transformers import AutoModel, AutoTokenizer
# GPU设置
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# 加载模型与tokenizer
model_name_or_path = 'scutcyr/BianQue-2'
model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True).half()
model.to(device)
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)

请问如果我想使用这种方式加载模型，可以对该项目进行修改吗
谢谢大佬解答！

[ssbuild]

如果不会改的话，可以提个issue ， 说明需要支持的模型 ， 有时间的话就会加上。

[Hapluckyy]

请您看一下这两个，放不方便加一下，谢谢
https://huggingface.co/scutcyr/BianQue-2
https://huggingface.co/deeplang-ai/LingoWhale-8B

[ssbuild]

it should be ok
注意 lingowhale 不是聊天模型，建议自己微调之后再加载到服务。对应模型模版自行修改。

[Hapluckyy]

非常感谢！

[Hapluckyy]

您好，我又遇到问题了 在部署bianque2（扁鹊2）模型时，我遇到了下面的问题：

Traceback (most recent call last):
  File "/aigc_serving/serving/model_handler/base/infer.py", line 136, in worker_ds
    self.init_model()
  File "/aigc_serving/serving/model_handler/base/infer.py", line 113, in init_model
    self.model, self.config, self.tokenizer = call_method(device_id)
  File "/aigc_serving/serving/model_handler/chatglm/infer.py", line 34, in _load_model
    tokenizer, config, _, _ = dataHelper.load_tokenizer_and_config(tokenizer_class_name=ChatGLMTokenizer,
  File "/usr/local/lib/python3.10/dist-packages/deep_training/data_helper/data_helper.py", line 190, in load_tokenizer_and_config
    tokenizer = load_tokenizer(tokenizer_name=tokenizer_name or model_args.tokenizer_name,
  File "/usr/local/lib/python3.10/dist-packages/numpy_io/pytorch_loader/tokenizer_config_helper.py", line 37, in load_tokenizer
    tokenizer = class_name.from_pretrained(tokenizer_name or model_name_or_path, **tokenizer_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 2024, in from_pretrained
    return cls._from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 2256, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/aigc_zoo/model_zoo/chatglm/tokenization_chatglm.py", line 195, in __init__
    super().__init__(
  File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py", line 367, in __init__
    self._add_tokens(
  File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py", line 467, in _add_tokens
    current_vocab = self.get_vocab().copy()
  File "/usr/local/lib/python3.10/dist-packages/aigc_zoo/model_zoo/chatglm/tokenization_chatglm.py", line 247, in get_vocab
    vocab = {self._convert_id_to_token(i): i for i in range(self.vocab_size)}
  File "/usr/local/lib/python3.10/dist-packages/aigc_zoo/model_zoo/chatglm/tokenization_chatglm.py", line 243, in vocab_size
    return self.sp_tokenizer.num_tokens
AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'
INFO:root:'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'

请问您知道是为什么吗

[ssbuild]

#131

[Hapluckyy]

问题解决啦，谢谢大佬