RobertaTokenizer从本地路径加载模型报错
收藏
tokenizer.save_pretrained(args.output_dir) # 加载保存的模型和tokenizer model = model_class.from_pretrained(args.output_dir) tokenizer = tokenizer_class.from_pretrained(args.output_dir) model.to(args.device)
报错的是这一句:model = model_class.from_pretrained(args.output_dir), 这里model_class我设置为了RobertaModel,tokenizer_class设置为RobertaTokenizer。 报错信息如下:
[2022-11-10 16:20:49,907] [ INFO] - tokenizer config file saved in model_files/chinese_model/mrc/tokenizer_config.json [2022-11-10 16:20:49,908] [ INFO] - Special tokens file saved in model_files/chinese_model/mrc/special_tokens_map.json Traceback (most recent call last): File "ote_model/mrc_paddle/main.py", line 413, in main() File "ote_model/mrc_paddle/main.py", line 390, in main tokenizer = tokenizer_class.from_pretrained(args.output_dir) File "/home/liangqiao2022/miniconda3/envs/paddle_env/lib/python3.8/site-packages/paddlenlp/transformers/roberta/tokenizer.py", line 665, in from_pretrained return RobertaBPETokenizer.from_pretrained( File "/home/liangqiao2022/miniconda3/envs/paddle_env/lib/python3.8/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 1638, in from_pretrained tokenizer = cls(*init_args, **init_kwargs) File "/home/liangqiao2022/miniconda3/envs/paddle_env/lib/python3.8/site-packages/paddlenlp/transformers/utils.py", line 159, in __impl__ init_func(self, *args, **kwargs) TypeError: __init__() missing 2 required positional arguments: 'vocab_file' and 'merges_file'
为什么会这样?按道理说from_pretrained()函数后面两个参数不要也行。求大佬解释
0
收藏
请登录后评论
可以尝试降低PaddlePaddle的版本至2.3.0