kettle 批量导入到 doris 报错。
187*****902 发布于2021-10 浏览:2484 回复:6
0
收藏

版本  V0.14

我们生产环境的问题,救急!!!!

kettle 抓取数据以后 导入到 doris 使用的是mysql协议。 每次导入到 500条的 时候 必报错。请问是什么原因?

2021-10-15 15:27:43,872 WARN (thrift-server-pool-3|546) [Coordinator.updateFragmentExecStatus():1369] one instance report fail, query_id=b85d66d45c3c4441-8f2ccf697660bf3a instance_id=b85d66d45c3c4441-8f2ccf697660bf3b
2021-10-15 15:27:43,872 WARN (thrift-server-pool-3|546) [Coordinator.updateStatus():670] one instance report fail throw updateStatus(), need cancel. job id: -1, query id: b85d66d45c3c4441-8f2ccf697660bf3a, instance id: b85d66d45c3c4441-8f2ccf697660bf3b
2021-10-15 15:27:43,873 WARN (doris-mysql-nio-pool-153|570) [StmtExecutor.handleInsertStmt():846] insert failed: already stopped, skip waiting for close. cancelled/!eos: : 1/0
2021-10-15 15:27:43,873 WARN (doris-mysql-nio-pool-153|570) [StmtExecutor.handleInsertStmt():894] handle insert stmt fail: insert_b85d66d45c3c4441-8f2ccf697660bf3a
org.apache.doris.common.DdlException: errCode = 2, detailMessage = already stopped, skip waiting for close. cancelled/!eos: : 1/0
at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:67) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.handleInsertStmt(StmtExecutor.java:847) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:326) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-10-15 15:27:43,875 WARN (doris-mysql-nio-pool-153|570) [StmtExecutor.execute():380] errors when abort txn
org.apache.doris.transaction.TransactionNotFoundException: errCode = 2, detailMessage = transaction not found
at org.apache.doris.transaction.DatabaseTransactionMgr.abortTransaction(DatabaseTransactionMgr.java:949) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:210) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:205) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:376) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-10-15 15:27:43,880 WARN (thrift-server-pool-1|151) [Coordinator.updateFragmentExecStatus():1369] one instance report fail, query_id=ce663cab6e747bc-a0bf4b95266cd3fb instance_id=ce663cab6e747bc-a0bf4b95266cd3fc
2021-10-15 15:27:43,880 WARN (thrift-server-pool-1|151) [Coordinator.updateStatus():670] one instance report fail throw updateStatus(), need cancel. job id: -1, query id: ce663cab6e747bc-a0bf4b95266cd3fb, instance id: ce663cab6e747bc-a0bf4b95266cd3fc
2021-10-15 15:27:43,880 WARN (doris-mysql-nio-pool-152|569) [StmtExecutor.handleInsertStmt():846] insert failed: already stopped, skip waiting for close. cancelled/!eos: : 1/0
2021-10-15 15:27:43,881 WARN (doris-mysql-nio-pool-152|569) [StmtExecutor.handleInsertStmt():894] handle insert stmt fail: insert_ce663cab6e747bc-a0bf4b95266cd3fb
org.apache.doris.common.DdlException: errCode = 2, detailMessage = already stopped, skip waiting for close. cancelled/!eos: : 1/0
at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:67) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.handleInsertStmt(StmtExecutor.java:847) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:326) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-10-15 15:27:43,882 WARN (doris-mysql-nio-pool-152|569) [StmtExecutor.execute():380] errors when abort txn
org.apache.doris.transaction.TransactionNotFoundException: errCode = 2, detailMessage = transaction not found
at org.apache.doris.transaction.DatabaseTransactionMgr.abortTransaction(DatabaseTransactionMgr.java:949) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:210) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:205) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:376) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]

收藏
点赞
0
个赞
共6条回复 最后由简一Zero回复于22天前
#9简一Zero回复于22天前
#3 187*****902回复
好的 已经解决了,刚开始以为使用 sql insert 批量插入没有啥问题,后来仔细看了下文档,不支持这种方式的频繁导入。
展开

咨询一下,是怎么解决的?

0
#8DwayneLIU回复于2022-08
#3 187*****902回复
好的 已经解决了,刚开始以为使用 sql insert 批量插入没有啥问题,后来仔细看了下文档,不支持这种方式的频繁导入。
展开

请问你是怎么解决的

 

0
#7客人画册回复于2022-08

务必用这种sql做批量导入

insert into table (id, name, desc)
values
(?, ?, ?),
(?, ?, ?),
(?, ?, ?),
(?, ?, ?)

0
#4小刚night回复于2022-03

你用kettle连接doris了吗,我用kettle连接doris总是提示报这个错。You don't seem to be getting a connection to the server. Please check the path you're using and make sure the server is up and running.

0
#3187*****902回复于2021-10

好的 已经解决了,刚开始以为使用 sql insert 批量插入没有啥问题,后来仔细看了下文档,不支持这种方式的频繁导入。

1
#2IamStrangers回复于2021-10

doris暂不支持实时数据导入,使用时需要按批导入,在业务侧自行控制导入频率,建议5秒间隔以上

0
快速回复
TOP
切换版块