翻译:索引文档中包含了一个巨大的字段
java.lang.IllegalArgumentException: Document contains at least one immense term in field="responseData" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[34, 91, 123, 92, 34, 99, 97, 112, 116, 105, 111, 110, 92, 34, 58, 92, 34, -26, -75, -117, -24, -81, -107, -26, -75, -117, -24, -81, -107, -29]...', original message: bytes can be at most 32766 in length; got 41756at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:772)at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417)at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373)at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:558)at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:520)at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:409)at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:556)at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:546)at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:191)at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:144)at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:63)at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:75)at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:48)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:905)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:875)at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113)at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:323)at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:258)at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:855)at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:852)at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:142)at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1655)at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:864)at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:90)at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:275)at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:254)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:246)at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:577)at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527)at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
Caused by: NotSerializableExceptionWrapper[max_bytes_length_exceeded_exception: bytes can be at most 32766 in length; got 41756]at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:263)at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:149)at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:762)... 38 more
java.lang.IllegalArgumentException: Document contains at least one immense term in field="responseData" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[34, 123, 92, 34, 100, 97, 116, 97, 92, 34, 58, 91, 123, 92, 34, 105, 100, 92, 34, 58, 92, 34, 97, 57, 57, 51, 48, 48, 100, 56]...', original message: bytes can be at most 32766 in length; got 62581at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:772)at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417)at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373)at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:558)at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:520)at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:409)at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:556)at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:546)at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:191)at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:144)at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:63)at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:75)at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:48)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:905)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:875)at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113)at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:323)at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:258)at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:855)at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:852)at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:142)at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1655)at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:864)at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:90)at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:275)at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:254)at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:246)at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:577)at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527)at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
Caused by: NotSerializableExceptionWrapper[max_bytes_length_exceeded_exception: bytes can be at most 32766 in length; got 62581]at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:263)at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:149)at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:762)... 38 more
提示信息说 responseData 超出了最大长度(the max length 32766)
解决方法:设置字段长度 ignore_above :256 (长度超过256个字节部分忽视)
ElasticSearch已经提供了解决方案,就是ignore_above示例配置如下:curl -XPUT 'http://localhost:9200/twitter' -d '
{
"mappings":{"tweet" : {"properties" : {"message" : {"type" : "string", "index":"not_analyzed","ignore_above":256 }}}}}
}
其中tweet下的message字段不做分词等处理,直接将原始内容来做索引,当内容长度大于256字节时,只索引前面256个字符,后面的内容被丢弃。这样就不会出现前文所提的immense term的错误了。一般ignore_above设置就是为not_analyzed字段存在的,不可滥用。
参考:https://blog.csdn.net/iteye_6322/article/details/82647099