`
侯上校
  • 浏览: 217100 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

Flume写数据换行

阅读更多
在Flume写入HDFS的时候会自动换行问题, 项目要求写入HDFS中数据不需要换行, 查看源码及配置如下:
BodyTextEventSerializer.java
  @Override
  public void write(Event e) throws IOException {
    out.write(e.getBody());
    if (appendNewline) {
      out.write('\n');
    }
  }
  // for legacy reasons, by default, append a newline to each event written out
  private final String APPEND_NEWLINE = "appendNewline";
  private final boolean APPEND_NEWLINE_DFLT = false;

  private final OutputStream out;
  private final boolean appendNewline;

  private BodyTextEventSerializer(OutputStream out, Context ctx) {
    this.appendNewline = ctx.getBoolean(APPEND_NEWLINE, APPEND_NEWLINE_DFLT);
    this.out = out;
  }

http://flume.apache.org/FlumeUserGuide.html中查看此配置项为:
Body Text Serializer

Alias: text. This interceptor writes the body of the event to an output stream without any transformation or modification. The event headers are ignored. Configuration options are as follows:

Property Name	Default	Description
appendNewline	true	Whether a newline will be appended to each event at write time. The default of true assumes that events do not contain newlines, for legacy reasons.
Example for agent named a1:

a1.sinks = k1
a1.sinks.k1.type = file_roll
a1.sinks.k1.channel = c1
a1.sinks.k1.sink.directory = /var/log/flume
a1.sinks.k1.sink.serializer = text
a1.sinks.k1.sink.serializer.appendNewline = false

 

分享到:
评论

相关推荐

Global site tag (gtag.js) - Google Analytics