作者 kevin

add english readme

  1 +[English](readme.md) | 简体中文
  2 +
  3 +# go-stash简介
  4 +
  5 +go-stash是一个高效的从Kafka获取,根据配置的规则进行处理,然后发送到ElasticSearch集群的工具。
  6 +
  7 +go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可执行文件即可。
  8 +
  9 +![go-stash](doc/flow.png)
  10 +
  11 +## Quick Start
  12 +
  13 +```shell
  14 +gostash -f etc/config.yaml
  15 +```
  16 +
  17 +config.yaml示例如下:
  18 +
  19 +```yaml
  20 +Processors:
  21 +- Input:
  22 + Kafka:
  23 + Name: gostash
  24 + Brokers:
  25 + - "172.16.186.16:19092"
  26 + - "172.16.186.17:19092"
  27 + Topic: k8slog
  28 + Group: pro
  29 + NumProducers: 16
  30 + Filters:
  31 + - Action: drop
  32 + Conditions:
  33 + - Key: k8s_container_name
  34 + Value: "-rpc"
  35 + Type: contains
  36 + - Key: level
  37 + Value: info
  38 + Type: match
  39 + Op: and
  40 + - Action: remove_field
  41 + Fields:
  42 + - message
  43 + - _source
  44 + - _type
  45 + - _score
  46 + - _id
  47 + - "@version"
  48 + - topic
  49 + - index
  50 + - beat
  51 + - docker_container
  52 + - offset
  53 + - prospector
  54 + - source
  55 + - stream
  56 + Output:
  57 + ElasticSearch:
  58 + Hosts:
  59 + - "172.16.141.4:9200"
  60 + - "172.16.141.5:9200"
  61 + Index: {.event}-2006.01.02
  62 +```
  63 +
  64 +### 微信交流群
  65 +
  66 +添加我的微信:kevwan,请注明go-stash,我拉进go-stash社区群🤝
1 -# go-stash简介 1 +English | [简体中文](readme-cn.md)
2 2
3 -go-stash是一个高效的从Kafka获取,根据配置的规则进行处理,然后发送到ElasticSearch集群的工具。 3 +# go-stash
4 4
5 -go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可执行文件即可。 5 +go-stash is a high performance, free and open source server-side data processing pipeline that ingests data from Kafka, processes it, and then sends it to ElasticSearch.
  6 +
  7 +go-stash is about 4x throughput more than logstash, and easy to deploy, only one executable file.
6 8
7 ![go-stash](doc/flow.png) 9 ![go-stash](doc/flow.png)
8 10
@@ -12,7 +14,7 @@ go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可 @@ -12,7 +14,7 @@ go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可
12 gostash -f etc/config.yaml 14 gostash -f etc/config.yaml
13 ``` 15 ```
14 16
15 -config.yaml示例如下: 17 +config.yaml example as below:
16 18
17 ```yaml 19 ```yaml
18 Processors: 20 Processors:
@@ -58,7 +60,3 @@ Processors: @@ -58,7 +60,3 @@ Processors:
58 - "172.16.141.5:9200" 60 - "172.16.141.5:9200"
59 Index: {.event}-2006.01.02 61 Index: {.event}-2006.01.02
60 ``` 62 ```
61 -  
62 -### 微信交流群  
63 -  
64 -添加我的微信:kevwan,请注明go-stash,我拉进go-stash社区群🤝