正在显示
2 个修改的文件
包含
72 行增加
和
8 行删除
readme-cn.md
0 → 100644
1 | +[English](readme.md) | 简体中文 | ||
2 | + | ||
3 | +# go-stash简介 | ||
4 | + | ||
5 | +go-stash是一个高效的从Kafka获取,根据配置的规则进行处理,然后发送到ElasticSearch集群的工具。 | ||
6 | + | ||
7 | +go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可执行文件即可。 | ||
8 | + | ||
9 | +![go-stash](doc/flow.png) | ||
10 | + | ||
11 | +## Quick Start | ||
12 | + | ||
13 | +```shell | ||
14 | +gostash -f etc/config.yaml | ||
15 | +``` | ||
16 | + | ||
17 | +config.yaml示例如下: | ||
18 | + | ||
19 | +```yaml | ||
20 | +Processors: | ||
21 | +- Input: | ||
22 | + Kafka: | ||
23 | + Name: gostash | ||
24 | + Brokers: | ||
25 | + - "172.16.186.16:19092" | ||
26 | + - "172.16.186.17:19092" | ||
27 | + Topic: k8slog | ||
28 | + Group: pro | ||
29 | + NumProducers: 16 | ||
30 | + Filters: | ||
31 | + - Action: drop | ||
32 | + Conditions: | ||
33 | + - Key: k8s_container_name | ||
34 | + Value: "-rpc" | ||
35 | + Type: contains | ||
36 | + - Key: level | ||
37 | + Value: info | ||
38 | + Type: match | ||
39 | + Op: and | ||
40 | + - Action: remove_field | ||
41 | + Fields: | ||
42 | + - message | ||
43 | + - _source | ||
44 | + - _type | ||
45 | + - _score | ||
46 | + - _id | ||
47 | + - "@version" | ||
48 | + - topic | ||
49 | + - index | ||
50 | + - beat | ||
51 | + - docker_container | ||
52 | + - offset | ||
53 | + - prospector | ||
54 | + - source | ||
55 | + - stream | ||
56 | + Output: | ||
57 | + ElasticSearch: | ||
58 | + Hosts: | ||
59 | + - "172.16.141.4:9200" | ||
60 | + - "172.16.141.5:9200" | ||
61 | + Index: {.event}-2006.01.02 | ||
62 | +``` | ||
63 | + | ||
64 | +### 微信交流群 | ||
65 | + | ||
66 | +添加我的微信:kevwan,请注明go-stash,我拉进go-stash社区群🤝 |
1 | -# go-stash简介 | 1 | +English | [简体中文](readme-cn.md) |
2 | 2 | ||
3 | -go-stash是一个高效的从Kafka获取,根据配置的规则进行处理,然后发送到ElasticSearch集群的工具。 | 3 | +# go-stash |
4 | 4 | ||
5 | -go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可执行文件即可。 | 5 | +go-stash is a high performance, free and open source server-side data processing pipeline that ingests data from Kafka, processes it, and then sends it to ElasticSearch. |
6 | + | ||
7 | +go-stash is about 4x throughput more than logstash, and easy to deploy, only one executable file. | ||
6 | 8 | ||
7 | ![go-stash](doc/flow.png) | 9 | ![go-stash](doc/flow.png) |
8 | 10 | ||
@@ -12,7 +14,7 @@ go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可 | @@ -12,7 +14,7 @@ go-stash有大概logstash 5倍的吞吐性能,并且部署简单,一个可 | ||
12 | gostash -f etc/config.yaml | 14 | gostash -f etc/config.yaml |
13 | ``` | 15 | ``` |
14 | 16 | ||
15 | -config.yaml示例如下: | 17 | +config.yaml example as below: |
16 | 18 | ||
17 | ```yaml | 19 | ```yaml |
18 | Processors: | 20 | Processors: |
@@ -58,7 +60,3 @@ Processors: | @@ -58,7 +60,3 @@ Processors: | ||
58 | - "172.16.141.5:9200" | 60 | - "172.16.141.5:9200" |
59 | Index: {.event}-2006.01.02 | 61 | Index: {.event}-2006.01.02 |
60 | ``` | 62 | ``` |
61 | - | ||
62 | -### 微信交流群 | ||
63 | - | ||
64 | -添加我的微信:kevwan,请注明go-stash,我拉进go-stash社区群🤝 |
-
请 注册 或 登录 后发表评论