Skip to content

Commit 11171dc

Browse files
zsoltgyulai94zsoltgyulai94
authored
loki() destination (#18)
Documented Loki Grafana destination. Included new reusable option snippets.
2 parents 8489a4e + b0d222c commit 11171dc

File tree

6 files changed

+246
-0
lines changed

6 files changed

+246
-0
lines changed

‎_data/external_links.yml

+20
Original file line numberDiff line numberDiff line change
@@ -152,3 +152,23 @@ lunr-search-help:
152152
id: lunr-search-help
153153
url: https://lunrjs.com/guides/searching.html
154154
title: [ "Lunar search help" ]
155+
156+
grafana-loki:
157+
id: grafana-loki
158+
url: https://grafana.com/docs/loki/latest/
159+
title: [ "Grafana Loki" ]
160+
161+
grafana-loki-endpoint:
162+
id: grafana-loki-endpoint
163+
url: https://grafana.com/docs/loki/latest/reference/loki-http-api/#push-log-entries-to-loki
164+
title: [ "Grafana Loki HTTP endpoint" ]
165+
166+
app-default-cred:
167+
id: app-default-cred
168+
url: https://cloud.google.com/docs/authentication/application-default-credentials
169+
title: [ "Application Default Credentials" ]
170+
171+
grpc-core:
172+
id: grpc-core
173+
url: https://grpc.github.io/grpc/core/group__grpc__arg__keys.html
174+
title: [ "GRPC Core library documentation" ]

‎_includes/doc/admin-guide/options/channel-args.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
## channel-args()
2+
3+
| Type:| |
4+
|Default:| |
5+
6+
*Description:* The `channel-args()` option is available in gRPC-based drivers. The option accepts name-value pairs and sets channel arguments defined in the GRPC Core library documentation.
7+
8+
### Example: channel-args() declaration
9+
10+
```config
11+
channel-args(
12+
"grpc.loadreporting" => 1
13+
"grpc.minimal_stack" => 0
14+
)
15+
```

‎_includes/doc/admin-guide/options/gRPC-keep-alive.md

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
## keep-alive()
2+
3+
This option configures the forwarding of [gRPC keepalive pings](https://grpc.io/docs/guides/keepalive/) in syslog-ng OSE.
4+
5+
### max-pings-without-data()
6+
7+
| Type:| integer|
8+
|Default:| |
9+
10+
*Description:* This option definies the maximum number of gRPC pings that are allowed to be sent when there is no data/header frame. Any pings succeeding this limit are not sent. Setting this option to `0` disables this restriction and keep sending pings.
11+
12+
### time()
13+
14+
| Type:| number[milliseconds]|
15+
|Default:| |
16+
17+
*Description:* syslog-ng OSE sends a gRPC keepalive ping after the amount of time defined in the `time()` option elapsed.
18+
19+
### timeout()
20+
21+
| Type:| number[milliseconds]|
22+
|Default:| 10|
23+
24+
*Description:* The time syslog-ng OSE waits for an acknowledgement.

‎doc/_admin-guide/060_Sources/035_Jellyfin/README.md

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
title: Jellyfin log source
3+
short_title: Jellyfin
4+
id: adm-src-jfin
5+
description: >-
6+
In syslog-ng OSE 4.7 and later versions it is possible to use the `jellyfin()` source to read [Jellyfin](https://jellyfin.org/) logs from its log file output.
7+
---
8+
9+
### Example: minimal configuration of jellyfin()
10+
11+
```config
12+
source s_jellyfin {
13+
jellyfin(
14+
base-dir("/path/to/jellyfin/root/log/dir")
15+
filename-pattern("log_*.log")
16+
);
17+
};
18+
```
19+
20+
The `jellyfin()` source can use wildcard-file() source options, since it is based on the `wildcard-file()` source.
21+
22+
The `jellyfin()` driver is a reusable configuration snippet. For more information on using configuration snippets, see Reusing configuration blocks. The source of this configuration snippet can be accessed as the Jellyfish config file on GitHub. [GitHub](https://github.com/syslog-ng/syslog-ng/blob/master/scl/jellyfin/jellyfin.conf).
23+
24+
For more information about Jellyfin logs, see the Jellyfin Main Configuration and Jellyfin Log Directory section in the Jellyfin documentation.
25+
* [https://jellyfin.org/docs/general/administration/configuration/#main-configuration](https://jellyfin.org/docs/general/administration/configuration/#main-configuration)
26+
* [https://jellyfin.org/docs/general/administration/configuration/#log-directory](https://jellyfin.org/docs/general/administration/configuration/#log-directory)

‎doc/_admin-guide/070_Destinations/125_Loki/001_Loki_options.md

+136
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
---
2+
title: loki() destination options
3+
id: adm-dest-loki-opt
4+
---
5+
6+
The `loki()` driver sends messages to a Loki Grafana database and has the following options:
7+
8+
## auth()
9+
10+
The `auth()` option can be used to set the authentication of the driver. The default state of this option is `insecure`, as it is not defined.
11+
12+
The following sub-options are available for `auth()`.
13+
14+
### adc()
15+
16+
This option is an authentication method that is only available for destinations. For more information, see Application Default Credentials.
17+
18+
### alts()
19+
20+
This option is an accessible authentication available for Google infrastructures. Service accounts can be listed with the nested `target-service-account()` option, to match these against the server.
21+
22+
#### Example: configure a Loki destination using auth(alts())
23+
24+
```config
25+
destination {
26+
loki(
27+
port(1234)
28+
auth(alts())
29+
);
30+
};
31+
```
32+
33+
### insecure()
34+
35+
This option can be used to disable authentication: `auth(insecure())`.
36+
37+
### tls()
38+
39+
The `tls()` option accepts the following nested sub-options.
40+
* ca-file()
41+
* key-file()
42+
* cert-file()
43+
44+
#### Example: configure a Loki destination using auth(tls())
45+
46+
```config
47+
destination {
48+
loki(
49+
url("loki-server:123")
50+
auth(
51+
tls(
52+
ca-file("/path/to/ca.pem")
53+
key-file("/path/to/key.pem")
54+
cert-file("/path/to/cert.pem")
55+
)
56+
)
57+
);
58+
};
59+
```
60+
61+
{% include doc/admin-guide/options/batch-bytes.md %}
62+
63+
{% include doc/admin-guide/options/batch-lines.md %}
64+
65+
{% include doc/admin-guide/options/batch-timeout.md %}
66+
67+
{% include doc/admin-guide/options/channel-args.md %}
68+
69+
{% include doc/admin-guide/options/gRPC-keep-alive.md %}
70+
71+
## labels()
72+
73+
This option defines the labels applied to the message as they are sent to the destination.
74+
75+
**Declaration:**
76+
77+
```config
78+
labels(
79+
"label-output-name" => "field-of-the-message"
80+
)
81+
```
82+
83+
## template()
84+
85+
| Type:| template or template-function|
86+
|Default:| ${ISODATE} ${HOST} ${MSGHDR} ${MSG}|
87+
88+
*Description:* This option specifies a template that defines the logformat to be used in the destination. For more information on macros and template functions, see Macros of syslog-ng OSE and Template functions of syslog-ng OSE.
89+
90+
## tenant-id()
91+
92+
| Type:| string|
93+
|Default:| |
94+
95+
Available in syslog-ng OSE 4.7 and later versions.
96+
97+
*Description:* This option sets the tenant ID for multi-tenant cases.
98+
99+
**Declaration:**
100+
101+
```config
102+
loki(
103+
url("localhost:9096")
104+
labels(
105+
"app" => "$PROGRAM",
106+
"host" => "$HOST",
107+
)
108+
109+
tenant-id("testTenant")
110+
);
111+
```
112+
113+
## timestamp()
114+
115+
| Type:| `current`, `received`, `msg`|
116+
|Default:| `current`|
117+
118+
*Description:* This option sets the timestamp type to be used for messages sent to a Loki destination.
119+
120+
**NOTE:** Loki destinations only accept subsequent messages with increasing timestamps. Messages with timestamps deviating from this are rejected.
121+
{: .notice--info}
122+
123+
The timestamp types are the following.
124+
125+
* `current`: The message procession output timestamp is used. This type guarantees an increasing timestamp order, but can deviate significantly from the message generation time.
126+
* `msg`: The original timestamp of the message is used.
127+
* `received`: The timestamp of message reception is used.
128+
129+
## url()
130+
131+
| Type:| string|
132+
|Default:| localhost:9095|
133+
134+
*Description:* This option specifies the URL of the Loki endpoint.
135+
136+
{% include doc/admin-guide/options/workers.md %}

‎doc/_admin-guide/070_Destinations/125_Loki/README.md

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
title: 'loki(): Storing messages in a Grafana Loki database'
3+
short_title: Loki
4+
id: adm-dest-loki
5+
description: >-
6+
In syslog-ng OSE 4.4 and later versions the `loki()` destination can be used to send log data to Grafana Loki.
7+
8+
For more information on the message format, see Grafna Loki HTTP endpoint.
9+
---
10+
11+
### Example: loki() destination configuration
12+
13+
```config
14+
loki(
15+
url("localhost:9096")
16+
labels(
17+
"app" => "$PROGRAM",
18+
"host" => "$HOST",
19+
)
20+
21+
workers(16)
22+
batch-timeout(10000)
23+
batch-lines(1000)
24+
);
25+
```

0 commit comments

Comments
 (0)