[MDEV-27147] JSON Error log to STDERR/STDOUT as an option Created: 2021-12-01 Updated: 2023-04-03 |
|
| Status: | Stalled |
| Project: | MariaDB Server |
| Component/s: | N/A |
| Fix Version/s: | N/A |
| Type: | Task | Priority: | Major |
| Reporter: | Richard Bensley (Inactive) | Assignee: | Daniel Black |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Description |
|
In the world of container orchestration, we have a common scenario in which we must collect and forward all log data produced by MariaDB, that isn't already forwarded to a file descriptor (slow query log, audit log, sql error log, etc). It is not always possible, practical or secure for a log tailing process to mount the same data volumes that a VM or container is using. Ideally we would want all the log data to be piped out to STDOUT or STDERR so we can capture all the data from a running container process. If the log data was formatted, using JSON for example, we could easily separate out the different log types using a key, for example "log"="slow". |
| Comments |
| Comment by Daniel Black [ 2021-12-01 ] | ||
|
The use of a table within the server has limitations, especially where security assessments want to see separation of audit logs. A straight 'log-output=JSON' to cover error log, slow query, general log all to stdout allows stream based message queue and event processors rather than a restricted table storage option. The stdout capture by other containers in the pod keeps a good separation and still gains the sidecar option without volume sharing. If this streamed data is ingested and stored elsewhere the analytics can be security separated and running on different hardware. Does this suit your needs? | ||
| Comment by Richard Bensley (Inactive) [ 2021-12-02 ] | ||
|
danblackyes! A structured log output, would be excellent. Especially if I could separate the messages by type/source. We are using fluentbit instead of fluentd. | ||
| Comment by Richard Bensley (Inactive) [ 2022-07-01 ] | ||
|
Closing this as it doesn't make sense. I shall create a new ticket with my ideas on structured log output instead. | ||
| Comment by Birk Bohne [ 2023-03-31 ] | ||
|
In my opinion we need an option to send all generated logs to stdout/stderr in the JSON format. The structure should be based on a schema and contain all relevant info's in dedicated fields. The Elastic Common Schema and the Splunk Common Information Model are good examples for that. That log line
should become something like this
| ||
| Comment by Richard Bensley (Inactive) [ 2023-04-03 ] | ||
|
danblack I updated the description to hopefully be more clear. |