@repeatedly , In #1856 issue, I'm very happy to see you add a nice feature to backup for bad chunk. In our production environment, I found it works. But when I check the bad chunk I don't know what tag is it. So I suggest also save the meta information. It is useful for check and repair data.

Bug Report

The [STREAM_TASK] configuration block was documented for v1.3, but I'm unable to find documentation for this in the v1.4 docs.

我修改了config.filebeat
cat >> $FILEBEAT_CONFIG << EOF
$(base)
output.elasticsearch:
hosts: [$ELASTICSEARCH_HOSTS]
index: ${ELASTICSEARCH_INDEX:-filebeat}-%{+yyyy.MM}
${ELASTICSEARCH_SCHEME:+protocol: ${ELASTICSEARCH_SCHEME}}
${ELASTICSEARCH_USER:+username: ${ELASTICSEARCH_USER}}
${ELASTICSEARCH_PASSWORD:+password: ${ELASTICSEARCH_PASSWORD}}
${ELASTICSEARCH_WORKER:+wor

can you add documentation to deploy loghouse on docker only (just front end and backend without clickhouse and kubenetes deployement) ?
like docker run command ?

Consider a scenario where:

• master process attaches fluent Sender or logging Handler
• master process forks off a bunch of worker processes
• workers use handler

Will the socket get cloned or reopened?
Does fluent logger expect a response from fluentd?
If it does, can the response be read out by wrong sender process?

related to fabric8io/fluent-plugin-kubernetes_metadata_filter#41
maybe this comment can fix the issue: fabric8io/fluent-plugin-kubernetes_metadata_filter@c9fa728#diff-83453022bdb83378e6c454e99caacd36
thus I have to verify whether plugin version bump will help

Problem

IAM user in readme.md requires too many permissions

Steps to replicate

Currently, README.md recommends creating a IAM user with the following policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:*",
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:logs:us-east-1:*:*",

I was testing this project (and it's pretty cool! - great job!) but it took me some time to get the right apache2 log message. It might be helpful to show one just so folks can see how it works? Anyway, you can close this issue if you don't think it's necessary, but I'm just thinking about the developer experience.

Without further ado -- I used this string:

127.0.0.1 - frank [10/Oct/20

Forced termination caused corruption of pos file. Mutations to pos file could be done atomically to increase reliability.

fluent-plugin-systemd version 1.0.1

From Logs

2018-07-05 07:02:41 +0000 [error]: [systemd.log] failed to read data from plugin storage file path="/var/log/systemd.log.pos" error_class=Yajl::ParseError error="lexical error: invalid char in json text.\n           

The module currently does not support cognito_options config block.
See https://www.terraform.io/docs/providers/aws/r/elasticsearch_domain.html for details.

It would be good to have this feature supported.