We are experiencing a recurring parser error in Fluent Bit.

[IanIkeda]

Bug Report

Describe the bug
Fluent Bit is intermittently reporting the following parser error:

[error] [parser] time string length is too long

This error occurred once on Feb 5th, then did not appear for about a month, and started happening again from Mar 11th onward.
No configuration changes were made to Fluent Bit, EKS, or our application during this period.
AWS Support suggested that audit.log rotation around Mar 11 might have triggered the issue, but the root cause remains unclear.

---

To Reproduce

• Rubular link if applicable: N/A
• Example log message:

[error] [parser] time string length is too long

• Steps to reproduce the problem:
  1. Use Fluent Bit (tail + parser) to read /var/log/audit/audit.log
  2. After audit.log rotates, the parser error begins to appear again
  3. The issue does not occur consistently for every line

---

Expected behavior

• Fluent Bit should parse audit.log timestamps without producing
  time string length is too long
• Log rotation should not cause parsing failures

---

Screenshots

• CloudWatch Logs showing the parser error

Your Environment

• Version used: 2.12.0
• Configuration:
  • Full configuration (all INPUT / FILTER / OUTPUT definitions) is attached as containers.conf（containersconf.txt）
  • Main config (fluent-bit.conf) only contains SERVICE section and @include directives
• Environment name and version (e.g. Kubernetes? What version?):
  • Amazon EKS / Kubernetes Version: 1.31
• Server type and version:
  • Node instance type: m5.large
• Operating System and version:
  • OS architecture: linux (amd64)
  • OS image: Amazon Linux 2
  • Kernel version: 5.10.228-219.884.amzn2.x86_64
• Filters and plugins (summary):

[INPUT]

• tail: fluentbit.*
• tail: cloudwatch-agent.*
• tail: kube-proxy.*
• tail: aws-node.*
• tail: coredns.*

[FILTER]

• kubernetes (for each tag)
• nest / modify filters for restructuring log fields

[OUTPUT]

• cloudwatch (for each tag)

---

Additional context

• The error reappeared immediately after audit.log rotation on Mar 11.
• Even if rotation is related, we cannot explain why the error continues afterward.
• AWS Support could not identify a root cause and recommended checking with Fluent Bit maintainers.

---

Questions

1. Under what conditions does the parser return
   time string length is too long?
2. Could audit.log rotation trigger this behavior?
3. Are there any known issues with parsing audit.log timestamps?
4. What additional information would be helpful for further investigation?

[IanIkeda]

The file upload failed in the main issue body, so I am pasting the full containers.conf here.

[INPUT]
    Name                tail
    Tag                 fluentbit.*
    Path                /var/log/containers/fluentbit*
    Parser              docker
    DB                  /var/fluent-bit/state/flb_log.db
    Mem_Buf_Limit       5MB
    Skip_Long_Lines     On
    Refresh_Interval    10
    Read_from_Head      ${READ_FROM_HEAD}

[INPUT]
    Name                tail
    Tag                 cloudwatch-agent.*
    Path                /var/log/containers/cloudwatch-agent*
    Docker_Mode         On
    Docker_Mode_Flush   5
    Docker_Mode_Parser  cwagent_firstline
    Parser              docker
    DB                  /var/fluent-bit/state/flb_cwagent.db
    Mem_Buf_Limit       5MB
    Skip_Long_Lines     On
    Refresh_Interval    10
    Read_from_Head      ${READ_FROM_HEAD}

[INPUT]
    Name                tail
    Tag                 kube-proxy.*
    Path                /var/log/containers/kube-proxy*
    Parser              docker
    DB                  /var/fluent-bit/state/flb_kubeproxy.db
    Mem_Buf_Limit       5MB
    Skip_Long_Lines     On
    Refresh_Interval    10
    Read_from_Head      ${READ_FROM_HEAD}

[INPUT]
    Name                tail
    Tag                 aws-node.*
    Path                /var/log/containers/aws-node*
    Parser              docker
    DB                  /var/fluent-bit/state/flb_aws-node.db
    Mem_Buf_Limit       5MB
    Skip_Long_Lines     On
    Refresh_Interval    10
    Read_from_Head      ${READ_FROM_HEAD}

[INPUT]
    Name                tail
    Tag                 coredns.*
    Path                /var/log/containers/coredns*
    Parser              docker
    DB                  /var/fluent-bit/state/flb_coredns.db
    Mem_Buf_Limit       5MB
    Skip_Long_Lines     On
    Refresh_Interval    10
    Read_from_Head      ${READ_FROM_HEAD}

[FILTER]
    Name                kubernetes
    Match               fluentbit.*
    Kube_URL            ${KUBE_MASTER_URL}
    Kube_Tag_Prefix     fluentbit.var.log.containers.
    Merge_Log           On
    Merge_Log_Key       log_processed
    K8S-Logging.Parser  On
    K8S-Logging.Exclude Off
    Annotations         Off

[FILTER]
    Name                kubernetes
    Match               cloudwatch-agent.*
    Kube_URL            ${KUBE_MASTER_URL}
    Kube_Tag_Prefix     cloudwatch-agent.var.log.containers.
    Merge_Log           On
    Merge_Log_Key       log_processed
    K8S-Logging.Parser  On
    K8S-Logging.Exclude Off
    Annotations         Off

[FILTER]
    Name                kubernetes
    Match               kube-proxy.*
    Kube_URL            ${KUBE_MASTER_URL}
    Kube_Tag_Prefix     kube-proxy.var.log.containers.
    Merge_Log           On
    Merge_Log_Key       log_processed
    K8S-Logging.Parser  On
    K8S-Logging.Exclude Off
    Annotations         Off

[FILTER]
    Name                kubernetes
    Match               aws-node.*
    Kube_URL            ${KUBE_MASTER_URL}
    Kube_Tag_Prefix     aws-node.var.log.containers.
    Merge_Log           On
    Merge_Log_Key       log_processed
    K8S-Logging.Parser  On
    K8S-Logging.Exclude Off
    Annotations         Off

[FILTER]
    Name                kubernetes
    Match               coredns.*
    Kube_URL            ${KUBE_MASTER_URL}
    Kube_Tag_Prefix     coredns.var.log.containers.
    Merge_Log           On
    Merge_Log_Key       log_processed
    K8S-Logging.Parser  On
    K8S-Logging.Exclude Off
    Annotations         Off

[FILTER]
    Name                nest
    Match               *
    Operation           lift
    Nested_under        kubernetes
    Add_prefix          Nested.

[FILTER]
    Name                modify
    Match               *
    Rename              Nested.docker_id            Docker.container_id

[FILTER]
    Name                nest
    Match               *
    Operation           nest
    Wildcard            Nested.*
    Nested_under        kubernetes
    Remove_prefix       Nested.

[FILTER]
    Name                nest
    Match               *
    Operation           nest
    Wildcard            Docker.*
    Nested_under        docker
    Remove_prefix       Docker.

[FILTER]
    Name                nest
    Match               *
    Operation           lift
    Nested_under        kubernetes
    Add_prefix          Kube.

[FILTER]
    Name                modify
    Match               *
    Remove              Kube.container_hash
    Remove              Kube.container_image
    Remove              Kube.pod_id

[FILTER]
    Name                nest
    Match               *
    Operation           nest
    Wildcard            Kube.*
    Nested_under        kubernetes
    Remove_prefix       Kube.

[OUTPUT]
    Name                cloudwatch
    Match               fluentbit.*
    region              ${AWS_REGION}
    log_group_name      /rf/${ENV_ID}/ec2/logs/eksworkernode_${WORKER_NODE_GROUP}_fluentd
    log_stream_name     ${HOST_NODE_NAME}_$(tag[4])
    auto_create_group   true
    extra_user_agent    container-insights

[OUTPUT]
    Name                cloudwatch
    Match               cloudwatch-agent.*
    region              ${AWS_REGION}
    log_group_name      /rf/${ENV_ID}/ec2/logs/eksworkernode_${WORKER_NODE_GROUP}_cwagent
    log_stream_name     ${HOST_NODE_NAME}_$(tag[4])
    auto_create_group   true
    extra_user_agent    container-insights

[OUTPUT]
    Name                cloudwatch
    Match               kube-proxy.*
    region              ${AWS_REGION}
    log_group_name      /rf/${ENV_ID}/ec2/logs/eksworkernode_${WORKER_NODE_GROUP}_kube-proxy
    log_stream_name     ${HOST_NODE_NAME}_$(tag[4])
    auto_create_group   true
    extra_user_agent    container-insights

[OUTPUT]
    Name                cloudwatch
    Match               aws-node.*
    region              ${AWS_REGION}
    log_group_name      /rf/${ENV_ID}/ec2/logs/eksworkernode_${WORKER_NODE_GROUP}_aws-node
    log_stream_name     ${HOST_NODE_NAME}_$(tag[4])
    auto_create_group   true
    extra_user_agent    container-insights
    
[OUTPUT]
    Name                cloudwatch
    Match               coredns.*
    region              ${AWS_REGION}
    log_group_name      /rf/${ENV_ID}/ec2/logs/eksworkernode_${WORKER_NODE_GROUP}_coredns
    log_stream_name     ${HOST_NODE_NAME}_$(tag[4])
    auto_create_group   true
    extra_user_agent    container-insights

[IanIkeda]

I realized that I mistakenly reported the wrong version.
The actual Fluent Bit version running in my environment is v1.7.2.

The container image tag was 2.12.0, but the Fluent Bit core version inside the image is v1.7.2.

Sorry for the confusion. Please consider the issue as occurring on Fluent Bit v1.7.2.

One more question regarding this issue.

Even with the fix in PR #11682, Fluent Bit will still output the error message
"time string length is too long" when the time field is too long.

Is there any plan or recommended way to avoid emitting this error message?
For example:

• ignoring the time field,
• disabling time parsing for specific inputs,
• or treating long time strings as normal strings instead of errors.

My goal is to prevent Fluent Bit from continuously logging this error to CloudWatch Logs.