Skip to content

fix(conn): update no-op logger to scheduler logger - #656

Open
thijmv wants to merge 1 commit into
masterfrom
thijmv/conn-noop-logger
Open

fix(conn): update no-op logger to scheduler logger#656
thijmv wants to merge 1 commit into
masterfrom
thijmv/conn-noop-logger

Conversation

@thijmv

@thijmv thijmv commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Currently, we have no observability on the scheduler's connection logs, since a no-op logger is being passed.

Therefore, this PR:

  • Passes the handshaker's logger when creating a connection, improving observability.
  • Downgrades the info logs to debug to avoid spiking ingestion rates. Socket read and write errors are expected when the sockets close.
  • Makes the log messages static, rather than dynamic, so they become easier to group.


func (c *Conn) readPayload(length int32) ([]byte, error) {
if err := c.bandwidth.ReserveIngress(int64(length)); err != nil {
c.log().Errorf("Error reserving ingress bandwidth for piece payload: %s", err)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this log tends to show up A LOT for the origin. It's one of our most common logs that we have. I have 2 concerns:

  1. What if we get it a lot with agent too, causing a lot of noise logging?
  2. I'm not sure if it's an error, as we are just throttling ourselves intentionally. It should be an Info/Warn or even maybe a Debug log if the amount this happens is a lot.

WDYT?

@thijmv thijmv Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. It may happen when the agents are under high load, but I would expect that on the control plane. Furthermore, we currently have a nocollect rule on the agent's scheduler logs, so we can let it soak without spiking log ingestion rates.
  2. I think this is a valid point. It is not necessarily an error if we deliberately limit the bandwidth. I have downgraded the log level to INFO, as this is more appropriate than ERROR.

isPeerOrigin,
info,
openedByRemote,
zap.NewNop().Sugar())

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we have a no-op logger in the past and are we 100% sure that we didn't miss any logs (that we shd move from Info/Error to Debug) that this logger is used for?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it was refactored but never wired up correctly. The logger is only used for the four logs that this PR touches.

@thijmv
thijmv force-pushed the thijmv/conn-noop-logger branch from 2e187d3 to 656d0cb Compare August 11, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants