Skip to content

Get required_grad check before the gradient calculation #17

@tomiock

Description

@tomiock

Currently, the required grad happens in the gradient accumulation function, after the actual gradient calculation.

Changing this would saved some operations. For example when having data (that do not require a gradient), the gradient for all the datapoints is going to get calculated. This can be very expensive, especially for high dim data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    backwardBackward pass relatedengineWork the tensor engine for forward and backwardenhancementNew feature or request

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions