LRU cache hit rate doesn't seem good enough for real use cases

[ebroder]

I had an occasion today to pull some data off of an EBS snapshot and tried to use ebuse for it. It was exhaustingly slow, and it seemed like it was because it had to keep re-fetching the same blocks. I don't know for sure, but I suspect it was fetching things like directory records. I increased the LRU cache size here:

ebuse/snapshot_backend.go

Line 49 in 4424289

blockCache, err := lru.NewARC(100)

Bumping it to 20000 (enough to store the entire 10G EBS snapshot in memory if necessary) resulted in a significant speed improvement.

It'd be great to have more data about what blocks the filesystem access frequently (say, for an ext4 filesystem) and if there's a different caching pattern that would work more effectively. (Maybe a cache pattern that's aware of frequency of access instead of pure LRU?)