Don't construct lat/long arrays before scattering.#59
Don't construct lat/long arrays before scattering.#59
Conversation
|
Thanks for these optimizations! I'll review them soon and do timing/regression tests. I have some other changes that need to go in ASAP, so this may be a few days, and need to be rebased, but I'll take care of that. |
|
Should the assumption be made that latitude and longitude have the same dimensions? |
No, we can't assume that. For example, in GFS, the latitude dimension is 1536 and the longitude is 3072. |
|
by dimensions, I meant number of dimensions. sorry. |
I still think that assumption isn't safe; in some input files the latitude and longitude fields are 2D, and in some there is no time dimension on the climate fields. |
The reason I asked is because that assumption was being made implicitly in the code. I added some checks that hopefully can catch odd inputs. |
|
@rcabell are the regression tests publicly available? |
No, the tests are currently a bit ad-hoc and not yet available publicly. |
This should reduce the overhead of scattering the arrays. Using broadcasting also reduces memory usage.