Open
Conversation
gzagaris
commented
Sep 19, 2021
src/LocalMesh.C
Outdated
| pc.postRecvs_direct(qbuf_d2,nfields_d,rcvmap,ireq,mycomm,&reqcount); | ||
|
|
||
| // wait for pack kernel to finish packing buffers? | ||
| FVSAND::gpu::synchronize(); |
Collaborator
Author
There was a problem hiding this comment.
Note: I think we need to synchronize here since the pack kernel, updateHost in line 336 may have not finished packing the send buffers in qbuf_d
gzagaris
commented
Sep 19, 2021
src/parallelComm.h
Outdated
| inline | ||
| void postRecvs_direct(double *qbuf, int nfields, | ||
| std::map <int, std::vector<int>> rcvmap, | ||
| const std::map <int, std::vector<int>>& rcvmap, |
Collaborator
Author
There was a problem hiding this comment.
This avoids deep copies when passing the maps to this function.
gzagaris
commented
Sep 19, 2021
src/parallelComm.h
Outdated
| inline | ||
| void postSends_direct(double *qbuf, int nfields, | ||
| std::map <int, std::vector<int>> sndmap, | ||
| const std::map <int, std::vector<int>>& sndmap, |
gzagaris
force-pushed
the
bugfix/use-cudaaware-mpi
branch
6 times, most recently
from
f380460 to
d0c425e
Compare
Owner
|
These all look ok to me. Go ahead and merge if you feel comfortable with it |
gzagaris
force-pushed
the
bugfix/use-cudaaware-mpi
branch
from
d0c425e to
9bfda6c
Compare
Pass maps to the post-send and post receive methods by const reference to avoid deep-copies. Also, mark those methods as "inline", which might help the compiler.
gzagaris
force-pushed
the
bugfix/use-cudaaware-mpi
branch
from
9bfda6c to
95f45f3
Compare
Collaborator
Author
|
Adding the |
This works on Ascent, but, it requires the `--smpiargs="-gpu"` flag to be passed to `jsrun`.
gzagaris
force-pushed
the
bugfix/use-cudaaware-mpi
branch
from
95f45f3 to
59ae40a
Compare
Adds cmd line option to enable/disable CUDA-aware MPI.
Collaborator
Author
|
I added a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.