[hipFile] POC: hipFile Python API using Cython bindings#4865
Open
systems-assistant[bot] wants to merge 30 commits intodevelopfrom
Open
[hipFile] POC: hipFile Python API using Cython bindings#4865systems-assistant[bot] wants to merge 30 commits intodevelopfrom
systems-assistant[bot] wants to merge 30 commits intodevelopfrom
Conversation
Also move _hipfile driver functions out of public __init__.py
This exception will be thrown by the Python hipFile module rather than returning an error struct. The hipFile C error macros are unused since they aren't very Pythonic in the first place. If the error is a hipFile error (even if it is a wrapped HIP error), we raise HipFileException. If it is a different error, a different Exception will be raised. We also won't be exposing the error struct/tuple to Python users, though it can be accessed via the HipFileException when raised.
This module will contain the functions related to querying driver/library configuration/properties in a Python friendly manner.
A fairly rudimentary interface for opening/closing files with hipFile.
A simple wrapper. Still need to figure out how exactly a hipFile Buffer object should look like and how it is tied to the underlying device memory.
We need some GPU memory allocated for registered a buffer with hipFile.
This isn't perfect, but it provides a starting point to at least running IO.
Not meant to be permanent. Just a quick and dirty script to verify running IO works.
Also adds limited typing hint for this method only. This import is only run if TYPE_CHECKING is defined. At runtime, if the ctypes package is not found, no error is raised since __future__.annotations is imported which treats the hint as a string literal and does not try to evaluate the name.
hipFileOpError and hipFileFileHandleType are exposed to the Python layer to provide the user direct access to these enum values.
A proper fix will come next where the C enum values are namespace qualified so that enum values exposed to Python do not need to have their names modified.
This puts the C hipFile symbols being imported by Cython into a "_c" namespace to avoid a namespace conflict when trying to set a Python variable/function with the same name.
The C functions hipFileRead()/hipFileWrite() report error conditions through a negative return value. If this return value is -1 or -5011, it indicates additional error information needs to be fetched as it cannot be included in a single return value. However, this error information needs to be fetched prior to control returning to Python. Otherwise there is no guarantee that these error values are still valid for the error that occurred.
Cleans up what is imported in __init__.py to not pollute the namespace when a user imports 'hipfile'.
Originally these functions were given different names when the initial Cython code was generated. In part, this was to avoid namespace conflicts. Now that the C functions are imported into their own "_c" namespace, we can reuse the same function names without conflict. Reusing the names improves consistency of what Cython function calls which C function.
jordan-turbofish
changed the title
Removes supporting Win32 from FileHandle. FileHandle was designed to manage opening & closing its own file reference. Python does not provide a public API to open a file which returns a Win32 handle. This leads to a contrived process of opening a file, getting a file descriptor, and converting it to a Win32 handle. To be clear, os.open() on Windows provides a POSIX FD. FileHandle does not currently support a user providing an already open file resource, which at this time seems to be the only plausible use case of a Win32 handle being used. For the sake of simplicity and getting out an initial release, we will just error if the OPAQUE_WIN32 handle type is set. (This however will leave Win32 support in the Cython layer as that is a trivial switch that gets passed into the C library.)
riley-dixon
force-pushed
the
import/develop/ROCm_hipFile/pr-248
branch
from
e366dc3 to
f4a5cf9
Compare
…evelop/ROCm_hipFile/pr-248
riley-dixon
force-pushed
the
import/develop/ROCm_hipFile/pr-248
branch
from
f4a5cf9 to
96726e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A proof of concept to see if we can create Python bindings for hipFile. This is still in its early stages and will likely change substantially between now and what could be called a "beta" release. The generated Cython bindings should be considered temporary as we work with another team for a long-term solution, but they at least give a base to build off of for the higher-level Python API.
Addresses ROCm/hipFile#201
Test Plan
Can we load the C hipFile library from Python and run select functions?
Can we run IO?
Test Result
It works :)
AIHIPFILE-140
🔁 Imported from ROCm/hipFile#248
🧑💻 Originally authored by @riley-dixon