VMEM file: use file object directly instead of RAM buffer#78
VMEM file: use file object directly instead of RAM buffer#78jeanbaptistelab wants to merge 3 commits into
Conversation
This reduces artificail RAM requirement for processes using VMEM file objects, at the (relative) expense of speed, but if speed is needed, then VMEM block objects (which offer configurable caching) can be used
| #if __64BIT__ | ||
| #define VMEM_FILE_VADDR_INITIALIZER(name_in) \ | ||
| .vaddr = (uint64_t)vmem_##name_in##_buf | ||
| .vaddr = (uint64_t)&vmem_##name_in##_driver |
There was a problem hiding this comment.
I can see this is going to be an issue: previously, the vaddr was generated by the compiler, and we wouldn't get overlap because each vaddr would be the size of the corresponding allocated static buffer.
I naively use the driver object's address, but depending on the link map, these objects might be placed one after the other in memory, effectively causing overlaps, which will have, let's say, "unintended effects" at runtime :)
I'll try to come up with a solution, but I don't think we can auto-magically solve that one....
Note that though the previous implementation was convenient, it had severe drawbacks when a piece of code serializes the vaddr to reuse it later (across reboots etc): the process address space is not guaranteed to start at the same address for every invocation, this is actually why the VMEM_DEFINE_FILE_VADDR macro was introduced, allowing manual setting the vaddr to values that don't overlap (the "don't overlap" part is currently part of the manual process, a check could be introduced to detect overlaps at run-time, or use our partool process that can detect these overlaps by processing the ELF file)
|
The concept behind VMEM is to have random access, directly even by peek and poke. The entire file thus needs to live inside virtual memory, and have a full memory or virtual allocation. If you need to open and close file handles and have dynamic access to a file system, it sounds more like its an FTP protocol you are looking for. |
johandc
left a comment
johandc
left a comment
There was a problem hiding this comment.
Create another VMEM type if you need to, but dont change default behaviour for VMEM_FILE,
|
@johandc I've now updated this PR to keep the original VMEM file implementation and introduced a new, separate VMEM file vaddr type |
|
ping ? @troelsjessen @kivkiv12345 @johandc |
troelsjessen
left a comment
troelsjessen
left a comment
There was a problem hiding this comment.
The implementation generally seems like a useful extension. I suggest we always set one of the upper bits in the macro when the vaddr is assigned, to avoid that the user accidentally overlaps with a dynamically alloced address allocated to the application.
Something like this @troelsjessen ? |
3 participants
This reduces artificial RAM requirement for processes using VMEM file objects, at the (relative) expense of speed, but if speed is needed, then VMEM block objects (which offer configurable caching) can be used