Skip to content

Implement Vulkan compute infrastructure and GDeflate format foundation#12

Merged
infinityabundance merged 11 commits intomainfrom
copilot/investigate-directstorage-io
Feb 19, 2026
Merged

Implement Vulkan compute infrastructure and GDeflate format foundation#12
infinityabundance merged 11 commits intomainfrom
copilot/investigate-directstorage-io

Conversation

Copy link
Contributor

Copilot AI commented Feb 16, 2026

Implements GPU compute capabilities and compression format structures for DirectStorage-style I/O pipeline on Linux. Completes Phase 1.2 (Vulkan compute) and begins Phase 2.1 (GDeflate decompression).

Vulkan Compute Infrastructure

Shader Management (src/ds_runtime_vulkan.cpp):

  • load_spirv_from_file(): Validates magic number (0x07230203), 4-byte alignment
  • ShaderModule: RAII wrapper, move-only semantics
  • ShaderModuleCache: Path-based O(1) lookup
  • CMake integration: Auto-compiles .comp.spv with glslangValidator

Descriptor System:

  • Factory functions: create_buffer_copy_layout() (2 bindings), create_decompression_layout() (3 bindings)
  • DescriptorPool: Pre-allocated (32 sets, 96 storage buffers), individual free support
  • descriptor_updates::update_storage_buffers(): Batch buffer binding
  • Convenience wrappers for common patterns (copy, decompression)

Pipeline Creation:

  • PipelineLayout: Combines descriptor layouts + push constants
  • ComputePipeline: Links shader to layout, supports pipeline cache
  • ComputePipelineBundle: Complete package (layout + pipeline + descriptors)
  • pipeline_factory: One-call creation for buffer_copy and decompression pipelines

Dispatch & Synchronization (compute_dispatch namespace):

  • record_compute_dispatch(): Single-call pipeline bind + descriptor bind + push constants + dispatch
  • calculate_workgroup_count_1d(): Ceiling division for workgroup sizing
  • Barrier helpers: barrier_transfer_to_compute(), barrier_compute_to_transfer(), barrier_compute_to_compute()
  • MemoryBarrierInfo abstraction for access masks and pipeline stages

Example Usage:

// Load shader and create pipeline
ShaderModuleCache cache(device);
auto pipeline = pipeline_factory::create_buffer_copy_pipeline(
    device, cache, "shaders/buffer_copy.comp.spv");

// Allocate and bind
DescriptorPool pool(device);
VkDescriptorSet desc = pool.allocate(pipeline.descriptor_layout.layout);
descriptor_updates::update_copy_buffers(device, desc, in_buf, in_sz, out_buf, out_sz);

// Dispatch
compute_dispatch::DispatchInfo info{
    .pipeline = pipeline.pipeline->get(),
    .pipeline_layout = pipeline.pipeline_layout->get(),
    .descriptor_set = desc,
    .workgroup_count_x = compute_dispatch::calculate_workgroup_count_1d(elements, 256),
    .push_constants_data = &elements,
    .push_constants_size = sizeof(uint32_t)
};
compute_dispatch::record_compute_dispatch(cmd, info);

GDeflate Format Foundation

Structures (include/gdeflate_format.h):

  • FileHeader: Magic (0x4744464C), version, sizes, block count
  • BlockInfo: Per-block offset, compressed/uncompressed sizes, checksum
  • StreamInfo: Aggregate with validation and size calculations
  • Inline parsers: parse_stream_info() validates entire chain

Testing (tests/gdeflate_format_test.cpp):

  • Header validation (valid/invalid magic)
  • Block parsing (single/multiple blocks)
  • Complete stream parsing with size checks

Example Shaders

  • shaders/buffer_copy.comp: GPU copy with push constants, 256 thread workgroups
  • shaders/uppercase.comp: ASCII transformation demonstrating uint8_t operations
  • shaders/README.md: Conventions and build instructions

Stats

  • ~1,340 LOC across Vulkan compute (~1,020) and GDeflate format (~320)
  • 11 incremental commits following investigation roadmap
  • 5/5 tests passing (added ds_gdeflate_format_test)
  • Zero compiler warnings with -Wconversion

Next Steps

  • Phase 2.1.2: DEFLATE integration (zlib/miniz evaluation)
  • Phase 2.1.3: Wire decoder into CPU backend, remove ENOTSUP stub
  • Phase 2.2: io_uring multi-worker architecture (parallel track)

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 16, 2026 06:26
Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
…etion

Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate phased build plan for DirectStorage implementation Investigation: DirectStorage I/O pipeline implementation roadmap Feb 16, 2026
Copilot AI and others added 5 commits February 16, 2026 08:07
Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
…eCache

Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
…lementation

Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
Copilot AI changed the title Investigation: DirectStorage I/O pipeline implementation roadmap Implement Vulkan compute infrastructure: shader modules and descriptor management Feb 16, 2026
Copilot AI and others added 3 commits February 16, 2026 08:25
…structure

Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
…nchronization

Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
Co-authored-by: infinityabundance <255699974+infinityabundance@users.noreply.github.com>
Copilot AI changed the title Implement Vulkan compute infrastructure: shader modules and descriptor management Implement Vulkan compute infrastructure and GDeflate format foundation Feb 16, 2026
@infinityabundance infinityabundance marked this pull request as ready for review February 19, 2026 20:56
@chatgpt-codex-connector
Copy link

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@infinityabundance infinityabundance merged commit b90fa00 into main Feb 19, 2026
6 checks passed
@infinityabundance infinityabundance deleted the copilot/investigate-directstorage-io branch February 19, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants