Add datasetio support by sachintu47 · Pull Request #1 · zopencommunity/zoslib

sachintu47 · 2026-04-14T13:41:59Z

Adapted majority of the work from https://github.com/zopencommunity/libdio.

Code changes added

Override open/close/write/read syscalls to use f* syscalls which has support for Dataset I/O.
Add runtime option to enable dataset (ZOSLIB_DATASET_SUPPORT)
Only tested support for tool which does read I/O (less, cat, head, tail, grep, diff)

Pending

Test write I/O
use configuration file similar to libdio, which maps dataset name to ccsid.
Add testing in zoslib

sachintu47 · 2026-04-14T13:44:17Z

augment review

sachintu47 · 2026-04-14T14:20:57Z

augment review

augmentcode · 2026-04-14T14:26:02Z

🤖 Augment PR Summary

Summary: Adds optional MVS dataset I/O support to zoslib, allowing paths with a // prefix to be treated as datasets when enabled at runtime.

Changes:

Adds build-time switch ZOSLIB_ENABLE_DATASETIO (CMake define + build.sh -d)
Introduces include/zos-datasetio.h and src/zos-datasetio.c implementing dataset open/read/write/lseek/stat using stdio record I/O
Overrides libc entry points (open/fopen/mkstemp/read/write/lseek/stat/fstat/close) to dispatch between regular files and dataset-backed descriptors
Adds runtime toggle via ZOSLIB_DATASET_SUPPORT (new field in zoslib_config_t + envar help text)
Uses a dummy /dev/null fd + a descriptor table to map POSIX fds to dataset state

Technical Notes: Fixed-record datasets are reopened in binary mode for more efficient multi-record I/O, while variable/undefined formats use type=record; optional debug logging is gated by ZOSLIB_DEBUG.

_{🤖 Was this summary useful? React with 👍 or 👎}

augmentcode

Review completed. 5 suggestions posted.

Comment augment review to trigger a new review at any time.

augmentcode · 2026-04-14T14:26:03Z


  // Write the formatted string to the specified file descriptor
-  written = write(fd, buffer, length);
+  written = __write_orig(fd, buffer, length);


src/zos-io.cc:1414: dprintf() now writes via __write_orig, which bypasses the dataset-fd dispatch and will send output to the dummy /dev/null fd for dataset-backed descriptors.

Severity: high

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

augmentcode · 2026-04-14T14:26:03Z

+    metadata->is_pds_member = entry->is_pds_member;
+    metadata->readonly = entry->readonly;
+
+    strncpy(metadata->member_name, entry->member_name, sizeof(metadata->member_name) - 1);


src/zos-datasetio.c:1570: dsio_get_metadata() uses strncpy(..., sizeof-1) without explicitly NUL-terminating metadata->member_name/hlq/llq, so an 8-character field can yield unterminated strings.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

IgorTodorovskiIBM · 2026-04-14T15:53:39Z

Great job! A few things to consider:

Is it thread-safe? The descriptor_table is one area where maybe we might need to use a mutex. Similarly for static struct dirent entry.
We should probably also update the manpage to document __DATASET_SUPPORT

sachintu47 · 2026-04-14T16:16:28Z

Great job! A few things to consider:

Is it thread-safe? The descriptor_table is one area where maybe we might need to use a mutex. Similarly for static struct dirent entry.

We should probably also update the manpage to document __DATASET_SUPPORT

Thanks. I'll check on the thread-safety. Will also update man-page

sachintu47 · 2026-04-20T12:35:31Z

augment review

augmentcode

Review completed. 6 suggestions posted.

Comment augment review to trigger a new review at any time.

sachintu47 · 2026-04-20T14:17:22Z

augment review

augmentcode

Review completed. 5 suggestions posted.

Comment augment review to trigger a new review at any time.

sachintu47 · 2026-04-20T16:35:20Z

augment review

augmentcode

Review completed. 4 suggestions posted.

Comment augment review to trigger a new review at any time.

augmentcode · 2026-04-20T16:43:34Z

+
+
+#define GET_DUMMY_FD(flags) (open("/dev/null", (flags) & (O_RDONLY | O_WRONLY | O_RDWR), 0))
+#define IS_DATASET(name) ((name) && ((name)[0] == '/') && ((name)[1] == '/'))


include/zos-datasetio.h:361: IS_DATASET(name) treats any path starting with // as a dataset, but POSIX paths can legitimately contain double-leading slashes (often equivalent to /). With dataset support enabled, this could misroute normal files (e.g., //tmp/foo) into open_dataset() and fail unexpectedly; consider tightening the predicate to the actual dataset syntax.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

sachintu47 · 2026-04-22T12:15:58Z

augment review

sachintu47 · 2026-04-22T15:46:34Z

augment review

augmentcode

Review completed. 4 suggestions posted.

Comment augment review to trigger a new review at any time.

v1gnesh · 2026-04-23T02:38:25Z

@sachintu47, if it helps, if you're able to use Augment in your workplace, I can ask if Augment Code still honour their "free for OSS" plan for general Augment IDE/CLI use (not just Review).

sachintu47 · 2026-04-27T11:54:00Z

@sachintu47, if it helps, if you're able to use Augment in your workplace, I can ask if Augment Code still honour their "free for OSS" plan for general Augment IDE/CLI use (not just Review).

Sure will check. @IgorTodorovskiIBM

sachintu47 · 2026-04-27T12:51:28Z

augment review

augmentcode

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

sachintu47 · 2026-04-27T17:02:52Z

augment review

- Refactor DatasetEntry to support byte-stream emulation with record buffering. - Implement newline insertion/stripping and record padding for FB datasets. - Optimize FB dataset I/O by reopening in binary mode for multi-record access. - Add VB size caching to improve performance of fstat and SEEK_END. - Fix write_dataset to prevent data loss at record boundaries and add bounds checking. - Use length-safe dsio_convert_buffer for all CCSID conversions. - Consolidate entry initialization and simplify redundant size checks.

…et to prevent tools like ggrep from incorrectly identifying datasets as stdout

…os in dataset I/O

…g FB seeks. Enables runtime debug logging via ZOSLIB_DEBUG and implements size caching.

- Added ZOSLIB_ENABLE_DATASETIO compilation flag (defaults to 0). - Added __DATASET_SUPPORT environment variable for runtime control. - Consolidated DatasetEntry struct and cleaned up redundant functions. - Defaulted dataset support to disabled at both compile and runtime.

- Added ZOSLIB_ENABLE_DATASETIO option to CMakeLists.txt (default OFF). - Added -d flag to build.sh to enable dataset I/O support during build.

…tat/dsio_get_size Harden error paths: fix zos_fcntl null-deref, close_dataset write-mode check, generate_name errno, and reduce noisy debug logging.

- Enable dataset I/O by default at runtime (opt-out instead of opt-in) - Replace mutex macros with inline helper functions for better readability - Consolidate 20+ error codes into 5 categories with backward-compatible aliases - Simplify logging macros to use centralized dsio_log() function

Dataset I/O now always compiled in, controlled at runtime via ZOSLIB_DATASET_SUPPORT env var

sachintu47 · 2026-05-08T06:13:57Z

/run tests

sachintu47 requested review from HarithaIBM, IgorTodorovskiIBM and sabi789 April 14, 2026 13:42

sachintu47 changed the title ~~Zopen2~~ Add datasetio support Apr 14, 2026

sachintu47 mentioned this pull request Apr 14, 2026

Add datasetio support ibmruntimes/zoslib#110

Closed

augmentcode Bot reviewed Apr 14, 2026

View reviewed changes

augmentcode Bot reviewed Apr 20, 2026

View reviewed changes

Comment thread src/zos-io.cc

Comment thread src/zos-io.cc Outdated

Comment thread src/zos-datasetio.c

Comment thread src/zos-datasetio.c Outdated

Comment thread src/zos-datasetio.c Outdated

Comment thread src/zos-datasetio.c Outdated

sachintu47 force-pushed the zopen2 branch from 0c13333 to 7ad242d Compare April 20, 2026 14:17

augmentcode Bot reviewed Apr 20, 2026

View reviewed changes

Comment thread src/zos-datasetio.c

Comment thread src/zos-datasetio.c

Comment thread include/zos-datasetio.h Outdated

Comment thread src/zos-datasetio.c

Comment thread src/zos-datasetio.c Outdated

sachintu47 force-pushed the zopen2 branch from 7ad242d to 9179453 Compare April 20, 2026 16:34

augmentcode Bot reviewed Apr 20, 2026

View reviewed changes

sachintu47 force-pushed the zopen2 branch from 9179453 to 7f35f62 Compare April 22, 2026 12:15

augmentcode Bot reviewed Apr 22, 2026

View reviewed changes

Comment thread src/zos-datasetio.c Outdated

Comment thread src/zos-datasetio.c

Comment thread src/zos-datasetio.c

Comment thread src/zos-io.cc Outdated

sachintu47 force-pushed the zopen2 branch from 7f35f62 to ddec4b5 Compare April 27, 2026 11:51

augmentcode Bot reviewed Apr 27, 2026

View reviewed changes

Comment thread src/zos-io.cc

Comment thread src/zos-datasetio.c Outdated

Comment thread src/zos-datasetio.c

sachintu47 force-pushed the zopen2 branch from ddec4b5 to 34df8d4 Compare April 27, 2026 17:02

sachintu47 added 28 commits May 8, 2026 00:18

Integrate from datasetio. Modify open and close syscall

4ee14ed

Add write, read, mkstemp for dataset

1f81165

fopen override

69885a0

lseek override

d8f3bfc

Add default unix fds to database

cad67d2

override stat, fstat

353de87

Fix fseek and filesize

97bf2b5

refactor datasetentry structure

6100fb8

fixes

0db5380

Implement unique st_dev/st_ino generation for datasets in fstat_datas…

2087f63

…et to prevent tools like ggrep from incorrectly identifying datasets as stdout

fix dataset record alignment

dd0c148

refactor: replace DEBUG_PRINT macros with conditional DSIO_LOG_* macr…

50c873b

…os in dataset I/O

refactor: remove ISPF implementation (deferred to future phase)

8663b08

cleanup dataset I/O logging and remove incomplete features

a7e990b

complete error handling API with DSIO-to-errno mapping

6e58f6c

fix: resolve dataset I/O hang by clearing EOF on lseek and refactorin…

2c600ab

…g FB seeks. Enables runtime debug logging via ZOSLIB_DEBUG and implements size caching.

clean up unused

770bfb6

build: add ZOSLIB_ENABLE_DATASETIO configuration option

03edcb0

- Added ZOSLIB_ENABLE_DATASETIO option to CMakeLists.txt (default OFF). - Added -d flag to build.sh to enable dataset I/O support during build.

fix(datasetio): avoid fseek(SEEK_END) on VB/U record-mode files in fs…

1b19b80

…tat/dsio_get_size Harden error paths: fix zos_fcntl null-deref, close_dataset write-mode check, generate_name errno, and reduce noisy debug logging.

augment review fixes

2764606

Resolved Potential Data Loss on Seek

df5d89a

fix: add thread safety and prevent memory leaks in dataset I/O

a5e5cfe

feat: enable dataset I/O support by default

d1bab12

Add documentation for ZOSLIB_DATASET_SUPPORT environment variable

4a29b17

refactor: remove ZOSLIB_ENABLE_DATASETIO build-time conditional

6bff782

Dataset I/O now always compiled in, controlled at runtime via ZOSLIB_DATASET_SUPPORT env var

sachintu47 force-pushed the zopen2 branch from d84e2d7 to 6bff782 Compare May 8, 2026 04:19



		#define GET_DUMMY_FD(flags) (open("/dev/null", (flags) & (O_RDONLY \| O_WRONLY \| O_RDWR), 0))
		#define IS_DATASET(name) ((name) && ((name)[0] == '/') && ((name)[1] == '/'))

Conversation

sachintu47 commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sachintu47 commented Apr 14, 2026

Uh oh!

sachintu47 commented Apr 14, 2026

Uh oh!

augmentcode Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

augmentcode Bot Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

augmentcode Bot Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

IgorTodorovskiIBM commented Apr 14, 2026

Uh oh!

sachintu47 commented Apr 14, 2026

Uh oh!

sachintu47 commented Apr 20, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sachintu47 commented Apr 20, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sachintu47 commented Apr 20, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sachintu47 commented Apr 22, 2026

Uh oh!

sachintu47 commented Apr 22, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

v1gnesh commented Apr 23, 2026

Uh oh!

sachintu47 commented Apr 27, 2026

Uh oh!

sachintu47 commented Apr 27, 2026

Uh oh!

augmentcode Bot left a comment

sachintu47 commented Apr 14, 2026 •

edited

Loading

augmentcode Bot commented Apr 14, 2026 •

edited

Loading

augmentcode Bot Apr 14, 2026 •

edited

Loading

augmentcode Bot Apr 14, 2026 •

edited

Loading

augmentcode Bot Apr 20, 2026 •

edited

Loading