Changes thats need to be included for 0.48.1 patch release (#52)

* Struct2Tensor 0.48.0 Release

PiperOrigin-RevId: 743712449

* Build documentation with `mkdocs` (#38)

* Add minimum viable mkdocs configuration

* Add examples section

* Add links to api docs and overviews

* Get api docs up and running

* Add `__all__` to `__init__.py` files to include correct classes

This is just a first step to including the correct items in the api docs

* Exclude test modules from docs

* Use correct module

* Add docs deployment workflow

* Include all correct modules/classes/functions

* Fix button rendering

* Fix formatting, including admonitions

* Fix broken link

* Use README.md as homepage for docs

* Fix links

* Remove debug triggers from docs deployment workflow

* Remove deprecated api docs

* Fix triggers

* Move docs requirements to common location

* Only trigger on pushes to master

* Remove unnecessary newline characters

* Move docs location to default `docs` directory

* only internal visibility change, no external change

PiperOrigin-RevId: 755419474

* Automated Code Change

PiperOrigin-RevId: 796979132

* Automated Code Change

PiperOrigin-RevId: 803375108

* no-op

PiperOrigin-RevId: 809038483

* Added pytype None checks to calculate.py

PiperOrigin-RevId: 845226580

* Replace unicode escaped characters in ipynb files

PiperOrigin-RevId: 856192266

* Fixes that needs to be available in the main branch in order to cherry-pick into Release branch (#49)

* Update WORKSPACE

* Update workspace.bzl

* Migrate to Protobuf 4.23.4: custom Bazel rule for Python proto generation

Protobuf 4.x removed py_proto_library and changed proto_library behavior. This commit:

- Creates _py_proto_library_rule: a custom Bazel rule that accepts ProtoInfo or PyInfo,
  runs protoc to generate Python _pb2.py files, and provides PyInfo for Python deps
- Replaces deprecated py_proto_library calls with custom rule implementation
- Configures protoc proto_path to include workspace dirs and external dependencies
- Adds local any.proto copies to bypass sandbox limitations in proto imports
- Applies compatibility patches to TensorFlow and TensorFlow Metadata dependencies
- Updates all s2t_proto_library_py calls to work with new implementation

Fixes: Protobuf 4.23.4 compatibility for struct2tensor build system

* Add compatibility patches for older Abseil and Protobuf 4.x

This patch extends tensorflow.patch with changes to support:

1. Abseil backward compatibility:
   - Add absl_base_prefetch.h compatibility shim for older Abseil
     versions that lack absl/base/prefetch.h
   - Update prefetch includes across TensorFlow/TSL to use the shim
   - Add inline implementation of NullTerminatedMessage for older
     Abseil versions without StatusMessageAsCStr

2. Protobuf 4.x support:
   - Refactor cc_proto_library to use native proto_library and
     cc_proto_library rules instead of custom proto_gen
   - Implement custom _tsl_py_proto_library_rule to replace the
     built-in py_proto_library removed in Protobuf 4.x
   - Update proto library generation to depend on generated
     cc_proto_library targets

3. Build cleanup:
   - Remove unused absl/strings:string_view dependency
   - Update BUILD files to reflect new dependency structure

These changes enable building struct2tensor with older Abseil versions
while supporting Protobuf 4.x, improving compatibility across different
dependency versions.

* chore: upgrade Protobuf dependency from 4.23.4 to 4.25.6

* Move TFMD patch to tfmd.patch; Protobuf 4.x compatibility

Remove vendored google/protobuf/any.proto and obsolete third_party/README.md

* fix: Use native prefix for Bazel rules in macros

Prefix cc_binary and cc_library with native. in struct2tensor.bzl

* Update Dockerfile

* Fix struct2tensor integration in TensorFlow Serving Docker build

Apply rules_cc patch before adding struct2tensor local repository to ensure proper Bazel configuration.

* Update RELEASE.md to sync with main branch

* Update version.py

---------

Co-authored-by: Gagandeep Singh <gdp.1807@gmail.com>

* docs(release): Update RELEASE.md and version for 0.48.0

* Update Dockerfile to build with serving version 2.17.1

* Build and stage dynamic op libraries before wheel creation (#51)

Add explicit build and copy steps for dynamic op libraries (.so files) in
build_common.sh. This ensures all six dynamic library targets are built and
copied to struct2tensor/ops/ before the wheel is packaged.

Previously, the stamp_wheel step in build_manylinux.sh would fail with
patchelf errors ("No such file or directory") because the .so files were
expected in struct2tensor/ops/ but were not being copied from bazel-bin.

This fix:
- Explicitly builds each dynamic library target
- Copies the built .so files from bazel-bin to the staging directory
- Runs before bazel run :build_pip_package to ensure files are present

Fixes the missing _decode_proto_map_op.so, _decode_proto_sparse_op.so and
related dynamic library warnings during wheel packaging.

* Enable OSS Bazel builds by fixing dependencies and proto paths (#50)

- Add py_proto_library macro for OSS compatibility
- Replace gunit_main with googletest gtest_main
- Comment out Google-internal dependencies and missing test files
- Fix proto import to use workspace-relative path
- Remove invalid third_party load statements

Fixes build errors when running `bazel build //...` in OSS environment.

Co-authored-by: Madhur Karampudi <142544288+vkarampudi@users.noreply.github.com>

---------

Co-authored-by: struct2tensor-team <tensorflow-extended-nonhuman@googlegroups.com>
Co-authored-by: Gagandeep Singh <gdp.1807@gmail.com>

Author: 3 people

RELEASE.md

@@ -16,5 +16,4 @@
 
 ## Deprecations
 
-*   N/A
-
+*   N/A

WORKSPACE

@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
 # 3. Request the new archive to be mirrored on mirror.bazel.build for more
 #    reliable downloads.
 
-_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b"  # tf 2.16.1
-_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
+_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95"  # tf 2.17.1
+_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"
 
 http_archive(
     name = "org_tensorflow",
@@ -51,6 +51,8 @@ http_archive(
         "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
     ],
     strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
+    patches = ["//third_party:tensorflow.patch"],
+    patch_args = ["-p1"],
 )
 
 load("//third_party:python_configure.bzl", "local_python_configure")
@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
 load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
 struct2tensor_workspace()
 
+# ===== Protobuf 4.25.6 dependency =====
+# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
+http_archive(
+    name = "com_google_protobuf",
+    sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
+    strip_prefix = "protobuf-4.25.6",
+    urls = [
+        "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
+    ],
+)
+
+# Load Protobuf dependencies
+load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
+protobuf_deps()
+
 # Initialize TensorFlow's external dependencies.
 load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
 tf_workspace3()

build_common.sh

@@ -62,6 +62,21 @@ function install_tensorflow() {
   "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
 }
 
+function build_dynamic_libraries() {
+  # Explicitly build the dynamic library targets that are needed for the wheel.
+  # These are required by the stamp_wheel function.
+  bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
+  bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
+  bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
+  bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
+  bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
+  bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;
+
+  RUNFILES_DIR=$(pwd)
+  cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/
+
+}
+
 set -x
 
 for i in "$@"; do
@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]
 
 fi
 
+build_dynamic_libraries
+
 # :build_pip_package builds and links struct2tensor ops against a TF
 # installation and packages the result dynamic libraries.
 bazel run -c opt \

examples/prensor_playground.ipynb

@@ -15,6 +15,7 @@
 # Prensors provide a way to parse protocol buffers and other structured data in tensorflow
 # simply and efficiently.
 
+load("@rules_cc//cc:cc_library.bzl", "cc_library")
 # Placeholder: load py_test
 
 load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
@@ -83,7 +84,7 @@ s2t_pytype_library(
         "path.py",
     ],
     deps = [
-        "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
+        "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
         "@com_google_protobuf//:protobuf_python",
     ],
 )

struct2tensor/BUILD

@@ -12,12 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# Placeholder: load py_proto_library
-# Placeholder: load py_binary
-# Placeholder: load py_library
-# Placeholder: load py_test
-load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
-load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
+load("@rules_cc//cc:cc_test.bzl", "cc_test")
+load("//struct2tensor:struct2tensor.bzl", "py_proto_library")
 
 licenses(["notice"])
 
@@ -33,26 +29,28 @@ py_proto_library(
     deps = [":benchmark_proto"],
 )
 
+# Use native cc_proto_library instead of custom macro since we already have proto_library defined
 cc_proto_library(
     name = "benchmark_cc_proto",
     deps = [":benchmark_proto"],
 )
 
-cc_test(
-    name = "serialization_benchmark",
-    srcs = ["serialization_benchmark.cc"],
-    deps = [
-        ":benchmark_cc_proto",
-        "//testing/base/public:gunit",
-        "@com_google_absl//absl/flags:flag",
-        "@com_google_absl//absl/flags:parse",
-        "@com_google_absl//absl/random",
-        "@com_google_absl//absl/random:distributions",
-        "@com_google_protobuf//:protobuf",
-        "@org_tensorflow//tensorflow/core:feature_util",
-        "@org_tensorflow//tensorflow/core:protos_all_cc",
-    ],
-)
+# Google-internal benchmark - commented out for OSS
+# cc_test(
+#     name = "serialization_benchmark",
+#     srcs = ["serialization_benchmark.cc"],
+#     deps = [
+#         ":benchmark_cc_proto",
+#         "//testing/base/public:gunit",
+#         "@com_google_absl//absl/flags:flag",
+#         "@com_google_absl//absl/flags:parse",
+#         "@com_google_absl//absl/random",
+#         "@com_google_absl//absl/random:distributions",
+#         "@com_google_protobuf//:protobuf",
+#         "@org_tensorflow//tensorflow/core:feature_util",
+#         "@org_tensorflow//tensorflow/core:protos_all_cc",
+#     ],
+# )
 
 py_library(
     name = "struct2tensor_benchmark_lib",
@@ -61,10 +59,11 @@ py_library(
         ":benchmark_proto_py_pb2",
         ":struct2tensor_benchmark_util",
         "//struct2tensor",
-        "//testing/pybase",
-        "//testing/pybase:parameterized",
-        "//third_party/py/cpuinfo",
-        "//third_party/py/psutil",
+        # Google-internal dependencies - not available in OSS
+        # "//testing/pybase",
+        # "//testing/pybase:parameterized",
+        # "//third_party/py/cpuinfo",
+        # "//third_party/py/psutil",
     ],
 )
 
@@ -110,9 +109,10 @@ py_library(
     name = "struct2tensor_benchmark_util",
     srcs = ["struct2tensor_benchmark_util.py"],
     deps = [
-        "//file/colossus/public:cns",
-        "//third_party/py/cpuinfo",
-        "//third_party/py/psutil",
+        # Google-internal dependencies - not available in OSS
+        # "//file/colossus/public:cns",
+        # "//third_party/py/cpuinfo",
+        # "//third_party/py/psutil",
         "@absl_py//absl/flags",
         "@absl_py//absl/testing:parameterized",
     ],

struct2tensor/benchmarks/BUILD

@@ -259,6 +259,7 @@ def __str__(self) -> str:
 
   def _create_value_error(self) -> ValueError:
     """Creates a ValueError, assuming there should be one for this node."""
+    assert self.value is not None
     return ValueError("Expression {} returned the wrong type:"
                       " expected: {}"
                       " actual: {}.".format(
@@ -353,8 +354,14 @@ def calculate_values(
       options: calculate_options.Options,
       feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
   ) -> None:
+    """Calculates the values of the expressions in the graph."""
     for node in self.ordered_node_list:
-      source_values = [self._node[id(x)].value for x in node.sources]
+      source_values = [
+          self._node[id(x)].value
+          for x in node.sources
+          if self._node[id(x)] is not None
+          and self._node[id(x)].value is not None
+      ]
       side_info = feed_dict[node.expression] if feed_dict and (
           node.expression in feed_dict) else None
       node.calculate(source_values, options, side_info=side_info)

struct2tensor/calculate.py

@@ -14,6 +14,7 @@
 
 # CPU kernels for struct2tensors.
 
+load("@rules_cc//cc:cc_library.bzl", "cc_library")
 load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")
 
 package(
@@ -68,7 +69,6 @@ s2t_dynamic_library(
         "@com_google_absl//absl/base:endian",
         "@com_google_absl//absl/container:flat_hash_map",
         "@com_google_absl//absl/container:inlined_vector",
-        "@com_google_absl//absl/container:inlined_vector_internal",
         "@com_google_absl//absl/memory",
         "@com_google_absl//absl/strings",
         "@com_google_absl//absl/types:span",
@@ -104,7 +104,6 @@ s2t_dynamic_library(
         "@com_google_absl//absl/base:endian",
         "@com_google_absl//absl/container:flat_hash_map",
         "@com_google_absl//absl/container:inlined_vector",
-        "@com_google_absl//absl/container:inlined_vector_internal",
         "@com_google_absl//absl/strings",
         "@com_google_absl//absl/types:span",
     ],

struct2tensor/kernels/BUILD

@@ -14,6 +14,8 @@
 #
 # Parquet Dataset Kernel Implementation.
 
+load("@rules_cc//cc:cc_library.bzl", "cc_library")
+load("@rules_cc//cc:cc_test.bzl", "cc_test")
 load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")
 
 package(
@@ -60,6 +62,7 @@ cc_library(
     deps = [
         ":parent_indices_builder_lib",
         ":parquet_reader_util_lib",
+        "@com_google_absl//absl/status",
         "@com_google_absl//absl/strings",
         "@arrow//:arrow",
         "//struct2tensor/kernels:vector_to_tensor",
@@ -96,38 +99,40 @@ cc_library(
     ],
 )
 
-cc_test(
-    name = "parent_indices_builder_test",
-    srcs = ["parent_indices_builder_test.cc"],
-    deps = [
-        ":parent_indices_builder_lib",
-        "//testing/base/public:gunit_main",
-        "@org_tensorflow//tensorflow/core:lib",
-        "@org_tensorflow//tensorflow/core:test",
-    ],
-)
+# Test file parent_indices_builder_test.cc does not exist, commenting out
+# cc_test(
+#     name = "parent_indices_builder_test",
+#     srcs = ["parent_indices_builder_test.cc"],
+#     deps = [
+#         ":parent_indices_builder_lib",
+#         "@com_google_googletest//:gtest_main",  # Replaces Google-internal //testing/base/public:gunit_main
+#         "@org_tensorflow//tensorflow/core:lib",
+#         "@org_tensorflow//tensorflow/core:test",
+#     ],
+# )
 
-cc_test(
-    name = "parquet_reader_test",
-    srcs = ["parquet_reader_test.cc"],
-    data = [
-        "//struct2tensor/testdata/parquet_testdata",
-    ],
-    deps = [
-        ":parquet_reader_lib",
-        "//testing/base/public:gunit_main",
-        "@org_tensorflow//tensorflow/core:framework",
-    ],
-)
-
-cc_test(
-    name = "parquet_reader_util_test",
-    srcs = ["parquet_reader_util_test.cc"],
-    data = [
-        "//struct2tensor/testdata/parquet_testdata",
-    ],
-    deps = [
-        ":parquet_reader_util_lib",
-        "//testing/base/public:gunit_main",
-    ],
-)
+# Test file parquet_reader_test.cc does not exist, commenting out
+# cc_test(
+#     name = "parquet_reader_test",
+#     srcs = ["parquet_reader_test.cc"],
+#     data = [
+#         "//struct2tensor/testdata/parquet_testdata",
+#     ],
+#     deps = [
+#         ":parquet_reader_lib",
+#         "@com_google_googletest//:gtest_main",  # Replaces Google-internal //testing/base/public:gunit_main
+#         "@org_tensorflow//tensorflow/core:framework",
+#     ],
+# )
+# Test file parquet_reader_util_test.cc does not exist, commenting out
+# cc_test(
+#     name = "parquet_reader_util_test",
+#     srcs = ["parquet_reader_util_test.cc"],
+#     data = [
+#         "//struct2tensor/testdata/parquet_testdata",
+#     ],
+#     deps = [
+#         ":parquet_reader_util_lib",
+#         "@com_google_googletest//:gtest_main",  # Replaces Google-internal //testing/base/public:gunit_main
+#     ],
+# )

struct2tensor/kernels/parquet/BUILD

@@ -16,6 +16,7 @@ limitations under the License.
 
 #include <memory>
 
+#include "absl/status/status.h"
 #include "absl/strings/str_cat.h"
 #include "struct2tensor/kernels/parquet/parquet_reader_util.h"
 #include "struct2tensor/kernels/vector_to_tensor.h"
@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
         column_index, def_levels, rep_levels, &cumulative_values);
 
     ++(*messages_read);
-    if (tensorflow::errors::IsOutOfRange(s)) {
+    if (absl::IsOutOfRange(s)) {
       break;
     }
     TF_RETURN_IF_ERROR(s);