TalkBack support for ScrollView accessibility announcements (list and grid) - JAVA ONLY CHANGES (#33180)

Summary:
This is the Java-only changes from D34518929 (dd6325b), split out for push safety. Original summary and test plan below:

This issue fixes [30977][17] . The Pull Request was previously published by [intergalacticspacehighway][13] with [31666][19].
The solution consists of:
1. Adding Javascript logic in the [FlatList][14], SectionList, VirtualizedList components to provide accessibility information (row and column position) for each cell in the method [renderItem][20] as a fourth parameter [accessibilityCollectionItem][21]. The information is saved on the native side in the AccessibilityNodeInfo and announced by TalkBack when changing row, column, or page ([video example][12]). The prop accessibilityCollectionItem is available in the View component which wraps each FlatList cell.
2. Adding Java logic in [ReactScrollView.java][16] and HorizontalScrollView to announce pages with TalkBack when scrolling up/down. The missing AOSP logic in [ScrollView.java][10] (see also the [GridView][11] example) is responsible for announcing Page Scrolling with TalkBack.

Relevant Links:
x [Additional notes on this PR][18]
x [discussion on the additional container View around each FlatList cell][22]
x [commit adding prop getCellsInItemCount to VirtualizedList][23]

## Changelog

[Android] [Added] - Accessibility announcement for list and grid in FlatList

Pull Request resolved: #33180

Test Plan:
[1]. TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer ([link][1])
[2]. TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer ([link][2])
[3]. `FlatList numColumns={undefined}` Should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack. ([link][3])
[4]. TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app ([link][4])

[1]: fabOnReact/react-native-notes#6 (comment)
[2]: fabOnReact/react-native-notes#6 (comment)
[3]: fabOnReact/react-native-notes#6 (comment)
[4]: fabOnReact/react-native-notes#6 (comment)
[10]:https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/AdapterView.java#L1027-L1029 "GridView.java method responsible for calling setFromIndex and setToIndex"
[11]:fabOnReact/react-native-notes#6 (comment) "test case on Android GridView"
[12]:fabOnReact/react-native-notes#6 (comment) "TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer"
[13]:https://github.com/intergalacticspacehighway "github intergalacticspacehighway"
[14]:https://github.com/fabriziobertoglio1987/react-native/blob/80acf523a4410adac8005d5c9472fb87f78e12ee/Libraries/Lists/FlatList.js#L617-L636 "FlatList accessibilityCollectionItem"
[16]:https://github.com/fabriziobertoglio1987/react-native/blob/5706bd7d3ee35dca48f85322a2bdcaec0bce2c85/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L183-L184 "logic added to ReactScrollView.java"
[17]: #30977
[18]: fabOnReact/react-native-notes#6
[19]: #31666
[20]: https://reactnative.dev/docs/next/flatlist#required-renderitem "FlatList renderItem documentation"
[21]: fabOnReact@7514735 "commit that introduces fourth param accessibilityCollectionItem in callback renderItem"
[22]: #33180 (comment) "discussion on the additional container View around each FlatList cell"
[23]: fabOnReact@d50fd1a "commit adding prop getCellsInItemCount to VirtualizedList"

Reviewed By: kacieb

Differential Revision: D37186697

Pulled By: blavalla

fbshipit-source-id: 7bb95274326ded417c6f1365cc8633391f589d1a

Author: fabOnReact

ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManager.java

@@ -249,6 +249,20 @@ public void setAccessibilityRole(@NonNull T view, @Nullable String accessibility
     view.setTag(R.id.accessibility_role, AccessibilityRole.fromValue(accessibilityRole));
   }
 
+  @Override
+  @ReactProp(name = ViewProps.ACCESSIBILITY_COLLECTION)
+  public void setAccessibilityCollection(
+      @NonNull T view, @Nullable ReadableMap accessibilityCollection) {
+    view.setTag(R.id.accessibility_collection, accessibilityCollection);
+  }
+
+  @Override
+  @ReactProp(name = ViewProps.ACCESSIBILITY_COLLECTION_ITEM)
+  public void setAccessibilityCollectionItem(
+      @NonNull T view, @Nullable ReadableMap accessibilityCollectionItem) {
+    view.setTag(R.id.accessibility_collection_item, accessibilityCollectionItem);
+  }
+
   @Override
   @ReactProp(name = ViewProps.ACCESSIBILITY_STATE)
   public void setViewState(@NonNull T view, @Nullable ReadableMap accessibilityState) {

ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManagerAdapter.java

@@ -31,6 +31,14 @@ public void setAccessibilityLiveRegion(@NonNull T view, @Nullable String liveReg
   @Override
   public void setAccessibilityRole(@NonNull T view, @Nullable String accessibilityRole) {}
 
+  @Override
+  public void setAccessibilityCollection(
+      @NonNull T view, @Nullable ReadableMap accessibilityCollection) {}
+
+  @Override
+  public void setAccessibilityCollectionItem(
+      @NonNull T view, @Nullable ReadableMap accessibilityCollectionItem) {}
+
   @Override
   public void setViewState(@NonNull T view, @Nullable ReadableMap accessibilityState) {}
 

ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManagerDelegate.java

@@ -48,6 +48,12 @@ public void setProperty(T view, String propName, @Nullable Object value) {
       case ViewProps.ACCESSIBILITY_STATE:
         mViewManager.setViewState(view, (ReadableMap) value);
         break;
+      case ViewProps.ACCESSIBILITY_COLLECTION:
+        mViewManager.setAccessibilityCollection(view, (ReadableMap) value);
+        break;
+      case ViewProps.ACCESSIBILITY_COLLECTION_ITEM:
+        mViewManager.setAccessibilityCollectionItem(view, (ReadableMap) value);
+        break;
       case ViewProps.BACKGROUND_COLOR:
         mViewManager.setBackgroundColor(
             view, value == null ? 0 : ColorPropConverter.getColor(value, view.getContext()));

ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManagerInterface.java

@@ -28,6 +28,10 @@ public interface BaseViewManagerInterface<T extends View> {
 
   void setAccessibilityRole(T view, @Nullable String accessibilityRole);
 
+  void setAccessibilityCollection(T view, @Nullable ReadableMap accessibilityCollection);
+
+  void setAccessibilityCollectionItem(T view, @Nullable ReadableMap accessibilityCollectionItem);
+
   void setViewState(T view, @Nullable ReadableMap accessibilityState);
 
   void setBackgroundColor(T view, int backgroundColor);

ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java

@@ -122,6 +122,7 @@ public enum AccessibilityRole {
     TABLIST,
     TIMER,
     LIST,
+    GRID,
     TOOLBAR;
 
     public static String getValue(AccessibilityRole role) {
@@ -152,6 +153,8 @@ public static String getValue(AccessibilityRole role) {
           return "android.widget.Switch";
         case LIST:
           return "android.widget.AbsListView";
+        case GRID:
+          return "android.widget.GridView";
         case NONE:
         case LINK:
         case SUMMARY:
@@ -242,6 +245,22 @@ public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfoCo
     }
     final ReadableArray accessibilityActions =
         (ReadableArray) host.getTag(R.id.accessibility_actions);
+
+    final ReadableMap accessibilityCollectionItem =
+        (ReadableMap) host.getTag(R.id.accessibility_collection_item);
+    if (accessibilityCollectionItem != null) {
+      int rowIndex = accessibilityCollectionItem.getInt("rowIndex");
+      int columnIndex = accessibilityCollectionItem.getInt("columnIndex");
+      int rowSpan = accessibilityCollectionItem.getInt("rowSpan");
+      int columnSpan = accessibilityCollectionItem.getInt("columnSpan");
+      boolean heading = accessibilityCollectionItem.getBoolean("heading");
+
+      AccessibilityNodeInfoCompat.CollectionItemInfoCompat collectionItemCompat =
+          AccessibilityNodeInfoCompat.CollectionItemInfoCompat.obtain(
+              rowIndex, rowSpan, columnIndex, columnSpan, heading);
+      info.setCollectionItemInfo(collectionItemCompat);
+    }
+
     if (accessibilityActions != null) {
       for (int i = 0; i < accessibilityActions.size(); i++) {
         final ReadableMap action = accessibilityActions.getMap(i);
@@ -466,6 +485,7 @@ public static void setDelegate(
             || view.getTag(R.id.accessibility_state) != null
             || view.getTag(R.id.accessibility_actions) != null
             || view.getTag(R.id.react_test_id) != null
+            || view.getTag(R.id.accessibility_collection_item) != null
             || view.getTag(R.id.accessibility_links) != null)) {
       ViewCompat.setAccessibilityDelegate(
           view,

ReactAndroid/src/main/java/com/facebook/react/uimanager/ViewProps.java

@@ -146,6 +146,8 @@ public class ViewProps {
   public static final String Z_INDEX = "zIndex";
   public static final String RENDER_TO_HARDWARE_TEXTURE = "renderToHardwareTextureAndroid";
   public static final String ACCESSIBILITY_LABEL = "accessibilityLabel";
+  public static final String ACCESSIBILITY_COLLECTION = "accessibilityCollection";
+  public static final String ACCESSIBILITY_COLLECTION_ITEM = "accessibilityCollectionItem";
   public static final String ACCESSIBILITY_HINT = "accessibilityHint";
   public static final String ACCESSIBILITY_LIVE_REGION = "accessibilityLiveRegion";
   public static final String ACCESSIBILITY_ROLE = "accessibilityRole";

ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactHorizontalScrollView.java

@@ -25,13 +25,10 @@
 import android.view.MotionEvent;
 import android.view.View;
 import android.view.ViewGroup;
-import android.view.accessibility.AccessibilityEvent;
 import android.widget.HorizontalScrollView;
 import android.widget.OverScroller;
 import androidx.annotation.Nullable;
-import androidx.core.view.AccessibilityDelegateCompat;
 import androidx.core.view.ViewCompat;
-import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
 import com.facebook.common.logging.FLog;
 import com.facebook.infer.annotation.Assertions;
 import com.facebook.react.common.ReactConstants;
@@ -122,22 +119,7 @@ public ReactHorizontalScrollView(Context context, @Nullable FpsListener fpsListe
     mReactBackgroundManager = new ReactViewBackgroundManager(this);
     mFpsListener = fpsListener;
 
-    ViewCompat.setAccessibilityDelegate(
-        this,
-        new AccessibilityDelegateCompat() {
-          @Override
-          public void onInitializeAccessibilityEvent(View host, AccessibilityEvent event) {
-            super.onInitializeAccessibilityEvent(host, event);
-            event.setScrollable(mScrollEnabled);
-          }
-
-          @Override
-          public void onInitializeAccessibilityNodeInfo(
-              View host, AccessibilityNodeInfoCompat info) {
-            super.onInitializeAccessibilityNodeInfo(host, info);
-            info.setScrollable(mScrollEnabled);
-          }
-        });
+    ViewCompat.setAccessibilityDelegate(this, new ReactScrollViewAccessibilityDelegate());
 
     mScroller = getOverScrollerFromParent();
     mReactScrollViewScrollState =
@@ -147,6 +129,10 @@ public void onInitializeAccessibilityNodeInfo(
                 : ViewCompat.LAYOUT_DIRECTION_LTR);
   }
 
+  public boolean getScrollEnabled() {
+    return mScrollEnabled;
+  }
+
   @Nullable
   private OverScroller getOverScrollerFromParent() {
     OverScroller scroller;
@@ -408,7 +394,7 @@ private boolean isScrolledInView(View descendent) {
   }
 
   /** Returns whether the given descendent is partially scrolled in view */
-  private boolean isPartiallyScrolledInView(View descendent) {
+  public boolean isPartiallyScrolledInView(View descendent) {
     int scrollDelta = getScrollDelta(descendent);
     descendent.getDrawingRect(mTempRect);
     return scrollDelta != 0 && Math.abs(scrollDelta) < mTempRect.width();

ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java

@@ -75,6 +75,7 @@ public class ReactScrollView extends ScrollView
   private final @Nullable OverScroller mScroller;
   private final VelocityHelper mVelocityHelper = new VelocityHelper();
   private final Rect mRect = new Rect(); // for reuse to avoid allocation
+  private final Rect mTempRect = new Rect();
   private final Rect mOverflowInset = new Rect();
 
   private boolean mActivelyScrolling;
@@ -120,6 +121,8 @@ public ReactScrollView(Context context, @Nullable FpsListener fpsListener) {
     mScroller = getOverScrollerFromParent();
     setOnHierarchyChangeListener(this);
     setScrollBarStyle(SCROLLBARS_OUTSIDE_OVERLAY);
+
+    ViewCompat.setAccessibilityDelegate(this, new ReactScrollViewAccessibilityDelegate());
   }
 
   @Override
@@ -191,6 +194,10 @@ public void setScrollEnabled(boolean scrollEnabled) {
     mScrollEnabled = scrollEnabled;
   }
 
+  public boolean getScrollEnabled() {
+    return mScrollEnabled;
+  }
+
   public void setPagingEnabled(boolean pagingEnabled) {
     mPagingEnabled = pagingEnabled;
   }
@@ -299,6 +306,19 @@ public void requestChildFocus(View child, View focused) {
     super.requestChildFocus(child, focused);
   }
 
+  private int getScrollDelta(View descendent) {
+    descendent.getDrawingRect(mTempRect);
+    offsetDescendantRectToMyCoords(descendent, mTempRect);
+    return computeScrollDeltaToGetChildRectOnScreen(mTempRect);
+  }
+
+  /** Returns whether the given descendent is partially scrolled in view */
+  public boolean isPartiallyScrolledInView(View descendent) {
+    int scrollDelta = getScrollDelta(descendent);
+    descendent.getDrawingRect(mTempRect);
+    return scrollDelta != 0 && Math.abs(scrollDelta) < mTempRect.width();
+  }
+
   private void scrollToChild(View child) {
     Rect tempRect = new Rect();
     child.getDrawingRect(tempRect);

ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollViewAccessibilityDelegate.java

@@ -0,0 +1,154 @@
+/*
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+package com.facebook.react.views.scroll;
+
+import android.view.View;
+import android.view.ViewGroup;
+import android.view.accessibility.AccessibilityEvent;
+import androidx.core.view.AccessibilityDelegateCompat;
+import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
+import com.facebook.react.R;
+import com.facebook.react.bridge.AssertionException;
+import com.facebook.react.bridge.ReactSoftExceptionLogger;
+import com.facebook.react.bridge.ReadableMap;
+import com.facebook.react.uimanager.ReactAccessibilityDelegate;
+
+public class ReactScrollViewAccessibilityDelegate extends AccessibilityDelegateCompat {
+  private final String TAG = ReactScrollViewAccessibilityDelegate.class.getSimpleName();
+
+  @Override
+  public void onInitializeAccessibilityEvent(View host, AccessibilityEvent event) {
+    super.onInitializeAccessibilityEvent(host, event);
+    if (host instanceof ReactScrollView || host instanceof ReactHorizontalScrollView) {
+      onInitializeAccessibilityEventInternal(host, event);
+    } else {
+      ReactSoftExceptionLogger.logSoftException(
+          TAG,
+          new AssertionException(
+              "ReactScrollViewAccessibilityDelegate should only be used with ReactScrollView or ReactHorizontalScrollView, not with class: "
+                  + host.getClass().getSimpleName()));
+    }
+  }
+
+  @Override
+  public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfoCompat info) {
+    super.onInitializeAccessibilityNodeInfo(host, info);
+    if (host instanceof ReactScrollView || host instanceof ReactHorizontalScrollView) {
+      onInitializeAccessibilityNodeInfoInternal(host, info);
+    } else {
+      ReactSoftExceptionLogger.logSoftException(
+          TAG,
+          new AssertionException(
+              "ReactScrollViewAccessibilityDelegate should only be used with ReactScrollView or ReactHorizontalScrollView, not with class: "
+                  + host.getClass().getSimpleName()));
+    }
+  };
+
+  private void onInitializeAccessibilityEventInternal(View view, AccessibilityEvent event) {
+    final ReadableMap accessibilityCollection =
+        (ReadableMap) view.getTag(R.id.accessibility_collection);
+
+    if (accessibilityCollection != null) {
+      event.setItemCount(accessibilityCollection.getInt("itemCount"));
+      View contentView;
+      if (view instanceof ViewGroup) {
+        ViewGroup viewGroup = (ViewGroup) view;
+        contentView = viewGroup.getChildAt(0);
+      } else {
+        return;
+      }
+      Integer firstVisibleIndex = null;
+      Integer lastVisibleIndex = null;
+
+      if (!(contentView instanceof ViewGroup)) {
+        return;
+      }
+
+      for (int index = 0; index < ((ViewGroup) contentView).getChildCount(); index++) {
+        View nextChild = ((ViewGroup) contentView).getChildAt(index);
+        boolean isVisible;
+        if (view instanceof ReactScrollView) {
+          ReactScrollView scrollView = (ReactScrollView) view;
+          isVisible = scrollView.isPartiallyScrolledInView(nextChild);
+        } else if (view instanceof ReactHorizontalScrollView) {
+          ReactHorizontalScrollView scrollView = (ReactHorizontalScrollView) view;
+          isVisible = scrollView.isPartiallyScrolledInView(nextChild);
+        } else {
+          return;
+        }
+        ReadableMap accessibilityCollectionItem =
+            (ReadableMap) nextChild.getTag(R.id.accessibility_collection_item);
+
+        if (!(nextChild instanceof ViewGroup)) {
+          return;
+        }
+
+        int childCount = ((ViewGroup) nextChild).getChildCount();
+
+        // If this child's accessibilityCollectionItem is null, we'll check one more
+        // nested child.
+        // Happens when getItemLayout is not passed in FlatList which adds an additional
+        // View in the hierarchy.
+        if (childCount > 0 && accessibilityCollectionItem == null) {
+          View nestedNextChild = ((ViewGroup) nextChild).getChildAt(0);
+          if (nestedNextChild != null) {
+            ReadableMap nestedChildAccessibility =
+                (ReadableMap) nestedNextChild.getTag(R.id.accessibility_collection_item);
+            if (nestedChildAccessibility != null) {
+              accessibilityCollectionItem = nestedChildAccessibility;
+            }
+          }
+        }
+
+        if (isVisible == true && accessibilityCollectionItem != null) {
+          if (firstVisibleIndex == null) {
+            firstVisibleIndex = accessibilityCollectionItem.getInt("itemIndex");
+          }
+          lastVisibleIndex = accessibilityCollectionItem.getInt("itemIndex");
+        }
+
+        if (firstVisibleIndex != null && lastVisibleIndex != null) {
+          event.setFromIndex(firstVisibleIndex);
+          event.setToIndex(lastVisibleIndex);
+        }
+      }
+    }
+  }
+
+  private void onInitializeAccessibilityNodeInfoInternal(
+      View view, AccessibilityNodeInfoCompat info) {
+    final ReactAccessibilityDelegate.AccessibilityRole accessibilityRole =
+        (ReactAccessibilityDelegate.AccessibilityRole) view.getTag(R.id.accessibility_role);
+
+    if (accessibilityRole != null) {
+      ReactAccessibilityDelegate.setRole(info, accessibilityRole, view.getContext());
+    }
+
+    final ReadableMap accessibilityCollection =
+        (ReadableMap) view.getTag(R.id.accessibility_collection);
+
+    if (accessibilityCollection != null) {
+      int rowCount = accessibilityCollection.getInt("rowCount");
+      int columnCount = accessibilityCollection.getInt("columnCount");
+      boolean hierarchical = accessibilityCollection.getBoolean("hierarchical");
+
+      AccessibilityNodeInfoCompat.CollectionInfoCompat collectionInfoCompat =
+          AccessibilityNodeInfoCompat.CollectionInfoCompat.obtain(
+              rowCount, columnCount, hierarchical);
+      info.setCollectionInfo(collectionInfoCompat);
+    }
+
+    if (view instanceof ReactScrollView) {
+      ReactScrollView scrollView = (ReactScrollView) view;
+      info.setScrollable(scrollView.getScrollEnabled());
+    } else if (view instanceof ReactHorizontalScrollView) {
+      ReactHorizontalScrollView scrollView = (ReactHorizontalScrollView) view;
+      info.setScrollable(scrollView.getScrollEnabled());
+    }
+  }
+};

ReactAndroid/src/main/res/views/uimanager/values/ids.xml

@@ -15,6 +15,12 @@
   <!--tag is used to store accessibilityRole tag-->
   <item type="id" name="accessibility_role"/>
 
+  <!--tag is used to store accessibilityCollection -->
+  <item type="id" name="accessibility_collection"/>
+
+  <!--tag is used to store accessibilityCollectionItem -->
+  <item type="id" name="accessibility_collection_item"/>
+
   <!--tag is used to store accessibilityState -->
   <item type="id" name="accessibility_state"/>