bigtable: many tests failed

[flaky-bot]

Many tests failed at the same time in this package.

• I will close this issue when there are no more failures in this package and
  there is at least one pass.
• No new issues will be filed for this package until this issue is closed.
• If there are already issues for individual test cases, I will close them when
  the corresponding test passes. You can close them earlier, if you prefer, and
  I won't reopen them while this issue is still open.

Here are the tests that failed:

• TestIntegration_TieredStorage
• TestIntegration_Pinger
• TestIntegration_UpdateFamilyValueType (bigtable: TestIntegration_UpdateFamilyValueType failed #11182)
• TestIntegration_Aggregates (bigtable: TestIntegration_Aggregates failed #14029)
• TestIntegration_HighlyConcurrentReadsAndWrites (bigtable: TestIntegration_HighlyConcurrentReadsAndWrites failed #10391)
• TestIntegration_NoopMetricsProvider
• TestIntegration_ExportBuiltInMetrics (bigtable: TestIntegration_ExportBuiltInMetrics failed #13732)
• TestIntegration_LargeReadsWritesAndScans (bigtable: TestIntegration_LargeReadsWritesAndScans failed #10909)
• TestIntegration_Presidents
• TestIntegration_FullReadStats (bigtable: TestIntegration_FullReadStats failed #9365)
• TestIntegration_SampleRowKeys (bigtable: TestIntegration_SampleRowKeys failed #11394)

---

commit: db45f2d
buildURL: Build Status, Sponge
status: failed

[flaky-bot]

13 tests failed in this package for commit 5ca653a (Build Status, Sponge).

---

commit: 5ca653a
buildURL: Build Status, Sponge
status: failed

[flaky-bot]

11 tests failed in this package for commit cfdd7ba (Build Status, Sponge).

---

commit: cfdd7ba
buildURL: Build Status, Sponge
status: failed

[flaky-bot]

13 tests failed in this package for commit dc245af (Build Status, Sponge).

---

commit: dc245af
buildURL: Build Status, Sponge
status: failed

[shollyman]

I suspect #14319 may be related here.

[bhshkh]

The failures occurred because the newly introduced connection pool's priming mechanism (PingAndWarm RPC) fails with a NotFound error when running against a production instance that does not have any tables created yet. The error message was:

integration_test.go:5621: Error creating client: bigtable_connpool:  connection factory prime failed: factory prime failed after 3 attempts: rpc error: code = NotFound desc = No tables found for instance projects/dulcet-port-762/instances/gc-bt-it-instance
    integration_test.go:196: bigtable_connpool:  connection factory prime failed: factory prime failed after 3 attempts: rpc error: code = NotFound desc = No tables found for instance projects/dulcet-port-762/instances/gc-bt-it-instance
--- FAIL: TestIntegration_TieredStorage (0.76s)

For now, I have created a table in the project used by kokoro tests pipeline.

Created #14424 to track the actual fix

[flaky-bot]

1 test failed in this package for commit 9d47d1c (Build Status, Sponge).

---

commit: 9d47d1c
buildURL: Build Status, Sponge
status: failed

[flaky-bot]

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

---

When run at the same commit (9d47d1c), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).