Improve test creation performance (#3518)

Test insertion is the most performance-critical part of CDash, simply
because there are often 2-4 orders of magnitude more tests than builds.
Each of those tests then has an additional order of magnitude more
measurements and/or labels associated with it. This PR improves two
things:

1. Only the test output ID is queried instead of the entire model. This
reduces the amount of network I/O to the database.
2. The entire test creation process is wrapped in a transaction. In
addition to ensuring consistency, this also avoids the overhead of
committing after many small transactions.

There's more to be done here, particularly relating to bulk inserts of
labels, measurements, and images, but some substantial refactoring needs
to be done first.

Author: williamjallen

app/Utils/TestCreator.php

@@ -24,6 +24,7 @@
 use CDash\Model\Build;
 use CDash\Model\Image;
 use ErrorException;
+use Illuminate\Support\Facades\DB;
 use Illuminate\Support\Facades\Log;
 
 /**
@@ -136,46 +137,46 @@ public function create(Build $build): void
             }
         }
 
-        $outputid = TestOutput::firstOrCreate([
-            'path' => $this->testPath,
-            'command' => $this->testCommand,
-            'output' => $this->testOutput,
-        ])->id;
-
-        // build2test
-        $buildtest = new Test();
-        $buildtest->buildid = $build->Id;
-        $buildtest->outputid = $outputid;
-        $buildtest->status = $this->testStatus;
-        $buildtest->details = $this->testDetails;
-        $buildtest->time = "$this->buildTestTime";
-        $buildtest->testname = $this->testName;
-        $buildtest->starttime = $this->testStartTime;
-
-        // Note: the newstatus column is currently handled in
-        // ctestparserutils::compute_test_difference. This gets updated when we call
-        // Build::ComputeTestTiming.
-        $buildtest->save();
-
-        foreach ($this->measurements as $measurement) {
-            $measurement->testid = $buildtest->id;
-            $measurement->save();
-        }
+        DB::transaction(function () use ($build): void {
+            $outputid = TestOutput::select('id')->firstOrCreate([
+                'path' => $this->testPath,
+                'command' => $this->testCommand,
+                'output' => $this->testOutput,
+            ])->id;
+
+            // Note: the newstatus column is currently handled in
+            // ctestparserutils::compute_test_difference. This gets updated when we call
+            // Build::ComputeTestTiming.
+            $buildtest = Test::create([
+                'buildid' => $build->Id,
+                'outputid' => $outputid,
+                'status' => $this->testStatus,
+                'details' => $this->testDetails,
+                'time' => "$this->buildTestTime",
+                'testname' => $this->testName,
+                'starttime' => $this->testStartTime,
+            ]);
+
+            foreach ($this->measurements as $measurement) {
+                $measurement->testid = $buildtest->id;
+                $measurement->save();
+            }
 
-        // Give measurements to the buildtest model so we can properly calculate
-        // proctime later on.
-        $buildtest->measurements = $this->measurements;
-        $build->AddTest($buildtest);
+            // Give measurements to the buildtest model so we can properly calculate
+            // proctime later on.
+            $buildtest->measurements = $this->measurements;
+            $build->AddTest($buildtest);
 
-        foreach ($this->labels as $label) {
-            $label->Test = $buildtest;
-            $label->Insert();
-            $buildtest->addLabel($label);
-        }
+            foreach ($this->labels as $label) {
+                $label->Test = $buildtest;
+                $label->Insert();
+                $buildtest->addLabel($label);
+            }
 
-        // test2image
-        foreach ($this->images as $image) {
-            $this->saveImage($image, $buildtest->id);
-        }
+            // test2image
+            foreach ($this->images as $image) {
+                $this->saveImage($image, $buildtest->id);
+            }
+        });
     }
 }

phpstan-baseline.neon

@@ -7062,12 +7062,6 @@ parameters:
 			count: 1
 			path: app/Utils/TestCreator.php
 
-		-
-			rawMessage: 'Property App\Models\Test::$time (float) does not accept string.'
-			identifier: assign.propertyType
-			count: 1
-			path: app/Utils/TestCreator.php
-
 		-
 			rawMessage: Property App\Utils\TestCreator::$alreadyCompressed has no type specified.
 			identifier: missingType.property