Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
77d6548
vide coding prototype of greedysnake
tiye Mar 27, 2026
3df1c20
finished prototype of snake
tiye Mar 28, 2026
fad5fee
get interactions more intuitive
tiye Mar 31, 2026
5967495
style refinements to get game easier
tiye Apr 1, 2026
de97e60
refine for colors
tiye Apr 2, 2026
1c23e3a
add a demo trying 4D stereographic projection
tiye Apr 10, 2026
c76eaa7
improve projection rotation
tiye Apr 10, 2026
4ce959b
drafting more 4d shapes
tiye Apr 10, 2026
26a26dc
reduce 4d shape triangles
tiye Apr 10, 2026
19764e3
vibe add cell highlight mode
tiye Apr 11, 2026
2f48266
SDF rendering demo with spheres and rhombic-mirror
tiye May 2, 2026
9154730
working on 3dgs rendering, not yet
tiye May 2, 2026
56a3499
still trying 3dgs
tiye May 2, 2026
b2f00fb
still trying 3dgs; stop here since os might crash
tiye May 2, 2026
8f6a44d
port shadertoy reflection demos
tiye May 3, 2026
578942b
still vibing container based raymarching arts, translating shadertoy …
tiye May 3, 2026
c25634f
refine shadertoy migrated demos
tiye May 3, 2026
939a5cb
get flyingthrough demo roughly ok
tiye May 3, 2026
270d78e
trying to translate more shadertoy demos
tiye May 3, 2026
f36901d
raymarching explained
tiye May 4, 2026
05aaea5
translating some more shadertoy demos
tiye May 5, 2026
2ecdaec
porting more shadertoys demos
tiye May 5, 2026
be42bdb
random attempts related to 3dgs
tiye May 7, 2026
9016ebb
improve glassBox detail in close distance
tiye May 7, 2026
4012425
tiling issue, sadly
tiye May 7, 2026
9e5ee89
translating more shadertoy demos
tiye May 8, 2026
f55e695
vide translating...(not checked)
tiye May 8, 2026
ab3c661
random fixes of raymarching shaders
tiye May 9, 2026
94312cb
random fixes of raymarching demos
tiye May 9, 2026
1f045f0
translating more shadertoy demos
tiye May 10, 2026
d33308f
fix shield demo and digital lines demo
tiye May 10, 2026
3ae7ad1
add trails demo, but slow
tiye May 11, 2026
ec30e8b
apollonian twist demo
tiye May 13, 2026
d0c0e16
translate shadertoys demos steampunk orb and apollonian wires
tiye May 14, 2026
e565989
translating more shadertoy demos
tiye May 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,8 @@ ignore/
.trae/

Packages/RealityKitContent/.build/
.build/
.build/

buildServer.json

resources/
132 changes: 132 additions & 0 deletions notes/05-08-tile-artifacts-and-stereo-bugs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# 05-08 瓦片伪影与立体渲染 Bug 排查记录

## 症状

多个 demo(PongWar、Snake3D、Stereographic 多胞体等)在 visionOS Simulator 上,
几何体周围出现明显的彩色方块(瓦片 / tile artifacts)。同时发现过右眼黑屏的问题。

---

## 根本原因 1:瓦片伪影(tile artifacts)

### 机制

visionOS 的 foveation(注视渲染)通过 `rasterizationRateMap` 将渲染目标分成密度
不均的 tile 块。启用 foveation 后,Metal 要求渲染器在每个 draw call 之前**显式**调
用 `setVertexAmplificationCount`,即使 viewCount == 1 时也必须调用,否则 GPU 不
知道如何将虚拟 viewport 坐标(如 4338×3478)映射到物理 texture(如 1888×1792),
未覆盖的 tile 会以 clear color 的颜色暴露出来。

### 直接触发条件

`RendererTypes.swift` 的 `applyViewConfiguration` 的 `else` 分支被误删:

```swift
// 错误(会导致 tile artifacts):
if viewData.viewCount > 1 {
encoder.setVertexAmplificationCount(...)
}
// ← 缺少 else,单视图时 GPU 拿不到 amplification count

// 正确:
if viewData.viewCount > 1 {
encoder.setVertexAmplificationCount(viewData.viewCount, viewMappings: &viewMappings)
} else {
encoder.setVertexAmplificationCount(1, viewMappings: nil) // ← 必须保留!
}
```

### 次要暴露条件:clearColor 不是纯黑

Foveation tile 被 clear 到 clearColor。如果 clearColor 是纯黑 `(0,0,0,1)`,tile
在黑色背景中不可见。如果 clearColor 有任何颜色分量,tile 就会以彩色方块出现。

**当前解决方案**:Renderer.swift 中 clearColor 硬编码为纯黑,与 `main` 分支保持
一致,完全忽略各 demo 的 `preferredClearColor` 属性。

```swift
// Renderer.swift - encodeDrawable 内
// ⚠️ 必须是纯黑。foveation 的 rasterizationRateMap 会把渲染目标分成 tile,
// 未被几何体覆盖的区域会被 clear 到此颜色。任何非黑色都会造成可见的彩色瓦片。
// 各 demo 的 preferredClearColor 属性目前被忽略(见各 renderer 文件)。
let clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
```

---

## 根本原因 2:右眼黑屏(两眼画面叠到左眼)

### 机制

visionOS layered layout 下,左右眼在**同一张 texture** 的不同 array slice(左眼
slice=0,右眼 slice=1)。

- `textureMap.textureIndex`:该 view 对应 drawable 的哪个 texture(两眼都是 0)
- `textureMap.sliceIndex`:该 texture 内的哪个 array slice(左眼 0,右眼 1)

在 `makeViewRenderingData` 中必须用 `sliceIndex`:

```swift
// 错误(导致两眼都渲染到 layer 0,右眼黑屏):
renderTargetLayers.append(UInt32(textureMap.textureIndex))

// 正确:
renderTargetLayers.append(UInt32(textureMap.sliceIndex))
```

---

## 根本原因 3:`cp_frame_end_submission` before `encodePresent` 崩溃

### 机制

CompositorServices 规定:调用 `queryDrawables()` 后,每个 drawable 都必须经过
`encodePresent(commandBuffer:)` 才能调用 `endSubmission()`。

当 world tracking 尚未就绪(无 deviceAnchor)时,原代码走 `completeEmptySubmission`
路径,只调用 `startSubmission/endSubmission`,跳过了 `encodePresent`,导致崩溃。

**修复**:`completeEmptySubmissionIfPossible` 接受 `drawables` 参数,对每个 drawable
提交一个空的命令 buffer:

```swift
private func completeEmptySubmissionIfPossible(
for frame: LayerRenderer.Frame,
drawables: [LayerRenderer.Drawable] = []
) {
guard layerRenderer.state == .running else { return }
frame.startSubmission()
for drawable in drawables {
guard let cmdBuf = commandQueue.makeCommandBuffer() else { continue }
drawable.encodePresent(commandBuffer: cmdBuf)
cmdBuf.commit()
}
frame.endSubmission()
}
```

---

## 排查过程中的误操作记录

以下操作**不是解决方案**,记录以避免重蹈:

1. **`isFoveationEnabled = false`**:不能消除 tile。即使禁用 foveation,
simulator 仍可能返回非 trivial 的 rate map。
2. **`rasterizationRateMap = nil`**:单独设为 nil 不能解决问题,且会破坏
foveation 的虚拟坐标到物理坐标的映射。
3. **用物理 texture 尺寸覆盖 viewport**:`textureMap.viewport` 是虚拟 foveation
坐标,不应替换为物理尺寸,这会破坏 foveation 的工作方式。
4. **`renderTargetArrayLength = max(min(...), 1)` 而非 `colorTexture.arrayLength`**:
clamp 版本会导致 render target 层数不足。

---

## 关键文件位置

| 文件 | 关键位置 | 说明 |
| ------------------------------ | ----------------------------------- | ---------------------------------------------------------------- |
| `Renderer/RendererTypes.swift` | `applyViewConfiguration` | **必须**保留 else 分支调用 `setVertexAmplificationCount(1, nil)` |
| `Renderer/Renderer.swift` | `encodeDrawable` clearColor | 硬编码黑色,不得改为读取 `preferredClearColor` |
| `Renderer/Renderer.swift` | `makeViewRenderingData` | 用 `sliceIndex` 不是 `textureIndex` |
| `Renderer/Renderer.swift` | `completeEmptySubmissionIfPossible` | 必须传入 drawables 并 encodePresent |
4 changes: 3 additions & 1 deletion vr-dive.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@
attributes = {
BuildIndependentTargetsInParallel = 1;
LastSwiftUpdateCheck = 2620;
LastUpgradeCheck = 2620;
LastUpgradeCheck = 2640;
TargetAttributes = {
9A9B19C32ECF8284003DA309 = {
CreatedOnToolsVersion = 26.2;
Expand Down Expand Up @@ -270,6 +270,7 @@
MTL_FAST_MATH = YES;
ONLY_ACTIVE_ARCH = YES;
SDKROOT = xros;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_ACTIVE_COMPILATION_CONDITIONS = "DEBUG $(inherited)";
SWIFT_OPTIMIZATION_LEVEL = "-Onone";
XROS_DEPLOYMENT_TARGET = 26.1;
Expand Down Expand Up @@ -327,6 +328,7 @@
MTL_ENABLE_DEBUG_INFO = NO;
MTL_FAST_MATH = YES;
SDKROOT = xros;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_COMPILATION_MODE = wholemodule;
VALIDATE_PRODUCT = YES;
XROS_DEPLOYMENT_TARGET = 26.1;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<?xml version="1.0" encoding="UTF-8"?>
<Bucket
uuid = "5A2B2B4B-2118-433C-9142-CFC3AB28334E"
type = "1"
version = "2.0">
<Breakpoints>
<BreakpointProxy
BreakpointExtensionID = "Xcode.Breakpoint.FileBreakpoint">
<BreakpointContent
uuid = "4E46E4BB-290B-4DC1-92B1-EB8C4C3F13A9"
shouldBeEnabled = "No"
ignoreCount = "0"
continueAfterRunningActions = "No"
filePath = "vr-dive/Demos/RayMarchingDemo/RayMarchingDemoRenderer.swift"
startingColumnNumber = "9223372036854775807"
endingColumnNumber = "9223372036854775807"
startingLineNumber = "240"
endingLineNumber = "240"
landmarkName = "rmdAppendRectFrame(_:center:halfWidth:halfHeight:radius:color:)"
landmarkType = "9">
</BreakpointContent>
</BreakpointProxy>
</Breakpoints>
</Bucket>
141 changes: 108 additions & 33 deletions vr-dive/ContentView.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,65 +13,140 @@ struct ContentView: View {
@Environment(AppModel.self) private var appModel

var body: some View {
VStack(spacing: 24) {
PatternMenuView(model: appModel.patternMenuModel)
ToggleImmersiveSpaceButton()
VStack(spacing: 28) {
HStack(alignment: .bottom, spacing: 20) {
PatternMenuView(model: appModel.patternMenuModel)
ToggleImmersiveSpaceButton()
}
ControlButtonsView(model: appModel.patternMenuModel)
}
.padding(.horizontal, 24)
.padding(.vertical, 28)
.frame(maxWidth: 420)
.padding(.horizontal, 28)
.padding(.vertical, 32)
.frame(maxWidth: 760, minHeight: 280)
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .center)
}
}

struct PatternMenuView: View {
@Bindable var model: PatternMenuModel

private var nextPattern: VisualPatternKind {
let all = VisualPatternKind.allCases
let idx = all.firstIndex(of: model.selectedPattern) ?? 0
return all[(idx + 1) % all.count]
}

var body: some View {
VStack(alignment: .leading, spacing: 8) {
Text("图案切换")
.font(.headline)
Picker("当前图案", selection: $model.selectedPattern) {
ForEach(VisualPatternKind.allCases) { pattern in
Text(pattern.displayName).tag(pattern)

HStack(spacing: 10) {
Picker("当前图案", selection: $model.selectedPattern) {
ForEach(VisualPatternKind.allCases) { pattern in
Text(pattern.displayName).tag(pattern)
}
}
.pickerStyle(.menu)

Button(action: { model.selectedPattern = nextPattern }) {
VStack(spacing: 1) {
Image(systemName: "chevron.right")
.font(.caption.weight(.semibold))
Text(nextPattern.displayName)
.font(.system(size: 9))
.lineLimit(1)
}
.frame(minWidth: 64)
}
.buttonStyle(.bordered)
}
.pickerStyle(.menu)
}
.frame(maxWidth: .infinity, alignment: .leading)
.frame(minWidth: 360, maxWidth: .infinity, alignment: .leading)
}
}

struct ControlButtonsView: View {
@Bindable var model: PatternMenuModel

var body: some View {
HStack(spacing: 12) {
Button(action: {
model.reset()
}) {
Label("Reset", systemImage: "arrow.counterclockwise")
VStack(spacing: 18) {
HStack(spacing: 18) {
Button(action: {
model.reset()
}) {
Label("Reset", systemImage: "arrow.counterclockwise")
}
.buttonStyle(.bordered)

Button(action: {
model.isPaused.toggle()
}) {
Label(
model.isPaused ? "Resume" : "Pause",
systemImage: model.isPaused ? "play.fill" : "pause.fill")
}
.buttonStyle(.bordered)

Button(action: {
model.toggleSpeed()
}) {
Label(
model.speedMultiplier > 1.0 ? "x5" : "x1",
systemImage: model.speedMultiplier > 1.0 ? "hare.fill" : "tortoise.fill")
}
.buttonStyle(.bordered)
}
.buttonStyle(.bordered)

Button(action: {
model.isPaused.toggle()
}) {
Label(
model.isPaused ? "Resume" : "Pause",
systemImage: model.isPaused ? "play.fill" : "pause.fill")

if model.selectedPattern == .rayMarchingDemo {
Button(action: {
model.cycleRayMarchingProbeDimTarget()
}) {
Label(model.rayMarchingProbeDimTarget.buttonTitle, systemImage: "circle.lefthalf.filled")
}
.buttonStyle(.bordered)
}

if model.selectedPattern == .huashan {
VStack(alignment: .leading, spacing: 8) {
Text("华山点比例")
.font(.headline)

HStack(spacing: 14) {
Button(action: {
model.adjustHuashanSampleRatio(by: -0.05)
}) {
Label("减少 5%", systemImage: "minus")
}
.buttonStyle(.bordered)

Text(model.huashanSampleRatioPercentText)
.font(.system(.body, design: .monospaced).weight(.semibold))
.frame(minWidth: 52)

Button(action: {
model.adjustHuashanSampleRatio(by: 0.05)
}) {
Label("增加 5%", systemImage: "plus")
}
.buttonStyle(.bordered)
}
}
.frame(maxWidth: .infinity, alignment: .leading)
}
.buttonStyle(.bordered)

Button(action: {
model.toggleSpeed()
}) {
Label(
model.speedMultiplier > 1.0 ? "x8" : "x1",
systemImage: model.speedMultiplier > 1.0 ? "hare.fill" : "tortoise.fill")

if model.selectedPattern.supportsOriginCellInspection {
Toggle(isOn: $model.originCellInspectionEnabled) {
VStack(alignment: .leading, spacing: 2) {
Text("原点胞高亮")
Text("自动暂停并加粗高亮包含原点的一个胞")
.font(.caption)
.foregroundStyle(.secondary)
}
}
.toggleStyle(.switch)
.padding(.top, 2)
}
.buttonStyle(.bordered)
}
.frame(maxWidth: .infinity, alignment: .center)
}
Expand Down
Loading