Skip to content

Commit a9a150e

Browse files
LordLuceusclaude
andauthored
feat: Add braille display support via UniversalSpeech (#2)
Add braille output functionality that sends text to braille displays through the screen reader's braille support. Braille is enabled by default and can be toggled via SpeechManager.EnableBraille. Changes: - Add brailleDisplay P/Invoke to UniversalSpeechWrapper - Add DisplayBraille() method for direct braille output - Update SpeechManager.Output() and RepeatLast() to output to braille - Document braille support, custom text replacements, and extensibility Closes #1 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 5bab72f commit a9a150e

4 files changed

Lines changed: 147 additions & 13 deletions

File tree

CLAUDE.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
44

55
## Project Overview
66

7-
MelonAccessibilityLib is a C# library that adds screen reader accessibility to Unity games via MelonLoader mods. It provides speech management, text cleaning utilities, and P/Invoke wrappers for UniversalSpeech with SAPI fallback.
7+
MelonAccessibilityLib is a C# library that adds screen reader accessibility to Unity games via MelonLoader mods. It provides speech and braille output, text cleaning utilities, and P/Invoke wrappers for UniversalSpeech with SAPI fallback.
88

99
## Build Commands
1010

@@ -34,9 +34,9 @@ No test framework is currently configured. If adding tests, use standard `dotnet
3434

3535
### Component Overview
3636

37-
- **SpeechManager** (`SpeechManager.cs`): High-level static API for speech output with duplicate prevention, repeat functionality, and text formatting
38-
- **UniversalSpeechWrapper** (`UniversalSpeechWrapper.cs`): Low-level P/Invoke wrapper for UniversalSpeech.dll with SAPI fallback
39-
- **TextCleaner** (`TextCleaner.cs`): Removes Unity rich text tags and normalizes text for screen reader output
37+
- **SpeechManager** (`SpeechManager.cs`): High-level static API for speech and braille output with duplicate prevention, repeat functionality, and text formatting
38+
- **UniversalSpeechWrapper** (`UniversalSpeechWrapper.cs`): Low-level P/Invoke wrapper for UniversalSpeech.dll with SAPI fallback; provides `Speak()` and `DisplayBraille()` methods
39+
- **TextCleaner** (`TextCleaner.cs`): Removes Unity rich text tags and normalizes text; supports custom string and regex replacements via `AddReplacement()` and `AddRegexReplacement()`
4040
- **AccessibilityLog/IAccessibilityLogger** (`IAccessibilityLogger.cs`): Logging facade with pluggable logger interface
4141
- **Net35Extensions** (`Net35Extensions.cs`): Polyfills for .NET 3.5 compatibility (e.g., `IsNullOrWhiteSpace`)
4242

@@ -50,8 +50,9 @@ Consumer (MelonMod)
5050
└─ Calls SpeechManager.Output()
5151
5252
├─ Duplicate suppression (time-based)
53-
├─ TextCleaner.Clean() (strips rich text)
54-
└─ UniversalSpeechWrapper.Speak() (P/Invoke)
53+
├─ TextCleaner.Clean() (strips rich text, applies custom replacements)
54+
├─ UniversalSpeechWrapper.Speak() (P/Invoke)
55+
└─ UniversalSpeechWrapper.DisplayBraille() (if EnableBraille)
5556
```
5657

5758
### Extensibility Points
@@ -60,6 +61,8 @@ Consumer (MelonMod)
6061
- **Text Formatting**: Set `SpeechManager.FormatTextOverride` delegate
6162
- **Repeat Logic**: Set `SpeechManager.ShouldStoreForRepeatPredicate` delegate
6263
- **Custom Text Types**: Use constants starting from `TextType.CustomBase` (100)
64+
- **Text Cleaning**: Use `TextCleaner.AddReplacement()` and `TextCleaner.AddRegexReplacement()` for custom text transformations
65+
- **Braille Control**: Set `SpeechManager.EnableBraille` to toggle braille output (default: true)
6366

6467
## Key Conventions
6568

README.md

Lines changed: 88 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@ A reusable library for adding screen reader accessibility to Unity games via Mel
55
## Features
66

77
- **UniversalSpeech integration** - P/Invoke wrapper for the UniversalSpeech library with SAPI fallback
8+
- **Braille display support** - Automatic output to braille displays via screen reader
89
- **High-level speech manager** - Duplicate prevention, repeat functionality, speaker formatting
9-
- **Text cleaning** - Strips Unity rich text tags (`<color>`, `<size>`, `<b>`, etc.)
10+
- **Text cleaning** - Strips Unity rich text tags (`<color>`, `<size>`, `<b>`, etc.) with extensible custom replacements
1011
- **Logging abstraction** - Integrate with MelonLoader or any logging system
1112
- **Multi-target support** - Builds for net6.0, net472, and net35 for broad compatibility with various games
1213

@@ -123,13 +124,16 @@ SpeechManager.RepeatLast();
123124
| `Stop()` | Stop current speech. |
124125
| `ClearRepeatBuffer()` | Clear stored repeat text. |
125126

126-
| Property | Description |
127-
| ------------------------------- | ----------------------------------------------------- |
128-
| `DuplicateWindowSeconds` | Time window for duplicate suppression (default: 0.5s) |
129-
| `EnableLogging` | Whether to log speech output (default: true) |
130-
| `ShouldStoreForRepeatPredicate` | Custom predicate for repeat storage |
127+
| Property | Description |
128+
| ------------------------------- | -------------------------------------------------------------- |
129+
| `DuplicateWindowSeconds` | Time window for duplicate suppression (default: 0.5s) |
130+
| `EnableLogging` | Whether to log speech output (default: true) |
131+
| `EnableBraille` | Whether to output to braille displays (default: true) |
132+
| `FormatTextOverride` | Custom delegate for text formatting (see Extensibility) |
133+
| `ShouldStoreForRepeatPredicate` | Custom predicate for repeat storage (see Extensibility) |
134+
| `TextTypeNames` | Dictionary mapping text type IDs to names for logging |
131135

132-
### TextType Enum
136+
### TextType Constants
133137

134138
| Value | Description |
135139
| ------------ | ------------------------------------------------- |
@@ -138,6 +142,7 @@ SpeechManager.RepeatLast();
138142
| `Menu` | Menu item text |
139143
| `MenuChoice` | Menu selection |
140144
| `System` | System messages |
145+
| `CustomBase` | Base value (100) for defining custom text types |
141146

142147
### UniversalSpeechWrapper
143148

@@ -146,12 +151,15 @@ Low-level access to UniversalSpeech:
146151
```csharp
147152
UniversalSpeechWrapper.Initialize(); // Initialize (called by SpeechManager)
148153
UniversalSpeechWrapper.Speak("text", true); // Speak with interrupt
154+
UniversalSpeechWrapper.DisplayBraille("text"); // Output to braille display
149155
UniversalSpeechWrapper.Stop(); // Stop speech
150156
UniversalSpeechWrapper.IsScreenReaderActive(); // Check if screen reader is running
151157
```
152158

153159
### TextCleaner
154160

161+
Basic usage:
162+
155163
```csharp
156164
string clean = TextCleaner.Clean("<color=#ff0000>Red text</color>");
157165
// Result: "Red text"
@@ -160,6 +168,22 @@ string combined = TextCleaner.CombineLines("Line 1", "<b>Line 2</b>", "Line 3");
160168
// Result: "Line 1 Line 2 Line 3"
161169
```
162170

171+
Custom text replacements (applied after tag removal):
172+
173+
```csharp
174+
// Simple string replacement
175+
TextCleaner.AddReplacement("", "heart");
176+
TextCleaner.AddReplacement("", "arrow");
177+
178+
// Regex replacement
179+
TextCleaner.AddRegexReplacement(@"\[(\d+)\]", "footnote $1");
180+
181+
// Clear custom replacements
182+
TextCleaner.ClearReplacements(); // Clear string replacements only
183+
TextCleaner.ClearRegexReplacements(); // Clear regex replacements only
184+
TextCleaner.ClearAllCustomReplacements(); // Clear all
185+
```
186+
163187
### AccessibilityLog
164188

165189
```csharp
@@ -183,6 +207,63 @@ The library includes `Net35Extensions` with polyfills for methods not available
183207

184208
- `Net35Extensions.IsNullOrWhiteSpace(string)` - Use instead of `string.IsNullOrWhiteSpace`
185209

210+
## Extensibility
211+
212+
### Custom Text Types
213+
214+
Define custom text types for game-specific content:
215+
216+
```csharp
217+
public static class MyTextTypes
218+
{
219+
public const int Tutorial = TextType.CustomBase + 1; // 101
220+
public const int Combat = TextType.CustomBase + 2; // 102
221+
public const int Inventory = TextType.CustomBase + 3; // 103
222+
}
223+
224+
// Register names for logging
225+
SpeechManager.TextTypeNames = new Dictionary<int, string>
226+
{
227+
{ TextType.Dialogue, "Dialogue" },
228+
{ TextType.Narrator, "Narrator" },
229+
{ MyTextTypes.Tutorial, "Tutorial" },
230+
{ MyTextTypes.Combat, "Combat" },
231+
};
232+
233+
// Use custom types
234+
SpeechManager.Announce("Press A to jump", MyTextTypes.Tutorial);
235+
```
236+
237+
### Custom Text Formatting
238+
239+
Override how text is formatted before output:
240+
241+
```csharp
242+
SpeechManager.FormatTextOverride = (speaker, text, textType) =>
243+
{
244+
// Custom formatting logic
245+
if (textType == MyTextTypes.Combat)
246+
return $"Combat: {text}";
247+
if (!string.IsNullOrEmpty(speaker))
248+
return $"{speaker} says: {text}";
249+
return text;
250+
};
251+
```
252+
253+
### Custom Repeat Storage
254+
255+
Control which text types are stored for repeat functionality:
256+
257+
```csharp
258+
SpeechManager.ShouldStoreForRepeatPredicate = (textType) =>
259+
{
260+
// Store dialogue, narrator, and tutorial text for repeat
261+
return textType == TextType.Dialogue
262+
|| textType == TextType.Narrator
263+
|| textType == MyTextTypes.Tutorial;
264+
};
265+
```
266+
186267
## UniversalSpeech Setup
187268

188269
1. Download UniversalSpeech from [GitHub](https://github.com/qtnc/UniversalSpeech)

SpeechManager.cs

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,13 @@ public static class SpeechManager
2828
/// </summary>
2929
public static bool EnableLogging { get; set; } = true;
3030

31+
/// <summary>
32+
/// Whether to send output to braille displays. Default is true.
33+
/// When enabled, all speech output is also sent to the braille display
34+
/// via the screen reader's braille support.
35+
/// </summary>
36+
public static bool EnableBraille { get; set; } = true;
37+
3138
/// <summary>
3239
/// Initialize the speech system.
3340
/// </summary>
@@ -74,6 +81,12 @@ public static void Output(string speaker, string text, int textType = TextType.D
7481
// Output via speech
7582
UniversalSpeechWrapper.Speak(formattedText);
7683

84+
// Output via braille if enabled
85+
if (EnableBraille)
86+
{
87+
UniversalSpeechWrapper.DisplayBraille(formattedText);
88+
}
89+
7790
if (EnableLogging)
7891
{
7992
string typeName =
@@ -104,6 +117,11 @@ public static void RepeatLast()
104117
string formattedText = FormatText(_currentSpeaker, _currentText, _currentType);
105118
UniversalSpeechWrapper.Speak(formattedText);
106119

120+
if (EnableBraille)
121+
{
122+
UniversalSpeechWrapper.DisplayBraille(formattedText);
123+
}
124+
107125
if (EnableLogging)
108126
{
109127
AccessibilityLog.Msg($"Repeating: '{formattedText}'");

UniversalSpeechWrapper.cs

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,15 @@ int interrupt
2828
[DllImport(DLL_NAME, CallingConvention = CallingConvention.Cdecl)]
2929
private static extern int speechStop();
3030

31+
[DllImport(
32+
DLL_NAME,
33+
CallingConvention = CallingConvention.Cdecl,
34+
CharSet = CharSet.Unicode
35+
)]
36+
private static extern int brailleDisplay(
37+
[MarshalAs(UnmanagedType.LPWStr)] string str
38+
);
39+
3140
[DllImport(DLL_NAME, CallingConvention = CallingConvention.Cdecl)]
3241
private static extern int speechSetValue(int what, int value);
3342

@@ -104,6 +113,29 @@ public static void Speak(string text, bool interrupt = false)
104113
}
105114
}
106115

116+
/// <summary>
117+
/// Display the given text on a braille display via the current screen reader.
118+
/// </summary>
119+
/// <param name="text">The text to display on braille</param>
120+
public static void DisplayBraille(string text)
121+
{
122+
if (!_initialized || !_dllAvailable || Net35Extensions.IsNullOrWhiteSpace(text))
123+
return;
124+
125+
try
126+
{
127+
brailleDisplay(text);
128+
}
129+
catch (DllNotFoundException)
130+
{
131+
_dllAvailable = false;
132+
}
133+
catch (Exception ex)
134+
{
135+
AccessibilityLog.Error($"Braille display error: {ex.Message}");
136+
}
137+
}
138+
107139
/// <summary>
108140
/// Stop any currently playing speech.
109141
/// </summary>

0 commit comments

Comments
 (0)