From ccfd3ec35a535c1341d515b14153d7c266cf6b00 Mon Sep 17 00:00:00 2001 From: Anton Shalin Date: Wed, 21 Jan 2026 20:47:26 +0800 Subject: [PATCH] feat: support multiple root folders selection --- .gitignore | 2 + README.md | 169 ++-- README_ru.md | 153 ++++ app.go | 826 ++++++++++++++---- auto_context_service.go | 201 ++++- design/architecture.md | 374 ++++---- design/architecture_ru.md | 161 ++++ frontend/src/components/CentralPanel.vue | 7 +- frontend/src/components/FileTree.vue | 18 +- frontend/src/components/LeftSidebar.vue | 31 +- frontend/src/components/MainLayout.vue | 424 ++++++--- .../src/components/common/LargeTextViewer.vue | 127 ++- .../components/steps/Step1PrepareContext.vue | 118 ++- .../components/steps/Step2ComposePrompt.vue | 7 +- frontend/wailsjs/go/main/App.d.ts | 16 +- frontend/wailsjs/go/main/App.js | 16 + frontend/wailsjs/go/models.ts | 22 + frontend/wailsjs/runtime/runtime.js | 4 - go.mod | 14 +- go.sum | 32 +- 20 files changed, 2017 insertions(+), 705 deletions(-) create mode 100644 README_ru.md create mode 100644 design/architecture_ru.md diff --git a/.gitignore b/.gitignore index be0c599..97d5387 100644 --- a/.gitignore +++ b/.gitignore @@ -62,6 +62,8 @@ logs/ *.bak *.tmp Show-DirectoryTree.ps1 +shotgun.desktop +run-shotgun.sh # If you have specific large files or data that shouldn't be in git # data/ diff --git a/README.md b/README.md index af49d98..e079658 100644 --- a/README.md +++ b/README.md @@ -2,82 +2,77 @@ ![Shotgun App Banner](https://github.com/user-attachments/assets/6dd15389-4ad9-493a-a0e7-9813eb143e38) -**Tired of Cursor cutting off context, missing your files, and spitting out empty responses?** - -**Shotgun** is the bridge between your local codebase and the world's most powerful LLMs. -It doesn't just copy files; it **intelligently packages your project context** and can **execute prompts directly** against OpenAI (GPT-4o/GPT-5), Google Gemini, or OpenRouter. - -> **Stop copy-pasting 50 files manually.** -> 1. Select your repo. -> 2. Let AI pick the relevant files (Auto-Context). -> 3. Blast the payload directly to the model or copy it for use in Cursor/Windsurf. +Shotgun is a desktop app for preparing project context and working with LLMs. +It collects selected files into a structured context, can auto‑select relevant files, and can send prompts directly to OpenAI, Gemini, or OpenRouter. --- -## 1. What Shotgun Does -Shotgun is a desktop power-tool that **explodes your project into a structured payload** designed for AI reasoning. - -It has evolved from a simple "context dumper" into a full-fledged **LLM Client for Codebases**: -* **Smart Selection:** Uses AI ("Auto-Context") to analyze your task and automatically select only the relevant files from your tree. -* **Direct Execution:** Configurable API integration with **OpenAI**, **Gemini**, and **OpenRouter**. -* **Prompt Engineering:** Built-in templates for different roles (Developer, Architect, Bug Hunter). -* **History & Audit:** Keeps a full log of every prompt sent and response received. +## What Shotgun Does +Shotgun turns your repository into an LLM‑friendly context: +- builds a file tree and collects file contents; +- respects `.gitignore` and custom rules; +- shows size and token counts per file/folder; +- outputs XML‑like blocks `...`. --- -## 2. Key Features +## Key Features -### 🧠 AI-Powered Context -* **Auto-Context:** Don't know which files are needed for a bug fix? Type your task, and Shotgun uses an LLM to scan your tree and select the relevant files for you. -* **Repo Scan:** supplement context retrieval with a `shotgun_reposcan.md` summary of your architecture to give the LLM high-level awareness before diving into code. +### 🧠 Auto Context +- **Auto‑Context**: the LLM selects relevant files based on your task. +- **Repo Scan**: optional `shotgun_reposcan.md` for high‑level architecture context. -### ⚡ Workflow Speed -* **Fast Tree Scan:** Go + Wails backend scans thousands of files in milliseconds. -* **Interactive Tree:** Manually toggle files/folders or use `.gitignore` and custom rule sets to filter noise. -* **One-Click Blast:** Generate a massive context payload instantly. +### ⚡ Large‑project performance +- **Multiple project roots**: add several root folders in one workspace. +- **Project tree** with manual selection. +- **Symlink traversal** with cycle protection. +- **Offline token counting** (`o200k_base`) without API calls. +- **Context size limit** via `settings.json` (`maxContextBytes`, default 50MB). +- **Auto context generation** with debounce + queue (no constant restarts). -### 🔌 Direct Integrations -* **OpenAI:** Support for GPT-4o and experimental support for **GPT-5** family models. -* **Google Gemini:** Native integration for Gemini 2.5/3 Pro & Flash. -* **OpenRouter:** Access hundreds of LLM's via a unified API. +### 🔌 Integrations +Supported providers: +- OpenAI +- Google Gemini +- OpenRouter -### 🛠 Developer Experience -* **Prompt Templates:** Switch modes easily (e.g., "Find Bug" vs "Refactor" vs "Write Docs"). -* **History Tracking:** Never lose a generated patch. Browse past prompts, responses, and raw API payloads. -* **Privacy Focused:** Your code goes only to the API provider you choose. No intermediate servers. +You can pick a model and run requests directly from the app. --- -## 3. The Workflow - -Shotgun guides you through a 3-step process: +## Workflow ### Step 1: Prepare Context -* **Select Project:** Open your local repository. -* **Filter:** Use the checkbox tree, `.gitignore`, or the **Auto-Context** button to define the scope. -* **Repo Scan:** Edit or load the high-level repository summary for better AI grounding. -* **Result:** A structured XML-like dump of your selected codebase. +1. Add one or more project roots. +2. Select files manually or use Auto‑Context. +3. Context is generated automatically with progress. +4. **Generated Project Context** shows an exact token count. ### Step 2: Compose & Execute -* **Define Task:** Describe what you need (e.g., "Refactor the auth middleware to use JWT"). -* **Select Template:** Choose a persona (Dev, Architect, QA). -* **Execute:** Click **"Execute Prompt"** to send it to the configured LLM API immediately, OR copy the full payload to your clipboard for use in external tools like ChatGPT or Cursor. +1. Describe your task. +2. Choose a prompt template. +3. Execute the request or copy the prompt. -### Step 3: History & Apply -* **Review:** View the AI's response alongside your original prompt. -* **Diffs:** The AI output is optimized for `diff` generation. -* **Audit:** Inspect raw API calls for debugging or token usage analysis. +### Step 3: History +History of requests, responses, and call parameters. --- -## 4. Installation +## UI +- **Resizable sidebar** (drag to resize). +- **Resizable bottom console**. -### Prerequisites -* **Go ≥ 1.20** -* **Node.js LTS** -* **Wails CLI:** `go install github.com/wailsapp/wails/v2/cmd/wails@latest` +--- -### Clone & Build +## Installation & Run + +### Requirements +- **Go ≥ 1.20** +- **Node.js LTS** +- **Wails CLI** + `go install github.com/wailsapp/wails/v2/cmd/wails@latest` + +### Build & Run ```bash git clone https://github.com/glebkudr/shotgun_code cd shotgun_code @@ -87,32 +82,42 @@ cd frontend npm install cd .. -# Run in Development Mode (Hot Reload) +# Dev mode (hot reload) wails dev -# Build Production Binary +# Production build wails build ``` -*Binaries will be located in `build/bin/`.* +Binaries are placed in `build/bin/`. --- -## 5. Configuration +## Configuration -### LLM Setup -Click the **Settings** (gear icon) in the app to configure providers: -1. **Provider:** Select OpenAI, Gemini, or OpenRouter. -2. **API Key:** Paste your key (stored locally). -3. **Model:** Select your preferred model (e.g., `gpt-4o`, `gemini-2.5-pro`, `claude-3.5-sonnet`). +### Settings file +Uses the XDG path `shotgun-code/settings.json`. +On Linux this is typically: +``` +~/.config/shotgun-code/settings.json +``` -### Custom Rules -You can define global excludes (like `node_modules`, `dist`, `.git`) and custom prompt instructions that are appended to every request. +Example: +```json +{ + "maxContextBytes": 50000000, + "customIgnoreRules": "...", + "customPromptRules": "..." +} +``` ---- +### Key options +- **`maxContextBytes`** — context size limit (default 50MB). +- **`customIgnoreRules`** — file exclusion rules. +- **`customPromptRules`** — global prompt rules. -## 6. Output Format +--- -Shotgun generates context optimized for LLM parsing: +## Output Format ```xml @@ -127,30 +132,22 @@ package main ``` -This format allows models to understand file boundaries perfectly, enabling accurate multi-file refactoring suggestions. - --- -## 7. ⚖️ License & Usage - -My name is Gleb Curly, and I am an indie developer making software for a living. - -Shotgun is developed and maintained by **Curly's Technology Tmi**. - -This project uses a **Community License** model: - -### 1. Free for Small Teams & Non-Commercial Use -You can use Shotgun for free (including modification and internal use) if: -- Your company/team generates **less than $1M USD** in annual revenue. -- You do **not** use the code to build a competing public product. +## License -### 2. Commercial License (Enterprise) -If your annual revenue exceeds **$1M USD**, you are required to purchase a commercial license with a pretty reasonable price. +Shotgun is developed and maintained by **Curly's Technology Tmi**. -Please contact me at **glebkudr@gmail.com** for pricing. +### 1. Free for small teams and non‑commercial use +You may use Shotgun for free if: +- your annual revenue is **below $1M**; +- you do **not** use the code to build a competing product. -See [LICENSE.md](LICENSE.md) for the full legal text. +### 2. Commercial license +If your revenue exceeds $1M, a commercial license is required. +Contact: **glebkudr@gmail.com** +See `LICENSE.md` for details. --- -*Shotgun – Load, Aim, Blast your code into the future.* +*Shotgun — Load, Aim, Blast your code into the future.* diff --git a/README_ru.md b/README_ru.md new file mode 100644 index 0000000..244c923 --- /dev/null +++ b/README_ru.md @@ -0,0 +1,153 @@ +# Shotgun App + +![Shotgun App Banner](https://github.com/user-attachments/assets/6dd15389-4ad9-493a-a0e7-9813eb143e38) + +Shotgun — десктопное приложение для подготовки контекста проекта и работы с LLM. +Оно собирает выбранные файлы в структурированный контекст, умеет автоматически выбирать релевантные файлы, а также отправлять готовый промпт прямо в OpenAI, Gemini или OpenRouter. + +--- + +## Что делает Shotgun +Shotgun превращает ваш репозиторий в удобный для LLM контекст: +- собирает дерево файлов и их содержимое; +- уважает `.gitignore` и кастомные правила; +- показывает размер и токены по каждому файлу/папке; +- формирует контекст в XML‑подобном формате `...`. + +--- + +## Основные возможности + +### 🧠 Автоматический контекст +- **Auto‑Context**: LLM сам выбирает файлы по описанию задачи. +- **Repo Scan**: подключаемый `shotgun_reposcan.md` с архитектурным обзором. + +### ⚡ Быстрая работа с большими проектами +- **Несколько корневых папок**: можно добавить несколько корней проекта в один workspace. +- **Дерево проектов** с ручным выбором файлов и папок. +- **Симлинки** обходятся рекурсивно с защитой от циклов. +- **Офлайн‑подсчёт токенов** (кодировка `o200k_base`) без обращения к API. +- **Ограничение размера контекста** через `settings.json` (`maxContextBytes`, дефолт 50MB). +- **Авто‑генерация контекста** с дебаунсом и очередью (без постоянных перезапусков). + +### 🔌 Интеграции +Поддерживаются провайдеры: +- OpenAI +- Google Gemini +- OpenRouter + +Можно выбрать модель и выполнять запрос прямо из приложения. + +--- + +## Процесс работы + +### Шаг 1: Prepare Context +1. Добавьте один или несколько корней проекта. +2. Выберите файлы вручную или через Auto‑Context. +3. Контекст генерируется автоматически с прогресс‑баром. +4. В блоке **Generated Project Context** показывается точное число токенов. + +### Шаг 2: Compose & Execute +1. Опишите задачу. +2. Выберите шаблон промпта. +3. Выполните запрос или скопируйте промпт. + +### Шаг 3: History +История запросов, ответов и параметров вызова. + +--- + +## Интерфейс +- **Сайдбар** регулируется по ширине перетаскиванием. +- **Нижняя консоль** регулируется по высоте. + +--- + +## Установка и запуск + +### Требования +- **Go ≥ 1.20** +- **Node.js LTS** +- **Wails CLI** + `go install github.com/wailsapp/wails/v2/cmd/wails@latest` + +### Сборка и запуск +```bash +git clone https://github.com/glebkudr/shotgun_code +cd shotgun_code + +# Установить зависимости фронтенда +cd frontend +npm install +cd .. + +# Режим разработки (hot reload) +wails dev + +# Продакшен‑сборка +wails build +``` +Бинарники появляются в `build/bin/`. + +--- + +## Конфигурация + +### Файл настроек +Используется XDG‑путь `shotgun-code/settings.json`. +На Linux это обычно: +``` +~/.config/shotgun-code/settings.json +``` + +Пример: +```json +{ + "maxContextBytes": 50000000, + "customIgnoreRules": "...", + "customPromptRules": "..." +} +``` + +### Важные параметры +- **`maxContextBytes`** — лимит размера контекста (по умолчанию 50MB). +- **`customIgnoreRules`** — правила исключения файлов. +- **`customPromptRules`** — глобальные правила для промптов. + +--- + +## Формат вывода + +```xml + +package main +... + + + + + +``` + +--- + +## Лицензия + +Shotgun разрабатывается и поддерживается **Curly's Technology Tmi**. + +### 1. Бесплатно для небольших команд и некоммерческого использования +Можно использовать бесплатно, если: +- годовой оборот компании **меньше $1M**; +- вы не используете код для создания конкурирующего продукта. + +### 2. Коммерческая лицензия +Если оборот выше $1M — требуется коммерческая лицензия. +Контакт: **glebkudr@gmail.com** +Подробности в `LICENSE.md`. + +--- + +*Shotgun — Load, Aim, Blast your code into the future.* diff --git a/app.go b/app.go index 2ce808e..478964e 100644 --- a/app.go +++ b/app.go @@ -2,6 +2,7 @@ package main import ( "context" + "crypto/sha1" _ "embed" "encoding/json" "errors" @@ -17,18 +18,20 @@ import ( "github.com/adrg/xdg" "github.com/fsnotify/fsnotify" gitignore "github.com/sabhiram/go-gitignore" + "github.com/tmc/tokencount/openaitokenizer" "github.com/wailsapp/wails/v2/pkg/runtime" "shotgun_code/internal/labgradient" ) -const maxOutputSizeBytes = 10_000_000 // 10MB +const defaultMaxOutputSizeBytes = 50_000_000 // 50MB var ErrContextTooLong = errors.New("context is too long") //go:embed ignore.glob var defaultCustomIgnoreRulesContent string -const defaultCustomPromptRulesContent = "no additional rules" +const defaultCustomPromptRulesContent = "пиши по-русски" +const legacyDefaultCustomPromptRulesContent = "no additional rules" const ( LLMProviderOpenAI = "openai" @@ -49,6 +52,8 @@ type AppSettings struct { CustomIgnoreRules string `json:"customIgnoreRules"` CustomPromptRules string `json:"customPromptRules"` LLMSettings LLMSettings `json:"llmSettings"` + ProjectRoots []string `json:"projectRoots"` + MaxContextBytes int64 `json:"maxContextBytes"` } type App struct { @@ -60,11 +65,13 @@ type App struct { configPath string useGitignore bool useCustomIgnore bool - projectGitignore *gitignore.GitIgnore // Compiled .gitignore for the current project + projectGitignores map[string]*gitignore.GitIgnore // Compiled .gitignore per project root autoContextService *AutoContextService historyManager *HistoryManager llmCache cachedProvider autoContextButtonTexture string + tokenEncoder *openaitokenizer.Encoder + tokenEncoderErr error } func NewApp() *App { @@ -100,6 +107,7 @@ func (a *App) startup(ctx context.Context) { } a.initAutoContextButtonTexture() + a.initTokenEncoder() } func (a *App) initAutoContextButtonTexture() { @@ -122,6 +130,16 @@ func (a *App) initAutoContextButtonTexture() { runtime.LogDebug(a.ctx, "auto-context LAB texture generated successfully") } +func (a *App) initTokenEncoder() { + enc, err := openaitokenizer.NewEncoder("o200k_base") + if err != nil { + a.tokenEncoderErr = err + runtime.LogErrorf(a.ctx, "failed to initialize tokenizer (o200k_base): %v", err) + return + } + a.tokenEncoder = enc +} + // GetAutoContextButtonTexture returns a data URL with the LAB gradient texture for the Auto context button. func (a *App) GetAutoContextButtonTexture() string { return a.autoContextButtonTexture @@ -131,62 +149,345 @@ type FileNode struct { Name string `json:"name"` Path string `json:"path"` // Full path RelPath string `json:"relPath"` // Path relative to selected root + RootPath string `json:"rootPath"` + RootLabel string `json:"rootLabel"` + SizeBytes int64 `json:"sizeBytes"` + TokensApprox int64 `json:"tokensApprox"` IsDir bool `json:"isDir"` Children []*FileNode `json:"children,omitempty"` IsGitignored bool `json:"isGitignored"` // True if path matches a .gitignore rule IsCustomIgnored bool `json:"isCustomIgnored"` // True if path matches a ignore.glob rule } +type ProjectRoot struct { + Path string `json:"path"` + Label string `json:"label"` +} + // SelectDirectory opens a dialog to select a directory and returns the chosen path func (a *App) SelectDirectory() (string, error) { return runtime.OpenDirectoryDialog(a.ctx, runtime.OpenDialogOptions{}) } -// ListFiles lists files and folders in a directory, parsing .gitignore if present -func (a *App) ListFiles(dirPath string) ([]*FileNode, error) { - runtime.LogDebugf(a.ctx, "ListFiles called for directory: %s", dirPath) - - a.projectGitignore = nil // Reset for the new directory - var gitIgn *gitignore.GitIgnore // For .gitignore in the project directory - gitignorePath := filepath.Join(dirPath, ".gitignore") - runtime.LogDebugf(a.ctx, "Attempting to find .gitignore at: %s", gitignorePath) - if _, err := os.Stat(gitignorePath); err == nil { - runtime.LogDebugf(a.ctx, ".gitignore found at: %s", gitignorePath) - gitIgn, err = gitignore.CompileIgnoreFile(gitignorePath) +func normalizeAndValidateRoots(input []string, requireExists bool) ([]string, error) { + if len(input) == 0 { + return []string{}, nil + } + seen := make(map[string]bool, len(input)) + normalized := make([]string, 0, len(input)) + for _, raw := range input { + trimmed := strings.TrimSpace(raw) + if trimmed == "" { + return nil, errors.New("project root is required") + } + absPath, err := filepath.Abs(trimmed) if err != nil { - runtime.LogWarningf(a.ctx, "Error compiling .gitignore file at %s: %v", gitignorePath, err) - gitIgn = nil - } else { - a.projectGitignore = gitIgn // Store the compiled project-specific gitignore - runtime.LogDebug(a.ctx, ".gitignore compiled successfully.") + return nil, fmt.Errorf("failed to resolve project root %q: %w", trimmed, err) } - } else { - runtime.LogDebugf(a.ctx, ".gitignore not found at %s (os.Stat error: %v)", gitignorePath, err) - gitIgn = nil + absPath = filepath.Clean(absPath) + if seen[absPath] { + return nil, fmt.Errorf("duplicate project root: %s", absPath) + } + if requireExists { + info, err := os.Stat(absPath) + if err != nil { + return nil, fmt.Errorf("project root not accessible: %w", err) + } + if !info.IsDir() { + return nil, fmt.Errorf("project root is not a directory: %s", absPath) + } + } + seen[absPath] = true + normalized = append(normalized, absPath) + } + return normalized, nil +} + +func computeRootLabels(rootDirs []string) map[string]string { + labels := make(map[string]string, len(rootDirs)) + baseCounts := make(map[string]int, len(rootDirs)) + for _, rootDir := range rootDirs { + baseCounts[filepath.Base(rootDir)]++ + } + for _, rootDir := range rootDirs { + base := filepath.Base(rootDir) + if baseCounts[base] == 1 { + labels[rootDir] = base + continue + } + sum := sha1.Sum([]byte(rootDir)) + hash := fmt.Sprintf("%x", sum) + labels[rootDir] = fmt.Sprintf("%s~%s", base, hash[:6]) + } + return labels +} + +func buildProjectRootList(rootDirs []string) []ProjectRoot { + labels := computeRootLabels(rootDirs) + result := make([]ProjectRoot, 0, len(rootDirs)) + for _, rootDir := range rootDirs { + result = append(result, ProjectRoot{ + Path: rootDir, + Label: labels[rootDir], + }) + } + return result +} + +func sumChildSizes(children []*FileNode) int64 { + var total int64 + for _, child := range children { + if child == nil { + continue + } + total += child.SizeBytes + } + return total +} +func sumChildTokens(children []*FileNode) int64 { + var total int64 + for _, child := range children { + if child == nil { + continue + } + total += child.TokensApprox + } + return total +} + +func resolveDirPath(path string) (string, error) { + resolved, err := filepath.EvalSymlinks(path) + if err != nil { + return "", err + } + abs, err := filepath.Abs(resolved) + if err != nil { + return "", err + } + return filepath.Clean(abs), nil +} +func (a *App) getTokenEncoder() (*openaitokenizer.Encoder, error) { + if a.tokenEncoderErr != nil { + return nil, fmt.Errorf("tokenizer initialization failed: %w", a.tokenEncoderErr) + } + if a.tokenEncoder == nil { + return nil, errors.New("tokenizer is not initialized") + } + return a.tokenEncoder, nil +} + +func (a *App) CountTokens(text string) (int64, error) { + encoder, err := a.getTokenEncoder() + if err != nil { + return 0, err + } + if text == "" { + return 0, nil + } + return int64(encoder.Count(text)), nil +} + +func (a *App) contextLimitBytes() int64 { + if a.settings.MaxContextBytes > 0 { + return a.settings.MaxContextBytes + } + return defaultMaxOutputSizeBytes +} + +func computeDirStats(encoder *openaitokenizer.Encoder, rootPath string, visited map[string]bool) (int64, int64, error) { + if encoder == nil { + return 0, 0, errors.New("tokenizer is not initialized") + } + if visited == nil { + return 0, 0, errors.New("visited map is required") } + rootResolved, err := resolveDirPath(rootPath) + if err != nil { + return 0, 0, err + } + if visited[rootResolved] { + return 0, 0, nil + } + visited[rootResolved] = true + defer delete(visited, rootResolved) - // App-level custom ignore patterns are in a.currentCustomIgnorePatterns + var totalBytes int64 + var totalTokens int64 + entries, err := os.ReadDir(rootPath) + if err != nil { + return 0, 0, err + } + for _, entry := range entries { + entryPath := filepath.Join(rootPath, entry.Name()) + entryIsDir := entry.IsDir() + isSymlink := entry.Type()&os.ModeSymlink != 0 + if isSymlink && !entryIsDir { + info, statErr := os.Stat(entryPath) + if statErr != nil { + return 0, 0, statErr + } + if info.IsDir() { + entryIsDir = true + } + } + if entryIsDir { + resolved, resolveErr := resolveDirPath(entryPath) + if resolveErr != nil { + return 0, 0, resolveErr + } + if visited[resolved] { + continue + } + subBytes, subTokens, subErr := computeDirStats(encoder, entryPath, visited) + if subErr != nil { + return 0, 0, subErr + } + totalBytes += subBytes + totalTokens += subTokens + continue + } + content, readErr := os.ReadFile(entryPath) + if readErr != nil { + return 0, 0, readErr + } + totalBytes += int64(len(content)) + totalTokens += int64(encoder.Count(string(content))) + } + return totalBytes, totalTokens, nil +} - rootNode := &FileNode{ - Name: filepath.Base(dirPath), - Path: dirPath, - RelPath: ".", - IsDir: true, - IsGitignored: false, // Root itself is not gitignored by default - // IsCustomIgnored for root is also false by default, specific patterns would be needed - IsCustomIgnored: a.currentCustomIgnorePatterns != nil && a.currentCustomIgnorePatterns.MatchesPath("."), +func (a *App) GetProjectRoots() ([]ProjectRoot, error) { + roots, err := normalizeAndValidateRoots(a.settings.ProjectRoots, false) + if err != nil { + return nil, err } + return buildProjectRootList(roots), nil +} - children, err := buildTreeRecursive(context.TODO(), dirPath, dirPath, gitIgn, a.currentCustomIgnorePatterns, 0) +func (a *App) AddProjectRoot(path string) ([]ProjectRoot, error) { + normalized, err := normalizeAndValidateRoots([]string{path}, true) + if err != nil { + return nil, err + } + newRoot := normalized[0] + existing, err := normalizeAndValidateRoots(a.settings.ProjectRoots, false) if err != nil { - return []*FileNode{rootNode}, fmt.Errorf("error building children tree for %s: %w", dirPath, err) + return nil, err + } + for _, root := range existing { + if root == newRoot { + return nil, fmt.Errorf("project root already added: %s", newRoot) + } + } + updated := append(existing, newRoot) + a.settings.ProjectRoots = updated + if err := a.saveSettings(); err != nil { + return nil, err + } + return buildProjectRootList(updated), nil +} + +func (a *App) RemoveProjectRoot(path string) ([]ProjectRoot, error) { + normalized, err := normalizeAndValidateRoots([]string{path}, false) + if err != nil { + return nil, err + } + target := normalized[0] + existing, err := normalizeAndValidateRoots(a.settings.ProjectRoots, false) + if err != nil { + return nil, err + } + updated := make([]string, 0, len(existing)) + removed := false + for _, root := range existing { + if root == target { + removed = true + continue + } + updated = append(updated, root) + } + if !removed { + return nil, fmt.Errorf("project root not found: %s", target) + } + a.settings.ProjectRoots = updated + if err := a.saveSettings(); err != nil { + return nil, err + } + return buildProjectRootList(updated), nil +} + +// ListFiles lists files and folders in directories, parsing .gitignore if present. +func (a *App) ListFiles(rootDirs []string) ([]*FileNode, error) { + normalizedRoots, err := normalizeAndValidateRoots(rootDirs, true) + if err != nil { + return nil, err + } + if len(normalizedRoots) == 0 { + return []*FileNode{}, nil + } + tokenEncoder, err := a.getTokenEncoder() + if err != nil { + return nil, err + } + + rootLabels := computeRootLabels(normalizedRoots) + a.projectGitignores = make(map[string]*gitignore.GitIgnore, len(normalizedRoots)) + + rootNodes := make([]*FileNode, 0, len(normalizedRoots)) + for _, rootDir := range normalizedRoots { + runtime.LogDebugf(a.ctx, "ListFiles called for directory: %s", rootDir) + + var gitIgn *gitignore.GitIgnore + gitignorePath := filepath.Join(rootDir, ".gitignore") + runtime.LogDebugf(a.ctx, "Attempting to find .gitignore at: %s", gitignorePath) + if _, statErr := os.Stat(gitignorePath); statErr == nil { + runtime.LogDebugf(a.ctx, ".gitignore found at: %s", gitignorePath) + compiled, compileErr := gitignore.CompileIgnoreFile(gitignorePath) + if compileErr != nil { + runtime.LogWarningf(a.ctx, "Error compiling .gitignore file at %s: %v", gitignorePath, compileErr) + gitIgn = nil + } else { + gitIgn = compiled + runtime.LogDebug(a.ctx, ".gitignore compiled successfully.") + } + } else { + runtime.LogDebugf(a.ctx, ".gitignore not found at %s (os.Stat error: %v)", gitignorePath, statErr) + gitIgn = nil + } + a.projectGitignores[rootDir] = gitIgn + + rootLabel := rootLabels[rootDir] + rootNode := &FileNode{ + Name: rootLabel, + Path: rootDir, + RelPath: ".", + RootPath: rootDir, + RootLabel: rootLabel, + IsDir: true, + IsGitignored: false, // Root itself is not gitignored by default + IsCustomIgnored: a.currentCustomIgnorePatterns != nil && a.currentCustomIgnorePatterns.MatchesPath("."), + } + + visited := make(map[string]bool) + resolvedRoot, resolveErr := resolveDirPath(rootDir) + if resolveErr != nil { + return nil, fmt.Errorf("failed to resolve root directory %s: %w", rootDir, resolveErr) + } + visited[resolvedRoot] = true + children, err := buildTreeRecursive(context.TODO(), a.ctx, rootDir, rootDir, rootLabel, gitIgn, a.currentCustomIgnorePatterns, tokenEncoder, visited, 0) + if err != nil { + return nil, fmt.Errorf("error building children tree for %s: %w", rootDir, err) + } + rootNode.Children = children + rootNode.SizeBytes = sumChildSizes(children) + rootNode.TokensApprox = sumChildTokens(children) + rootNodes = append(rootNodes, rootNode) } - rootNode.Children = children - return []*FileNode{rootNode}, nil + return rootNodes, nil } -func buildTreeRecursive(ctx context.Context, currentPath, rootPath string, gitIgn *gitignore.GitIgnore, customIgn *gitignore.GitIgnore, depth int) ([]*FileNode, error) { +func buildTreeRecursive(ctx context.Context, logCtx context.Context, currentPath, rootPath, rootLabel string, gitIgn *gitignore.GitIgnore, customIgn *gitignore.GitIgnore, tokenEncoder *openaitokenizer.Encoder, visited map[string]bool, depth int) ([]*FileNode, error) { select { case <-ctx.Done(): return nil, ctx.Err() @@ -207,8 +508,27 @@ func buildTreeRecursive(ctx context.Context, currentPath, rootPath string, gitIg isGitignored := false isCustomIgnored := false + entryIsDir := entry.IsDir() + isSymlink := entry.Type()&os.ModeSymlink != 0 + if isSymlink && !entryIsDir { + info, statErr := os.Stat(nodePath) + if statErr != nil { + runtime.LogWarningf(logCtx, "Failed to stat symlink %s: %v", nodePath, statErr) + } else if info.IsDir() { + entryIsDir = true + } + } + var resolvedDirPath string + if entryIsDir { + resolved, resolveErr := resolveDirPath(nodePath) + if resolveErr != nil { + return nil, fmt.Errorf("failed to resolve directory %s: %w", nodePath, resolveErr) + } + resolvedDirPath = resolved + } + pathToMatch := relPath - if entry.IsDir() { + if entryIsDir { if !strings.HasSuffix(pathToMatch, string(os.PathSeparator)) { pathToMatch += string(os.PathSeparator) } @@ -222,34 +542,66 @@ func buildTreeRecursive(ctx context.Context, currentPath, rootPath string, gitIg } if depth < 2 || strings.Contains(relPath, "node_modules") || strings.HasSuffix(relPath, ".log") { - fmt.Printf("Checking path: '%s' (original relPath: '%s'), IsDir: %v, Gitignored: %v, CustomIgnored: %v\n", pathToMatch, relPath, entry.IsDir(), isGitignored, isCustomIgnored) + fmt.Printf("Checking path: '%s' (original relPath: '%s'), IsDir: %v, Gitignored: %v, CustomIgnored: %v\n", pathToMatch, relPath, entryIsDir, isGitignored, isCustomIgnored) + } + var sizeBytes int64 + var tokenCount int64 + if entryIsDir { + if isCustomIgnored { + dirSize, dirTokens, statsErr := computeDirStats(tokenEncoder, nodePath, visited) + if statsErr != nil { + runtime.LogErrorf(logCtx, "Failed to compute stats for directory %s: %v", nodePath, statsErr) + return nil, statsErr + } + sizeBytes = dirSize + tokenCount = dirTokens + } + } else { + content, readErr := os.ReadFile(nodePath) + if readErr != nil { + runtime.LogErrorf(logCtx, "Failed to read file %s: %v", nodePath, readErr) + return nil, readErr + } + sizeBytes = int64(len(content)) + tokenCount = int64(tokenEncoder.Count(string(content))) } node := &FileNode{ Name: entry.Name(), Path: nodePath, RelPath: relPath, - IsDir: entry.IsDir(), + RootPath: rootPath, + RootLabel: rootLabel, + SizeBytes: sizeBytes, + TokensApprox: tokenCount, + IsDir: entryIsDir, IsGitignored: isGitignored, IsCustomIgnored: isCustomIgnored, } - - if entry.IsDir() { + if entryIsDir { // If it's a directory, recursively call buildTree. // Recursion stops only for folders ignored by custom rules. // For folders matched by .gitignore, recursion continues so the UI can show their contents // and allow selective inclusion of files inside gitignored folders. if !isCustomIgnored { - children, err := buildTreeRecursive(ctx, nodePath, rootPath, gitIgn, customIgn, depth+1) - if err != nil { - if errors.Is(err, context.Canceled) { - return nil, err // Propagate cancellation - } - // runtime.LogWarnf(ctx, "Error building subtree for %s: %v", nodePath, err) // Use ctx if available - runtime.LogWarningf(context.Background(), "Error building subtree for %s: %v", nodePath, err) // Fallback for now - // Decide: skip this dir or return error up. For now, skip with log. + if visited[resolvedDirPath] { + runtime.LogWarningf(logCtx, "Skipping directory %s (resolved to %s) to avoid symlink cycle", nodePath, resolvedDirPath) } else { - node.Children = children + visited[resolvedDirPath] = true + children, err := buildTreeRecursive(ctx, logCtx, nodePath, rootPath, rootLabel, gitIgn, customIgn, tokenEncoder, visited, depth+1) + delete(visited, resolvedDirPath) + if err != nil { + if errors.Is(err, context.Canceled) { + return nil, err // Propagate cancellation + } + // runtime.LogWarnf(ctx, "Error building subtree for %s: %v", nodePath, err) // Use ctx if available + runtime.LogWarningf(logCtx, "Error building subtree for %s: %v", nodePath, err) // Fallback for now + // Decide: skip this dir or return error up. For now, skip with log. + } else { + node.Children = children + node.SizeBytes = sumChildSizes(children) + node.TokensApprox = sumChildTokens(children) + } } } } @@ -283,7 +635,7 @@ func NewContextGenerator(app *App) *ContextGenerator { // RequestShotgunContextGeneration is called by the frontend to start/restart generation. // This method itself is not bound to Wails directly if it's part of App. // Instead, a wrapper method in App struct will be bound. -func (cg *ContextGenerator) requestShotgunContextGenerationInternal(rootDir string, excludedPaths []string) { +func (cg *ContextGenerator) requestShotgunContextGenerationInternal(rootDirs []string, excludedByRoot map[string][]string) { cg.mu.Lock() if cg.currentCancelFunc != nil { runtime.LogDebug(cg.app.ctx, "Cancelling previous context generation job.") @@ -294,7 +646,8 @@ func (cg *ContextGenerator) requestShotgunContextGenerationInternal(rootDir stri myToken := new(struct{}) // Create a unique token for this generation job cg.currentCancelFunc = cancel cg.currentCancelToken = myToken - runtime.LogInfof(cg.app.ctx, "Starting new shotgun context generation for: %s. Max size: %d bytes.", rootDir, maxOutputSizeBytes) + limit := cg.app.contextLimitBytes() + runtime.LogInfof(cg.app.ctx, "Starting new shotgun context generation for %d roots. Max size: %d bytes.", len(rootDirs), limit) cg.mu.Unlock() go func(tokenForThisJob interface{}) { @@ -313,65 +666,98 @@ func (cg *ContextGenerator) requestShotgunContextGenerationInternal(rootDir stri }() if genCtx.Err() != nil { // Check for immediate cancellation - runtime.LogInfo(cg.app.ctx, fmt.Sprintf("Context generation for %s cancelled before starting: %v", rootDir, genCtx.Err())) + runtime.LogInfo(cg.app.ctx, fmt.Sprintf("Context generation cancelled before starting: %v", genCtx.Err())) return } - output, err := cg.app.generateShotgunOutputWithProgress(genCtx, rootDir, excludedPaths) + output, err := cg.app.generateShotgunOutputWithProgress(genCtx, rootDirs, excludedByRoot) + var tokenCount int64 + if err == nil { + tokenCount, err = cg.app.CountTokens(output) + if err != nil { + err = fmt.Errorf("failed to count tokens for context: %w", err) + } + } select { case <-genCtx.Done(): - errMsg := fmt.Sprintf("Shotgun context generation cancelled for %s: %v", rootDir, genCtx.Err()) + errMsg := fmt.Sprintf("Shotgun context generation cancelled: %v", genCtx.Err()) runtime.LogInfo(cg.app.ctx, errMsg) // Changed from LogWarn runtime.EventsEmit(cg.app.ctx, "shotgunContextError", errMsg) default: if err != nil { - errMsg := fmt.Sprintf("Error generating shotgun output for %s: %v", rootDir, err) + errMsg := fmt.Sprintf("Error generating shotgun output: %v", err) runtime.LogError(cg.app.ctx, errMsg) runtime.EventsEmit(cg.app.ctx, "shotgunContextError", errMsg) } else { finalSize := len(output) - successMsg := fmt.Sprintf("Shotgun context generated successfully for %s. Size: %d bytes.", rootDir, finalSize) - if finalSize > maxOutputSizeBytes { // Should have been caught by ErrContextTooLong, but as a safeguard - runtime.LogWarningf(cg.app.ctx, "Warning: Generated context size %d exceeds max %d, but was not caught by ErrContextTooLong.", finalSize, maxOutputSizeBytes) + successMsg := fmt.Sprintf("Shotgun context generated successfully. Size: %d bytes. Tokens: %d.", finalSize, tokenCount) + if int64(finalSize) > limit { // Should have been caught by ErrContextTooLong, but as a safeguard + runtime.LogWarningf(cg.app.ctx, "Warning: Generated context size %d exceeds max %d, but was not caught by ErrContextTooLong.", finalSize, limit) } runtime.LogInfo(cg.app.ctx, successMsg) - runtime.EventsEmit(cg.app.ctx, "shotgunContextGenerated", output) + runtime.EventsEmit(cg.app.ctx, "shotgunContextGenerated", map[string]interface{}{ + "output": output, + "tokens": tokenCount, + }) } } }(myToken) // Pass the token to the goroutine } // RequestShotgunContextGeneration is the method bound to Wails. -func (a *App) RequestShotgunContextGeneration(rootDir string, excludedPaths []string) { +func (a *App) RequestShotgunContextGeneration(rootDirs []string, excludedByRoot map[string][]string) { if a.contextGenerator == nil { // This should not happen if startup initializes it correctly runtime.LogError(a.ctx, "ContextGenerator not initialized") runtime.EventsEmit(a.ctx, "shotgunContextError", "Internal error: ContextGenerator not initialized") return } - a.contextGenerator.requestShotgunContextGenerationInternal(rootDir, excludedPaths) + a.contextGenerator.requestShotgunContextGenerationInternal(rootDirs, excludedByRoot) } -func (a *App) RequestAutoContextSelection(rootDir string, excludedPaths []string, userTask string) ([]string, error) { +func (a *App) RequestAutoContextSelection(rootDirs []string, excludedByRoot map[string][]string, userTask string) (map[string][]string, error) { if a.autoContextService == nil { return nil, errors.New("auto-context service is not initialized") } - rootDir = strings.TrimSpace(rootDir) - if rootDir == "" { + normalizedRoots, err := normalizeAndValidateRoots(rootDirs, true) + if err != nil { + return nil, err + } + if len(normalizedRoots) == 0 { return nil, errors.New("project root is required") } if !a.HasActiveLlmKey() { return nil, errors.New("no active LLM configuration found") } - // Prepare excluded paths map - excludedMap := make(map[string]bool) - for _, p := range excludedPaths { - excludedMap[normalizeRelativePath(p)] = true + rootLabels := computeRootLabels(normalizedRoots) + excludedMaps := make(map[string]map[string]bool, len(normalizedRoots)) + for _, rootDir := range normalizedRoots { + excludedMaps[rootDir] = make(map[string]bool) + } + for rootDir, paths := range excludedByRoot { + normalizedRoot, err := normalizeAndValidateRoots([]string{rootDir}, false) + if err != nil { + return nil, err + } + if len(normalizedRoot) == 0 { + return nil, errors.New("project root is required") + } + rootKey := normalizedRoot[0] + if _, ok := excludedMaps[rootKey]; !ok { + return nil, fmt.Errorf("excluded paths provided for unknown root: %s", rootKey) + } + for _, p := range paths { + normalized := normalizeRelativePath(p) + if normalized == "" { + continue + } + excludedMaps[rootKey][normalized] = true + } } - tree, err := buildAutoContextTree(rootDir, excludedMap) + tree, err := buildAutoContextTreeForRoots(normalizedRoots, rootLabels, excludedMaps) if err != nil { a.emitAutoContextError(fmt.Sprintf("failed to build project tree: %v", err)) return nil, err @@ -426,13 +812,18 @@ func (a *App) RequestAutoContextSelection(rootDir string, excludedPaths []string return nil, err } - selected, err := resolveLLMSelection(rootDir, parsed.Files) + selected, err := resolveLLMSelectionForRoots(normalizedRoots, rootLabels, parsed.Files) if err != nil { a.emitAutoContextError(fmt.Sprintf("unable to match LLM selection to files: %v", err)) return nil, err } - runtime.LogInfof(a.ctx, "Auto-context selected %d files via %s (%s)", len(selected), cfg.Provider, cfg.Model) + selectedCount := 0 + for _, items := range selected { + selectedCount += len(items) + } + + runtime.LogInfof(a.ctx, "Auto-context selected %d files via %s (%s)", selectedCount, cfg.Provider, cfg.Model) return selected, nil } @@ -497,37 +888,67 @@ func (a *App) emitProgress(state *generationProgressState) { } // generateShotgunOutputWithProgress generates the TXT output with progress reporting and size limits -func (a *App) generateShotgunOutputWithProgress(jobCtx context.Context, rootDir string, excludedPaths []string) (string, error) { +func (a *App) generateShotgunOutputWithProgress(jobCtx context.Context, rootDirs []string, excludedByRoot map[string][]string) (string, error) { if err := jobCtx.Err(); err != nil { // Check for cancellation at the beginning return "", err } - excludedMap := make(map[string]bool) - for _, p := range excludedPaths { - excludedMap[p] = true + normalizedRoots, err := normalizeAndValidateRoots(rootDirs, true) + if err != nil { + return "", err } + if len(normalizedRoots) == 0 { + return "", nil + } + limit := a.contextLimitBytes() - totalItems, err := a.countProcessableItems(jobCtx, rootDir, excludedMap) - if err != nil { - return "", fmt.Errorf("failed to count processable items: %w", err) + rootLabels := computeRootLabels(normalizedRoots) + excludedMaps := make(map[string]map[string]bool, len(normalizedRoots)) + for _, rootDir := range normalizedRoots { + excludedMaps[rootDir] = make(map[string]bool) + } + for rootDir, paths := range excludedByRoot { + normalizedRoot, err := normalizeAndValidateRoots([]string{rootDir}, false) + if err != nil { + return "", err + } + if len(normalizedRoot) == 0 { + return "", errors.New("project root is required") + } + rootKey := normalizedRoot[0] + if _, ok := excludedMaps[rootKey]; !ok { + return "", fmt.Errorf("excluded paths provided for unknown root: %s", rootKey) + } + for _, p := range paths { + if p == "" { + continue + } + excludedMaps[rootKey][p] = true + } + } + + totalItems := 0 + for _, rootDir := range normalizedRoots { + count, err := a.countProcessableItems(jobCtx, rootDir, excludedMaps[rootDir]) + if err != nil { + return "", fmt.Errorf("failed to count processable items: %w", err) + } + totalItems += count } progressState := &generationProgressState{processedItems: 0, totalItems: totalItems} a.emitProgress(progressState) // Initial progress (0 / total) + rootOrder := make([]string, len(normalizedRoots)) + copy(rootOrder, normalizedRoots) + sort.SliceStable(rootOrder, func(i, j int) bool { + return rootLabels[rootOrder[i]] < rootLabels[rootOrder[j]] + }) + var output strings.Builder var fileContents strings.Builder - // Root directory line - output.WriteString(filepath.Base(rootDir) + string(os.PathSeparator) + "\n") - progressState.processedItems++ - a.emitProgress(progressState) - if output.Len() > maxOutputSizeBytes { - return "", fmt.Errorf("%w: content limit of %d bytes exceeded after root dir line (size: %d bytes)", ErrContextTooLong, maxOutputSizeBytes, output.Len()) - } - - // buildShotgunTreeRecursive is a recursive helper for generating the tree string and file contents - var buildShotgunTreeRecursive func(pCtx context.Context, currentPath, prefix string) error - buildShotgunTreeRecursive = func(pCtx context.Context, currentPath, prefix string) error { + var buildShotgunTreeRecursive func(pCtx context.Context, rootDir, rootLabel, currentPath, prefix string, excludedMap map[string]bool) error + buildShotgunTreeRecursive = func(pCtx context.Context, rootDir, rootLabel, currentPath, prefix string, excludedMap map[string]bool) error { select { case <-pCtx.Done(): return pCtx.Err() @@ -590,12 +1011,12 @@ func (a *App) generateShotgunOutputWithProgress(jobCtx context.Context, rootDir progressState.processedItems++ // For tree entry a.emitProgress(progressState) - if output.Len()+fileContents.Len() > maxOutputSizeBytes { - return fmt.Errorf("%w: content limit of %d bytes exceeded during tree generation (size: %d bytes)", ErrContextTooLong, maxOutputSizeBytes, output.Len()+fileContents.Len()) + if int64(output.Len()+fileContents.Len()) > limit { + return fmt.Errorf("%w: content limit of %d bytes exceeded during tree generation (size: %d bytes)", ErrContextTooLong, limit, output.Len()+fileContents.Len()) } if entry.IsDir() { - err := buildShotgunTreeRecursive(pCtx, path, nextPrefix) + err := buildShotgunTreeRecursive(pCtx, rootDir, rootLabel, path, nextPrefix, excludedMap) if err != nil { if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { return err @@ -614,27 +1035,43 @@ func (a *App) generateShotgunOutputWithProgress(jobCtx context.Context, rootDir content = []byte(fmt.Sprintf("Error reading file: %v", err)) } - // Ensure forward slashes for the name attribute, consistent with documentation. relPathForwardSlash := filepath.ToSlash(relPath) + filePath := relPathForwardSlash + if rootLabel != "" { + filePath = rootLabel + "/" + relPathForwardSlash + } - fileContents.WriteString(fmt.Sprintf("\n", relPathForwardSlash)) + fileContents.WriteString(fmt.Sprintf("\n", filePath)) fileContents.WriteString(string(content)) fileContents.WriteString("\n\n") // Each file block ends with a newline progressState.processedItems++ // For file content a.emitProgress(progressState) - if output.Len()+fileContents.Len() > maxOutputSizeBytes { // Final check after append - return fmt.Errorf("%w: content limit of %d bytes exceeded after appending file %s (total size: %d bytes)", ErrContextTooLong, maxOutputSizeBytes, relPath, output.Len()+fileContents.Len()) + if int64(output.Len()+fileContents.Len()) > limit { // Final check after append + return fmt.Errorf("%w: content limit of %d bytes exceeded after appending file %s (total size: %d bytes)", ErrContextTooLong, limit, relPath, output.Len()+fileContents.Len()) } } } return nil } - err = buildShotgunTreeRecursive(jobCtx, rootDir, "") - if err != nil { - return "", fmt.Errorf("failed to build tree for shotgun: %w", err) + for _, rootDir := range rootOrder { + if err := jobCtx.Err(); err != nil { + return "", err + } + rootLabel := rootLabels[rootDir] + output.WriteString(rootLabel + string(os.PathSeparator) + "\n") + progressState.processedItems++ + a.emitProgress(progressState) + if int64(output.Len()+fileContents.Len()) > limit { + return "", fmt.Errorf("%w: content limit of %d bytes exceeded after root dir line (size: %d bytes)", ErrContextTooLong, limit, output.Len()+fileContents.Len()) + } + + err := buildShotgunTreeRecursive(jobCtx, rootDir, rootLabel, rootDir, "", excludedMaps[rootDir]) + if err != nil { + return "", fmt.Errorf("failed to build tree for shotgun: %w", err) + } } if err := jobCtx.Err(); err != nil { // Check for cancellation before final string operations @@ -650,9 +1087,51 @@ func (a *App) generateShotgunOutputWithProgress(jobCtx context.Context, rootDir // --- Watchman Implementation --- +func copyGitignoreMap(src map[string]*gitignore.GitIgnore) map[string]*gitignore.GitIgnore { + if src == nil { + return nil + } + dst := make(map[string]*gitignore.GitIgnore, len(src)) + for key, value := range src { + dst[key] = value + } + return dst +} + +func isRelativeWithinRoot(rel string) bool { + if rel == "" || rel == "." { + return true + } + if rel == ".." { + return false + } + return !strings.HasPrefix(rel, ".."+string(os.PathSeparator)) +} + +func findRootForPath(path string, roots []string) (string, string) { + bestLen := -1 + bestRoot := "" + bestRel := "" + for _, root := range roots { + rel, err := filepath.Rel(root, path) + if err != nil { + continue + } + if !isRelativeWithinRoot(rel) { + continue + } + if len(root) > bestLen { + bestLen = len(root) + bestRoot = root + bestRel = rel + } + } + return bestRoot, bestRel +} + type Watchman struct { app *App - rootDir string + rootDirs []string fsWatcher *fsnotify.Watcher watchedDirs map[string]bool // Tracks directories explicitly added to fsnotify @@ -661,8 +1140,8 @@ type Watchman struct { cancelFunc context.CancelFunc // Store current patterns to be used by scanDirectoryStateInternal - currentProjectGitignore *gitignore.GitIgnore - currentCustomPatterns *gitignore.GitIgnore + currentProjectGitignores map[string]*gitignore.GitIgnore + currentCustomPatterns *gitignore.GitIgnore } func NewWatchman(app *App) *Watchman { @@ -672,13 +1151,13 @@ func NewWatchman(app *App) *Watchman { } } -// StartFileWatcher is called by JavaScript to start watching a directory. -func (a *App) StartFileWatcher(rootDirPath string) error { - runtime.LogInfof(a.ctx, "StartFileWatcher called for: %s", rootDirPath) +// StartFileWatcher is called by JavaScript to start watching directories. +func (a *App) StartFileWatcher(rootDirs []string) error { + runtime.LogInfof(a.ctx, "StartFileWatcher called for %d roots.", len(rootDirs)) if a.fileWatcher == nil { return fmt.Errorf("file watcher not initialized") } - return a.fileWatcher.Start(rootDirPath) + return a.fileWatcher.Start(rootDirs) } // StopFileWatcher is called by JavaScript to stop the current watcher. @@ -691,38 +1170,34 @@ func (a *App) StopFileWatcher() error { return nil } -func (w *Watchman) Start(newRootDir string) error { +func (w *Watchman) Start(newRootDirs []string) error { w.Stop() // Stop any existing watcher - w.mu.Lock() - w.rootDir = newRootDir - if w.rootDir == "" { - w.mu.Unlock() + normalizedRoots, err := normalizeAndValidateRoots(newRootDirs, true) + if err != nil { + return err + } + if len(normalizedRoots) == 0 { runtime.LogInfo(w.app.ctx, "Watchman: Root directory is empty, not starting.") return nil } - w.mu.Unlock() - // Initialize patterns based on App's current state + w.mu.Lock() + w.rootDirs = normalizedRoots if w.app.useGitignore { - w.currentProjectGitignore = w.app.projectGitignore + w.currentProjectGitignores = copyGitignoreMap(w.app.projectGitignores) } else { - w.currentProjectGitignore = nil + w.currentProjectGitignores = nil } if w.app.useCustomIgnore { w.currentCustomPatterns = w.app.currentCustomIgnorePatterns } else { w.currentCustomPatterns = nil } - - w.mu.Lock() - // Ensure settings are loaded if they haven't been (e.g. if called before startup completes, though unlikely) - // However, loadSettings is called in startup, so this should generally be populated. ctx, cancel := context.WithCancel(w.app.ctx) // Use app's context as parent w.cancelFunc = cancel w.mu.Unlock() - var err error w.fsWatcher, err = fsnotify.NewWatcher() if err != nil { runtime.LogErrorf(w.app.ctx, "Watchman: Error creating fsnotify watcher: %v", err) @@ -730,8 +1205,10 @@ func (w *Watchman) Start(newRootDir string) error { } w.watchedDirs = make(map[string]bool) // Initialize/clear - runtime.LogInfof(w.app.ctx, "Watchman: Starting for directory %s", newRootDir) - w.addPathsToWatcherRecursive(newRootDir) // Add initial paths + runtime.LogInfof(w.app.ctx, "Watchman: Starting for %d roots", len(normalizedRoots)) + for _, rootDir := range normalizedRoots { + w.addPathsToWatcherRecursive(rootDir, rootDir) // Add initial paths + } go w.run(ctx) return nil @@ -753,7 +1230,7 @@ func (w *Watchman) Stop() { } w.fsWatcher = nil } - w.rootDir = "" + w.rootDirs = nil w.watchedDirs = make(map[string]bool) // Clear watched directories } @@ -767,17 +1244,17 @@ func (w *Watchman) run(ctx context.Context) { }() w.mu.Lock() - currentRootDir := w.rootDir + currentRootDirs := append([]string{}, w.rootDirs...) w.mu.Unlock() - runtime.LogInfof(w.app.ctx, "Watchman: Monitoring goroutine started for %s", currentRootDir) + runtime.LogInfof(w.app.ctx, "Watchman: Monitoring goroutine started for %d roots", len(currentRootDirs)) for { select { case <-ctx.Done(): w.mu.Lock() - shutdownRootDir := w.rootDir // Re-fetch rootDir under lock as it might have changed + shutdownRoots := append([]string{}, w.rootDirs...) w.mu.Unlock() - runtime.LogInfof(w.app.ctx, "Watchman: Context cancelled, shutting down watcher for %s.", shutdownRootDir) + runtime.LogInfof(w.app.ctx, "Watchman: Context cancelled, shutting down watcher for %d roots.", len(shutdownRoots)) return case event, ok := <-w.fsWatcher.Events: @@ -788,22 +1265,25 @@ func (w *Watchman) run(ctx context.Context) { runtime.LogDebugf(w.app.ctx, "Watchman: fsnotify event: %s", event) w.mu.Lock() - currentRootDir = w.rootDir // Update currentRootDir under lock - // Safely copy ignore patterns - projIgn := w.currentProjectGitignore + currentRootDirs = append([]string{}, w.rootDirs...) + projIgns := w.currentProjectGitignores custIgn := w.currentCustomPatterns w.mu.Unlock() - if currentRootDir == "" { // Watcher might have been stopped + if len(currentRootDirs) == 0 { // Watcher might have been stopped continue } - relEventPath, err := filepath.Rel(currentRootDir, event.Name) - if err != nil { - runtime.LogWarningf(w.app.ctx, "Watchman: Could not get relative path for event %s (root: %s): %v", event.Name, currentRootDir, err) + rootDir, relEventPath := findRootForPath(event.Name, currentRootDirs) + if rootDir == "" { continue } + var projIgn *gitignore.GitIgnore + if projIgns != nil { + projIgn = projIgns[rootDir] + } + // Check if the event path is ignored isIgnoredByGit := projIgn != nil && projIgn.MatchesPath(relEventPath) isIgnoredByCustom := custIgn != nil && custIgn.MatchesPath(relEventPath) @@ -815,8 +1295,8 @@ func (w *Watchman) run(ctx context.Context) { // Handle relevant events (excluding Chmod) if event.Op&fsnotify.Chmod == 0 { - runtime.LogInfof(w.app.ctx, "Watchman: Relevant change detected for %s in %s", event.Name, currentRootDir) - w.app.notifyFileChange(currentRootDir) + runtime.LogInfof(w.app.ctx, "Watchman: Relevant change detected for %s in %s", event.Name, rootDir) + w.app.notifyFileChange(rootDir) } // Dynamic directory watching @@ -828,7 +1308,7 @@ func (w *Watchman) run(ctx context.Context) { isNewDirIgnoredByCustom := custIgn != nil && custIgn.MatchesPath(relEventPath) if !isNewDirIgnoredByGit && !isNewDirIgnoredByCustom { runtime.LogDebugf(w.app.ctx, "Watchman: New directory created %s, adding to watcher.", event.Name) - w.addPathsToWatcherRecursive(event.Name) // This will add event.Name and its children + w.addPathsToWatcherRecursive(rootDir, event.Name) // This will add event.Name and its children } else { runtime.LogDebugf(w.app.ctx, "Watchman: New directory %s is ignored, not adding to watcher.", event.Name) } @@ -861,23 +1341,27 @@ func (w *Watchman) run(ctx context.Context) { } } -func (w *Watchman) addPathsToWatcherRecursive(baseDirToAdd string) { +func (w *Watchman) addPathsToWatcherRecursive(rootDir, baseDirToAdd string) { w.mu.Lock() // Lock to access watcher and ignore patterns fsW := w.fsWatcher - projIgn := w.currentProjectGitignore + projIgns := w.currentProjectGitignores custIgn := w.currentCustomPatterns - overallRoot := w.rootDir w.mu.Unlock() - if fsW == nil || overallRoot == "" { + if fsW == nil || rootDir == "" { runtime.LogWarningf(w.app.ctx, "Watchman.addPathsToWatcherRecursive: fsWatcher is nil or rootDir is empty. Skipping add for %s.", baseDirToAdd) return } + var projIgn *gitignore.GitIgnore + if projIgns != nil { + projIgn = projIgns[rootDir] + } + filepath.WalkDir(baseDirToAdd, func(path string, d fs.DirEntry, walkErr error) error { if walkErr != nil { runtime.LogWarningf(w.app.ctx, "Watchman scan error accessing %s: %v", path, walkErr) - if d != nil && d.IsDir() && path != overallRoot { // Changed scanRootDir to overallRoot for clarity + if d != nil && d.IsDir() && path != rootDir { // Changed scanRootDir to rootDir for clarity return filepath.SkipDir } return nil // Try to continue @@ -887,16 +1371,16 @@ func (w *Watchman) addPathsToWatcherRecursive(baseDirToAdd string) { return nil } - relPath, errRel := filepath.Rel(overallRoot, path) + relPath, errRel := filepath.Rel(rootDir, path) if errRel != nil { - runtime.LogWarningf(w.app.ctx, "Watchman.addPathsToWatcherRecursive: Could not get relative path for %s (root: %s): %v", path, overallRoot, errRel) + runtime.LogWarningf(w.app.ctx, "Watchman.addPathsToWatcherRecursive: Could not get relative path for %s (root: %s): %v", path, rootDir, errRel) return nil // Continue with other paths } - // Skip .git directory at the top level of overallRoot + // Skip .git directory at the top level of rootDir if d.IsDir() && d.Name() == ".git" { parentDir := filepath.Dir(path) - if parentDir == overallRoot { + if parentDir == rootDir { runtime.LogDebugf(w.app.ctx, "Watchman.addPathsToWatcherRecursive: Skipping .git directory: %s", path) return filepath.SkipDir } @@ -931,25 +1415,25 @@ func (a *App) notifyFileChange(rootDir string) { // RefreshIgnoresAndRescan is called when ignore settings change in the App. func (w *Watchman) RefreshIgnoresAndRescan() error { w.mu.Lock() - if w.rootDir == "" { + if len(w.rootDirs) == 0 { w.mu.Unlock() - runtime.LogInfo(w.app.ctx, "Watchman.RefreshIgnoresAndRescan: No rootDir, skipping.") + runtime.LogInfo(w.app.ctx, "Watchman.RefreshIgnoresAndRescan: No rootDirs, skipping.") return nil } runtime.LogInfo(w.app.ctx, "Watchman.RefreshIgnoresAndRescan: Refreshing ignore patterns and re-scanning.") // Update patterns based on App's current state if w.app.useGitignore { - w.currentProjectGitignore = w.app.projectGitignore + w.currentProjectGitignores = copyGitignoreMap(w.app.projectGitignores) } else { - w.currentProjectGitignore = nil + w.currentProjectGitignores = nil } if w.app.useCustomIgnore { w.currentCustomPatterns = w.app.currentCustomIgnorePatterns } else { w.currentCustomPatterns = nil } - currentRootDir := w.rootDir + currentRoots := append([]string{}, w.rootDirs...) defer w.mu.Unlock() // Stop existing watcher (closes, clears watchedDirs) @@ -969,8 +1453,10 @@ func (w *Watchman) RefreshIgnoresAndRescan() error { return fmt.Errorf("failed to create new fsnotify watcher: %w", err) } - w.addPathsToWatcherRecursive(currentRootDir) // Add paths with new rules - w.app.notifyFileChange(currentRootDir) // Notify frontend to refresh its view + for _, rootDir := range currentRoots { + w.addPathsToWatcherRecursive(rootDir, rootDir) // Add paths with new rules + w.app.notifyFileChange(rootDir) // Notify frontend to refresh its view + } return nil } @@ -1003,6 +1489,7 @@ func (a *App) compileCustomIgnorePatterns() error { func (a *App) loadSettings() { // Default to embedded rules a.settings.CustomIgnoreRules = defaultCustomIgnoreRulesContent + shouldSaveSettings := false if a.configPath == "" { runtime.LogWarningf(a.ctx, "Config path is empty, using default custom ignore rules (embedded).") @@ -1015,11 +1502,8 @@ func (a *App) loadSettings() { data, err := os.ReadFile(a.configPath) if err != nil { if os.IsNotExist(err) { - runtime.LogInfo(a.ctx, "Settings file not found. Using default custom ignore rules (embedded) and attempting to save them.") - // Save default settings to create the file. compileCustomIgnorePatterns will be called after this. - if errSave := a.saveSettings(); errSave != nil { // saveSettings will use a.settings.CustomIgnoreRules which is currently default - runtime.LogErrorf(a.ctx, "Failed to save default settings: %v", errSave) - } + runtime.LogInfo(a.ctx, "Settings file not found. Using default custom ignore rules (embedded).") + shouldSaveSettings = true } else { runtime.LogErrorf(a.ctx, "Error reading settings file %s: %v. Using default custom ignore rules (embedded).", a.configPath, err) } @@ -1039,15 +1523,31 @@ func (a *App) loadSettings() { if strings.TrimSpace(a.settings.CustomPromptRules) == "" { runtime.LogInfo(a.ctx, "Custom prompt rules are empty or missing, using default.") a.settings.CustomPromptRules = defaultCustomPromptRulesContent + shouldSaveSettings = true + } else if strings.TrimSpace(a.settings.CustomPromptRules) == legacyDefaultCustomPromptRulesContent { + runtime.LogInfo(a.ctx, "Custom prompt rules match legacy default, updating to new default.") + a.settings.CustomPromptRules = defaultCustomPromptRulesContent + shouldSaveSettings = true } } } + if a.settings.MaxContextBytes <= 0 { + runtime.LogWarningf(a.ctx, "MaxContextBytes is missing or invalid, defaulting to %d.", defaultMaxOutputSizeBytes) + a.settings.MaxContextBytes = defaultMaxOutputSizeBytes + shouldSaveSettings = true + } a.ensureLLMSettingsDefaults() if errCompile := a.compileCustomIgnorePatterns(); errCompile != nil { // Error already logged in compileCustomIgnorePatterns } + + if shouldSaveSettings { + if err := a.saveSettings(); err != nil { + runtime.LogErrorf(a.ctx, "Failed to save settings: %v", err) + } + } } func (a *App) saveSettings() error { @@ -1101,7 +1601,7 @@ func (a *App) SetCustomIgnoreRules(rules string) error { return fmt.Errorf("rules saved, but failed to compile custom ignore patterns: %w", compileErr) } - if a.fileWatcher != nil && a.fileWatcher.rootDir != "" { + if a.fileWatcher != nil { return a.fileWatcher.RefreshIgnoresAndRescan() } return nil @@ -1130,8 +1630,7 @@ func (a *App) SetCustomPromptRules(rules string) error { func (a *App) SetUseGitignore(enabled bool) error { a.useGitignore = enabled runtime.LogInfof(a.ctx, "App setting useGitignore changed to: %v", enabled) - if a.fileWatcher != nil && a.fileWatcher.rootDir != "" { - // Assuming watcher is for the current project if active. + if a.fileWatcher != nil { return a.fileWatcher.RefreshIgnoresAndRescan() } return nil @@ -1141,8 +1640,7 @@ func (a *App) SetUseGitignore(enabled bool) error { func (a *App) SetUseCustomIgnore(enabled bool) error { a.useCustomIgnore = enabled runtime.LogInfof(a.ctx, "App setting useCustomIgnore changed to: %v", enabled) - if a.fileWatcher != nil && a.fileWatcher.rootDir != "" { - // Assuming watcher is for the current project if active. + if a.fileWatcher != nil { return a.fileWatcher.RefreshIgnoresAndRescan() } return nil diff --git a/auto_context_service.go b/auto_context_service.go index a3a842c..793a1d1 100644 --- a/auto_context_service.go +++ b/auto_context_service.go @@ -148,9 +148,32 @@ func parseAutoContextJSON(text string) (AutoContextResult, error) { return result, nil } -func buildAutoContextTree(rootDir string, excludedMap map[string]bool) (string, error) { - var builder strings.Builder - builder.WriteString(filepath.Base(rootDir) + string(os.PathSeparator) + "\n") +func writeAutoContextTree(builder *strings.Builder, rootDir, rootLabel string, excludedMap map[string]bool) error { + if builder == nil { + return errors.New("tree builder is required") + } + if strings.TrimSpace(rootDir) == "" { + return errors.New("project root is required") + } + label := strings.TrimSpace(rootLabel) + if label == "" { + label = filepath.Base(rootDir) + } + if label == "" || label == "." { + return errors.New("project root label is required") + } + + appendLine := func(line string) error { + builder.WriteString(line) + if builder.Len() > maxAutoContextTreeChars { + return errAutoContextTreeTooLarge + } + return nil + } + + if err := appendLine(filepath.ToSlash(label) + "/\n"); err != nil { + return err + } var walk func(string, string) error walk = func(currentPath, prefix string) error { @@ -184,9 +207,8 @@ func buildAutoContextTree(rootDir string, excludedMap map[string]bool) (string, branch = "└── " nextPrefix = prefix + " " } - builder.WriteString(prefix + branch + entry.Name() + "\n") - if builder.Len() > maxAutoContextTreeChars { - return errAutoContextTreeTooLarge + if err := appendLine(prefix + branch + entry.Name() + "\n"); err != nil { + return err } if entry.IsDir() { @@ -198,9 +220,47 @@ func buildAutoContextTree(rootDir string, excludedMap map[string]bool) (string, return nil } - if err := walk(rootDir, ""); err != nil { + return walk(rootDir, "") +} + +func buildAutoContextTree(rootDir string, excludedMap map[string]bool) (string, error) { + var builder strings.Builder + if err := writeAutoContextTree(&builder, rootDir, filepath.Base(rootDir), excludedMap); err != nil { return "", err } + return builder.String(), nil +} + +func buildAutoContextTreeForRoots(rootDirs []string, rootLabels map[string]string, excludedByRoot map[string]map[string]bool) (string, error) { + if len(rootDirs) == 0 { + return "", errors.New("project root is required") + } + rootOrder := make([]string, len(rootDirs)) + copy(rootOrder, rootDirs) + sort.SliceStable(rootOrder, func(i, j int) bool { + return rootLabels[rootOrder[i]] < rootLabels[rootOrder[j]] + }) + + var builder strings.Builder + for idx, rootDir := range rootOrder { + label := strings.TrimSpace(rootLabels[rootDir]) + if label == "" { + return "", fmt.Errorf("missing label for root: %s", rootDir) + } + excludedMap, ok := excludedByRoot[rootDir] + if !ok { + return "", fmt.Errorf("excluded paths missing for root: %s", rootDir) + } + if idx > 0 { + builder.WriteString("\n") + if builder.Len() > maxAutoContextTreeChars { + return "", errAutoContextTreeTooLarge + } + } + if err := writeAutoContextTree(&builder, rootDir, label, excludedMap); err != nil { + return "", err + } + } if builder.Len() > maxAutoContextTreeChars { return "", errAutoContextTreeTooLarge } @@ -220,47 +280,54 @@ func normalizeRelativePath(rel string) string { // normalizeCandidateForRoot brings an LLM-returned path into the canonical // "relative to rootDir" form. It accepts either strictly relative paths like -// "frontend/src/..." or paths prefixed with the project root name, e.g.: +// "frontend/src/..." or paths prefixed with the project root label/name, e.g.: // "shotgun_code/frontend/src/..." when rootDir == ".../shotgun_code". -func normalizeCandidateForRoot(rootDir, candidate string) string { +func normalizeCandidateForRoot(rootDir, rootLabel, candidate string) string { candidate = normalizeRelativePath(candidate) if candidate == "" { return "" } - - rootBase := filepath.Base(rootDir) - if rootBase == "" || rootBase == "." { - return candidate + prefixes := make([]string, 0, 2) + if label := strings.TrimSpace(rootLabel); label != "" && label != "." { + prefixes = append(prefixes, filepath.ToSlash(label)) } - rootBase = filepath.ToSlash(rootBase) - - // Common case: "shotgun_code/frontend/src/..." → "frontend/src/..." - prefix := rootBase + "/" - if strings.HasPrefix(candidate, prefix) { - return strings.TrimPrefix(candidate, prefix) - } - - // Also accept "./shotgun_code/..." just in case the model prepends "./". - dotPrefix := "./" + prefix - if strings.HasPrefix(candidate, dotPrefix) { - return strings.TrimPrefix(candidate, dotPrefix) + rootBase := filepath.Base(rootDir) + if rootBase != "" && rootBase != "." && rootBase != rootLabel { + prefixes = append(prefixes, filepath.ToSlash(rootBase)) } - // If the candidate is exactly the root name, it is not a file path. - if candidate == rootBase { - return "" + for _, prefix := range prefixes { + if candidate == prefix { + return "." + } + prefixWithSlash := prefix + "/" + if strings.HasPrefix(candidate, prefixWithSlash) { + trimmed := strings.TrimPrefix(candidate, prefixWithSlash) + if trimmed == "" { + return "." + } + return trimmed + } + dotPrefix := "./" + prefixWithSlash + if strings.HasPrefix(candidate, dotPrefix) { + trimmed := strings.TrimPrefix(candidate, dotPrefix) + if trimmed == "" { + return "." + } + return trimmed + } } return candidate } -func resolveLLMSelection(rootDir string, candidates []string) ([]string, error) { +func resolveLLMSelection(rootDir, rootLabel string, candidates []string, allowEmpty bool) ([]string, error) { if len(candidates) == 0 { return nil, errors.New("no candidate paths provided") } selected := make(map[string]struct{}) for _, candidate := range candidates { - candidate = normalizeCandidateForRoot(rootDir, candidate) + candidate = normalizeCandidateForRoot(rootDir, rootLabel, candidate) if candidate == "" { continue } @@ -293,6 +360,9 @@ func resolveLLMSelection(rootDir string, candidates []string) ([]string, error) } if len(selected) == 0 { + if allowEmpty { + return []string{}, nil + } return nil, errors.New("no existing files matched the LLM selection") } @@ -304,6 +374,77 @@ func resolveLLMSelection(rootDir string, candidates []string) ([]string, error) return sorted, nil } +func resolveLLMSelectionForRoots(rootDirs []string, rootLabels map[string]string, candidates []string) (map[string][]string, error) { + if len(rootDirs) == 0 { + return nil, errors.New("project root is required") + } + labelToRoot := make(map[string]string, len(rootDirs)) + for _, rootDir := range rootDirs { + label := strings.TrimSpace(rootLabels[rootDir]) + if label == "" { + return nil, fmt.Errorf("missing label for root: %s", rootDir) + } + if _, exists := labelToRoot[label]; exists { + return nil, fmt.Errorf("duplicate root label: %s", label) + } + labelToRoot[label] = rootDir + } + + normalized := make([]string, 0, len(candidates)) + for _, candidate := range candidates { + if normalizedCandidate := normalizeRelativePath(candidate); normalizedCandidate != "" { + normalized = append(normalized, normalizedCandidate) + } + } + if len(normalized) == 0 { + return nil, errors.New("no candidate paths provided") + } + + grouped := make(map[string][]string, len(rootDirs)) + for _, rootDir := range rootDirs { + grouped[rootDir] = []string{} + } + + if len(rootDirs) == 1 { + rootDir := rootDirs[0] + grouped[rootDir] = append(grouped[rootDir], normalized...) + } else { + for _, candidate := range normalized { + parts := strings.SplitN(candidate, "/", 2) + label := parts[0] + rootDir, ok := labelToRoot[label] + if !ok { + return nil, fmt.Errorf("path %q does not include a known root label", candidate) + } + grouped[rootDir] = append(grouped[rootDir], candidate) + } + } + + selectedByRoot := make(map[string][]string) + totalSelected := 0 + for _, rootDir := range rootDirs { + rootCandidates := grouped[rootDir] + if len(rootCandidates) == 0 { + continue + } + selected, err := resolveLLMSelection(rootDir, rootLabels[rootDir], rootCandidates, true) + if err != nil { + return nil, err + } + if len(selected) == 0 { + continue + } + selectedByRoot[rootDir] = selected + totalSelected += len(selected) + } + + if totalSelected == 0 { + return nil, errors.New("no existing files matched the LLM selection") + } + + return selectedByRoot, nil +} + func buildProviderConfig(settings LLMSettings) provider.Config { return provider.Config{ Provider: settings.ActiveProvider, diff --git a/design/architecture.md b/design/architecture.md index 295132e..5701c95 100644 --- a/design/architecture.md +++ b/design/architecture.md @@ -1,227 +1,161 @@ -# Shotgun Application Architecture +# Shotgun Architecture ## 1. Overview +Shotgun is a desktop application built with **Wails v2** and a **Vue 3** frontend. +Stack: +- **Backend**: Go (file scanning, tree building, context generation, tokenization). +- **Frontend**: Vue + Vite + Tailwind (UI, state, event handling). +- **Integration**: Wails bridges Go methods to JS. -Shotgun App is a desktop application built with Wails (v2) and Vue.js (v3). -Wails makes it possible to build cross‑platform desktop applications with a Go backend and a web frontend. +Key capabilities: +- multiple project roots in a single workspace; +- **offline** token counting via `o200k_base`; +- recursive symlink traversal with cycle protection; +- context size limit configured via `settings.json`; +- auto generation with debounce + queue. -- **Backend:** Go. Handles filesystem operations, logic for determining files/folders to exclude, and generation of the textual "Shotgun" output. -- **Frontend:** Vue.js (with Vite and Tailwind CSS). Provides the user interface for selecting a folder, displaying the file/folder tree, marking items to exclude, and showing the generated output. -- **Integration:** Wails provides the bridge between Go functions and JavaScript calls from the frontend. +--- ## 2. Backend (Go) -The Go backend is structured mainly inside the `main` package. - -### Key components (`app.go`): - -- **`App` struct**: Stores the application state and its dependencies. - - `ctx context.Context`: Wails context. - - `contextGenerator *ContextGenerator`: Instance of the context generator. - - `fileWatcher *Watchman`: Instance of the filesystem watcher. - - `settings AppSettings`: Current application settings (ignore rules, prompt rules). - - `currentCustomIgnorePatterns *gitignore.GitIgnore`: Compiled custom ignore rules. - - `configPath string`: Path to the `settings.json` configuration file. - - `useGitignore bool`: Flag for using `.gitignore` rules. - - `useCustomIgnore bool`: Flag for using custom ignore rules. - - `projectGitignore *gitignore.GitIgnore`: Compiled rules from the project's `.gitignore`. -- **`startup(ctx context.Context)`**: Wails lifecycle hook. Initializes `ctx`, `contextGenerator`, `fileWatcher`, and loads settings. -- **`FileNode` struct**: Represents a file or folder in the tree. Includes `Name`, `Path`, `RelPath`, `IsDir`, `Children`, `IsGitignored`, `IsCustomIgnored`. -- **`SelectDirectory() (string, error)`**: Opens the system dialog to select a directory. - - **`ListFiles(dirPath string) ([]*FileNode, error)`**: - - Accepts a directory path. - - Loads and compiles `.gitignore` from the given directory (if it exists) and stores it in `projectGitignore`. - - Creates the root `FileNode` representing `dirPath`. - - Recursively scans `dirPath` using `buildTreeRecursive` to build the tree of child `FileNode`s, taking into account rules from `.gitignore` (if `useGitignore` is enabled) and custom rules (if `useCustomIgnore` is enabled). - - Returns a slice containing only the root `FileNode`. - -- **`ContextGenerator` struct**: Manages asynchronous context generation. - - `requestShotgunContextGenerationInternal(rootDir string, excludedPaths []string)`: Internal method for starting/restarting generation in a separate goroutine. Handles cancellation of previous jobs. -- **`RequestShotgunContextGeneration(rootDir string, excludedPaths []string) error`**: Method exposed to the frontend via Wails. Delegates the call to `contextGenerator`. -- **`countProcessableItems(jobCtx context.Context, rootDir string, excludedMap map[string]bool) (int, error)`**: Recursively counts the number of items (directories, files to be listed, files whose contents will be read) to estimate total progress for context generation. -- **`generateShotgunOutputWithProgress(jobCtx context.Context, rootDir string, excludedPaths []string) (string, error)`**: - - Main function for generating the textual project context. - - Accepts the job context `jobCtx` for cancellation, `rootDir`, and the list of `excludedPaths`. - - Builds a textual representation of the file tree and aggregates the contents of non‑excluded files in an XML‑like format (`...`). - - Periodically calls `emitProgress` to send the `shotgunContextGenerationProgress` event to the frontend. - - Checks the overall size of the generated output against `maxOutputSizeBytes`. If the limit is exceeded, returns the `ErrContextTooLong` error. - - On completion (or on error/cancellation) sends either `shotgunContextGenerated` or `shotgunContextError` to the frontend. -- **`Watchman` struct**: Component for monitoring filesystem changes. - - `StartFileWatcher(rootDirPath string) error` / `StopFileWatcher() error`: Methods exposed to the frontend. - - `Start(newRootDir string)`: Initializes and starts watching using `fsnotify`. - - `Stop()`: Stops the current watcher. - - `run(ctx context.Context)`: Main monitoring loop running in a goroutine. Reacts to `fsnotify` events, filters them using ignore rules, manages adding/removing directories from watching, and notifies the frontend via `App.notifyFileChange`. - - `addPathsToWatcherRecursive(baseDirToAdd string)`: Recursively adds directories to `fsnotify.Watcher`, skipping ignored paths. - - `RefreshIgnoresAndRescan()`: Reloads ignore rules and restarts `fsnotify.Watcher` with updated paths. -- **`AppSettings` struct**: Structure for storing settings (`CustomIgnoreRules`, `CustomPromptRules`). - -- **Configuration management**: - - `compileCustomIgnorePatterns()`: Compiles textual ignore rules into a `gitignore.GitIgnore` object. - - `loadSettings()`: Loads settings from `settings.json` (using `xdg.ConfigFile`). If the file is missing or invalid, uses built‑in defaults (`defaultCustomIgnoreRulesContent` and `defaultCustomPromptRulesContent`). - - `saveSettings()`: Persists current settings to `settings.json`. - -- **Methods for managing rules and flags (exposed to the frontend)**: - - `GetCustomIgnoreRules() string` - - `SetCustomIgnoreRules(rules string) error` - - `GetCustomPromptRules() string` - - `SetCustomPromptRules(rules string) error` - - `SetUseGitignore(enabled bool) error` - - `SetUseCustomIgnore(enabled bool) error` - -- **`notifyFileChange(rootDir string)`**: Sends the `projectFilesChanged` event to the frontend. - -### `main.go`: - -- Initializes Wails. -- Configures the application window and its options (title, dimensions, background color). -- Embeds frontend assets (`embed`). -- Binds an instance of `App` so its public methods can be called from the frontend. -- Configures the system menu (for example, the standard macOS menu). - -## 3. Frontend (Vue.js) - -The frontend is a single‑page application (SPA) built with Vue 3 (Composition API), Vite, and Tailwind CSS. -It implements a multi‑step user interface for preparing project context, composing prompts, "executing" them, and "applying" patches. - -### Key components: - -- **`main.js`**: Entry point of the Vue application. -- **`App.vue`**: Root component that mounts `MainLayout.vue`. -- **`components/MainLayout.vue`**: - - **Structure**: Manages the main layout: horizontal stepper (`HorizontalStepper`), left sidebar (`LeftSidebar`), central content panel (`CentralPanel`), and bottom console (`BottomConsole`). - - **State management** (using `ref` and `reactive`): - - `currentStep`: Current active step (1–4). - - `steps`: Array of step objects with `id`, `title`, `completed`, `description`. - - `logMessages`: Array of log messages for `BottomConsole`. - - `projectRoot`, `fileTree`, `shotgunPromptContext`, `useGitignore`, `useCustomIgnore`, `loadingError`, `isGeneratingContext`, `generationProgressData`, `userTask`, `rulesContent`, `finalPrompt`: State related to the project, context, and user input. - - **Logic**: - - Navigation between steps (`navigateToStep`). - - Handling actions from step components (`handleStepAction`). - - Interaction with the Go backend (calling `SelectDirectoryGo`, `ListFiles`, `RequestShotgunContextGeneration`, settings methods). - - Subscribing to Wails events (`shotgunContextGenerated`, `shotgunContextError`, `shotgunContextGenerationProgress`, `projectFilesChanged`). - - Managing `Watchman` (start/stop). - - Debouncing context generation (`debouncedTriggerShotgunContextGeneration`). - - Updating excluded state for nodes in the tree (`updateAllNodesExcludedState`, `toggleExcludeNode`). -- **`components/HorizontalStepper.vue`**: Displays steps (1–4) at the top and enables navigation. -- **`components/LeftSidebar.vue`**: - - Displays the "Select Project Folder" button and the project path. - - Contains the "Use .gitignore rules" and "Use custom rules" checkboxes. - - Provides a button (⚙️) to open the modal for editing custom ignore rules (`CustomRulesModal.vue`). - - Displays the project file tree using `FileTree.vue`. - - Shows the list of steps for navigation. -- **`components/CentralPanel.vue`**: Dynamically renders the component for the current step. -- **`components/steps/Step1PrepareContext.vue`**: - - UI for the first step. Displays a progress bar while context is being generated. - - Shows the generated `generatedContext` in a read‑only `