-
Notifications
You must be signed in to change notification settings - Fork 4
Add deep research prompting analysis #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| ### 1. Executive Summary | ||
|
|
||
| - 1 new primary framework identified: Deep Research Prompting (DRP). | ||
| - Adds capabilities for structural archetyping (Typological Stack, Lens-Driven Analysis) and complex iteration (ReAct, Recursive Criticism). | ||
| - Expected impact: "foundational" for moving standard prompt workflows into research-grade, strategist-level pipelines. | ||
|
|
||
| ### 2. Document-Level Framework Extraction | ||
|
|
||
| #### 2.1 PromptResearch/Deep Research Prompt Engineering_.md | ||
|
|
||
| - **Framework Name**: Deep Research Prompting (DRP) | ||
| - **Core Innovation**: Moving beyond basic queries to designing prompts as instruments of epistemic control, structuring the analytical and cognitive workflow. | ||
| - **Key Components / Sub-Modules**: Typological Stack, Lens-Driven Analysis, Constraint-Directed Inquiry, Counterfactual Exploration, CoT, ReAct, Recursive Criticism. | ||
| - **Complexity Level**: Advanced | ||
| - **Primary Contribution**: Provides a taxonomy of advanced prompting patterns aimed at eliciting insight generation rather than mere summarization. | ||
| - **Intended Use Cases**: Complex research, strategic analysis, insight generation, policy analysis. | ||
| - **Prerequisites / Dependencies**: Large LLMs capable of multi-step reasoning. | ||
|
|
||
| ### 3. Normalized Framework Catalog | ||
|
|
||
| - **Name**: Deep Research Prompting (DRP) | ||
| - **Type**: Orchestration pattern / Cognitive scaffolding | ||
| - **Core Mechanisms**: | ||
| - Typological Stacking (multi-persona simulation) | ||
| - Lens-Driven Analysis (theoretical application) | ||
| - Explicit Constraints | ||
| - CoT / ReAct / RCI loops | ||
| - **Input / Output Contract**: Expects high-level research questions; produces structured, multi-dimensional analysis and insights. | ||
| - **Typical Integration Point in PEW**: Prompt design and multi-agent orchestration. | ||
| - **Complexity Tier**: Advanced | ||
|
|
||
| ### 4. Integration Mapping to Existing Workbench | ||
|
|
||
| #### 4.1 Extensions of Existing Frameworks | ||
|
|
||
| - **PALS Extension**: Enhances the Cognitive and Trust & Epistemics layers by providing concrete structural patterns (e.g., Typological Stacks) for directing attention and reasoning. | ||
| - **Context Engineering 2.0**: The DRP constraints map directly onto the CxEP post-conditions and invariants. | ||
| - **Role Prompting**: Directly extends Role Prompting with the "Typological Stack" pattern (simulating multiple interacting functions within a persona). | ||
|
|
||
| #### 4.2 New Modules and Services | ||
|
|
||
| - **New Services**: "Typological Stack Composer", "Lens-Driven Analysis Router" | ||
| - **Data Models**: New fields in the PRP schema to define "Analytical Lens" and "Counterfactual Variables". | ||
|
projectedanx marked this conversation as resolved.
|
||
| - **Workflow Slots**: During the initial prompt design phase and within ReAct/RCI loops in the execution engine. | ||
|
|
||
| ### 5. Proposed Workbench Architecture Updates | ||
|
|
||
| - **Core Engine Changes**: Add native support for ReAct and Recursive Criticism (RCI) loops within the execution engine. | ||
| - **Integration Layer**: Create a "Lens Application Engine" that maps specific theoretical frameworks to incoming prompts. | ||
| - **User Interface Additions**: Add a "Typological Stack Builder" to the Role Prompting workspace and a "Lens Selector" to the prompt builder. | ||
| - **Advanced Workflow Engine**: Support for recursive prompt execution (RCI loops). | ||
|
|
||
| ### 6. Implementation Roadmap | ||
|
|
||
| - **Phase 1: Foundation (0–2 months)** | ||
| - **Goals**: Implement basic Constraint-Directed Inquiry and Typological Stack patterns. | ||
| - **Deliverables**: Typological Stack Builder UI, constraint validation in CxEP. | ||
| - **Technical Risks**: Prompt length limits with complex typological stacks. | ||
| - **Dependencies**: Existing PALS and Role Prompting workspaces. | ||
|
|
||
| - **Phase 2: Intermediate (2–6 months)** | ||
| - **Goals**: Deploy the Lens-Driven Analysis routing and basic CoT scaffolding. | ||
| - **Deliverables**: Lens Selector UI, pre-built lens library, automated CoT injection. | ||
| - **Technical Risks**: Lens abstraction might be too vague for certain models. | ||
| - **Dependencies**: Phase 1 completion. | ||
|
|
||
| - **Phase 3: Advanced (6–12+ months)** | ||
| - **Goals**: Implement full ReAct and RCI (Recursive Criticism and Improvement) loops. | ||
| - **Deliverables**: Recursive execution engine, ReAct tool integration, automated self-refinement pipelines. | ||
| - **Technical Risks**: Infinite loops in recursive prompting, high API costs. | ||
| - **Dependencies**: Advanced Workflow Engine overhaul. | ||
|
|
||
| ### 7. Success Metrics and Evaluation | ||
|
|
||
| - **Quantitative Metrics**: | ||
| - Prompt quality improvements: Increase in "Insight Generation" scores vs. "Summarization" scores on a standardized test set. | ||
| - RCI loop efficiency: Number of iterations to reach passing state. | ||
| - **Qualitative Metrics**: | ||
| - Researcher satisfaction with Lens-Driven Analysis outputs. | ||
| - Novel insights generated via Counterfactual Exploration patterns. | ||
|
|
||
| ### 8. Research-to-Practice Translation | ||
|
|
||
| - **Recommended Starter Workflows**: Start with the Typological Stack Builder to define complex personas, then apply a specific Analytical Lens to a dataset. | ||
| - **Suggested Personas**: Researchers (using Lens-Driven Analysis), Strategists (using Counterfactual Exploration). | ||
| - **Collaboration**: Open repository of Analytical Lenses and Typological Stack templates. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.