Improved system prompt with LLM-based prompt engineering#12
Open
TaylorTurnerIT wants to merge 27 commits intostardothosting:mainfrom
Open
Improved system prompt with LLM-based prompt engineering#12TaylorTurnerIT wants to merge 27 commits intostardothosting:mainfrom
TaylorTurnerIT wants to merge 27 commits intostardothosting:mainfrom
Conversation
Apply fixes from StyleCI
… to amazon directly which was being throttled. This offloads that dependency to the browser end user session
Replace client-side expansion with secure backend API endpoint. Handles a.co/amzn.to redirects server-side to avoid CORS/blocking. Includes comprehensive tests and maintains all existing functionality. - Add UrlExpansionController with /api/expand-url - Update frontend to use backend expansion - Add security validation and CSRF protection - Update test suite (121 tests passing)
Minor responsive fix
Author
|
I was unable to get a local copy of the repo running on my system so all testing was done on ChatGPT.com and using Perplexity.ai with GPT 4.1. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I used Gemini, Perplexity, and Claude's research models to analyze 477 academic sources on prompt engineering and sentiment analysis, then created a new system prompt based on these findings.
Testing Results:
I compared the new prompt against our current one using an Amazon product with questionable reviews (https://www.amazon.com/LVB-Industrial-Cabinet-Farmhouse-Sideboard/dp/B0BPM2JSCD). This product is rated D by Fakespot, making it a good test case.
Results (mean scores across 84 reviews):
The new prompt produces more conservative scores, moving closer to the "Likely genuine" range while still flagging suspicious content. This result leans towards trusting a given review but uses more rigorous prompting to get those results.
Additionally, during testing the original prompt would occasionally output a single score for all reviews instead of providing a score for all of them, the new prompt always outputted correctly formatted JSON.
Note: Fakespot claims this product has 300 reviews, but I could only access 84 on the Amazon listing. Unclear where the discrepancy comes from.