Skip to content

Deploy conversational chat to production#306

Merged
mbklein merged 33 commits intomainfrom
deploy/staging
Apr 14, 2025
Merged

Deploy conversational chat to production#306
mbklein merged 33 commits intomainfrom
deploy/staging

Conversation

@mbklein
Copy link
Copy Markdown
Contributor

@mbklein mbklein commented Apr 14, 2025

No description provided.

mbklein and others added 30 commits April 2, 2025 15:10
Write API Token and Base URL to Secrets Manager
Cleanup model info from event config

Get agents working printing to logs

Add Azure API key to chat template

wip: temporary log to chat metric group

Fix secrets and build stuff in chat

Get agent callbacks working

Pass web socket to agent handler and add skeleton for callbacks

Exclude embeddings from hybrid query

Add websocket callbacks

Add responses over websocket

Send tool start/end, tokens, and stop/result messages over websocket

Teach the chat handler how to forget on demand

Update tools, remove examples

Improve prompting
Compress checkpoint data
Add more fields back to hybrid query

Add S3Saver and S3 infrastructure to chat template

Rename source_documents to search_result
Slightly change key pattern of checkpoints in s3

Indicate final message in socket
Make search agent a class
Wire model, stream_response input options back up
Add jupyter notebook for rapid iteration

Remove errant #* comment
Get sync chat function working

Refactor SearchAgent

Fix up mocking and SearchAgent test

Get chat and sync chat tests passing

Add additional tests

Add agent handler unit tests

Refactor chat package/module layout
Get all tests passing
Add tests for the S3 Checkpointer

Full test coverage for metrics callback

Extract SearchWorkflow class for readability and testing

Full test coverage for EventConfig

Add unit tests and fixture for the complex real-world keyword fields

Fix up ruff check

Full test coverage for tools

Full test coverage for setup.py with a slight refactor

Full test coverage for setup.py and OpenSearchNeuralSearch

Full test coverage for WebSocket class

Don't automatically load secrets on module import

Switch test runner to pytest
Leave individual test files as unittest

Add tests for core.secrets

Add test to make sure the chat handler writes metrics

Full test coverage for s3_checkpointer.py (marked as a slow test) and more coverage for OpenSearchNeuralSearch
Split checkpoint into interactions and filter all but the last one
Identify tool-related AIMessages by response_metadata
Add ability to selectively deploy stacks
Use cfn-lint instead of sam validate

Add DEFAULT_SEARCH_SIZE parameter

Add make features for env.json and samconfig.yaml and update documentation

Fix issues with templates & makefile and update documentation

Fix `start-with-step` to allow av-download stack to run
Update the prompt to limit tool calls and avoid recursion errors.
@mbklein mbklein merged commit eae572d into main Apr 14, 2025
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants