foundry local spitting out garbage regardless of which model being used

[ramifayad105]

So I am taking a summer class on agentic AI, and I am fairly new to Foundry Local. Our first assignment was to create our own simple AI chatbot using a video our professor made. The code runs on my professor's all-AMD desktop and my laptop, but not on my all-AMD desktop. Any help pointing me in the right direction to figure out why?
pc specs:
cpu: ryzen 7 9800x3d
gpu: RX 6800
32 gbs ddr5 6000MT/s

[natke]

Hi @ramifayad105, can you please share the version of Foundry Local you are running?

[ramifayad105]

Hi @ramifayad105, can you please share the version of Foundry Local you are running?

Thanks for the quick response! I'm on version 1.2.3

[natke]

Would you mind sharing the code for your application? Or a version with any private information redacted?

[ramifayad105]

https://www.dropbox.com/scl/fi/esl1kwjqmyz2az6l40g4o/handson-foundrylocal-simplechatbot-ramifayad105.zip?rlkey=3nqh0gtbwwmu1o9vob1wbwnal&st=8zkeb283&dl=0
Sorry for the late response! I had a busy week, The files are too big to just upload to github so I made a dropbox repo with a zip file inside. .csproj arent supported for upload through git disscusions :/.

[ramifayad105]

@natke
// done Add required packages

// Add using statements

// Create a Cancellation Token
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
using Microsoft.AI.Foundry.Local;
using Microsoft.Extensions.Logging.Abstractions;
CancellationToken ct = new CancellationToken();

// Create foundry local configuration
var config = new Configuration
{
AppName = "Simple Chatbot",
LogLevel = LogLevel.Information,
};
// Initialize FoundryLocalManager - singleton instance
await FoundryLocalManager.CreateAsync(config, NullLogger.Instance);
var mgr = FoundryLocalManager.Instance;

// Retrieve available EPs
var eps = mgr.DiscoverEps();
foreach (var ep in eps)
{
Console.WriteLine($"EP: {ep.Name}");
}

// Download and register EPs

await mgr.DownloadAndRegisterEpsAsync((epname, percent) =>
{
Console.WriteLine($"Percent downloaded {percent} - {epname}");
});

// Retrieve the model catalogue
var catalog = await mgr.GetCatalogAsync();
// show list of models
var models = await catalog.ListModelsAsync();
foreach (var m in models)
{
Console.WriteLine($"Model: {m.Alias}");
}
// retrive model info for a specific model for the catalog
var model = await catalog.GetModelAsync("qwen2.5-0.5b");
if(model is null)
{
Console.WriteLine("Model not found");
return;
}

// Download the model
await model.DownloadAsync(progress =>
{
Console.WriteLine($"Percent downloaded {progress:F2}");
});

// Load the model
await model.LoadAsync();

// Retrieve chat client from the model
var chatClient = await model.GetChatClientAsync();

// Create list of chat messages to maintain conversation history
List messages = new();

Console.WriteLine("\n Type your prompt and press Enter to chat.");
Console.WriteLine("Press Enter to exit.\n");

// Interactive loop - sentinel: pressing Enter with no input exits
while (true)
{
Console.Write("You: ");
string? userInput = Console.ReadLine();

// Sentinel event: empty input (just pressed Enter) terminates the loop
if (string.IsNullOrWhiteSpace(userInput))
{
    Console.WriteLine("No input detected. Exiting chatbot...");
    break;
}

// Add user message to conversation history
messages.Add(new ChatMessage { Role = "user", Content = userInput });

// Complete the chat - receive the LLM response
var response = await chatClient.CompleteChatAsync(messages, ct);
string assistantReply = response.Choices[0].Message.Content;

// Add assistant response to history so the model has context
messages.Add(new ChatMessage { Role = "assistant", Content = assistantReply });

Console.WriteLine($"\nAssistant: {assistantReply}\n");

}

// Unload the model before the application ends
await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");

//This is just the project.cs file

[baijumeswani]

Any chance you could also share the logs from ~/.Simple\ Chatbot/? What EP are you using to run the models (I am guessing VitisAI ep, but want to be sure)?