[Bug]: Tomato Penetration hangs for over 5 hours using Qwen3.5-27b

[zedyasuotalong]

Affected Component

AI Agents (Researcher/Developer/...)

Describe the bug

Using Qwen3.5-27b to perform a penetration test on the tomato target machine(https://www.vulnhub.com/entry/tomato-1,557/), it has been running for over 5 hours without stopping, and the task/process is less than half complete.

From the execution process of the flow, it can be observed that it repeatedly executes some commands, seemingly trapped in a loop. I'm not sure if it's due to the agent not being well-planned or the model used being too weak. It feels like it will take a long time to complete if it continues executing. Therefore, I temporarily stopped this flow. I would like to ask, how is the execution effect of Qwen3.5-27B in your use? Could you give me some suggestions? Thank you

Steps to Reproduce

1、The penetration testing process utilized the API_KEY from the Alibaba Bailian platform and the model includes qwen3.5-27b and text-embedding-v3. The testing process demonstrates that the model used is functioning properly.
2、The tomato host(https://www.vulnhub.com/entry/tomato-1,557/) is set up within the same local area network. During the penetration testing process, the pentagi terminal can access it normally, including IP access, port scanning, and so on.
3、Create an automation flow and the penetration prompt is "Perform penetration testing on the host 192.168.136.136"

System Configuration

[.env]
LLM_SERVER_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
LLM_SERVER_KEY=sk-xxx
LLM_SERVER_MODEL=qwen3.5-27b
LLM_SERVER_PROVIDER=
LLM_SERVER_CONFIG_PATH=/opt/pentagi/conf/custom.provider.yml

EMBEDDING_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
EMBEDDING_KEY=sk-xxx
EMBEDDING_MODEL=text-embedding-v3
EMBEDDING_PROVIDER=openai
EMBEDDING_BATCH_SIZE=10

[custom.provider.yml]
simple:
model: "qwen3.5-27b"
temperature: 0.7
top_k: 20
top_p: 0.8
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

simple_json:
model: "qwen3.5-27b"
temperature: 0.7
top_k: 20
top_p: 0.8
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
json: true
extra_body:
chat_template_kwargs:
enable_thinking: false

primary_agent:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

assistant:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

generator:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

refiner:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

adviser:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

reflector:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

searcher:
model: "qwen3.5-27b"
temperature: 0.7
top_k: 20
top_p: 0.8
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

enricher:
model: "qwen3.5-27b"
temperature: 0.7
top_k: 20
top_p: 0.8
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

coder:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

installer:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

pentester:
model: "qwen3.5-27b"
temperature: 1.0
top_k: 20
top_p: 0.95
min_p: 0.0
presence_penalty: 1.5
repetition_penalty: 1.0
n: 1
max_tokens: 32768
extra_body:
chat_template_kwargs:
enable_thinking: false

Logs and Artifacts

No response

Screenshots or Recordings

No response

Verification

• I have checked that this issue hasn't been already reported
• I have provided all relevant configuration files (with sensitive data removed)
• I have included relevant logs and error messages
• I am running the latest version of PentAGI

[zedyasuotalong]

Thanks for the recommendation @mason5052. PentAGI can find the LFI vulnerability in Tomato, but after several trying, pentagi couldn't obtain a rebound shell through remote code execution (RCE) Using the following prompt:

Conduct a penetration test on the target 192.168.136.136, enumerate ports and services using nmap, identify Web applications and directory structures, find LFI vulnerabilities and exploit them to read sensitive files, and ultimately obtain a rebound shell. Fail commands should not be repeated more than twice. Each utilization method should be attempted at most three times. Do not attempt to escalate privileges. Stop immediately after obtaining a rebound shell and output a summary report including the LFI entry point, payload, and confirmation of the rebound shell connection.