Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 166 additions & 0 deletions duui-open-webui/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@

[![Version](https://img.shields.io/static/v1?label=duui-multimodal\&message=0.1.0\&color=blue)](https://docker.texttechnologylab.org/v2/duui-multimodal/tags/list)
[![Python](https://img.shields.io/static/v1?label=Python\&message=3.12\&color=green)]()
[![Transformers](https://img.shields.io/static/v1?label=Transformers\&message=4.48.2\&color=yellow)]()
[![Torch](https://img.shields.io/static/v1?label=Torch\&message=2.6.0\&color=red)]()

# DUUI Open-WebUI

DUUI implementation for **multimodal Hugging Face models** that support combinations of:

* Text
* Image



---


## Supported Models services

| Model Name | Source | Dockerimage | Mode | Lang | Version |
|------------|--------------------------------------------------------------------------------------------------|-------------|------------|---------------------------------|---------|
| ollama | https://docs.ollama.com/api/openai-compatibility | NA | image/text | multi | 0.0.1 |

---

## Supported Modes

| Mode | Description |
|---------|---------------------------------------------------------------------|
| `text` | Process raw text prompts |
| `image` | Process images and prompt combinations |

---

## How To Use

Requires the [Docker Unified UIMA Interface (DUUI)](https://github.com/texttechnologylab/DockerUnifiedUIMAInterface).

### Start Docker Container

```bash
docker run -p 9714:9714 docker.texttechnologylab.org/duui-open-webui
```

Find available image tags: [Docker Registry](https://docker.texttechnologylab.org/v2/duui-mutlimodality-transformer/tags/list)

---

## Use within DUUI

### ollama setup
```java
composer.add(
new DUUIRemoteDriver.Component(url)
.withParameter("model_name", "qwen2.5vl:3b")
.withParameter("mode", "image")
.withParameter("language", "en")
.withParameter("ollama_host", "localhost") // https:/llm.example
// .withParameter("ollama_port", "8080")
.withParameter("ollama_auth_token", "")
.withParameter("system_prompt", "")
.build().withTimeout(1000)
);
```
### Transformer Models

```java
// Code before as it is..

List<String> prompts = Arrays.asList(
"Who is the current president of the USA?",
"Is Frankfurt the capital of EU finance?"
);

createCas("en", prompts);
composer.run(cas);

verifyNoImages();

// Print results
for (Result result : JCasUtil.select(cas, Result.class)) {
System.out.println(result.getMeta());
}
//
// Helper method to create CAS with prompts
public void createCas(String language, List<String> prompts) throws UIMAException {
cas.setDocumentLanguage(language);
StringBuilder sb = new StringBuilder();

for (String messageText : prompts) {
Prompt prompt = new Prompt(cas);
prompt.setArgs("{}");

Message message = new Message(cas);
message.setRole("user");
message.setContent(messageText);
message.addToIndexes();

FSArray messages = new FSArray(cas, 1);
messages.set(0, message);
prompt.setMessages(messages);
prompt.addToIndexes();

sb.append(messageText).append(" ");
}

inputView.setDocumentText(sb.toString().trim());
// cas.setDocumentText(sb.toString().trim());
}
```



---

## Parameters

| Name | Description |
| ------------ |--------------------------------------------------------|
| `model_name` | Name of the multimodal model to use (inside ollama) |
| `mode` | Processing mode: text, image |
| `ollama_host` | ollama host url |
| `ollama_port` | ollama port, default 8080 |
| `ollama_auth_token`| ollama auth token if exists, default empty |
| `system_prompt` | System prompt for all prompts if needed, default empty |

---

## Cite

If you want to use the DUUI image, please cite the following:

**Leonhardt et al. (2023)**
*"Unlocking the Heterogeneous Landscape of Big Data NLP with DUUI."*
Findings of the Association for Computational Linguistics: EMNLP 2023, 385–399.
\[[LINK](https://aclanthology.org/2023.findings-emnlp.29)] \[[PDF](https://aclanthology.org/2023.findings-emnlp.29.pdf)]

**Abusaleh (2026)**
*"OpenWebUI wrapper as {DUUI} Component"*
\[[LINK](https://github.com/texttechnologylab/duui-uima/tree/main/duui-open-webui])]
---

## BibTeX

```bibtex
@inproceedings{Leonhardt:et:al:2023,
title = {Unlocking the Heterogeneous Landscape of Big Data {NLP} with {DUUI}},
author = {Leonhardt, Alexander and Abrami, Giuseppe and Baumartz, Daniel and Mehler, Alexander},
booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2023},
year = {2023},
address = {Singapore},
publisher = {Association for Computational Linguistics},
url = {https://aclanthology.org/2023.findings-emnlp.29},
pages = {385--399},
pdf = {https://aclanthology.org/2023.findings-emnlp.29.pdf}
}

@misc{abusaleh:duui:openwebui:2026,
author = {Abusaleh, Ali},
title = {OpenWebUI Ollama wrapper as {DUUI} Component},
year = {2026},
howpublished = {https://github.com/texttechnologylab/duui-uima/tree/main/duui-open-webui}
}


26 changes: 26 additions & 0 deletions duui-open-webui/bash_dockers/docker_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail


export MM_ANNOTATOR_CUDA=transformer
#export DUUI_MM_CUDA="-cuda"

export MM_ANNOTATOR_NAME=duui-openwebui
export MM_ANNOTATOR_VERSION=0.2.0
export MM_LOG_LEVEL=DEBUG
export MM_MODEL_CACHE_SIZE=3
export DOCKER_REGISTRY="docker.texttechnologylab.org/"

cd ..

docker build \
--build-arg MM_ANNOTATOR_NAME \
--build-arg MM_ANNOTATOR_VERSION \
--build-arg MM_LOG_LEVEL \
-t ${DOCKER_REGISTRY}${MM_ANNOTATOR_NAME}-${MM_ANNOTATOR_CUDA}:${MM_ANNOTATOR_VERSION} \
-f src/main/docker/Dockerfile${MM_ANNOTATOR_CUDA} \
.

docker tag \
${DOCKER_REGISTRY}${MM_ANNOTATOR_NAME}-${MM_ANNOTATOR_CUDA}:${MM_ANNOTATOR_VERSION} \
${DOCKER_REGISTRY}${MM_ANNOTATOR_NAME}-${MM_ANNOTATOR_CUDA}:latest
26 changes: 26 additions & 0 deletions duui-open-webui/bash_dockers/docker_build_vlm.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail


export MM_ANNOTATOR_CUDA=vllm
#export DUUI_MM_CUDA="-cuda"

export MM_ANNOTATOR_NAME=duui-openwebui
export MM_ANNOTATOR_VERSION=0.1.0
export MM_LOG_LEVEL=DEBUG
export MM_MODEL_CACHE_SIZE=3
export DOCKER_REGISTRY="docker.texttechnologylab.org/"

cd ..

docker build \
--build-arg MM_ANNOTATOR_NAME \
--build-arg MM_ANNOTATOR_VERSION \
--build-arg MM_LOG_LEVEL \
-t ${DOCKER_REGISTRY}${MM_ANNOTATOR_NAME}-${MM_ANNOTATOR_CUDA}:${MM_ANNOTATOR_VERSION} \
-f src/main/docker/Dockerfile${MM_ANNOTATOR_CUDA} \
.

docker tag \
${DOCKER_REGISTRY}${MM_ANNOTATOR_NAME}-${MM_ANNOTATOR_CUDA}:${MM_ANNOTATOR_VERSION} \
${DOCKER_REGISTRY}${MM_ANNOTATOR_NAME}-${MM_ANNOTATOR_CUDA}:latest
162 changes: 162 additions & 0 deletions duui-open-webui/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.texttechnology</groupId>
<artifactId>duui-WebUI-Wrapper</artifactId>
<version>1.0-SNAPSHOT</version>
<licenses>
<license>
<name>AGPL-3.0-or-later</name>
<url>https://www.gnu.org/licenses/agpl.txt</url>
<distribution>repo</distribution>
<comments>GNU Affero General Public License v3.0 or later</comments>
</license>
</licenses>

<organization>
<name>Texttechnology Lab</name>
<url>https://www.texttechnologylab.org</url>
</organization>
<developers>
<developer>
<id>mehler</id>
<name>Prof. Dr. Alexander Mehler</name>
<email>mehler@em.uni-frankfurt.de</email>
<url>https://www.texttechnologylab.org/team/alexander-abrami/</url>
<organization>Goethe University Frankfurt / Texttechnology Lab</organization>
<organizationUrl>https://www.texttechnologylab.org</organizationUrl>
<roles>
<role>head of department</role>
</roles>
</developer>
<developer>
<id>aabusale</id>
<name>Ali Abusaleh</name>
<email>a.abusaleh@em.uni-frankfurt.de</email>
<url>https://www.texttechnologylab.org/team/ali-abusaleh/</url>
<organization>Goethe University Frankfurt / Texttechnology Lab</organization>
<organizationUrl>https://www.texttechnologylab.org</organizationUrl>
<roles>
<role>Research assistant</role>
</roles>
<timezone>Europe/Berlin</timezone>
</developer>
</developers>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<argLine>
--illegal-access=permit
--add-opens java.base/java.util=ALL-UNNAMED
<!-- add-opens for use in JUnit-Tests...-->
</argLine>
</configuration>
</plugin>
</plugins>
</build>

<properties>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<dkpro.core.version>2.4.0</dkpro.core.version>
</properties>

<repositories>
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
</repositories>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.dkpro.core</groupId>
<artifactId>dkpro-core-asl</artifactId>
<version>${dkpro.core.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<!--<dependency>
<groupId>com.github.texttechnologylab</groupId>
<artifactId>DockerUnifiedUIMAInterface</artifactId>
<version>${ttlab.duui.version}</version>
</dependency>-->

<dependency>
<groupId>com.github.texttechnologylab</groupId>
<artifactId>UIMATypeSystem</artifactId>
<version>3.0.10</version>
</dependency>
<dependency>
<groupId>com.github.texttechnologylab</groupId>
<artifactId>DockerUnifiedUIMAInterface</artifactId>
<version>1.4.6</version>
</dependency>
<!-- <dependency>-->
<!-- <groupId>com.github.texttechnologylab.textimager-uima</groupId>-->
<!-- <artifactId>textimager-uima-util</artifactId>-->
<!-- <version>${ttlab.textimager.typesystem.version}</version>-->
<!-- </dependency>-->



<!-- <dependency>-->
<!-- <groupId>org.texttechnologylab.annotation</groupId>-->
<!-- <artifactId>typesystem</artifactId>-->
<!-- <version>3.0.1</version>-->
<!-- </dependency>-->

<!-- <dependency>-->
<!-- <groupId>org.texttechnologylab</groupId>-->
<!-- <artifactId>DockerUnifiedUIMAInterface</artifactId>-->
<!-- <version>1.3</version>-->
<!-- </dependency>-->

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.9.0</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.dkpro.core</groupId>
<artifactId>dkpro-core-api-segmentation-asl</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.dkpro.core</groupId>
<artifactId>dkpro-core-io-xmi-asl</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.dkpro.core</groupId>
<artifactId>dkpro-core-io-json-asl</artifactId>
<version>2.0.0</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.dkpro.core</groupId>
<artifactId>dkpro-core-api-resources-asl</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

</project>
Loading