Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions Demos/Emoji-Gemma-on-Web/app-mediapipe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,28 @@ This app demonstrates how to generate emojis from a text input using a fine-tune
1. Download the app files in this directory and include your .task model bundle in the local app folder.
2. In the worker.js file, update the `modelPath` to point to the .task file.
3. Open terminal on your computer and navigate (`cd`) to the app folder.
4. Run `npx serve` to start the local server.
4. To run the app, you need a local web server. For development and testing, it is recommended to use a secure (HTTPS) server, as WebGPU may not be available in non-secure contexts.
1. **For a secure server:**
1. If you don't have a local certificate, create one by running this command in your terminal at the root of this project (/app-mediapipe):
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj "/C=US/ST=CA/L=Mountain View/O=Dev/CN=localhost"
```
Comment on lines +12 to +14
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The openssl command includes a very specific subject. While this works, a simpler subject focusing only on the Common Name (CN) is sufficient for a local development certificate and is less confusing for users who might think they need to change these values.

Suggested change
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj "/C=US/ST=CA/L=Mountain View/O=Dev/CN=localhost"
```
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj "/CN=localhost"
```

2. Start the server with the following command:
```bash
npx http-server -S -c-1 -p 8080
```
2. **For a non-secure server:**
```bash
npx serve
```
5. Open the provided `localhost` address in your browser to run the app.

## How it works
This demo sets up a simple web server to host a frontend where users can enter a text prompt. This starts a generation process in a web worker to avoid blocking the main UI thread. The worker uses a bundled version of the MediaPipe Tasks GenAI package ([@mediapipe/tasks-genai](https://www.npmjs.com/package/@mediapipe/tasks-genai)) to generate a response from the model and send it back to the user.

**Requirements:** Browser with [WebGPU support](https://caniuse.com/webgpu)
**Requirements:** Browser with [WebGPU support](https://caniuse.com/webgpu). Note that some platforms, like iOS Safari, require a secure context (HTTPS) to enable WebGPU.

**Device Limitations:** Running language models in the browser is memory-intensive. While this demo may work on high-end desktop machines, some memory-constrained devices, especially on iOS, may fail to load the model and cause the page to crash. This is due to strict memory limits imposed by the operating system on browser tabs. Even when running with 16-bit float precision, the Gemma 270M model can exceed these limits.

## Resources
* [Notebook: Fine-tune Gemma 3 270M](https://github.com/google-gemini/gemma-cookbook/blob/main/Demos/Emoji-Gemma-on-Web/resources/Fine_tune_Gemma_3_270M_for_emoji_generation.ipynb)
Expand Down
9 changes: 5 additions & 4 deletions Demos/Emoji-Gemma-on-Web/app-mediapipe/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Emoji generator</title>
<link rel="preload" href="worker.js" />
<link rel="preload" href="worker.js" as="worker" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Google+Sans:wght@400;600;800&family=Noto+Color+Emoji&display=swap"
Expand All @@ -23,13 +23,14 @@
<h1 class="google-sans-medium">Emoji generator</h1>
<p class="text-cta">Translate text to fun combinations of emoji with your own Gemma model ✨</p>
<div class="input-container">
<input type="text" id="prompt-input" aria-label="Type any phrase and get suggestions" placeholder="Type any phrase and get suggestions">
<input type="text" id="prompt-input" aria-label="Type any phrase and get suggestions"
placeholder="Type any phrase and get suggestions">
<button disabled type="submit" id="generate-btn" aria-label="Generate emojis">
<svg class="arrow" fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 10 16">
<path d="M.59 14.59 2 16l8-8-8-8L.59 1.41 7.17 8" fill="#fff" />
</svg>
<svg class="loader" width="18" height="18" viewBox="0 0 18 18" fill="none"
xmlns="http://www.w3.org/2000/svg">
xmlns="http://www.w3.org/2000/svg">
<circle cx="9" cy="9" r="8" stroke="white" stroke-width="2" />
</svg>
</button>
Expand All @@ -49,4 +50,4 @@ <h1 class="google-sans-medium">Emoji generator</h1>
<script type="module" src="script.js"></script>
</body>

</html>
</html>
23 changes: 16 additions & 7 deletions Demos/Emoji-Gemma-on-Web/app-mediapipe/script.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,20 @@ function createEmojiButton(result) {
// Check for WebGPU availability and displays an error message if it's not available
function checkWebGPU() {
if (!navigator.gpu) {
statusMessageContainer.innerHTML = `
let message = `
This application requires WebGPU to run the model in your browser.
Please update your browser to enable WebGPU support, try a different browser (like Chrome or Edge, version 113+), or ensure your GPU drivers are up to date.</br></br>
For more details, read the <a href="https://github.com/gpuweb/gpuweb/wiki/Implementation-Status" target="_blank" style="f; color: #286aac">WebGPU Implementation Status</a>.
For more details, read the <a href="https://github.com/gpuweb/gpuweb/wiki/Implementation-Status" target="_blank" style="color: #286aac">WebGPU Implementation Status</a>.
`;

if (!window.isSecureContext) {
message += `
<br/><br/>
<strong>Note:</strong> WebGPU requires a secure context. Please access this page over HTTPS or from 'localhost'. Accessing a local server via its IP address also requires an HTTPS setup in many instances (e.g. iOS26 Safari).
`;
Comment on lines +36 to +39
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There appears to be a typo "iOS26" in the note. It's better to either correct it to a specific version or make it more general, like 'iOS Safari', to avoid it becoming outdated.

Suggested change
message += `
<br/><br/>
<strong>Note:</strong> WebGPU requires a secure context. Please access this page over HTTPS or from 'localhost'. Accessing a local server via its IP address also requires an HTTPS setup in many instances (e.g. iOS26 Safari).
`;
message += `
<br/><br/>
<strong>Note:</strong> WebGPU requires a secure context. Please access this page over HTTPS or from 'localhost'. Accessing a local server via its IP address also requires an HTTPS setup in many instances (e.g. iOS Safari).
`;

}

statusMessageContainer.innerHTML = message;
generateBtn.disabled = true;
return false;
}
Expand All @@ -40,7 +49,7 @@ function checkWebGPU() {
// Main function to run worker
async function initializeModelInWorker() {
console.log("[UI] Initializing application...");

// Create a simulated progress loader that clears once the model loads
if (!checkWebGPU()) {
return;
Expand Down Expand Up @@ -70,18 +79,18 @@ async function initializeModelInWorker() {
statusMessageContainer.textContent = `Loading model (100%)`; // Show 100% briefly
modelReady = true;
generateBtn.disabled = false; // Enable the generation button upon model load
setTimeout(() => {
setTimeout(() => {
statusMessageContainer.innerHTML = ``; // Then empty status message
}, 500);
break;

case "result":
const line = data.trim();
if (line) {
createEmojiButton(line);
}
break;

case "complete":
responseComplete();
break;
Expand Down Expand Up @@ -110,7 +119,7 @@ async function initializeModelInWorker() {
function responseComplete() {
generateBtn.classList.remove('generating');
generateBtn.disabled = false;

if (responseOutput.childElementCount === 0) {
statusMessageContainer.textContent = "No results";
} else {
Expand Down
21 changes: 18 additions & 3 deletions Demos/Emoji-Gemma-on-Web/app-transformersjs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,29 @@ This app demonstrates how to generate emojis from a text input using a fine-tune
2. In the worker.js file, update the model string in the `pipeline()` function call to the model on Hugging Face Hub.
1. Alternatively, download and place the model files in a new subdirectory i.e. `app-transformersjs/myemoji-gemma-3-270m-it-onnx/` for full offline use.
3. Open terminal on your computer and navigate (`cd`) to the app folder.
4. Run `npx serve` to start the local server.
5. Open the provided `localhost` address in your browser to run the app.
4. To run the app, you need a local web server. For development and testing, it is recommended to use a secure (HTTPS) server, as WebGPU may not be available in non-secure contexts.
1. **For a secure server:**
1. If you don't have a local certificate, create one by running this command in your terminal at the root of this project (/app-transformersjs):
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj "/C=US/ST=CA/L=Mountain View/O=Dev/CN=localhost"
```
Comment on lines +13 to +15
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The openssl command includes a very specific subject. While this works, a simpler subject focusing only on the Common Name (CN) is sufficient for a local development certificate and is less confusing for users who might think they need to change these values.

Suggested change
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj "/C=US/ST=CA/L=Mountain View/O=Dev/CN=localhost"
```
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj "/CN=localhost"
```

2. Start the server with the following command:
```bash
npx http-server -S -c-1 -p 8080
```
2. **For a non-secure server:**
```bash
npx serve
```
5. Open the provided `localhost` address in your browser to run the app (you may have to accept untrusted SSL warning in your browser).

**Requirements:** Browser with [WebGPU support](https://caniuse.com/webgpu)
**Requirements:** Browser with [WebGPU support](https://caniuse.com/webgpu). Note that some platforms, like iOS Safari, require a secure context (HTTPS) to enable WebGPU.

## How it works
This demo sets up a simple web server to host a frontend where users can enter a text prompt. This starts a generation process in a web worker to avoid blocking the main UI thread. The worker uses [Transformers.js](https://huggingface.co/docs/transformers.js/index) to generate a response from the model and send it back to the user.

**Device Limitations:** Running language models in the browser is memory-intensive. While this demo may work on high-end desktop machines, some memory-constrained devices, especially on iOS, may fail to load the model and cause the page to crash. This is due to strict memory limits imposed by the operating system on browser tabs. Even when running with 16-bit float precision, the Gemma 270M model can exceed these limits.

## Resources
* [Notebook: Fine-tune Gemma 3 270M](https://github.com/google-gemini/gemma-cookbook/blob/main/Demos/Emoji-Gemma-on-Web/resources/Fine_tune_Gemma_3_270M_for_emoji_generation.ipynb)
* [Notebook: Convert Gemma 3 270M to ONNX](https://github.com/google-gemini/gemma-cookbook/blob/main/Demos/Emoji-Gemma-on-Web/resources/Convert_Gemma_3_270M_to_ONNX.ipynb)
Expand Down
5 changes: 3 additions & 2 deletions Demos/Emoji-Gemma-on-Web/app-transformersjs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,14 @@
<h1 class="google-sans-medium">Emoji generator</h1>
<p class="text-cta">Translate text to fun combinations of emoji with your own Gemma model ✨</p>
<div class="input-container">
<input type="text" id="prompt-input" aria-label="Type any phrase and get suggestions" placeholder="Type any phrase and get suggestions">
<input type="text" id="prompt-input" aria-label="Type any phrase and get suggestions"
placeholder="Type any phrase and get suggestions">
<button disabled type="submit" id="generate-btn" aria-label="Generate emojis">
<svg class="arrow" fill="none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 10 16">
<path d="M.59 14.59 2 16l8-8-8-8L.59 1.41 7.17 8" fill="#fff" />
</svg>
<svg class="loader" width="18" height="18" viewBox="0 0 18 18" fill="none"
xmlns="http://www.w3.org/2000/svg">
xmlns="http://www.w3.org/2000/svg">
<circle cx="9" cy="9" r="8" stroke="white" stroke-width="2" />
</svg>
</button>
Expand Down