From 0434a8734588ff37a4598750b156553472acd095 Mon Sep 17 00:00:00 2001 From: Brage Date: Tue, 4 Aug 2026 11:00:01 +0200 Subject: [PATCH 1/3] News: Vespa cloud instead of local. HF dataset fix --- .../tutorials/news-1-deploy-an-application.md | 162 +++++++----------- .../news-2-basic-feeding-and-query.md | 93 +++++++--- en/learn/tutorials/news-3-searching.md | 2 +- en/learn/tutorials/news-5-recommendation.md | 33 ++-- .../news-6-recommendation-with-searchers.md | 2 +- ...news-7-recommendation-with-parent-child.md | 6 +- 6 files changed, 154 insertions(+), 144 deletions(-) diff --git a/en/learn/tutorials/news-1-deploy-an-application.md b/en/learn/tutorials/news-1-deploy-an-application.md index ca51b8d1f6..4c725ac433 100644 --- a/en/learn/tutorials/news-1-deploy-an-application.md +++ b/en/learn/tutorials/news-1-deploy-an-application.md @@ -1,6 +1,6 @@ --- # Copyright Vespa.ai. All rights reserved. -title: "News search and recommendation tutorial - getting started on Docker" +title: "News search and recommendation tutorial - getting started on Vespa Cloud" redirect_from: - /en/tutorials/news-1-getting-started.html - /en/tutorials/news-1-deploy-an-application @@ -8,7 +8,7 @@ redirect_from: Our goal with this series is to set up a Vespa application for personalized -news recommendations. We will do this in stages, starting with a simple news +news recommendations on Vespa Cloud. We will do this in stages, starting with a simple news search system and gradually adding functionality as we go through the tutorial parts. @@ -22,32 +22,33 @@ The parts are: 6. [News recommendation with searchers](news-6-recommendation-with-searchers.html) - custom searchers, doc processors 7. [News recommendation with parent-child](news-7-recommendation-with-parent-child.html) - parent-child, tensor ranking -There are different entry points to this tutorial. This one is describing how to get -started using Docker on your local machine. You can also deploy the application we are creating on -[Vespa Cloud](https://cloud.vespa.ai). - -In this part, we will start with a minimal Vespa application to -get used to some basic operations for running the application on Docker. +In this part, we will start with a minimal Vespa application to get used to some basic +operations for deploying and running an application on Vespa Cloud. In the next part of the tutorial, we'll start developing our application. -{% include pre-req.html memory="4 GB" extra-reqs=' -
  • Python3 for converting the dataset to Vespa JSON.
    • -
  • curl to download the dataset and run the Vespa health-checks.
    • -
  • Java 17 in part 6.
    • -
  • Apache Maven in part 6.
    • ' %} - -{% include note.html content='4 GB Docker memory is sufficient for the demo dataset in part 2. -The full MIND dataset requires more, use 10 GB.' %} + In upcoming parts of this series, we will have some additional Python dependencies - we use [PyTorch](https://pytorch.org/) to train vector representations for news and users and train machine learning models for use in ranking. -## Installing vespa-cli +## Installing Vespa CLI This tutorial uses [Vespa-CLI](../../clients/vespa-cli.html), -Vespa CLI is the official command-line client for Vespa.ai. +Vespa CLI is the official command-line client for Vespa.ai. It is a single binary without any runtime dependencies and is available for Linux, macOS, and Windows.
    @@ -57,18 +58,21 @@ $ brew install vespa-cli
    -For the rest of this tutorial, you will be using localhost, so you need to configure your Vespa CLI to connect to localhost. -Run the following to use endpoints on localhost: +In this tutorial it is possible to use either a local or global +Vespa CLI configuration mode for configuration variables. Using +local configuration mode is generally recommended when working +with multiple distinct Vespa applications, but most parts of +this tutorial uses the same configuration values which makes +it easier to use global configuration mode: 
    -$ vespa config set target local
+$ vespa config set default_config_scope global
    - ## A minimal Vespa application This tutorial has a [companion sample application](https://github.com/vespa-engine/sample-apps/tree/master/news). @@ -105,58 +109,69 @@ There are two files there: We will revisit these files in the next part of the tutorial. -## Starting Vespa +## Configuring Vespa CLI for Vespa Cloud +Configure the Vespa CLI to use Vespa Cloud, and set the application name. +Replace `tenant-name` with your tenant name from [console.vespa-cloud.com](https://console.vespa-cloud.com): -This application doesn't contain much at the moment, -let's start up the application anyway by starting a Docker container to run it: +
    + +
    +$ vespa config set target cloud
+$ vespa config set application tenant-name.news
+
    +
    + +Usually its better to use local configuration for each application, +but this tutorial uses global configuration to avoid having to set the +configuration values for each part of the tutorial. + +Authenticate with Vespa Cloud:
    -
    -$ docker pull vespaengine/vespa
-$ docker run --detach --name vespa --hostname vespa-tutorial \
-  --publish 8080:8080 --publish 19071:19071 --publish 19092:19092 \
-  vespaengine/vespa
+
    +$ vespa auth login
    -First, we pull the latest [vespa-image](https://hub.docker.com/r/vespaengine/vespa/) -from the Docker hub, then we -start it with the name `vespa`. This starts the Docker container and the -initial Vespa services to be able to deploy an application. +Follow the browser instructions to complete authentication. -Starting the container can take a short while. Before continuing, make sure -that the configuration service is running by using `vespa status`. +Next, add a certificate for [data plane access](/en/security/guide#data-plane) to the application:
    -
    -$ vespa status deploy --wait 300 
+
    +$ vespa auth cert app-1-getting-started
    -With the config server up and running, deploy the application using vespa-cli: + + + +## Deploying to Vespa Cloud + +This application doesn't contain much at the moment, +but let's deploy it to Vespa Cloud anyway to get used to the basic operations. +The first deployment may take a few minutes while nodes are provisioned: 
    -$ vespa deploy --wait 300 app-1-getting-started 
+$ vespa deploy --wait 600 app-1-getting-started
    The command uploads the application and verifies the content. If anything is wrong with the application, this step will fail with a failure description; -Otherwise, this switches the application to a live status. +otherwise, this switches the application to a live status. -Whenever you have a new version of your application, +Whenever you have a new version of your application, run the same command to deploy the application. In most cases, there is no need to restart services. Vespa takes care of reconfiguring the system. -If a restart of services is required in some rare case, however, the output will notify -which services need restart to make the change effective. -In the upcoming parts of the tutorials, we'll frequently deploy the -application changes in this manner. +In the upcoming parts of the tutorials, we'll frequently deploy the +application changes in this manner. ## Feeding to Vespa @@ -167,7 +182,7 @@ For now, to test that everything is up and running, we'll feed in a single test 
    -$ vespa feed -t http://localhost:8080 doc.json
+$ vespa feed doc.json
    @@ -237,57 +252,12 @@ $ vespa document -v remove id:news:news::1 Well done! -## Stopping and starting Vespa - -Keep Vespa running to continue with the next steps in this tutorial set (skip the below). +## Managing the Vespa Cloud application -To stop Vespa, we can run the following commands: - -
    - -
    -$ docker exec vespa vespa-stop-services
-$ docker exec vespa vespa-stop-configserver
-
    -
    - -Likewise, to start the Vespa services: - -
    - -
    -$ docker exec vespa vespa-start-configserver
-$ docker exec vespa vespa-start-services
-
    -
    - -If a [restart is required](../../reference/schemas/schemas.html#changes-that-require-restart-but-not-re-feed) -due to changes in the application package, -these two steps are what you need to do. - -To wipe the index and restart: - -
    - -
    -$ docker exec vespa sh -c ' \
-  vespa-stop-services && \
-  vespa-remove-index -force && \
-  vespa-start-services'
-
    -
    - -You can stop and kill the Vespa container application like this: - -
    - -
    -$ docker stop vespa; docker rm -f vespa
-
    -
    +Application instances in the [dev zone](../operations/environments.html#dev) will by default keep running for 14 days after the last deployment. +You can control this in the [console](https://console.vespa-cloud.com/). -This will delete the Vespa application, including all data and configuration. See -[container tuning for production](../../operations/self-managed/docker-containers.html). +The [Vespa Cloud console](https://console.vespa-cloud.com) can also be used to delte the application instance. ## Conclusion diff --git a/en/learn/tutorials/news-2-basic-feeding-and-query.md b/en/learn/tutorials/news-2-basic-feeding-and-query.md index 7f765496ff..64b12d06c6 100644 --- a/en/learn/tutorials/news-2-basic-feeding-and-query.md +++ b/en/learn/tutorials/news-2-basic-feeding-and-query.md @@ -56,7 +56,33 @@ news content at first. We'll use the impression data as we begin building the recommendation system later in this series. Let's start by downloading the data. The `news` sample app directory will -be our starting point. We've included a script to download the data for us: +be our starting point. The MIND dataset is hosted on +[Hugging Face](https://huggingface.co/datasets/yjw1029/MIND) and requires +a free account and acceptance of the dataset's terms of use. + +### Accept the dataset terms + +1. Log in at [huggingface.co](https://huggingface.co) +2. Go to the [MIND dataset page](https://huggingface.co/datasets/yjw1029/MIND) +3. Click **Agree and access repository** to accept the terms + +### Create a Hugging Face access token + +1. Go to [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) +2. Click **New token**, give it a name, select **Read** role, and copy the token + +### Download the dataset + +Set your token as an environment variable: + +
    + +
    +$ export HF_TOKEN=hf_your_token_here
+
    +
    + +Then run the download script from the **news** sample app directory:
    @@ -110,12 +136,6 @@ ranking will be done and how data will be processed during feeding and indexing. The schema, e.g., `news.sd`, is a required part of an application package — the other file needed is `services.xml`. -For self-hosted multi-node deployments, a `hosts.xml` file is also needed. -For multi-node self-hosted deployments using `hosts.xml`, see -the [multinode high -availability](https://github.com/vespa-engine/sample-apps/tree/master/examples/operations/multinode-HA) -sample application. - We mentioned these files in the previous part but didn't really explain them at the time. We'll go through them here, starting with the specification of services. @@ -135,9 +155,7 @@ service. Write the following to `news/my-app/services.xml`: <container id="default" version="1.0"> <search /> <document-api /> - <nodes> - <node hostalias="node1" /> - </nodes> + <nodes count="1" /> </container> <content id="mind" version="1.0"> @@ -145,9 +163,7 @@ service. Write the following to `news/my-app/services.xml`: <documents> <document type="news" mode="index" /> </documents> - <nodes> - <node hostalias="node1" distribution-key="0" /> - </nodes> + <nodes count="1" /> </content> </services> @@ -163,13 +179,13 @@ Quite a lot is set up here: for how to use mTLS with Vespa. - `` sets up the [document endpoint](../../reference/api/document-v1.html) for feeding and visiting. -- `` defines the nodes required per service. (See the +- `` defines the number of nodes for the service. (See the [reference](../../reference/applications/services/container.html) for more on container cluster setup). - `` The stateful content cluster - `` denotes how many copies to store of each document. - `` assigns the document types in the _schema_ — the content - cluster capacity can be increased by adding node elements — see [elasticity](../../content/elasticity.html). (See also the + cluster capacity can be increased by raising the node count — see [elasticity](../../content/elasticity.html). (See also the [reference](../../reference/applications/services/content.html) for more on content cluster setup.) @@ -278,10 +294,40 @@ my-app/ └── services.xml +If not already done from the previous [news tutorial](news-1-deploy-an-application.html), set Vespa configuration variables. Use your tenant name from the [Vespa Cloud console](https://console.vespa-cloud.com/) instead of `tenant-name`: + +
    + +
    +$ vespa config set target cloud
+$ vespa config set application tenant-name.news
+
    +
    + +Log in to Vespa Cloud: + +
    + +
    +$ vespa auth login
+
    +
    + +Add security credentials to the application package for data plane access: + +
    + +
    +$ vespa auth cert my-app -f
+
    +
    + +Finally, deploy the application package to Vespa Cloud: + 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    @@ -307,7 +353,7 @@ file can now be fed to Vespa. Use the method described in the previous part: 
    -$ vespa feed mind/vespa.json --target http://localhost:8080
+$ vespa feed mind/vespa.json
    @@ -364,14 +410,19 @@ $ vespa query -v 'yql=select * from news where default contains "music"'
    -or a POST JSON query (Notice the *Content-Type* header specification): +or a POST JSON query. The values can be found by using the previous `vespa query -v` command. +The content type should be changed to `application/json`. +It's also possible to use `vespa status` to find the url of your cloud instance. +: 
    -$ curl -s -H "Content-Type: application/json" \
-  --data '{"yql" : "select * from sources * where default contains \"music\""}' \
-  http://localhost:8080/search/ | python3 -m json.tool
+$ curl --key /path/to/key \
+     --cert /path/to/cert \
+     -H 'Content-Type: application/json' \
+     --data '{"yql" : "select * from sources * where default contains \"music\""}' \
+     'https://your-vespa-url.vespa-app.cloud/search/' | python3 -m json.tool
    diff --git a/en/learn/tutorials/news-3-searching.md b/en/learn/tutorials/news-3-searching.md index 0513ed5f61..0140d5fa40 100644 --- a/en/learn/tutorials/news-3-searching.md +++ b/en/learn/tutorials/news-3-searching.md @@ -532,7 +532,7 @@ Deploy the _popularity_ rank profile: 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    diff --git a/en/learn/tutorials/news-5-recommendation.md b/en/learn/tutorials/news-5-recommendation.md index df6f55781c..1230b62015 100644 --- a/en/learn/tutorials/news-5-recommendation.md +++ b/en/learn/tutorials/news-5-recommendation.md @@ -177,9 +177,7 @@ so we modify `services.xml` and add it under `documents` in the `content` sectio <container id="default" version="1.0"> <search /> <document-api /> - <nodes> - <node hostalias="node1" /> - </nodes> + <nodes count="1" /> </container> <content id="mind" version="1.0"> @@ -188,9 +186,7 @@ so we modify `services.xml` and add it under `documents` in the `content` sectio <document type="news" mode="index" /> <document type="user" mode="index" /> </documents> - <nodes> - <node hostalias="node1" distribution-key="0" /> - </nodes> + <nodes count="1" /> </content> </services> @@ -200,7 +196,7 @@ so we modify `services.xml` and add it under `documents` in the `content` sectio 
    -$ vespa deploy --wait 300 my-app 
+$ vespa deploy --wait 600 my-app
    @@ -215,8 +211,8 @@ feed `mind/vespa_user_embeddings.json` and `mind/vespa_news_embeddings.json`: 
    -$ vespa feed mind/vespa_user_embeddings.json --target http://localhost:8080
-$ vespa feed mind/vespa_news_embeddings.json --target http://localhost:8080
+$ vespa feed mind/vespa_user_embeddings.json
+$ vespa feed mind/vespa_news_embeddings.json
    @@ -292,7 +288,7 @@ Deploy the updates to query profiles: 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    @@ -528,27 +524,20 @@ restart is required so that the index can be built: 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    -Introducing the HNSW `index` requires a content node restart, in this case we restart all services: +Introducing the HNSW `index` requires a content node restart. +On Vespa Cloud, this is handled automatically after deployment — +wait for the deployment to complete and the application to become ready: -
    - -
    -$ docker exec vespa /usr/bin/sh -c \
-  '/opt/vespa/bin/vespa-stop-services && /opt/vespa/bin/vespa-start-services'
-
    -
    - - 
     $ vespa status --wait 300
    -After doing this and waiting a bit for Vespa to start, we can query Vespa again: +After the deployment completes, we can query Vespa again:
    diff --git a/en/learn/tutorials/news-6-recommendation-with-searchers.md b/en/learn/tutorials/news-6-recommendation-with-searchers.md index 8eb524680c..8c5e391397 100644 --- a/en/learn/tutorials/news-6-recommendation-with-searchers.md +++ b/en/learn/tutorials/news-6-recommendation-with-searchers.md @@ -275,7 +275,7 @@ This contains the full Vespa application, with Java components - deploy it: 
    -$ vespa deploy --wait 300 app-6-recommendation-with-searchers
+$ vespa deploy --wait 600 app-6-recommendation-with-searchers
    diff --git a/en/learn/tutorials/news-7-recommendation-with-parent-child.md b/en/learn/tutorials/news-7-recommendation-with-parent-child.md index 8f26c61fef..6ad8d1ccf0 100644 --- a/en/learn/tutorials/news-7-recommendation-with-parent-child.md +++ b/en/learn/tutorials/news-7-recommendation-with-parent-child.md @@ -287,7 +287,7 @@ $ (cd app-7-parent-child && mvn package) 
    -$ vespa deploy --wait 300 app-7-parent-child
+$ vespa deploy --wait 600 app-7-parent-child
    @@ -320,8 +320,8 @@ Feed the created feed files: 
    -$ vespa feed mind/global_category_ctr.json --target http://localhost:8080
-$ vespa feed mind/news_category_ctr_update.json --target http://localhost:8080
+$ vespa feed mind/global_category_ctr.json
+$ vespa feed mind/news_category_ctr_update.json
    From e6c5487947ea6aa34cdd98f334d830bc39af54d8 Mon Sep 17 00:00:00 2001 From: Brage Kvamme <91749638+BrageHK@users.noreply.github.com> Date: Tue, 4 Aug 2026 15:05:26 +0200 Subject: [PATCH 2/3] Update en/learn/tutorials/news-1-deploy-an-application.md Co-authored-by: Sebastian August Berg <95385037+sebastiannberg@users.noreply.github.com> --- en/learn/tutorials/news-1-deploy-an-application.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en/learn/tutorials/news-1-deploy-an-application.md b/en/learn/tutorials/news-1-deploy-an-application.md index 4c725ac433..18e2c7caf0 100644 --- a/en/learn/tutorials/news-1-deploy-an-application.md +++ b/en/learn/tutorials/news-1-deploy-an-application.md @@ -257,7 +257,7 @@ Well done! Application instances in the [dev zone](../operations/environments.html#dev) will by default keep running for 14 days after the last deployment. You can control this in the [console](https://console.vespa-cloud.com/). -The [Vespa Cloud console](https://console.vespa-cloud.com) can also be used to delte the application instance. +The [Vespa Cloud console](https://console.vespa-cloud.com) can also be used to delete the application instance. ## Conclusion From 6c60ddb14a8c1cfdd94f18bbf7e6458ff9c00f7f Mon Sep 17 00:00:00 2001 From: Brage Date: Tue, 4 Aug 2026 15:12:57 +0200 Subject: [PATCH 3/3] docs: explanation of why to use global meta-config --- en/learn/tutorials/news-1-deploy-an-application.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/en/learn/tutorials/news-1-deploy-an-application.md b/en/learn/tutorials/news-1-deploy-an-application.md index 18e2c7caf0..bda89f9a04 100644 --- a/en/learn/tutorials/news-1-deploy-an-application.md +++ b/en/learn/tutorials/news-1-deploy-an-application.md @@ -63,7 +63,9 @@ Vespa CLI configuration mode for configuration variables. Using local configuration mode is generally recommended when working with multiple distinct Vespa applications, but most parts of this tutorial uses the same configuration values which makes -it easier to use global configuration mode: +it easier to use global configuration mode. The global +configuration mode is useful because you don't have to reapply +the same Vespa configurations for each part of this tutorial series: