diff --git a/en/learn/tutorials/news-1-deploy-an-application.md b/en/learn/tutorials/news-1-deploy-an-application.md index ca51b8d1f6..bda89f9a04 100644 --- a/en/learn/tutorials/news-1-deploy-an-application.md +++ b/en/learn/tutorials/news-1-deploy-an-application.md @@ -1,6 +1,6 @@ --- # Copyright Vespa.ai. All rights reserved. -title: "News search and recommendation tutorial - getting started on Docker" +title: "News search and recommendation tutorial - getting started on Vespa Cloud" redirect_from: - /en/tutorials/news-1-getting-started.html - /en/tutorials/news-1-deploy-an-application @@ -8,7 +8,7 @@ redirect_from: Our goal with this series is to set up a Vespa application for personalized -news recommendations. We will do this in stages, starting with a simple news +news recommendations on Vespa Cloud. We will do this in stages, starting with a simple news search system and gradually adding functionality as we go through the tutorial parts. @@ -22,32 +22,33 @@ The parts are: 6. [News recommendation with searchers](news-6-recommendation-with-searchers.html) - custom searchers, doc processors 7. [News recommendation with parent-child](news-7-recommendation-with-parent-child.html) - parent-child, tensor ranking -There are different entry points to this tutorial. This one is describing how to get -started using Docker on your local machine. You can also deploy the application we are creating on -[Vespa Cloud](https://cloud.vespa.ai). - -In this part, we will start with a minimal Vespa application to -get used to some basic operations for running the application on Docker. +In this part, we will start with a minimal Vespa application to get used to some basic +operations for deploying and running an application on Vespa Cloud. In the next part of the tutorial, we'll start developing our application. -{% include pre-req.html memory="4 GB" extra-reqs=' -
  • Python3 for converting the dataset to Vespa JSON.
    • -
  • curl to download the dataset and run the Vespa health-checks.
    • -
  • Java 17 in part 6.
    • -
  • Apache Maven in part 6.
    • ' %} - -{% include note.html content='4 GB Docker memory is sufficient for the demo dataset in part 2. -The full MIND dataset requires more, use 10 GB.' %} + In upcoming parts of this series, we will have some additional Python dependencies - we use [PyTorch](https://pytorch.org/) to train vector representations for news and users and train machine learning models for use in ranking. -## Installing vespa-cli +## Installing Vespa CLI This tutorial uses [Vespa-CLI](../../clients/vespa-cli.html), -Vespa CLI is the official command-line client for Vespa.ai. +Vespa CLI is the official command-line client for Vespa.ai. It is a single binary without any runtime dependencies and is available for Linux, macOS, and Windows.
    @@ -57,18 +58,23 @@ $ brew install vespa-cli
    -For the rest of this tutorial, you will be using localhost, so you need to configure your Vespa CLI to connect to localhost. -Run the following to use endpoints on localhost: +In this tutorial it is possible to use either a local or global +Vespa CLI configuration mode for configuration variables. Using +local configuration mode is generally recommended when working +with multiple distinct Vespa applications, but most parts of +this tutorial uses the same configuration values which makes +it easier to use global configuration mode. The global +configuration mode is useful because you don't have to reapply +the same Vespa configurations for each part of this tutorial series: 
    -$ vespa config set target local
+$ vespa config set default_config_scope global
    - ## A minimal Vespa application This tutorial has a [companion sample application](https://github.com/vespa-engine/sample-apps/tree/master/news). @@ -105,58 +111,69 @@ There are two files there: We will revisit these files in the next part of the tutorial. -## Starting Vespa +## Configuring Vespa CLI for Vespa Cloud +Configure the Vespa CLI to use Vespa Cloud, and set the application name. +Replace `tenant-name` with your tenant name from [console.vespa-cloud.com](https://console.vespa-cloud.com): -This application doesn't contain much at the moment, -let's start up the application anyway by starting a Docker container to run it: +
    + +
    +$ vespa config set target cloud
+$ vespa config set application tenant-name.news
+
    +
    + +Usually its better to use local configuration for each application, +but this tutorial uses global configuration to avoid having to set the +configuration values for each part of the tutorial. + +Authenticate with Vespa Cloud:
    -
    -$ docker pull vespaengine/vespa
-$ docker run --detach --name vespa --hostname vespa-tutorial \
-  --publish 8080:8080 --publish 19071:19071 --publish 19092:19092 \
-  vespaengine/vespa
+
    +$ vespa auth login
    -First, we pull the latest [vespa-image](https://hub.docker.com/r/vespaengine/vespa/) -from the Docker hub, then we -start it with the name `vespa`. This starts the Docker container and the -initial Vespa services to be able to deploy an application. +Follow the browser instructions to complete authentication. -Starting the container can take a short while. Before continuing, make sure -that the configuration service is running by using `vespa status`. +Next, add a certificate for [data plane access](/en/security/guide#data-plane) to the application:
    -
    -$ vespa status deploy --wait 300 
+
    +$ vespa auth cert app-1-getting-started
    -With the config server up and running, deploy the application using vespa-cli: + + + +## Deploying to Vespa Cloud + +This application doesn't contain much at the moment, +but let's deploy it to Vespa Cloud anyway to get used to the basic operations. +The first deployment may take a few minutes while nodes are provisioned: 
    -$ vespa deploy --wait 300 app-1-getting-started 
+$ vespa deploy --wait 600 app-1-getting-started
    The command uploads the application and verifies the content. If anything is wrong with the application, this step will fail with a failure description; -Otherwise, this switches the application to a live status. +otherwise, this switches the application to a live status. -Whenever you have a new version of your application, +Whenever you have a new version of your application, run the same command to deploy the application. In most cases, there is no need to restart services. Vespa takes care of reconfiguring the system. -If a restart of services is required in some rare case, however, the output will notify -which services need restart to make the change effective. -In the upcoming parts of the tutorials, we'll frequently deploy the -application changes in this manner. +In the upcoming parts of the tutorials, we'll frequently deploy the +application changes in this manner. ## Feeding to Vespa @@ -167,7 +184,7 @@ For now, to test that everything is up and running, we'll feed in a single test 
    -$ vespa feed -t http://localhost:8080 doc.json
+$ vespa feed doc.json
    @@ -237,57 +254,12 @@ $ vespa document -v remove id:news:news::1 Well done! -## Stopping and starting Vespa - -Keep Vespa running to continue with the next steps in this tutorial set (skip the below). +## Managing the Vespa Cloud application -To stop Vespa, we can run the following commands: - -
    - -
    -$ docker exec vespa vespa-stop-services
-$ docker exec vespa vespa-stop-configserver
-
    -
    - -Likewise, to start the Vespa services: - -
    - -
    -$ docker exec vespa vespa-start-configserver
-$ docker exec vespa vespa-start-services
-
    -
    - -If a [restart is required](../../reference/schemas/schemas.html#changes-that-require-restart-but-not-re-feed) -due to changes in the application package, -these two steps are what you need to do. - -To wipe the index and restart: - -
    - -
    -$ docker exec vespa sh -c ' \
-  vespa-stop-services && \
-  vespa-remove-index -force && \
-  vespa-start-services'
-
    -
    - -You can stop and kill the Vespa container application like this: - -
    - -
    -$ docker stop vespa; docker rm -f vespa
-
    -
    +Application instances in the [dev zone](../operations/environments.html#dev) will by default keep running for 14 days after the last deployment. +You can control this in the [console](https://console.vespa-cloud.com/). -This will delete the Vespa application, including all data and configuration. See -[container tuning for production](../../operations/self-managed/docker-containers.html). +The [Vespa Cloud console](https://console.vespa-cloud.com) can also be used to delete the application instance. ## Conclusion diff --git a/en/learn/tutorials/news-2-basic-feeding-and-query.md b/en/learn/tutorials/news-2-basic-feeding-and-query.md index 7f765496ff..64b12d06c6 100644 --- a/en/learn/tutorials/news-2-basic-feeding-and-query.md +++ b/en/learn/tutorials/news-2-basic-feeding-and-query.md @@ -56,7 +56,33 @@ news content at first. We'll use the impression data as we begin building the recommendation system later in this series. Let's start by downloading the data. The `news` sample app directory will -be our starting point. We've included a script to download the data for us: +be our starting point. The MIND dataset is hosted on +[Hugging Face](https://huggingface.co/datasets/yjw1029/MIND) and requires +a free account and acceptance of the dataset's terms of use. + +### Accept the dataset terms + +1. Log in at [huggingface.co](https://huggingface.co) +2. Go to the [MIND dataset page](https://huggingface.co/datasets/yjw1029/MIND) +3. Click **Agree and access repository** to accept the terms + +### Create a Hugging Face access token + +1. Go to [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) +2. Click **New token**, give it a name, select **Read** role, and copy the token + +### Download the dataset + +Set your token as an environment variable: + +
    + +
    +$ export HF_TOKEN=hf_your_token_here
+
    +
    + +Then run the download script from the **news** sample app directory:
    @@ -110,12 +136,6 @@ ranking will be done and how data will be processed during feeding and indexing. The schema, e.g., `news.sd`, is a required part of an application package — the other file needed is `services.xml`. -For self-hosted multi-node deployments, a `hosts.xml` file is also needed. -For multi-node self-hosted deployments using `hosts.xml`, see -the [multinode high -availability](https://github.com/vespa-engine/sample-apps/tree/master/examples/operations/multinode-HA) -sample application. - We mentioned these files in the previous part but didn't really explain them at the time. We'll go through them here, starting with the specification of services. @@ -135,9 +155,7 @@ service. Write the following to `news/my-app/services.xml`: <container id="default" version="1.0"> <search /> <document-api /> - <nodes> - <node hostalias="node1" /> - </nodes> + <nodes count="1" /> </container> <content id="mind" version="1.0"> @@ -145,9 +163,7 @@ service. Write the following to `news/my-app/services.xml`: <documents> <document type="news" mode="index" /> </documents> - <nodes> - <node hostalias="node1" distribution-key="0" /> - </nodes> + <nodes count="1" /> </content> </services> @@ -163,13 +179,13 @@ Quite a lot is set up here: for how to use mTLS with Vespa. - `` sets up the [document endpoint](../../reference/api/document-v1.html) for feeding and visiting. -- `` defines the nodes required per service. (See the +- `` defines the number of nodes for the service. (See the [reference](../../reference/applications/services/container.html) for more on container cluster setup). - `` The stateful content cluster - `` denotes how many copies to store of each document. - `` assigns the document types in the _schema_ — the content - cluster capacity can be increased by adding node elements — see [elasticity](../../content/elasticity.html). (See also the + cluster capacity can be increased by raising the node count — see [elasticity](../../content/elasticity.html). (See also the [reference](../../reference/applications/services/content.html) for more on content cluster setup.) @@ -278,10 +294,40 @@ my-app/ └── services.xml +If not already done from the previous [news tutorial](news-1-deploy-an-application.html), set Vespa configuration variables. Use your tenant name from the [Vespa Cloud console](https://console.vespa-cloud.com/) instead of `tenant-name`: + +
    + +
    +$ vespa config set target cloud
+$ vespa config set application tenant-name.news
+
    +
    + +Log in to Vespa Cloud: + +
    + +
    +$ vespa auth login
+
    +
    + +Add security credentials to the application package for data plane access: + +
    + +
    +$ vespa auth cert my-app -f
+
    +
    + +Finally, deploy the application package to Vespa Cloud: + 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    @@ -307,7 +353,7 @@ file can now be fed to Vespa. Use the method described in the previous part: 
    -$ vespa feed mind/vespa.json --target http://localhost:8080
+$ vespa feed mind/vespa.json
    @@ -364,14 +410,19 @@ $ vespa query -v 'yql=select * from news where default contains "music"'
    -or a POST JSON query (Notice the *Content-Type* header specification): +or a POST JSON query. The values can be found by using the previous `vespa query -v` command. +The content type should be changed to `application/json`. +It's also possible to use `vespa status` to find the url of your cloud instance. +: 
    -$ curl -s -H "Content-Type: application/json" \
-  --data '{"yql" : "select * from sources * where default contains \"music\""}' \
-  http://localhost:8080/search/ | python3 -m json.tool
+$ curl --key /path/to/key \
+     --cert /path/to/cert \
+     -H 'Content-Type: application/json' \
+     --data '{"yql" : "select * from sources * where default contains \"music\""}' \
+     'https://your-vespa-url.vespa-app.cloud/search/' | python3 -m json.tool
    diff --git a/en/learn/tutorials/news-3-searching.md b/en/learn/tutorials/news-3-searching.md index 0513ed5f61..0140d5fa40 100644 --- a/en/learn/tutorials/news-3-searching.md +++ b/en/learn/tutorials/news-3-searching.md @@ -532,7 +532,7 @@ Deploy the _popularity_ rank profile: 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    diff --git a/en/learn/tutorials/news-5-recommendation.md b/en/learn/tutorials/news-5-recommendation.md index df6f55781c..1230b62015 100644 --- a/en/learn/tutorials/news-5-recommendation.md +++ b/en/learn/tutorials/news-5-recommendation.md @@ -177,9 +177,7 @@ so we modify `services.xml` and add it under `documents` in the `content` sectio <container id="default" version="1.0"> <search /> <document-api /> - <nodes> - <node hostalias="node1" /> - </nodes> + <nodes count="1" /> </container> <content id="mind" version="1.0"> @@ -188,9 +186,7 @@ so we modify `services.xml` and add it under `documents` in the `content` sectio <document type="news" mode="index" /> <document type="user" mode="index" /> </documents> - <nodes> - <node hostalias="node1" distribution-key="0" /> - </nodes> + <nodes count="1" /> </content> </services> @@ -200,7 +196,7 @@ so we modify `services.xml` and add it under `documents` in the `content` sectio 
    -$ vespa deploy --wait 300 my-app 
+$ vespa deploy --wait 600 my-app
    @@ -215,8 +211,8 @@ feed `mind/vespa_user_embeddings.json` and `mind/vespa_news_embeddings.json`: 
    -$ vespa feed mind/vespa_user_embeddings.json --target http://localhost:8080
-$ vespa feed mind/vespa_news_embeddings.json --target http://localhost:8080
+$ vespa feed mind/vespa_user_embeddings.json
+$ vespa feed mind/vespa_news_embeddings.json
    @@ -292,7 +288,7 @@ Deploy the updates to query profiles: 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    @@ -528,27 +524,20 @@ restart is required so that the index can be built: 
    -$ vespa deploy --wait 300 my-app
+$ vespa deploy --wait 600 my-app
    -Introducing the HNSW `index` requires a content node restart, in this case we restart all services: +Introducing the HNSW `index` requires a content node restart. +On Vespa Cloud, this is handled automatically after deployment — +wait for the deployment to complete and the application to become ready: -
    - -
    -$ docker exec vespa /usr/bin/sh -c \
-  '/opt/vespa/bin/vespa-stop-services && /opt/vespa/bin/vespa-start-services'
-
    -
    - - 
     $ vespa status --wait 300
    -After doing this and waiting a bit for Vespa to start, we can query Vespa again: +After the deployment completes, we can query Vespa again:
    diff --git a/en/learn/tutorials/news-6-recommendation-with-searchers.md b/en/learn/tutorials/news-6-recommendation-with-searchers.md index 8eb524680c..8c5e391397 100644 --- a/en/learn/tutorials/news-6-recommendation-with-searchers.md +++ b/en/learn/tutorials/news-6-recommendation-with-searchers.md @@ -275,7 +275,7 @@ This contains the full Vespa application, with Java components - deploy it: 
    -$ vespa deploy --wait 300 app-6-recommendation-with-searchers
+$ vespa deploy --wait 600 app-6-recommendation-with-searchers
    diff --git a/en/learn/tutorials/news-7-recommendation-with-parent-child.md b/en/learn/tutorials/news-7-recommendation-with-parent-child.md index 8f26c61fef..6ad8d1ccf0 100644 --- a/en/learn/tutorials/news-7-recommendation-with-parent-child.md +++ b/en/learn/tutorials/news-7-recommendation-with-parent-child.md @@ -287,7 +287,7 @@ $ (cd app-7-parent-child && mvn package) 
    -$ vespa deploy --wait 300 app-7-parent-child
+$ vespa deploy --wait 600 app-7-parent-child
    @@ -320,8 +320,8 @@ Feed the created feed files: 
    -$ vespa feed mind/global_category_ctr.json --target http://localhost:8080
-$ vespa feed mind/news_category_ctr_update.json --target http://localhost:8080
+$ vespa feed mind/global_category_ctr.json
+$ vespa feed mind/news_category_ctr_update.json