diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index f8a9448..0af680c 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -1,7 +1,7 @@ * xref:hello-world:overview.adoc[Couchbase {name_platform} SDK Overview] * xref:hello-world:start-using-sdk.adoc[Hello World] -// ** xref:hello-world:sample-application.adoc[] +** xref:hello-world:sample-application.adoc[] // ** xref:hello-world:student-record-developer-tutorial.adoc[] ** xref:howtos:managing-connections.adoc[Connecting to Your Database] // *** xref:howtos:sdk-authentication.adoc[Authentication & Authorization] @@ -39,9 +39,9 @@ ** xref:howtos:health-check.adoc[Using Health Check] ** xref:concept-docs:response-time-observability.adoc[Observability] *** xref:howtos:slow-operations-logging.adoc[] -// *** xref:howtos:observability-metrics.adoc[] -// *** xref:howtos:observability-orphan-logger.adoc[] -// *** xref:howtos:observability-tracing.adoc[] +*** xref:howtos:observability-metrics.adoc[] +*** xref:howtos:observability-orphan-logger.adoc[] +*** xref:howtos:observability-tracing.adoc[] * xref:concept-docs:best-practices.adoc[] ** xref:howtos:concurrent-async-apis.adoc[Async APIs] ** xref:howtos:error-handling.adoc[] @@ -55,7 +55,7 @@ ** xref:project-docs:compatibility.adoc[] // *** xref:project-docs:migrating-sdk-code-to-3.n.adoc[] // *** xref:project-docs:distributed-acid-transactions-migration-guide.adoc[] -// *** xref:project-docs:third-party-integrations.adoc[] +*** xref:project-docs:third-party-integrations.adoc[] ** xref:project-docs:sdk-full-installation.adoc[] // *** xref:hello-world:platform-help.adoc[] ** xref:project-docs:get-involved.adoc[] diff --git a/modules/hello-world/assets/images/cli_output.png b/modules/hello-world/assets/images/cli_output.png new file mode 100644 index 0000000..5572a05 Binary files /dev/null and b/modules/hello-world/assets/images/cli_output.png differ diff --git a/modules/hello-world/pages/sample-application.adoc b/modules/hello-world/pages/sample-application.adoc index 382a455..fd4c194 100644 --- a/modules/hello-world/pages/sample-application.adoc +++ b/modules/hello-world/pages/sample-application.adoc @@ -1,51 +1,509 @@ -= Sample Application -:description: Discover how to program interactions with the Couchbase Server via the data, query, and search services -- using the Travel Sample Application with the built-in Travel Sample data Bucket. -// :page-aliases: ROOT:sample-application,ROOT:tutorial4,ROOT:sample-app-backend -:page-pagination: prev += Quickstart in Couchbase with {cpp} +:page-toclevels: 2 +:description: Quickstart app to build a REST API using Couchbase Capella in {cpp}. +:dev-portal-link: https://github.com/couchbase-examples/cxx-quickstart +:this-platform: (in C++) +:dir-name: cxx-quickstart +:clang: clang++ +:g: g++ +:example-source-lang: c++ -:travel-sample-git-project: try-cb-scala -// include::project-docs:partial$attributes.adoc[] +// abstract +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=abstract] +[abstract] +{description} +Discover how to program interactions with Couchbase via the Data, Query, and Search services. -//// -For 1.7: -https://github.com/couchbase-examples/scala-quickstart +// intro +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=intro] +After you have navigated through https://cloud.couchbase.com/sign-up[signing up to Capella], +if {name_platform} is entered as your chosen language, you will be pointed to a clonable quickstart app on GitHub. +If you were not, you can still find it {dev-portal-link}[here]. + + +Often, the first step developers take after creating their database is to create a REST API that can perform Create, Read, Update, and Delete (CRUD) operations for that database. +The {dev-portal-link}[quickstart project] is designed to teach you and give you a starter project {this-platform} to generate such a REST API. +After you have loaded the travel-sample bucket in your database, you can run this application which is a REST API with Swagger documentation so that you can learn: + +. How to create, read, update, and delete documents using xref:howto:kv-operations[Key-Value operations]. +KV operations are unique to Couchbase and provide super fast (under millisecond) operations. +. How to write simple parametrized xref:howtos:n1ql-queries-with-sdk.html[{sqlpp} queries] using the built-in travel-sample bucket. + +This documentation -- and a number of other useful developer tutorials -- +can be found on the {dev-portal-link}[Couchbase Developer Portal]. + + +== Prerequisites + +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=prereq] + +To run this prebuilt project, you will need: + +* A https://www.couchbase.com/products/capella/[Couchbase Capella] cluster with the xref:ref:travel-app-data-model.adoc[travel-sample] bucket loaded. ++ +To run this tutorial using a self-managed Couchbase cluster, refer to the <<#running-self-managed-couchbase-cluster>> section. + +* The Travel Sample Bucket is pre-loaded in Capella Free Tier. If you need to load it manually, see the xref:cloud:clusters:data-service/import-data-documents.adoc#import-sample-data[sample data impost page]. + +* https://cmake.org/[CMake] 3.9 or higher installed + +* https://en.cppreference.com/w/cpp/17[{cpp}17] and a compatible compiler, such as +https://clang.llvm.org/[{clang}] or https://gcc.gnu.org/[{g}]. + + +== App Setup + +We will walk through the different steps required to get the application running: -use some of https://developer.couchbase.com/tutorial-quickstart-scala-webservers ? +. Cloning the Repo ++ +[source,console,subs="+attributes"] +---- +$ git clone {dev-portal-link}.git +---- -point back to the previous version (Travel Sample) as an alternate version?? +. Navigate to the Project Directory ++ +[source,console,subs="+attributes"] +---- +$ cd {dir-name} +---- + +//// ++ +This project makes use of CMake and CPM to install dependencies. +We will cover this at the install and build stage, after setting up the database. //// +// ^ +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=setup] + + + +=== Setup Database Configuration + +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=db-config] + +To learn more about connecting to your Capella cluster, follow the https://docs.couchbase.com/cloud/get-started/connect.html[instructions]. +Specifically, you need to do the following: + +. Create the xref:cloud:clusters:manage-database-users.adoc[database credentials] to access the travel-sample bucket (Read and Write) used in the application. +. https://docs.couchbase.com/cloud/clusters/allow-ip-address.html[Allow access] to the Cluster from the IP on which the application is running. + +All configuration for communication with the database is read from the environment variables. +We have provided a convenience feature in this quickstart to set the required environment variables using the shell script `set_env_vars.sh`. +Change the values of the following lines: + +[source,sh] +---- +export DB_CONN_STR= +export DB_USERNAME= +export DB_PASSWORD= +export BUCKET_NAME= +export SCOPE_NAME= +export COL_NAME= +Note: The connection string expects the couchbases:// or couchbase:// part. +---- + +Run the command: + +[source,console] +---- +source set_env_vars.sh +---- + +This will set the environment variables for that session. + + +=== Install Dependencies and Build + +This project makes use of CMake and CPM to install dependencies. + +[source,console] +---- +$ mkdir build +---- + +[source,console] +---- +$ cd build +---- + +[source,console] +---- +$ cmake .. +---- + +[source,console] +---- +$ cmake --build . +---- + +This will download and install all of the dependencies required for the project to be built, +and it will build the executable required to run the application. + + + +== Running The Application + +// constrained network environment +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=dev-pc] + +.Running from a Development Machine +[TIP] +==== +Couchbase -- including Capella -- is designed to run in a LAN-like environment. +For development, connecting to a remote cloud instance from a local laptop instead of an application server in the same region, +may require you to adjust some timeouts. +For more information, see the xref:ref:client-settings.adoc#constrained-network-environments[constrained network environments] section of the docs. +==== + + +=== Directly on Your Local Machine + +At this point, we have installed the dependencies, loaded the travel-sample data, and configured the application with the credentials. +The application is now ready and you can run it by executing the following command from the `build` directory: + +[source,console] +---- +$ cmake --build . +$ ./cxx_quickstart +---- + + +=== Verifying the Application + +Once you run the executable, your terminal should fill up with the results of the executed statements written in the main function of the `main.cpp` and should look something like this: + +image::cli_output.png + + +=== Running Tests + +For running tests, a self-managed cluster is required with travel-sample bucket loaded or you need to update the connection details in the `tests/test.cpp` file. + +To run the tests, use the following command from the `build` directory: + +[source,console] +---- +$ ./tests/u_tests +---- + + +=== Schema and Usage Overview + +This quickstart utilizes two collections: airline and hotel. +The airline collection is used for CRUD operations, while the hotel collection is leveraged for Search indexes and Query execution. +The schemas for both collections can be found in the model folder. + + + +== Code Review + +To begin this tutorial, clone the repo and open it up in the IDE of your choice. +Now you can explore how to interact with Couchbase Server using the {cpp} SDK. + +We have separated out the SDK code and the main function. +The `db.h` and `db.cpp` contain the declaration and the implementation of utility functions we will use to parse environment variables and create a connection to the cluster. +`operations.h` and `operations.cpp` contain all the functions that perform operations on the database. +Both `db.cpp` and `operations.cpp` are combined to make a static library. +The tests are similarly separated out in the tests folder which utilize the library created earlier. +The `main.cpp` is the executable which is also built by linking the library and contains code that demonstrates the usage of the functions we defined earlier to interact with the database. -include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=abstract] -include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=quick-start] +=== Connecting to the Cluster -include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=bring-your-own] +In `db.h`, we include the required header files to work with {cpp} SDK in order to implement the functions required to initialize the database. +In the `db.cpp` we implement the functions that help us connect to the database. +We begin by implementing a few utility functions that will help us later. +The `parseEnvironmentVariables` serves as a utility to get the values set for a list of environment variables. +This enables us to get the connection parameters and credentials set by running source `set_env_vars.sh`. +Following this, `checkScopeAndColExists` and `checkSearchEnabled` are implemented to check for the existence of a scope and collection of a given name and to verify if the Search Service is enabled respectively. +Finally we have the `InitCluster` function which returns the connection objects as a tuple. -include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=using] +[source,{example-source-lang}] +.db.h +---- +... +std::vector parseEnvironmentVariables(const std::vector& keys); +bool checkScopeAndColExists(couchbase::bucket& bucket, const std::string& scope_name, const std::string& col_name); +bool checkSearchEnabled(couchbase::cluster& cluster, int min_nodes); +... +std::tuple InitCluster(); +---- -== Sample App Backend -The backend code shows Couchbase {name-sdk} in action with Query and Search, -but also how to plug together all of the elements and build an application with Couchbase Server and the {name-sdk}. +We recommend creating a single Couchbase connection when your application starts up, and sharing this instance throughout your application. +You should always set the default `BUCKET_NAME`, `SCOPE_NAME`, `COL_NAME` environment variables, and use the `InitCluster` function to get the instances. +You should share and use these instances throughout your application. -The https://github.com/couchbaselabs/{travel-sample-git-project}/tree/HEAD/app[`app` directory] contains the various components, classes and functions for handling users, registration, and {sqlpp} (formerly N1QL) queries. +The Couchbase connection is established in the `connectCluster` method defined in `db.h` and implemented in `db.cpp`. +There, we call the `connect` method defined in the SDK to create the Database connection. +If the connection is already established, we do not do anything. +Following connection to the cluster, get the reference to the bucket, scope and collection and return all the objects as a tuple. +.db.cpp +[source,{example-source-lang}] +.db.h +---- -include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=data-model] +... +auto [connect_err, cluster] = couchbase::cluster::connect(DB_CONN_STRING, options).get(); +... +auto bucket = cluster.bucket(BUCKET_NAME); +... +auto scope = bucket.scope(SCOPE_NAME); +auto col = scope.collection(COL_NAME); +return {cluster, bucket, scope, col}; +---- -include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=rest-api] +== Operations +Operations for interacting with the database are defined and implemented in `operations.h` and `operations.cpp`. -// TODO: we don't use the partial here, because of the location of swagger.json -// in try-cb-dotnet/wwwroot/ -== REST API -You can explore the REST API here in read-only mode, or once you are running the application, at the `/apidocs` endpoint. +=== Insert Document + +Insert function is the equivalent of the `POST` request and can be used to insert new documents to the collection. +We can pass the document to be inserted as a JSON string or as a JSON file path, the function takes in `file_flag` which is used to differentiate between the two. + +* The value gets converted to the type `tao::json::value` and inserts it to the collection if `file_flag=false`. +* If `file_flag=true`, it reads the content from the provided file and then converts it to `tao::json::value`. +* Performs an upsert operation on the collection using the `doc_id` and the converted document content. +* If successful, `return 1`. If an error occurs, prints an error message and `return 0`. + + +[source,{example-source-lang}] +.operations.cpp +---- +auto [in_error, in_res] = col.insert(doc_id, v).get(); +---- + +[source,{example-source-lang}] +.main.cpp +---- +auto insert_res = Insert(col, "quickstart_test", "{ \"test\": \"hello\"}", false); +auto insert_res2 = Insert(col, "quickstart_test2", "doc.json", true); +---- + +=== Upsert Document + +The `Upsert` function is the equivalent of the `PUT` request. +It can be used to update any existing document or to insert a new document to the collection if the `doc_id` does not already exist. +Similar to Insert -- we can pass the document to be inserted as a JSON string or as a JSON file path, the function takes in `file_flag` which is used to differentiate between the two. + +* The value gets converted to the type `tao::json::value` and inserts it to the collection if `file_flag=false`. +* If `file_flag=true`, it reads the content from the provided file and then converts it to `tao::json::value`. +* Performs an insert operation on the collection using the `doc_id` and the converted document content. +* If successful, `return 1`. If an error occurs, prints an error message and `return 0`. + +[source,{example-source-lang}] +.operations.cpp +---- +auto [up_error, up_res] = col.upsert(doc_id, v).get(); +---- + +[source,{example-source-lang}] +.main.cpp +---- +auto upsert_res = Upsert(col, "quickstart_test", "{ \"test\": \"hello\"}", false); +auto upsert_res2 = Upsert(col, "quickstart_test2", "doc.json", true); +---- + +=== Read + +The `Read` function is equivalent to `GET` requests and can be used to fetch documents using the `doc_id`. + +* First checks if the document exists using `col.exists(doc_id)`. +* If the document exists, it retrieves the document's content using `col.get(doc_id)` and returns it after converting it to `tao::json::value` for easier usage on return. +* If an error occurs (such as "document not found"), it prints an error message and returns an empty `tao::json::value` object. + +[source,{example-source-lang}] +.operations.cpp +---- +auto [ex_err, ex_res] = col.exists(doc_id).get(); +... +auto [get_err, get_res] = col.get(doc_id).get(); +... +auto doc = get_res.content_as(); +return doc; +---- + +[source,{example-source-lang}] +.main.cpp +---- + v = Read(col, "airline_10123"); +std::cout << tao::json::to_string(v) << std::endl; +---- + +=== Delete + +The `Delete` function attempts to remove a document with the given `doc_id`. + +* Attempts to remove the document with a given `doc_id` from the collection. +* If the deletion is successful, `return 1` and if an error occurs, it prints an error message and `return 0`. + +[source,{example-source-lang}] +.operations.cpp +---- +auto [delete_err, delete_res] = col.remove(doc_id).get(); +---- + +[source,{example-source-lang}] +.main.cpp +---- +auto res = Delete(col, doc_id); +---- + +=== Query + +We can use the `Query` function to execute any {sqlpp} query on a scope. + +* Executes the {sqlpp} query using the provided `scope.query(query, opts)`. +* Returns the result of the query if successful. The result is added to a `std::vector` object that contains the `id`, `country`, `avg_rating`, `title`. +* We can pass `opts` parameter, which can be used to insert positional parameters in the query. +* If there is an error, it prints an error message and returns an empty result object. + +[source,{example-source-lang}] +.operations.cpp +---- +std::string query{ R"( + SELECT META(h).id, h AS doc, + AVG(r.ratings.Overall) AS avg_rating + FROM hotel h + UNNEST h.reviews r + WHERE h.country IN $1 AND h.description LIKE "%cheap%" + GROUP BY META(h).id, h + ORDER BY avg_rating DESC + LIMIT 5; +)" }; +auto [q_err, q_res] = scope.query(query, couchbase::query_options{}.positional_parameters(std::vector{"United States", "United Kingdom"})).get(); +---- + +[source,{example-source-lang}] +.main.cpp +---- +for (auto& row : query_res) { + std::cout << row << std::endl; +} +---- + +=== Create a Search Index + +Search indexes in Couchbase are used for full-text search and efficient querying of documents based on specific fields or attributes. +The `CreateSearchIndex` function helps in creating a new Search index which can then be used. + +* Reads the index configuration from the `index_file`. +* Checks if an index with the same name already exists using the `searchIndexExists` function. +* If the index with same name exists, it returns the index name. +* If the index does not exist, it constructs a new search index object and upserts it into the Couchbase scope. +* Returns the name of the newly created index or an empty string if there was an error. + +[source,{example-source-lang}] +.operations.cpp +---- +auto err = scope_index_manager.upsert_index(i).get(); +---- + +[source,{example-source-lang}] +.main.cpp +---- +std::string index_name = CreateSearchIndex(scope, "hotel_search_index.json"); +---- + +=== Search By Name + +The `SearchByName` function aims to demonstrate the usage of a Search index to search for documents in a scope. Params: + +* `scope`: The Couchbase scope to search in. +* `index`: The Search index name to use for the query. +* `name`: The name to search for in the documents. +* `field`: The field where the name should be searched. +* `limit`: The maximum number of results to return. + +[source,{example-source-lang}] +.operations.cpp +---- +auto [s_err, s_res] = scope.search(index, searchQ, opts).get(); +... +std::vector rows_res{}; +// Reference is important since the copy constructor is deleted +for(auto &row:s_res.rows()){ + rows_res.push_back(row.id()); +} +return rows_res; +---- + +[source,{example-source-lang}] +.main.cpp +---- +auto search_res = SearchByName(scope, index_name, "swanky", "name", 50); +std::cout << "Search result contains:\t" << search_res.size() << std::endl; +---- + +=== Filter + +The `Filter` function aims to demo the construction and execution of a `conjunction_query` which can be described as an AND operation on two or more types of filters. +This particular implementation performs a conjunction on `couchbase::match_query("United States").field("country")` and `couchbase::term_query("San Diego").field("city")`. + + +[source,{example-source-lang}] +.operations.cpp +---- +auto query = couchbase::conjunction_query{ + couchbase::match_query("United States").field("country"), + couchbase::term_query("San Diego").field("city") + }; +... +auto [err,res] = scope.search(index_name, couchbase::search_request(query), opts).get(); + +for(auto &row:res.rows()){ + auto fields = row.fields_as(); + rows_res.push_back(fields["name"].as()); +} +return rows_res; +---- + + +[source,{example-source-lang}] +.main.cpp +---- +auto filter_res = Filter(scope, index_name, 50, 1); +std::cout << "Filter result contains:\t" << filter_res.size() << std::endl; +---- + + + +== Running Self-Managed Couchbase Cluster + +// include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=self-managed] + +If you are running this quickstart with a self-managed Couchbase cluster, you need to xref:server:manage:manage-settings/install-sample-buckets.adoc[load] the travel-sample data bucket in your cluster and generate the credentials for the bucket. + +You need to update the connection string and the credentials in the `application.properties` file in the `src/main/resources` folder. + + + +//// +=== Swagger Documentation + +include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=swagger] + + + +==== Trying Out the API + + +include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=try] + + +==== Models + +include::{version-common}@sdk:shared:partial$sample-application.adoc[tag=models] +//// -swagger_ui::https://raw.githubusercontent.com/couchbaselabs/{travel-sample-git-project}/HEAD/try-cb-dotnet/wwwroot/swagger.json[] diff --git a/modules/hello-world/pages/start-using-sdk.adoc b/modules/hello-world/pages/start-using-sdk.adoc index 86a9901..45ff7b4 100644 --- a/modules/hello-world/pages/start-using-sdk.adoc +++ b/modules/hello-world/pages/start-using-sdk.adoc @@ -294,7 +294,6 @@ Otherwise, changes might get lost if two people change the same document at the The remove method deletes a document from a collection: -[source,cxx] [source,{example-source-lang}] ---- include::{example-source}[indent=0,tag=remove] diff --git a/modules/howtos/pages/concurrent-async-apis.adoc b/modules/howtos/pages/concurrent-async-apis.adoc index 9020fe1..01e8ec9 100644 --- a/modules/howtos/pages/concurrent-async-apis.adoc +++ b/modules/howtos/pages/concurrent-async-apis.adoc @@ -14,7 +14,7 @@ // This page pulls in content from -sdk-common- // and code samples from -example-dir- // -// It can be seen built at wwww. +// It can be seen built at https://docs.couchbase.com/cxx-sdk/1.3/howtos/concurrent-async-apis.html [abstract] {description} @@ -58,169 +58,9 @@ provides a clean and modern way to access results of asynchronous operations. And you can use different APIs at different times. -//// -[abstract] -{description} - -The Scala SDK provides three APIs, which can be freely mixed: - -* A simple blocking one, that returns `Try`. -* An asynchronous one, that returns `Future`. -* A reactive one, that returns reactive primitives from the https://projectreactor.io/[Project Reactor] library, e.g. `Mono` and `Flux`. - - -== Using the Blocking API - -This is the simplest API, in which all operations are synchronous and blocking. -A simple upsert example looks like this: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=upsert-simple,indent=0] ----- - -Methods in the blocking API that can error will return a Scala `Try` object, -which can either be a `Success` containing the result, -or a `Failure` containing a _Throwable_ exception that derives from `CouchbaseException`. - -This can be pattern matched on, like this: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=upsert,indent=0] ----- - -Simple pattern matching like this works fine for simple examples, but starts to look complicated when doing multiple operations: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=get,indent=0] ----- - -Developers may prefer to use `flatMap`, which lets all the `Try` be checked in one place. -Here, the call to `collection.get` will only be attempted if the call to `collection.upsert` was successful -- that is, returned `Success`. - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=get-map,indent=0] ----- - - -== Using the Asynchronous API - -The asynchronous API returns Scala `Future`, representing the execution of an asynchronous task and the promise of a future result. - -Here's what a simple upsert looks like, handling `Success` or `Failure` -(note we are not actually blocking on the result here -- if you need that then you can use the methods in Scala's `Await`, or simply use the blocking Couchbase API): - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=upsert-async,indent=0] ----- - -But, the compiler will fail to compile this, reporting that an implicit `ExecutionContext` cannot be found for onComplete. - -As you may know, to do anything with a `Future` -- including `onComplete`, `map` and `flatMap` -- an `ExecutionContext` is required. -Here's how to create one that creates an unlimited thread-pool with named threads: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=async-import,indent=0] ----- - -Now there's an implicit ExecutionContext available, the upsert example will compile. - -Let's see a more complex example combining operations together. -`Future` compose well, with `flatMap` being the most common tool: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=get-async,indent=0] ----- - -Interestingly, you'll see a couple of calls to `Try.get()` here. -These throw exceptions if the Try is `Failure`, and exceptions are usually something we try to avoid in Scala. -But here there's no problem as the `Future` will capture the exception, -and raise in so it can be handled in a `Failure`, as in the example. - - -== Using the Reactive API - -Reactive Programming is an advanced paradigm designed to handle the challenges of building modern, network-aware and fault-tolerant programs. - -The reactive API uses primitives from https://projectreactor.io/[Project Reactor], namely `Mono` (returning at most one result) and `Flux` (returning many). -These are compliant with the https://www.reactive-streams.org/[reactive streams specification], and so can easily be converted into other reactive implementations such as RxJava. - -Here's how to do a simple reactive upsert operation, logging any errors: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=upsert-reactive,indent=0] ----- - -`upsert` returns `Mono[MutationResult]`. The `subscribe` starts the operation - without a subscribe, nothing will happen. The operation happens in the background. - -Normally with reactive programming you will chain multiple operations together, and it's often possible to continue handling the chain in a reactive manner too. -For instance, many web frameworks allow an endpoint to stream back a reactive result. - -In the rare cases where you need to block on a reactive primitive (say, in a unit test) you can do it like this: - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=upsert-reactive-block,indent=0] ----- - -The `block` call here also does a `subscribe` under the hood. - -Let's look at a more complex example, combining multiple operations. As with `Future`, reactive primitives can be composed, with `flatMap` being the most common tool. - -[source,scala] ----- -include::devguide:example$scala/MultipleAPIs.scala[tag=get-reactive,indent=0] ----- - -Similar to the `Future` example, you'll note some calls to .get, on `Option` and `Try` that will throw exceptions if they do not contain `Some` or `Success` respectively. -This is fine -- the reactive code will capture it, and raise it in the standard reactive way -- e.g. through `doOnError` and similar operators. - -While it's beyond the scope of this guide to teach reactive programming, it's important to touch on a handful of golden rules: - -1. Never do blocking calls inside operators (operators are `doOnNext`, `flatMap`, etc.). Those operators are executing on a limited number of threads, and blocking calls will limit concurrency. Instead, convert the blocking call into a `Mono` or `Flux`, and `flatMap` to it. -2. Never subscribe to a reactive primitive inside an operator. Instead, `flatMap` to it. -3. Always subscribe. A `Mono` or `Flux` will not start until it's subscribed to. === Bulk Operations -For bulk operations, and applying the golden rules above, -call `flatMap` inside `SFlux`, as in the `upsert` example following: - -[source,scala] ----- -val json = JsonObject("foo" -> "bar", "baz" -> "qux") -val parallelism = 32 - -val result: Seq[Either[Throwable, MutationResult]] = SFlux.fromIterable(Seq("doc1", "doc2", "doc3")) - .flatMap(docId => collection.reactive.upsert(docId, json) - .map(result => Right(result)) - .onErrorResume(err => SMono.just(Left(err))), - parallelism) - .collectSeq() - .block() ----- - - -== Choosing an API - -So which API should you choose? - -It's really down to you and the needs of your application. -If you're already writing code in a synchronous way already then it may make sense to continue that way. -If you're writing a web application that supports reactive streams, it may make sense to use the reactive API. -And you can use different APIs at different times. - -The most important thing to consider is when streaming back large queries from Query, Full Text Search, and Analytics. -Here, the reactive API will provide full backpressure: -that is, if your application is processing rows slower than the service is returning them, then automatically fewer rows will be requested from the service to give the application time to catch up. -The upshot of this is that few rows are ever buffered in-memory, and the application shouldn't get out-of-memory exceptions. - -By contrast, the blocking and asynchronous APIs will buffer all rows in-memory before returning them to the application. This will generally be fine, but if you're doing any large queries then you may want to consider the reactive API. -//// +The {cpp} source repo contains an example of +https://github.com/couchbase/couchbase-cxx-client/blob/{sdk_current_version}/examples/bulk_base_api.cxx[implementing a bulk get]. +Careful benchmarking would be required to ensure that this approach is beneficial for your particular workload. diff --git a/modules/howtos/pages/observability-metrics.adoc b/modules/howtos/pages/observability-metrics.adoc index 16a9e92..8ad3e22 100644 --- a/modules/howtos/pages/observability-metrics.adoc +++ b/modules/howtos/pages/observability-metrics.adoc @@ -1,9 +1,86 @@ = Metrics Reporting :description: Individual request tracing presents a very specific (though isolated) view of the system. +:example-source-lang: c++ + + [abstract] {description} +In the {name-sdk}, `couchbase::metrics::otel_meter` wraps an `opentelemetry::metrics::Meter`. + +A complete code example showing how to instrument a Couchbase {cpp} application with `OpenTelemetry`, distributed tracing, and metrics, can be found https://github.com/couchbaselabs/couchbase-cxx-client-demo/blob/main/examples/inventory_with_opentelemetry.cpp[in our GitHub repo]. +The snippets used on this page are taken from that example. + + +== Metrics Collection + +The SDK's internal metric collection is activated with `options.metrics().enable(true)`: + +[source,{example-source-lang}] +---- + options.metrics().enable(true); + options.metrics().meter(std::make_shared( + opentelemetry::metrics::Provider::GetMeterProvider()->GetMeter(k_service_name, + k_service_version))); +---- + + +The SDK reads the meter and tracer from the options object at connect time. +OTel metrics and tracing must be wired into cluster options before calling `couchbase::cluster::connect()`: + +[source,{example-source-lang}] +---- + auto meter_provider = apply_opentelemetry_meter_options(options, config.metrics_config); + auto tracer_provider = apply_opentelemetry_tracer_options(options, config.traces_config); + + auto [connect_err, cluster] = + couchbase::cluster::connect(config.connection_string, options).get(); + if (connect_err) { + std::cout << "Unable to connect to the cluster. ec: " << connect_err.message() << "\n"; + } else { + auto collection = cluster.bucket(config.bucket_name) + .scope(config.scope_name) + .collection(config.collection_name); +---- + + +== Exporting Metrics + +In the example, the SDK records per-operation latency histograms and retry/timeout counters, all labelled by bucket, scope, collection, and operation type. +A `PeriodicExportingMetricReader` then pushes those measurements to the configured OTLP endpoint. + +[source,{example-source-lang}] +---- + opentelemetry::sdk::metrics::PeriodicExportingMetricReaderOptions reader_options{}; + reader_options.export_interval_millis = metrics.reader_export_interval; + reader_options.export_timeout_millis = metrics.reader_export_timeout; + + std::unique_ptr reader{ + new opentelemetry::sdk::metrics::PeriodicExportingMetricReader(std::move(exporter), + reader_options) +---- + + +== Complete Code Listing + +=== {empty} + +.Complete tracing and metrics sample code: [*Click to open or collapse the listing*] +[%collapsible] +==== +[source,{example-source-lang}] +---- +include::devguide:example$cxx/src/inventory_with_opentelemetry.cpp[] +---- +==== + + + + +// Return to the below format with more info? + +//// In addition, it also makes sense to capture information that aggregates request data (i.e. requests per second), but also data which is not tied to a specific request at all (i.e. resource utilization). @@ -142,3 +219,4 @@ Once you've created the a Micrometer `MeterRegistry`, you need to wrap it and pa ---- At this point the metrics are hooked up to Micrometer and will be reported as `cb.requests` (as a `Counter`) and `cb.responses` (as a `DistributionSummary`). +//// diff --git a/modules/howtos/pages/observability-orphan-logger.adoc b/modules/howtos/pages/observability-orphan-logger.adoc index 8cb9f04..fc818cf 100644 --- a/modules/howtos/pages/observability-orphan-logger.adoc +++ b/modules/howtos/pages/observability-orphan-logger.adoc @@ -1,16 +1,17 @@ = Orphaned Requests Logging :description: In addition to request tracing and metrics reporting, logging orphaned requests provides additional insight into why an operation might have timed out (or got cancelled for a different reason). -:page-topic-type: howto [abstract] {description} While tracing and metrics can also be consumed through external interfaces, getting information about orphaned requests only works through the built-in mechanisms. -The way it works is that every time a response is in the process of being completed, when the SDK detects that the original caller is not listening anymore (likely because of a timeout), it will send this "orphaned" response to a reporting utility which aggregates all responses and in regular intervals logs them in a specific format. +The way it works is that every time a response is in the process of being completed, when the SDK detects that the original caller is not listening anymore (likely because of a timeout), +it will send this "orphaned" response to a reporting utility which aggregates all responses and in regular intervals logs them in a specific format. When you spot a `TimeoutException` in your log file, you can look for the output of the `OrphanReporter` and correlate the information. + == Output Format The `OrphanReporter` raises an `OrphansRecordedEvent` which in turn is going to be logged alongside all other SDK logs. @@ -119,6 +120,8 @@ Each individual request has the following format: } ---- + +//// .Experimental JSON Output Format Descriptions [options="header"] |==== @@ -137,20 +140,21 @@ Each individual request has the following format: |==== If a field is not available, it will not be included in the output. +//// -== Configuration -The orphan logger can be configured through the `OrphanReporterConfig`. +== Configuration +The orphan logger can be configured through `tracing_options`. The following properties can be configured: .OrphanReporterConfig Properties [options="header"] |==== | Property | Default | Description -| `emitInterval` | 10 seconds | The interval where found orphans are emitted. +| `emitInterval` | 10,000 milliseconds | The interval where found orphans are emitted. | `sampleSize` | 10 | The number of samples to store per service. -| `queueLength` | 1024 | Maximum buffer size of orphans to store to pick up for the reporter. |==== -In addition to those properties, if you want to try out the new logging format you can set the `com.couchbase.orphanReporterNewOutputFormat` system property to `true`. +Configuration details can be found in the +https://docs.couchbase.com/sdk-api/couchbase-cxx-client-{sdk_current_version}/classcouchbase_1_1tracing__options.html#a2872a66bcf62fa033181a3da656bec9e[API reference]. diff --git a/modules/howtos/pages/observability-tracing.adoc b/modules/howtos/pages/observability-tracing.adoc index 21c4f60..b603318 100644 --- a/modules/howtos/pages/observability-tracing.adoc +++ b/modules/howtos/pages/observability-tracing.adoc @@ -1,5 +1,8 @@ = Request Tracing :description: Collecting information about an individual request and its response is an essential feature of every observability stack. +:example-source-lang: c++ + + [abstract] {description} @@ -7,11 +10,14 @@ To give insight into a request/response flow, the SDK provides a `RequestTracer` interface and ships with both a default implementation as well as modules that can feed the traces to external systems (including OpenTelemetry). -== The Default ThresholdRequestTracer +== The Default TracerProvider By default, the SDK will emit information about requests that are over a configurable threshold every 10 seconds. -Note that if no requests are over the threshold then no event / log will be emitted. +Note that if no requests are over the threshold then no event or log will be emitted. + + +//// It is possible to customize this behavior by modifying the configuration: [source,scala] @@ -54,7 +60,91 @@ The JSON blob emitted looks similar to the following (prettified here for readab } ] ---- +//// + + + + +A complete code example showing how to instrument a Couchbase {cpp} application with `OpenTelemetry`, distributed tracing, and metrics, can be found https://github.com/couchbaselabs/couchbase-cxx-client-demo/blob/main/examples/inventory_with_opentelemetry.cpp[in our GitHub repo]. +The snippets used on this page are taken from that example. + +//// +== Metrics Collection + +The SDK's internal metric collection is activated with `options.metrics().enable(true)`: + +[source,{example-source-lang}] +---- + options.metrics().enable(true); + options.metrics().meter(std::make_shared( + opentelemetry::metrics::Provider::GetMeterProvider()->GetMeter(k_service_name, + k_service_version))); +---- + + +The SDK reads the meter and tracer from the options object at connect time. +OTel metrics and tracing must be wired into cluster options before calling `couchbase::cluster::connect()`: + +[source,{example-source-lang}] +---- + auto meter_provider = apply_opentelemetry_meter_options(options, config.metrics_config); + auto tracer_provider = apply_opentelemetry_tracer_options(options, config.traces_config); + + auto [connect_err, cluster] = + couchbase::cluster::connect(config.connection_string, options).get(); + if (connect_err) { + std::cout << "Unable to connect to the cluster. ec: " << connect_err.message() << "\n"; + } else { + auto collection = cluster.bucket(config.bucket_name) + .scope(config.scope_name) + .collection(config.collection_name); +---- + + +== Exporting Metrics + +In the example, the SDK records per-operation latency histograms and retry/timeout counters, all labelled by bucket, scope, collection, and operation type. +A `PeriodicExportingMetricReader` then pushes those measurements to the configured OTLP endpoint. + +[source,{example-source-lang}] +---- + opentelemetry::sdk::metrics::PeriodicExportingMetricReaderOptions reader_options{}; + reader_options.export_interval_millis = metrics.reader_export_interval; + reader_options.export_timeout_millis = metrics.reader_export_timeout; + + std::unique_ptr reader{ + new opentelemetry::sdk::metrics::PeriodicExportingMetricReader(std::move(exporter), + reader_options) +---- +//// + +== {empty} + +.Complete tracing and metrics sample code: [*Click to open or collapse the listing*] +[%collapsible] +==== +[source,{example-source-lang}] +---- +include::devguide:example$cxx/src/inventory_with_opentelemetry.cpp[] +---- +==== + + + + + + +=== Legacy OpenTracing Integration + +The {name-sdk} does not provide support for `OpenTracing`, since OpenTracing has been sunsetted. + + + + + + +//// For each service (e.g. Key-Value or Query), an entry exists in the outer JSON array. The top N (10 by default) slowest operations are collected and displayed, sorted by the total duration. This promotes quick visibility of the "worst offenders", and more efficient troubleshooting. @@ -202,3 +292,4 @@ val config: Try[ClusterEnvironment] = ClusterEnvironment.builder ---- OpenTracing spans can be wrapped with `OpenTracingRequestSpan.wrap` and passed as parent spans to all SDK operations, in the same way as with OpenTelemetry. +//// diff --git a/modules/project-docs/pages/third-party-integrations.adoc b/modules/project-docs/pages/third-party-integrations.adoc index 0bc12da..44ea5af 100644 --- a/modules/project-docs/pages/third-party-integrations.adoc +++ b/modules/project-docs/pages/third-party-integrations.adoc @@ -1,5 +1,5 @@ = Integrations & Ecosystem -:description: The Couchbase Scala SDK is often used with unofficial and third party tools and applications to integrate into broader language and platform ecosystems, and across data lakes in heterogeneous environments. +:description: The Couchbase {name-sdk} is often used with unofficial and third party tools and applications to integrate into broader language and platform ecosystems, and across data lakes in heterogeneous environments. :navtitle: Integrations :page-toclevels: 2 @@ -13,10 +13,6 @@ include::{version-common}@sdk:shared:partial$integrations.adoc[tag=intro] include::{version-common}@sdk:shared:partial$integrations.adoc[tag=official] -The Couchbase Scala SDK is a first class citizen in the https://spring.io/projects/spring-data-couchbase[Spring Data^] world, and the Scala SDK can leverage that through the JVM. - -Couchbase also supports integrating with xref:spark-connector:ROOT:getting-started.adoc.adoc[Spark]. - === IDE Integrations @@ -27,7 +23,7 @@ https://plugins.jetbrains.com/plugin/22131-couchbase[IntelliJ] family of IDEs an - +//// include::{version-common}@sdk:shared:partial$integrations.adoc[tag=important] Why not take advantage of compatible Java tools? @@ -39,7 +35,7 @@ and https://github.com/couchbaselabs/flink-connector-couchbase[Apache Flink^]. include::{version-common}@sdk:shared:partial$integrations.adoc[tag=other] https://docs.databricks.com/data/data-sources/couchbase.html[Databricks^] provides integration with the Couchbase Scala SDK and Spark. - +//// == See Also