Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
333 changes: 119 additions & 214 deletions source/aggregation.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,241 +18,146 @@ Aggregation
:depth: 2
:class: singlecol

.. toctree::

Filtered Subset </aggregation/filtered-subset/>
Group & Total </aggregation/group-total/>
Unpack Arrays & Group </aggregation/unpack-arrays/>
One-to-One Join </aggregation/one-to-one-join/>
Multi-Field Join </aggregation/multi-field-join/>

.. _nodejs-aggregation-overview:

Overview
--------

In this guide, you can learn how to use **aggregation operations** in
the MongoDB Node.js driver.
In this guide, you can learn how to use the MongoDB Node.js Driver to perform
**aggregation operations**.

Aggregation operations process data in your MongoDB collections and return
computed results. The MongoDB Aggregation framework is modeled on the
concept of data processing pipelines. Documents enter a pipeline comprised of one or
more stages, and this pipeline transforms the documents into an aggregated result.

Aggregation operations are expressions you can use to produce reduced
and summarized results in MongoDB. MongoDB's aggregation framework
allows you to create a pipeline that consists of one or more stages,
each of which performs a specific operation on your data.
To learn more about the aggregation stages supported by the Node.js Driver, see :ref:`Aggregation Stages <>`.
.. todo-- add link here

Analogy
~~~~~~~

You can think of the aggregation pipeline as similar to an automobile factory.
Automobile manufacturing requires the use of assembly stations organized
into assembly lines. Each station has specialized tools, such as
drills and welders. The factory transforms and
assembles the initial parts and materials into finished products.

The **aggregation pipeline** is the assembly line, **aggregation
stages** are the assembly stations, and **expression operators** are the
specialized tools.

Comparing Aggregation and Query Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using query operations, such as the ``find()`` method, you can perform the following actions:

- Select *which documents* to return
- Select *which fields* to return
- Sort the results

Using aggregation operations, you can perform the following actions:

- Perform all query operations
- Rename fields
- Calculate fields
- Summarize data
- Group values

Aggregation operations have some :manual:`limitations </core/aggregation-pipeline-limits/>`:

- Returned documents must not violate the :manual:`BSON-document size limit </reference/limits/#mongodb-limit-BSON-Document-Size>`
The aggregation pipeline is similar to an automobile factory assembly line. An
assembly lines has stations with specialized tools that are used to perform
specific tasks. For example, when building a car, the assembly line begins with
a frame. As the car frame moves though the assembly line, each station adds a
new part. The factory transforms and assembles the initial parts, resulting in
finished cars.

The *aggregation pipeline* is the assembly line, the *aggregation stages* are
the assembly stations, and the *expression operators* are the specialized tools.

Compare Aggregation and Find Operations
---------------------------------------

The following table lists the different tasks you can perform with find
operations compared to what you can achieve with aggregation
operations. The aggregation framework provides expanded functionality
that allows you to transform and manipulate your data.

.. list-table::
:header-rows: 1
:widths: 50 50

* - Find Operations
- Aggregation Operations

* - | Select *certain* documents to return
| Select *which* fields to return
| Sort the results
| Limit the results
| Count the results
- | Select *certain* documents to return
| Select *which* fields to return
| Sort the results
| Limit the results
| Count the results
| Group the results
| Rename fields
| Compute new fields
| Summarize data
| Connect and merge data sets

Server Limitations
------------------

Consider the following :manual:`limitations </core/aggregation-pipeline-limits/>` when performing aggregation operations:

- Returned documents must not violate the :manual:`BSON document size limit </reference/limits/#mongodb-limit-BSON-Document-Size>`
of 16 megabytes.

- Pipeline stages have a memory limit of 100 megabytes by default. You can exceed this
limit by setting the ``allowDiskUse`` property of ``AggregateOptions`` to ``true``. See
the `AggregateOptions API documentation <{+api+}/interfaces/AggregateOptions.html>`__
for more details.

.. important:: $graphLookup exception

The :manual:`$graphLookup
</reference/operator/aggregation/graphLookup/>` stage has a strict
memory limit of 100 megabytes and will ignore ``allowDiskUse``.

References
~~~~~~~~~~

To view a full list of expression operators, see :manual:`Aggregation
Operators </reference/operator/aggregation/>` in the Server manual.

To learn about assembling an aggregation pipeline and view examples, see
:manual:`Aggregation Pipeline </core/aggregation-pipeline/>` in the
Server manual.

To learn more about creating pipeline stages, see :manual:`Aggregation
Stages </reference/operator/aggregation-pipeline/>` in the Server manual.

Runnable Examples
-----------------

The example uses sample data about restaurants. The following code
inserts data into the ``restaurants`` collection of the ``aggregation``
database:

.. literalinclude:: /code-snippets/aggregation/agg.js
:start-after: begin data insertion
:end-before: end data insertion
:language: javascript
:dedent:

.. tip::

For more information on connecting to your MongoDB deployment, see the :doc:`Connection Guide </connect>`.
- Pipeline stages have a memory limit of 100 megabytes by default. If required,
you can exceed this limit by enabling the `AllowDiskUse
<https://mongodb.github.io/node-mongodb-native/6.17/interfaces/AggregateOptions.html#allowDiskUse>`__
property of the ``AggregateOptions`` object that you pass to the
``Aggregate()`` method.

..
Aggregation Example
~~~~~~~~~~~~~~~~~~~

-------------------
..
To perform an aggregation, pass a list of aggregation stages to the
``collection.aggregate()`` method.

In the example, the aggregation pipeline uses the following aggregation stages:

- A :manual:`$match </reference/operator/aggregation/match/>` stage to filter for documents whose
``categories`` array field contains the element ``Bakery``.

- A :manual:`$group </reference/operator/aggregation/group/>` stage to group the matching documents by the ``stars``
field, accumulating a count of documents for each distinct value of ``stars``.

..
.. note::
..
This example uses the ``sample_restaurants.restaurants`` collection
from the :atlas:`Atlas sample datasets </sample-data>`. To learn how to create a
free MongoDB Atlas cluster and load the sample datasets, see the :ref:`Get Started <node-get-started>` guide.
..
The following code example produces a count of the number of bakeries in each borough
of New York City. To do so, the aggregation pipeline uses the following aggregation stages:
..
- A :manual:`$match </reference/operator/aggregation/match/>` stage to filter
for documents whose ``cuisine`` field contains the element ``Bakery``.
..
- A :manual:`$group </reference/operator/aggregation/group/>` stage to group the
matching documents by the ``borough`` field, accumulating a count of documents
for each distinct value in the ``borough`` field.
..
.. literalinclude:: /code-snippets/aggregation/agg.js
:start-after: begin aggregation
:end-before: end aggregation
:language: javascript
:dedent:

..
This example produces the following output:

..
.. code-block:: json
:copyable: false

{ _id: 4, count: 2 }
{ _id: 3, count: 1 }
{ _id: 5, count: 1 }

For more information, see the `aggregate() API documentation <{+api+}/classes/Collection.html#aggregate>`__.

.. _node-aggregation-tutorials-landing:
.. _node-aggregation-tutorials:

Aggregation Tutorials
---------------------

Aggregation tutorials provide detailed explanations of common
aggregation tasks in a step-by-step format. The tutorials are adapted
from examples in the `Practical MongoDB Aggregations book
<https://www.practical-mongodb-aggregations.com/>`__ by Paul Done.

Each tutorial includes the following sections:

- **Introduction**, which describes the purpose and common use cases of the
aggregation type. This section also describes the example and desired
outcome that the tutorial demonstrates.

- **Before You Get Started**, which describes the necessary databases,
collections, and sample data that you must have before building the
aggregation pipeline and performing the aggregation.

- **Tutorial**, which describes how to build and run the aggregation
pipeline. This section describes each stage of the completed
aggregation tutorial, and then explains how to run and interpret the
output of the aggregation.

At the end of each aggregation tutorial, you can find a link to a fully
runnable Node.js code file that you can run in your environment.

.. tip::

To learn more about performing aggregations, see the
:ref:`node-aggregation` guide.

.. _node-agg-tutorial-template-app:

Aggregation Template App
~~~~~~~~~~~~~~~~~~~~~~~~

Before you begin following an aggregation tutorial, you must set up a
new Node.js app. You can use this app to connect to a MongoDB
deployment, insert sample data into MongoDB, and run the aggregation
pipeline in each tutorial.

.. tip::

To learn how to install the driver and connect to MongoDB,
see the :ref:`node-get-started-download-and-install` and
:ref:`node-get-started-create-deployment` steps of the
Quick Start guide.

Once you install the driver, create a file called
``agg_tutorial.js``. Paste the following code in this file to create an
app template for the aggregation tutorials:

.. literalinclude:: /includes/aggregation/template-app.js
:language: javascript
:copyable: true

.. important::

In the preceding code, read the code comments to find the sections of
the code that you must modify for the tutorial you are following.

If you attempt to run the code without making any changes, you will
encounter a connection error.

For every tutorial, you must replace the connection string placeholder with
your deployment's connection string.

.. tip::

To learn how to locate your deployment's connection string, see the
:ref:`node-get-started-connection-string` step of the Quick Start guide.

For example, if your connection string is
``"mongodb+srv://mongodb-example:27017"``, your connection string assignment resembles
the following:

.. code-block:: javascript
:copyable: false

const uri = "mongodb+srv://mongodb-example:27017";

To run the completed file after you modify the template for a
tutorial, run the following command in your shell:

.. code-block:: bash

node agg_tutorial.js

Available Tutorials
~~~~~~~~~~~~~~~~~~~

- :ref:`node-aggregation-filtered-subset`
- :ref:`node-aggregation-group-total`
- :ref:`node-aggregation-arrays`
- :ref:`node-aggregation-one-to-one`
- :ref:`node-aggregation-multi-field`

Additional Examples
~~~~~~~~~~~~~~~~~~~

To view step-by-step explanations of common aggregation tasks, see the
:ref:`node-aggregation-tutorials-landing`.

You can find another aggregation pipeline example in the `Aggregation
Framework with Node.js Tutorial
<https://www.mongodb.com/blog/post/quick-start-nodejs--mongodb--how-to-analyze-data-using-the-aggregation-framework>`_
blog post on the MongoDB website.
..
{ _id = 'Bronx', count = 71 }
{ _id = 'Brooklyn', count = 173 }
{ _id = 'Staten Island', count = 20 }
{ _id = 'Missing', count = 2 }
{ _id = 'Manhattan', count = 221 }
{ _id = 'Queens', count = 204 }

Additional information
----------------------

To view a full list of expression operators, see
:manual:`Aggregation Operators </reference/operator/aggregation/>`.

..
To learn more about assembling an aggregation pipeline and view examples, see
:manual:`Aggregation Pipeline </core/aggregation-pipeline/>`.
..
To learn more about creating pipeline stages and view examples, see
:manual:`Aggregation Stages </reference/operator/aggregation-pipeline/>`.

To learn about explaining MongoDB aggregation operations, see
:manual:`Explain Results </reference/explain-results/>` and
:manual:`Query Plans </core/query-plans/>`.

..
API Documentation
~~~~~~~~~~~~~~~~~
..
For more information about the aggregation operations discussed in this guide, see the
following API documentation:
..
- `Collection() <https://mongodb.github.io/node-mongodb-native/6.17/classes/Collection.html>`__
- `aggregate() <https://mongodb.github.io/node-mongodb-native/6.17/classes/Collection.html#aggregate>`__
- `AggregateOptions <https://mongodb.github.io/node-mongodb-native/6.17/interfaces/AggregateOptions.html>`__
.. try to find $match and $group api links ..
Loading
Loading