query.html

---
layout: default
---

<h1 id="introduction">Introduction</h1>
<h2 id="goals-of-the-query-ui">Goals of the Query UI</h2>
<p>
  This tool will help you translate your research questions into Timur queries,
  which you can run in your browser to generate a data frame. Most research
  questions begin with a root question and a set of desired data points, as well
  as some filtering criteria. For example, "I’m curious about patients with
  positive disease status, under the age of 50. In particular, I also want to
  know their sex at birth and IL-6 levels." Once you receive a data frame with
  this information, you can then run various analyses on your own computer to
  explore relationships among the data. In this document we’ll discuss:
</p>
<ol>
  <li>
    <p>
      The thought process behind converting your research question into a
      question that fits the Timur models.
    </p>
  </li>
  <li><p>How to input that question into the Query UI.</p></li>
  <li>
    <p>
      How to make sure you have a "flat" data frame, which will make your
      subsequent analyses simpler.
    </p>
  </li>
</ol>
<p>
  We’ll start with showing how to use the Map view to understand a project’s
  models, walk through the overall Query UI, and then step through some concrete
  examples, showing how to get from a research question to a data frame.
</p>
<h2 id="section:map">Map and Model Overview</h2>
<p>
  The Timur Map view is available by clicking on the Map link in the top
  navigation bar, once you navigate to your project.
</p>
<figure>
  <img
    src="/assets/images/timur/map/NavigationBar_Map.png"
    id="fig:navigation_bar_map"
    alt="A project’s Map link"
  />
  <figcaption aria-hidden="true">A project’s Map link</figcaption>
</figure>
<p>
  Clicking that brings up a view with a graphical representation of the project
  models, as well as a tabular listing of the selected model’s attributes. By
  default the project model is selected.
</p>
<figure>
  <img
    src="/assets/images/timur/map/ModelView.png"
    id="fig:ipi_map"
    alt="Map for the IPI project"
  />
  <figcaption aria-hidden="true">Map for the IPI project</figcaption>
</figure>
<p>
  We won’t describe the entire Map view here, only the relevant portions to
  constructing queries. Each box in the map represents a specific model, and
  each attribute represents some aspect of that model. You can think of a model
  as a high-level concept (i.e. Bulk RNA sequencing), and the attributes as
  specific details within that concept (i.e. Eisenberg score). Within the Timur
  database, each model contains individual records that are concrete data points
  identified by unique identifier strings, such as an individual library of
  RNASeq or tube of CyTOF.
</p>
<h3 id="paths-between-models">Paths Between Models</h3>
<p>
  Often in this document we will discuss paths between models, and intervening
  models that appear along this path. When you specify a filter or column, the
  Query UI will attempt to find the shortest path between the root model and
  your filter or column model – shortest path meaning the fewest number of
  intervening models between the starting point and the destination. You can see
  the paths on the map, and while you do not have to select the path yourself,
  you will want to note the intervening models, because they may affect how you
  toggle the filter and column settings in your query.
</p>
<p>
  Let’s look at  <a
    href="#fig:ipi_map_patient_subtree"
    data-reference-type="ref"
    data-reference="fig:ipi_map_patient_subtree"
    >1.3</a
  >, where we can use the Patient subtree for some examples. If your root model
  is Patient, and you want to add a filter based on some Flow attribute, the
  path would be Patient -&gt; Sample -&gt; Flow. If your root model is RnaSeq,
  and you want to filter based on some Demographic value, the path in that case
  is RnaSeq -&gt; Sample -&gt; Patient -&gt; Demographic. Links are
  non-parent-child relationships between models, and you can find them by
  viewing the attributes for a model (links currently do not appear visually on
  the graph). Paths may also traverse through these links, if they provide a
  shorter path between two models.
</p>
<figure>
  <img
    src="/assets/images/timur/map/Map_PatientSubtree.png"
    id="fig:ipi_map_patient_subtree"
    alt="Patient Subtree"
  />
  <figcaption aria-hidden="true">Patient Subtree</figcaption>
</figure>
<h3 id="travel-directionality">Travel Directionality</h3>
<p>
  You’ll notice that the visual representation of the map looks like a tree,
  with the project at the top of the tree. We’ll thus frequently use the terms
  "up the tree" and "down the tree" to describe relationships between models.
  This directionality will be important to understand if the relationship is
  one-to-one or one-to-many, which affects the shape of your final data frame.
</p>
<h4 id="section:up-the-tree">Up the tree</h4>
<p>
  The way models are designed, all relationships that go "up the tree" are
  one-to-one relationships. When doing queries, one-to-one relationships result
  in a single data point inside of a data frame cell, which makes analysis
  simpler.
</p>
<p>
  For example, looking at  <a
    href="#fig:ipi_map"
    data-reference-type="ref"
    data-reference="fig:ipi_map"
    >1.2</a
  >, we can see that Patient is above Sample in the tree. So going from Sample
  to Patient means we’re moving up the tree – and each Sample belongs to one and
  only one Patient. In a data frame, this might look like  <a
    href="#table:up-the-tree-example"
    data-reference-type="ref"
    data-reference="table:up-the-tree-example"
    >1.1</a
  >.
</p>
<div id="table:up-the-tree-example">
  <table>
    <caption>
      Example data frame going up the tree
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Sample</th>
        <th style="text-align: center">Patient</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">Patient001.T1</td>
        <td style="text-align: center">Patient001</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Patient001.N1</td>
        <td style="text-align: center">Patient001</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Patient002.T1</td>
        <td style="text-align: center">Patient002</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Patient003.N1</td>
        <td style="text-align: center">Patient003</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Note that knowing the relationship "up the tree" gives us no information about
  the inverse direction, down the tree. To determine the relationship type going
  down the tree, we’ll need to inspect the attributes.
</p>
<h4 id="down-the-tree">Down the tree</h4>
<p>
  Most relationships that are "down the tree" are one-to-many, which can result
  in data frames with nested information. If you take our previous example of
  Patients and Samples, we can look at the Patient attributes to see what kind
  of relationship exists between the two when going down the tree. This can be
  seen in  <a
    href="#fig:ipi_patient_model"
    data-reference-type="ref"
    data-reference="fig:ipi_patient_model"
    >1.4</a
  >.
</p>
<figure>
  <img
    src="/assets/images/timur/map/PatientModel.png"
    id="fig:ipi_patient_model"
    alt="Patient model"
  />
  <figcaption aria-hidden="true">Patient model</figcaption>
</figure>
<p>
  We see that Sample is a ‘collection‘ type attribute. This means that a single
  Patient has zero or more Samples. In a non-flat data frame, querying this out
  results in a nested data frame, like in  <a
    href="#table:nested-data-frame-example"
    data-reference-type="ref"
    data-reference="table:nested-data-frame-example"
    >1.2</a
  >.
</p>
<div id="table:nested-data-frame-example">
  <table>
    <caption>
      Example nested data frame
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Patient</th>
        <th style="text-align: center">Sample</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">Patient001</td>
        <td style="text-align: center">Patient001.T1, Patient001.N1</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Patient002</td>
        <td style="text-align: center">Patient002.T1</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Patient003</td>
        <td style="text-align: center">Patient003.N1</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  To analyze a nested data frame like this, you would have to extract the nested
  data yourself. To get a flat data frame, you can either restructure your query
  to take advantage of  <a href="#section:up-the-tree"
    >up the tree relationships</a
  >, or use  <a href="#section:column-slicing">column slicing</a> and extract
  only the data values you are interested in.
</p>
<p>
  For one-to-one "down the tree" relationships (they are rare, but do exist in
  several projects), you do not have to worry about nested data.
</p>
<h2 id="query-ui-overview">Query UI Overview</h2>
<p>
  The Timur Query UI is available by clicking on the Query link in the top
  navigation bar, once you navigate to your project.
</p>
<figure>
  <img
    src="/assets/images/timur/query/NavigationBar_Query.png"
    id="fig:navigation_bar_query"
    alt="A project’s Query link"
  />
  <figcaption aria-hidden="true">A project’s Query link</figcaption>
</figure>
<p>
  Clicking that brings up an empty form builder, with a selector for a "Root
  Model".
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RootModel.png"
    id="fig:query_unselected_root_model"
    alt="Query form with unselected root model"
  />
  <figcaption aria-hidden="true">
    Query form with unselected root model
  </figcaption>
</figure>
<p>Once you select a root model, the rest of the form builder appears.</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_BlankForm.png"
    id="fig:query_blank_form"
    alt="Blank query form"
  />
  <figcaption aria-hidden="true">Blank query form</figcaption>
</figure>
<h3 id="root-model">Root Model</h3>
<p>
  The root model forms the starting point of your query. The identifier of the
  model is the left-most column of your data frame and acts as the unique
  identifier for the output. Generally, it is the starting point of the question
  you’ve formulated. For example, if you want to know "All the patients that are
  COVID positive and their IL-6 levels", you would most likely want to start
  with Patient as your root model. We will explore a couple different ways to
  extract the same data with different root models, but we’ll start with this
  more straightforward approach in each example.
</p>
<p>
  Note that once you select a root model, its identifier is automatically added
  as a  <a href="#section:columns">column</a> and appears in the  <a
    href="#section:data-frame"
    >output data frame</a
  >. Unlike other columns which you will add later, this column can only be
  removed by selecting a different root model.
</p>
<h3 id="where-filters">Where Filters</h3>
<p>
  Most research questions have some sort of constraint on the data that you want
  to analyze. You can think of Where Filters as applying those constraints and
  limiting the rows that appear in your data frame. This section of the form
  builder allows you to specify the set of filters that will narrow down the
  returned data.
</p>
<p>
  As an added bonus, remember that where filters are one way that we might be
  able to narrow down a nested data frame into a flat data frame.
</p>
<h4 id="specifying-a-filter">Specifying a filter</h4>
<p>
  To specify a filter, you will need three or four basic pieces of information:
</p>
<ol>
  <li><p>The model you want to filter on.</p></li>
  <li>
    <p>
      Clauses that you want to apply to the filter model or its children models.
      Each clause is composed of:
    </p>
    <ol>
      <li><p>(sometimes) An "Any" or "Every" statement.</p></li>
      <li><p>The clause model (can be same as the filter model).</p></li>
      <li><p>The clause model’s attribute you want to filter on.</p></li>
      <li><p>The operator you want to apply.</p></li>
      <li><p>(sometimes) The operand to evaluate the operator against.</p></li>
    </ol>
  </li>
</ol>
<p>
  For example, if you wanted to filter on Patients with age greater than 50, you
  would need to know that "age" in IPI is contained in the demographic table.
  Note that only models that have a valid path from your root model will appear
  in the filter’s Model selector.
</p>
<p>
  Once you have selected a filter model, you will need to add one or more
  clauses to your filter. A clause is a condition on a specific model – either
  the filter model itself, or a child of the filter model. If the clause model
  has a one-to-many relationship with the filter model, you will also be able to
  select <code>Any</code> or <code>Every</code> as part of the condition.
</p>
<p>
  Once you determine the right clause model, you’ll have to determine the
  attribute name. For non-table models, you can use the  <a href="#section:map"
    >Map view</a
  >
  to determine the attribute you want. For tables, you will have to inspect the
  table and find the right "Name" or "Value" that you want to filter on. The
  simplest way to do that is probably with the Search page, to view the raw
  table data.
</p>
<p>
  One tricky operator to apply is <code>Is present</code> or
  <code>Is missing</code> on models. When you want to know if an attribute on a
  model is populated, you might specify a filter like "Samples where
  <code>tissue_type</code> Is present", to mean "Sample records where the
  <code>tissue_type</code> field has some data provided". When you attempt to do
  the same with a "collection" type attribute (i.e. Samples with RnaSeq data),
  you cannot do a "Samples where RnaSeq Is present" query, since
  <code>rna_seq</code> will not appear as an attribute for Sample. Instead, you
  have to add a filter like "RnaSeq where <code>tube_name</code> Is present",
  using the identifier of the RnaSeq model.
</p>
<p>
  If using the <code>In</code> or <code>Not in</code> operator, you have to
  provide a comma-separated string with no spaces. i.e. if you want Patient IDs
  in the set Patient 5, Patient 9, and Patient 11, you would construct the
  operand as:
</p>
<pre><code>    Patient5,Patient9,Patient11</code></pre>
<p>
  As you add and edit filters, you can see them appear in the  <a
    href="#section:query-preview"
    >query preview</a
  >. While the exact syntax of that section may not be very intuitive (and that
  is okay!), hopefully as you edit the filters you can see how your filters
  affect the query and the output data frame.
</p>
<p>
  You can check  <a
    href="#table:appendix-where-filter-operators"
    data-reference-type="ref"
    data-reference="table:appendix-where-filter-operators"
    >3.1</a
  >
  for a list of operators for each type of attribute.
</p>
<h4 id="any-vs.-every">Any vs. Every</h4>
<p>
  When filters traverse across models, any one-to-many relationship creates the
  opportunity to also specify an "Any" or "Every" operator. These models are
  calculated by the tool and will appear to the left of the filter model. For
  example, when using the Patient as a root model, if we add in a Sample filter
  to only return samples with <code>tissue_type</code> as "Primary" we’ll see a
  selector appear with "Any Sample" or "Every Sample" as options. This is seen
  in  <a
    href="#fig:query_any_every_example"
    data-reference-type="ref"
    data-reference="fig:query_any_every_example"
    >1.8</a
  >.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_AnyEveryExample.png"
    id="fig:query_any_every_example"
    alt="Any or every sample"
  />
  <figcaption aria-hidden="true">Any or every sample</figcaption>
</figure>
<p>
  By default, "Any" is selected. This means that you want records where Any of
  the Samples meet the criteria. If a Patient has three Samples, one or more of
  them must be labelled as Primary in order for the Patient to be included in
  the output data frame. Only Patients with zero Primary Samples will be left
  out of the data set. If you select "Every", that means you expect every single
  Sample attached to that Patient to meet the criteria. So if a Patient has
  three Samples, all of them must be labelled as Primary for that Patient to be
  included in the output data frame. Patients with one or more non-Primary
  Samples will be left out of the data set.
</p>
<p>
  When you have multiple models in the path from your root model to the filter
  model, you will have an Any / Every toggle for each one-to-many relationship
  in the path, and each combination of selections will result in a different set
  of output data. An example is shown in  <a
    href="#fig:query_multiple_any_every_example"
    data-reference-type="ref"
    data-reference="fig:query_multiple_any_every_example"
    >1.9</a
  >, where Patient has a one-to-many relationship with Sample, and Sample has a
  one-to-many relationship with RnaSeq, and so two Any / Every toggles appear
  when you add an RnaSeq filter.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_MultipleAnyEverySelectors.png"
    id="fig:query_multiple_any_every_example"
    alt="Multiple any or every selectors"
  />
  <figcaption aria-hidden="true">Multiple any or every selectors</figcaption>
</figure>
<p>
  Note that the same Any / Every reasoning applies to the clauses within a given
  filter, so you can control how a clause affects the filter’s output using
  those selectors.
</p>
<h4 id="and-and-or">And and Or</h4>
<p>
  Sometimes you may want to combine filters using AND or OR logic. For example,
  I want Patients who are older than 50 OR younger than 25 (or conversely, I
  want Patients who are younger than 50 AND older than 25). Currently, the Query
  UI supports a very simple version of this – if you need to construct more
  complicated queries, please let the Data Library Engineering team know, and we
  can add that ability as a feature request.
</p>
<p>
  By default, all filters are applied as AND filters. However, the checkboxes on
  the left of each filter, as shown in  <a
    href="#fig:query_any_every_example"
    data-reference-type="ref"
    data-reference="fig:query_any_every_example"
    >1.8</a
  >, let you add one layer of OR filters. All checked filters are aggregated
  into a single OR statement that is then ANDed with the other filters.
</p>
<p>
  For example, a query with filters as shown in  <a
    href="#fig:query_patient_between_25_50"
    data-reference-type="ref"
    data-reference="fig:query_patient_between_25_50"
    >1.10</a
  >
  would be translated as "Patients between the ages of 25 and 50".
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PatientBetween25_50.png"
    id="fig:query_patient_between_25_50"
    alt="Patients between ages of 25 and 50"
  />
  <figcaption aria-hidden="true">Patients between ages of 25 and 50</figcaption>
</figure>
<p>
  Whereas a query with filters as shown in  <a
    href="#fig:query_patient_not_between_25_50"
    data-reference-type="ref"
    data-reference="fig:query_patient_not_between_25_50"
    >1.11</a
  >
  would be translated as "Patients older than 50 OR younger than 25".
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PatientOlder50_Younger25.png"
    id="fig:query_patient_not_between_25_50"
    alt="Patients older than 50 or younger than 25"
  />
  <figcaption aria-hidden="true">
    Patients older than 50 or younger than 25
  </figcaption>
</figure>
<p>
  You can then combine these with other filters as shown in  <a
    href="#fig:query_patient_not_between_25_50_bmi_35"
    data-reference-type="ref"
    data-reference="fig:query_patient_not_between_25_50_bmi_35"
    >1.12</a
  >, which would be translated as "Patients older than 50 OR younger than 25,
  who have a bmi greater than 35".
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PatientOlder50_Younger25_bmi.png"
    id="fig:query_patient_not_between_25_50_bmi_35"
    alt="Patients older than 50 or younger than 25 and with bmi greater than 35"
  />
  <figcaption aria-hidden="true">
    Patients older than 50 or younger than 25 and with bmi greater than 35
  </figcaption>
</figure>
<p>
  Note that you can also combine "And"-type statements as multiple clauses on a
  single filter. Sometimes (especially with child-models), this is required to
  answer a specific question. The usage of multiple clauses versus multiple
  filters depends upon the question and in some cases, either may result in the
  same answer.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PatientBetween25_50_clauses.png"
    id="fig:query_patient_between_25_50_clauses"
    alt="Patients between ages of 25 and 50, using clauses"
  />
  <figcaption aria-hidden="true">
    Patients between ages of 25 and 50, using clauses
  </figcaption>
</figure>
<h3 id="section:columns">Columns</h3>
<p>
  Where filters adjust which records of data to traverse, and thus affect the
  contents of your data frame’s rows. But it takes more than rows to fill a data
  frame. In the Columns section, you’ll pick what attributes to use to fill your
  columns. This is the third section of the UI and appears auto-populated with
  the identifier of your root model. You cannot directly remove this column, but
  you can provide an alternate label to change how it appears in your data
  frame. The default label is simply <code>model_name.attribute_name</code>.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_ColumnSection.png"
    id="fig:query_column_section"
    alt="Column section with root model identifier"
  />
  <figcaption aria-hidden="true">
    Column section with root model identifier
  </figcaption>
</figure>
<p>
  Note that the columns are independent of the filters, so you can select
  columns on models that do not have filters.
</p>
<h4 id="specifying-a-column">Specifying a Column</h4>
<p>
  To specify a column, you only need the join model name and the attribute name:
</p>
<ol>
  <li><p>The model you want data from.</p></li>
  <li><p>The model’s attribute you want data for.</p></li>
</ol>
<p>
  Note that the first input box for each column is for a
  <code>Display Label</code> – this is optional, and it will default to
  <code>model_name.attribute_name</code> if you leave the input blank.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_NewColumnRow.png"
    id="fig:query_new_column_row"
    alt="New column row"
  />
  <figcaption aria-hidden="true">New column row</figcaption>
</figure>
<p>
  Once you define the column’s model and attribute, it will appear in the  <a
    href="#section:data-frame"
    >data frame</a
  >
  at the bottom of the page. If you edit the <code>Display Label</code> for a
  column, the data frame column heading should also change. You may need to do
  this to prevent duplicate colum headings, which would confuse downstream R or
  Python analysis.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_ColumnDisplayLabel.png"
    id="fig:query_column_display_label"
    alt="Edited column labels"
  />
  <figcaption aria-hidden="true">Edited column labels</figcaption>
</figure>
<p>
  As you add and edit columns and slices, you can also see them appear in the
   <a href="#section:query-preview">query preview</a>. While the exact syntax of
  that section may not be very intuitive (and that is okay!), hopefully as you
  edit the columns and slices you can intuit how they affect the query and the
  output data frame.
</p>
<h4 id="section:column-slicing">Slicing</h4>
<p>
  When your research question traverses across models that have one-to-many
  relationships, many times we are only interested in a subset of those
  relationships. In order to prevent nested data in your data frame, you can
  construct one or more Slices to select a subset of the nested data and get a
  flat data frame.
</p>
<p>
  For example, you may only be interested in Tumor samples, in which case you
  might slice on the Patient -&gt; Sample relationship. Column slicing gives you
  a subset of column data (as opposed to Filters, which give you a subset of row
  data). Going back to our example in  <a
    href="#table:nested-data-frame-example"
    data-reference-type="ref"
    data-reference="table:nested-data-frame-example"
    >1.2</a
  >
  where Patient is one-to-many with Sample, we said that one way to construct a
  flat data frame with the same information was to use Sample as the root model.
  However, perhaps we are collecting additional data, and we really want to keep
  Patient as our root model. If we only want Tumor samples, we could construct a
  slice to select only the Tumor samples out of each Sample column, as in  <a
    href="#fig:query_sample_slice_tumor"
    data-reference-type="ref"
    data-reference="fig:query_sample_slice_tumor"
    >1.17</a
  >.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_SampleSliceTumor.png"
    id="fig:query_sample_slice_tumor"
    alt="Slice out tumor samples"
  />
  <figcaption aria-hidden="true">Slice out tumor samples</figcaption>
</figure>
<p>
  Assuming only a single tumor sample per patient, this would also result in a
  flat data frame. Slicing columns in this fashion is particularly helpful with
  clinical data, which generally appear in Timur as tables.
</p>
<p>
  Note that slice construction requires the same set of information as a filter:
</p>
<ol>
  <li><p>The model you want to filter on.</p></li>
  <li><p>The model’s attribute you want to filter on.</p></li>
  <li><p>The operator you want to apply.</p></li>
  <li><p>(sometimes) The operand to evaluate the operator against.</p></li>
</ol>
<p>
  You can also slice out Matrix data, which is currently used for gene
  expression and gene count data in the RnaSeq model. The main difference is
  that the only slice operator in this case is <code>Slice</code>, and you then
  provide a comma-separated list of Ensembl Gene Ids, with no spaces:
</p>
<pre><code>    ENSG00000000003,ENSG00000000005,ENSG00000000419</code></pre>
<p>
  An example of a matrix slice can be seen in  <a
    href="#fig:query_matrix_slicing_example"
    data-reference-type="ref"
    data-reference="fig:query_matrix_slicing_example"
    >1.18</a
  >.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_MatrixSlicingExample.png"
    id="fig:query_matrix_slicing_example"
    alt="Matrix Slicing Example"
  />
  <figcaption aria-hidden="true">Matrix Slicing Example</figcaption>
</figure>
<p>
  You can check  <a
    href="#table:appendix-column-slice-operators"
    data-reference-type="ref"
    data-reference="table:appendix-column-slice-operators"
    >3.2</a
  >
  for a list of operators for each type of column slice attribute.
</p>
<h3 id="section:query-preview">Query Preview</h3>
<p>
  As you construct your query by adding filters and columns, there is a small
  text window that shows you what the raw Timur query will look like. It has a
  green outline, as seen in  <a
    href="#fig:query_preview"
    data-reference-type="ref"
    data-reference="fig:query_preview"
    >1.19</a
  >. While you should not expect to understand the exact syntax of this string,
  it should be helpful for you to see how changing the filters and columns
  changes this raw query. As you get more familiar with the Query UI, you may be
  able to intuit the kind of data in your data frame, based on this raw query
  string.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_Preview.png"
    id="fig:query_preview"
    alt="Query preview string"
  />
  <figcaption aria-hidden="true">Query preview string</figcaption>
</figure>
<h3 id="section:data-frame">Data Frame</h3>
<p>
  The bottom pane of the tool includes a set of control toggles and buttons as
  well as a data frame. While you contruct your query, the columns will appear
  in the data frame. This gives you an idea of what your final data will look
  like.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_EmptyDataFrame.png"
    id="fig:query_empty_data_frame"
    alt="Empty Data Frame"
  />
  <figcaption aria-hidden="true">Empty Data Frame</figcaption>
</figure>
<h4 id="reset-query">Reset query</h4>
<p>
  If you would like to reset the entire query, you can click this button to
  remove all form entries and start over.
</p>
<h4 id="query">Query</h4>
<p>
  Once you are satisfied with your data frame, you can click the
  <code>Query</code> button at the top right of the data frame. Once data is
  returned from the server, you will see it appear in the data frame. You can
  navigate between pages or set a different number of items per page. Note that
  setting a different number of items per page requires you to re-click the
  <code>Query</code> button to re-fetch data.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PopulatedDataFrame_Flattened_Expanded.png"
    id="fig:query_populated_data_frame_flattened_expanded"
    alt="Populated Data Frame"
  />
  <figcaption aria-hidden="true">Populated Data Frame</figcaption>
</figure>
<h4 id="download-tsv">Download TSV</h4>
<p>
  To download all records in a tab-separated file, you can click the
  <code>Download TSV</code> button. This will get you all records from the query
  results. Note that with larger data frames or matrix data, the download may
  take a while to complete.
</p>
<h4 id="copy-link">Copy link</h4>
<p>
  You can share and bookmark the query you’ve built, using the URL. For your
  convenience, clicking this button will put the entire URL into your clipboard,
  so you can paste it / share it with others. You can also bookmark the URL in
  your browser to save a specific query.
</p>
<h4 id="nesting-and-expanding-matrices">Nesting and Expanding Matrices</h4>
<p>
  One thing you may have noticed from  <a
    href="#fig:query_populated_data_frame_flattened_expanded"
    data-reference-type="ref"
    data-reference="fig:query_populated_data_frame_flattened_expanded"
    >1.21</a
  >
  is that we only specified two columns, but there are four columns in the data
  frame. This is because the default behavior of the tool is to expand matrix
  slices such that each data point is in its own, unique cell. This is generally
  more convenient for analysis.
</p>
<p>
  A toggle does exist for you to change that behavior. Toggling that to nested
  matrices results in the expected two columns, but leaves the matrix data all
  joined in a single cell, with no clear labelling of which data point belongs
  to which Ensembl gene code, as seen in  <a
    href="#fig:query_populated_data_frame_nested_matrix"
    data-reference-type="ref"
    data-reference="fig:query_populated_data_frame_nested_matrix"
    >1.22</a
  >.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PopulatedDataFrame_NestedMatrix.png"
    id="fig:query_populated_data_frame_nested_matrix"
    alt="Data Frame with Nested Matrix"
  />
  <figcaption aria-hidden="true">Data Frame with Nested Matrix</figcaption>
</figure>
<p>
  If you include a matrix attribute as a column but do not slice it, and if the
  toggle is switched to <code>Expand matrices</code>, the UI will expand the
  column with all possible gene codes. This results in over 58,000 columns! The
  UI will only render a maximum of 10, and to see the entire data set, you will
  have to download the TSV. You will see a warning for this, as seen in  <a
    href="#fig:query_column_limit_warning"
    data-reference-type="ref"
    data-reference="fig:query_column_limit_warning"
    >1.23</a
  >. Downloading this TSV may be a slow operation, since the TSV is generated in
  your browser. If you need to download an entire gene expression matrix, the
  Search page may offer faster performance, once you have identified the
  <code>rna_seq</code> records you want to pull data for. Alternatively, you can
  add a column slice to your query and specify just the genes you need.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_ColumnLimitWarning.png"
    id="fig:query_column_limit_warning"
    alt="Warning about unrendered columns"
  />
  <figcaption aria-hidden="true">Warning about unrendered columns</figcaption>
</figure>
<h4 id="nested-or-flattened-data-frame">Nested or Flattened Data Frame</h4>
<p>
  Another toggle for the data frame is to use a nested or flattened view – with
  the default being a flattened view. While your goal should be to construct the
  query in such a way as to get a flattened view, it may be difficult to realize
  when you have not done so. Changing the value of this toggle will reveal if
  your actual data is flat or not. If your data is flat, changing this value
  between <code>Nested</code> and <code>Flattened</code> will not change the
  data frame. However, if your query data is not flat, changing the value to
  <code>Nested</code> will reveal additional labels and data points, as can be
  seen in  <a
    href="#fig:query_populated_data_frame_nested_expanded"
    data-reference-type="ref"
    data-reference="fig:query_populated_data_frame_nested_expanded"
    >1.24</a
  >.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PopulatedDataFrame_Nested_Expanded.png"
    id="fig:query_populated_data_frame_nested_expanded"
    alt="Populated Data Frame, Nested"
  />
  <figcaption aria-hidden="true">Populated Data Frame, Nested</figcaption>
</figure>
<h3 id="section:errors">Messages and Errors</h3>
<p>
  As you explore queries, you may enounter messages at the top of the screen.
  You can dismiss the messages by clicking on the green checkmark on the left.
  Messages persist on the screen until you dismiss them, even if you take other
  actions on the page, like re-running a query.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_ErrorMessage.png"
    id="fig:query_error_message"
    alt="Error Notification"
  />
  <figcaption aria-hidden="true">Error Notification</figcaption>
</figure>
<p>
  If your query has no results, you will see a message like
  <code>Page 1 not found</code>.
</p>
<p>
  However, sometimes you may construct an invalid query due to a bug, or more
  likely, an invalid operand value (i.e. typing in a text operand for a date
  attribute). In that case, the message may be more obscure, like:
</p>
<pre><code>    Server Error</code></pre>
<p>
  In those cases you will want to check over your query to make sure you have
  valid selections for filters and slices. We will be working over time to
  improve error catching and messaging, and reduce how often obscure error
  messages are returned. If you are unsure how to proceed, or if you think there
  is a bug with a valid query, please reach out to the Data Library team.
</p>
<h1 id="examples">Examples</h1>
<p>
  In this section we will dissect several example queries across different
  projects. You may find that one query is similar to your research question,
  and you can build off of it. Each example will be structured with the
  following sections:
</p>
<ol>
  <li><p>Specify the research question.</p></li>
  <li><p>Identify the relevant models to answer the question.</p></li>
  <li><p>Input the query into the Query UI.</p></li>
  <li><p>Understand the output data frame.</p></li>
</ol>
<h2 id="ipi">IPI</h2>
<h3 id="gene-expression-from-rna-seq-single-compartment">
  Gene expression from RNA Seq, single compartment
</h3>
<h4 id="question">Question</h4>
<p>
  We are interested in exploring the gene expression from Bulk RNA Seq data in
  the IPI data set, specifically for the stroma compartment. We have a list of
  specific genes that we are interested in, so we will want to extract their
  data only.
</p>
<h4 id="models">Models</h4>
<p>
  From the IPI Map view, we can see that there is a model called
  <code>rna_seq</code>, so this seems like a good place to start. Once we click
  on it, we’ll see that there are a lot of attributes to get familiar with!
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_Model.png"
    id="fig:query_rna_seq_model"
    alt="IPI Bulk RNA Seq Model"
  />
  <figcaption aria-hidden="true">IPI Bulk RNA Seq Model</figcaption>
</figure>
<p>
  We know that IPI uses the <code>eisenberg_score</code> attribute as a quality
  control metric for Bulk RNA Seq data, and so we will want to only collect
  "good" records in our final data set. Let’s keep in mind that we will want to
  add in a filter on this attribute.
</p>
<p>
  We see that gene expression data is kept in the
  <code>gene_tpm</code> attribute. Since it is a matrix keyed to Ensembl gene
  ids, we will need to find the Ensembl ids (not Hugo names) of the genes we are
  interested in. For this example, we will simply use the first three options:
</p>
<pre><code>    ENSG00000000003,ENSG00000000005,ENSG00000000419</code></pre>
<p>
  Because we are only interested in exploring the stroma compartment, we will
  also have to use the <code>compartment</code> attribute to narrow down our
  results.
</p>
<p>
  Looking at the above criteria, it seems like our research question might be
  formulated along the lines of:
</p>
<pre><code>    From the rna_seq model, I want the gene_tpm data for genes ENSG00000000003, ENSG00000000005, and ENSG00000000419, but only in the stroma compartment and with an eisenberg_score greater than 7 (since someone told me that 7 is a good cutoff).</code></pre>
<h4 id="ui-input">UI Input</h4>
<p>Let’s translate the general question we’ve formulated into the Query UI.</p>
<pre><code>    From the rna_seq model</code></pre>
<p>indicates that the root model should be <code>rna_seq</code>.</p>
<pre><code>    I want the gene_tpm data for genes ENSG00000000003, ENSG00000000005, and ENSG00000000419,</code></pre>
<p>
  becomes a column, where the model is <code>rna_seq</code> and the attribute is
  <code>gene_tpm</code>. Because we want only a subset of the gene data, we’ll
  add a slice with the operand of
</p>
<pre><code>    ENSG00000000003,ENSG00000000005,ENSG00000000419</code></pre>
<p>Lastly,</p>
<pre><code>    but only in the stroma compartment and with an eisenberg_score greater than 7</code></pre>
<p>
  could become two different filters. They would have the following settings:
</p>
<div id="table:filter-settings-for-rna-seq-example">
  <table>
    <caption>
      Filter settings for Bulk RNA Seq example
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">stroma</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">eisenberg_score</td>
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">7</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Note that in this case, you could also create a single filter with two
  clauses:
</p>
<div id="table:filter-settings-for-rna-seq-example-clauses">
  <table>
    <caption>
      Filter settings for Bulk RNA Seq example, using clauses
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center" rowspan="2">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">stroma</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">eisenberg_score</td>
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">7</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_rna_seq_data_frame"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_data_frame"
    >2.2</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/ipi/query?rootModel=rna_seq&amp;recordFilters=%5B%7B%22modelName%22%3A%22rna_seq%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22stroma%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22rna_seq%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22eisenberg_score%22%2C%22operator%22%3A%22%3A%3A%3E%22%2C%22operand%22%3A%227%22%2C%22attributeType%22%3A%22number%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22rna_seq%22%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22rna_seq.tube_name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22gene_tpm%22%2C%22operator%22%3A%22%3A%3Aslice%22%2C%22operand%22%3A%22ENSG00000000003%2CENSG00000000005%2CENSG00000000419%22%2C%22attributeType%22%3A%22matrix%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame">Data Frame</h4>
<p>
  You should see the first page of data in your browser, and can check out other
  pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_DataFrame.png"
    id="fig:query_rna_seq_data_frame"
    alt="IPI Bulk RNA Seq gene eata"
  />
  <figcaption aria-hidden="true">IPI Bulk RNA Seq gene eata</figcaption>
</figure>
<h3 id="gene-expression-from-rna-seq-multiple-compartments">
  Gene expression from RNA Seq, multiple compartments
</h3>
<h4 id="question-1">Question</h4>
<p>
  We are interested in exploring the expression of Progranulin from Bulk RNA Seq
  data in the IPI data set, specifically for the T cell and Myeloid
  compartments. We want to see if they are well expressed or not, so we want to
  see the expression value across all records regardless of quality score.
</p>
<h4 id="models-1">Models</h4>
<p>
  From the IPI Map view, we can see that there is a model called
  <code>rna_seq</code>, so this seems like a good place to start. If you want to
  re-familiarize yourself with the model, you can check out  <a
    href="#fig:query_rna_seq_model"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_model"
    >2.1</a
  >.
</p>
<p>
  Recall that gene expression data is kept in the
  <code>gene_tpm</code> attribute. Since it is a matrix keyed to Ensembl gene
  ids, we will need to find the Ensembl id for Progranulin. From the  <a
    href="https://www.uniprot.org/uniprot/K7EKL3"
    >Uniprot website</a
  >, we find that the Ensembl id for Progranulin is
  <code>ENSG00000030582</code>.
</p>
<p>
  Because we are only interested in exploring the T cell and myeloid
  compartments, we will also have to use the <code>compartment</code> attribute
  to narrow down our results.
</p>
<p>
  Looking at the above criteria, it seems like our research question might be
  formulated along the lines of:
</p>
<pre><code>    From the rna_seq model, I want the gene_tpm data for gene ENSG00000030582, but only in the tcell and myeloid compartments.</code></pre>
<h4 id="ui-input-1">UI Input</h4>
<p>Let’s translate the general question we’ve formulated into the Query UI.</p>
<pre><code>    From the rna_seq model</code></pre>
<p>indicates that the root model should be <code>rna_seq</code>.</p>
<pre><code>    I want the gene_tpm data for gene ENSG00000030582</code></pre>
<p>
  becomes a column, where the model is <code>rna_seq</code> and the attribute is
  <code>gene_tpm</code>. Because we want only a subset of the gene data, we’ll
  add a slice with the operand of
</p>
<pre><code>    ENSG00000030582</code></pre>
<p>Lastly,</p>
<pre><code>    but only in the tcell and myeloid compartments</code></pre>
<p>
  can be implemented in a couple of ways, both of which result in the same data
  set. One way would be to use an <code>In</code> operator, like in  <a
    href="#table:in-operator-for-multiple-compartments"
    data-reference-type="ref"
    data-reference="table:in-operator-for-multiple-compartments"
    >2.3</a
  >.
</p>
<div id="table:in-operator-for-multiple-compartments">
  <table>
    <caption>
      In operator for multiple compartments
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">In</td>
        <td style="text-align: center">tcell,myeloid</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Another way to think about the filtering is to create two separate filters,
  but join them with an "OR" operator. This would look like  <a
    href="#table:or-multiple-compartments"
    data-reference-type="ref"
    data-reference="table:or-multiple-compartments"
    >2.4</a
  >.
</p>
<div id="table:or-multiple-compartments">
  <table>
    <caption>
      Or multiple compartment operators
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">tcell</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">myeloid</td>
      </tr>
    </tbody>
  </table>
</div>
<p>In the latter case, we need to make sure to check the OR checkboxes.</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_OrCompartments.png"
    id="fig:query_rna_seq_or_compartments"
    alt="Using OR with filters"
  />
  <figcaption aria-hidden="true">Using OR with filters</figcaption>
</figure>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_rna_seq_two_compartments_data_frame"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_two_compartments_data_frame"
    >2.4</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser-1">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/ipi/query?rootModel=rna_seq&amp;recordFilters=%5B%7B%22modelName%22%3A%22rna_seq%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22tcell%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22rna_seq%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22myeloid%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B1%2C0%5D&amp;columns=%5B%7B%22model_name%22%3A%22rna_seq%22%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22rna_seq.tube_name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22gene_tpm%22%2C%22operator%22%3A%22%3A%3Aslice%22%2C%22operand%22%3A%22ENSG00000030582%22%2C%22attributeType%22%3A%22matrix%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame-1">Data Frame</h4>
<p>
  You should see the first page of data in your browser, and can check out other
  pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_TwoCompartments_DataFrame.png"
    id="fig:query_rna_seq_two_compartments_data_frame"
    alt="IPI Bulk RNA Seq gene data from two compartments"
  />
  <figcaption aria-hidden="true">
    IPI Bulk RNA Seq gene data from two compartments
  </figcaption>
</figure>
<p>
  Note that records with no gene expression data will have blank data in the
  data frame. To remove such blank fields, we could also add a filter on
  <code>rna_seq</code> model where <code>gene_tpm</code> "Is Present".
</p>
<h4 id="expanding-on-this-query">Expanding on this query</h4>
<p>
  Note that the compartment is included in record names for this IPI data, but
  we might also wish to add a separate column with the
  <code>rna_seq</code> model’s <code>compartment</code> attribute, in order to
  make this easier to parse in our data frame.
</p>
<h3 id="section:gene-expression-specific-samples">
  Gene expression from RNA Seq, specific samples
</h3>
<h4 id="question-2">Question</h4>
<p>
  We are interested in exploring all of the gene expression data from Bulk RNA
  Seq data in the IPI data set, for a given set of samples that we’ve identified
  through other means.
</p>
<p>
  Note, we should be patient with getting results from this kind of query,
  because gene expression data is very large and may take awhile to load from
  the server when not selecting specific genes. Also, because the full gene
  expression data set is so large, we shouldn’t expect it to render inside our
  browser. Instead, we will have to download the TSV results without previewing
  the entire data frame.
</p>
<h4 id="models-2">Models</h4>
<p>
  From the IPI Map view, we can see that there is a model called
  <code>rna_seq</code>, so this seems like a good place to start. If you want to
  re-familiarize yourself with the model, you can check out  <a
    href="#fig:query_rna_seq_model"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_model"
    >2.1</a
  >.
</p>
<p>
  Because we are only interested in exploring the samples that we have
  pre-identified, we will use the <code>sample</code> attribute. Our target
  sample set could be much larger in practice, but for the purpose of this
  example, let’s say these are the only two samples in which we are interested:
</p>
<pre><code>    IPIADR001.T1
    IPIGYN070.T1</code></pre>
<p>
  Looking at the above criteria, it seems like our research question might be
  formulated along the lines of:
</p>
<pre><code>    From the rna_seq model, I want the gene_tpm data for all genes, but only for samples IPIADR001.T1 and IPIGYN070.T1.</code></pre>
<p>
  You may notice that this query format goes "up the tree", becuase
  <code>rna_seq</code> is a child of <code>sample</code>. In this example, let’s
  also explore what this might look like if we go "down the tree":
</p>
<pre><code>    From the sample model, I want the gene_tpm data for all the rna_seq records attached to samples IPIADR001.T1 and IPIGYN070.T1.</code></pre>
<p>
  Because Sample -&gt; RnaSeq is one-to-many, this will result in nested data
  frames unless we slice out individual records when we construct our columns.
  If you follow along with the down-the-tree example, you’ll learn how to slice
  columns to get a flat data frame.
</p>
<h4 id="ui-input---up-the-tree">UI Input - up the tree</h4>
<p>
  First, let’s translate the "up the tree" question we’ve formulated into the
  Query UI.
</p>
<pre><code>    From the rna_seq model</code></pre>
<p>indicates that the root model should be <code>rna_seq</code>.</p>
<pre><code>    I want the gene_tpm data for all genes</code></pre>
<p>
  becomes a column, where the model is <code>rna_seq</code> and the attribute is
  <code>gene_tpm</code>. We won’t add a slice, since we want all the gene
  expression data.
</p>
<p>Lastly,</p>
<pre><code>    but only for samples IPIADR001.T1 and IPIGYN070.T1</code></pre>
<p>
  will be a filter on the Sample model. You’ll notice that if you select
  <code>rna_seq</code> as the model, you don’t have the option to select a
  <code>sample</code> attribute – for relationship attributes, you have to
  select the target model, instead. So we’ll add a filter on Sample instead.
</p>
<div id="table:sample-filter-for-bulk-rna-seq">
  <table>
    <caption>
      Sample filter for Bulk RNA Seq
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center" rowspan="2">sample</td>
        <td style="text-align: center" rowspan="2">sample</td>
        <td style="text-align: center" rowspan="2">sample_name</td>
        <td style="text-align: center" rowspan="2">In</td>
        <td style="text-align: center">IPIADR001.T1,</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">IPIGYN070.T1</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  As in the previous example, we could also use multiple operators joined by an
  OR clause. However, with a lot of values, adding many filters manually can be
  tedious, and a single filter with an <code>In</code> operator will be simpler.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_rna_seq_specific_samples"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_specific_samples"
    >2.5</a
  >. Now hit <code>Query</code>! Because we are not slicing the matrix column,
  the render may take awhile and you may see a warning that the page is slowing
  down your browser.
</p>
<h4 id="view-in-browser---up-the-tree">View in browser - up the tree</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/ipi/query?rootModel=rna_seq&amp;recordFilters=%5B%7B%22modelName%22%3A%22sample%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22sample_name%22%2C%22operator%22%3A%22%3A%3Ain%22%2C%22operand%22%3A%22IPIADR001.T1%2CIPIGYN070.T1%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sample%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22rna_seq%22%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22rna_seq.tube_name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame---up-the-tree">Data Frame - up the tree</h4>
<p>
  You should see the first page of data in your browser, and can check out other
  pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_SpecificSamples.png"
    id="fig:query_rna_seq_specific_samples"
    alt="IPI Bulk RNA Seq gene data from two samples"
  />
  <figcaption aria-hidden="true">
    IPI Bulk RNA Seq gene data from two samples
  </figcaption>
</figure>
<p>
  You can see that the <code>IPIGYN070.T1</code> records have no gene expression
  data, so just show empty values. Also, you’ll notice that because we didn’t
  slice out data, all the possible gene codes are expanded, and performance is
  slow. If you want the full data set, you will have to download the TSV.
  Alternatively, if you know exactly the Ensembl gene ids that you want, you
  should add a column slice.
</p>
<h4 id="ui-input---down-the-tree">UI Input - down the tree</h4>
<p>
  Now let’s translate the "down the tree" question we’ve formulated for this
  example into the Query UI.
</p>
<pre><code>    From the sample model</code></pre>
<p>indicates that the root model should be <code>sample</code>.</p>
<pre><code>    I want the gene_tpm data for all the rna_seq records.</code></pre>
<p>
  is actually tricky to deal with. If we do nothing else and just include the
  single column, we will get nested data because there are multiple
  <code>rna_seq</code> records attached to each of the two desired samples.
  However, we don’t actually know how many <code>rna_seq</code> records there
  are for each sample, and it’s not a consistent number of records for every
  sample in the IPI data set. We can kind of guess that there should be one
  record per compartment, and construct the query in that fashion, but we may
  miss some data on samples with multiple <code>rna_seq</code> records for the
  same compartment. We’ll construct this example assuming one compartment per
  sample, to demonstrate how to use column slicing on collections.
</p>
<p>There are five standard <code>rna_seq</code> compartments for IPI:</p>
<ul>
  <li><p>live</p></li>
  <li><p>myeloid</p></li>
  <li><p>stroma</p></li>
  <li><p>tcell</p></li>
  <li><p>tumor</p></li>
</ul>
<p>Some samples may also have specialized compartments, like:</p>
<ul>
  <li><p>cd45neg</p></li>
  <li><p>cd45pos</p></li>
</ul>
<p>
  We’ll assume that the samples we’re looking at only use the five standard
  compartments. For each one, we’ll add a column with a slice, so we get the
  gene data for the "first" record of the given compartment (if there happen to
  be multiple). The columns would all be:
</p>
<div id="table:columns-for-rna-seq">
  <table>
    <caption>
      Bulk RNA Seq columns
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Model</th>
        <th style="text-align: center">Attribute</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">gene_tpm</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">gene_tpm</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">gene_tpm</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">gene_tpm</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">gene_tpm</td>
      </tr>
    </tbody>
  </table>
</div>
<p>With one slice per column:</p>
<div id="table:column-slices-for-rna-seq">
  <table>
    <caption>
      Bulk RNA Seq column slices
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">live</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">myeloid</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">stroma</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">tcell</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">tumor</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Because the columns are all from the same model and attribute, we’ll edit the
  Display Labels so we can differentiate the data in the final data frame.
</p>
<p>Lastly,</p>
<pre><code>    attached to samples IPIADR001.T1 and IPIGYN070.T1</code></pre>
<p>
  will be a filter on the Sample model’s <code>sample_name</code> attribute, as
  in  <a
    href="#table:sample-filter-for-bulk-rna-seq"
    data-reference-type="ref"
    data-reference="table:sample-filter-for-bulk-rna-seq"
    >2.5</a
  >.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_rna_seq_compartments"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_compartments"
    >2.6</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser---down-the-tree">View in browser - down the tree</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/ipi/query?rootModel=sample&amp;recordFilters=%5B%7B%22modelName%22%3A%22sample%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22sample_name%22%2C%22operator%22%3A%22%3A%3Ain%22%2C%22operand%22%3A%22IPIADR001.T1%2CIPIGYN070.T1%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sample%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22sample%22%2C%22attribute_name%22%3A%22sample_name%22%2C%22display_label%22%3A%22sample.sample_name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22live%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm.live%22%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22myeloid%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm.myeloid%22%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22stroma%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm.stroma%22%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22tcell%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm.tcell%22%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22tumor%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm.tumor%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame---down-the-tree">Data Frame - down the tree</h4>
<p>
  You should see the first page of data in your browser, though many, many
  columns will not be rendered, and you can check out other pages or download
  the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_Compartments.png"
    id="fig:query_rna_seq_compartments"
    alt="Sliced Bulk RNA Seq gene data from two samples"
  />
  <figcaption aria-hidden="true">
    Sliced Bulk RNA Seq gene data from two samples
  </figcaption>
</figure>
<p>
  You can see that the <code>IPIGYN070.T1</code> records have no gene expression
  data, so just show empty values. Also, you’ll notice that because we didn’t
  slice out data, all the possible gene codes are expanded, and performance is
  slow (many, many columns are not rendered, as shown in the warning). If you
  want the full data set, you will have to download the TSV (will be very slow).
  Alternatively, if you know exactly the Ensembl gene ids that you want, you
  should add a column slice.
</p>
<p>
  However, if <code>IPIADR001.T1</code> had multiple
  <code>live</code> compartment records, we wouldn’t know which one was
  processed for the data frame. Due to this weakness of the "down the tree"
  method for this query, we would highly recommend to use the "up the tree"
  method in this kind of query.
</p>
<h3 id="section:cell-populations-example">
  Cell populations in a collection of samples
</h3>
<h4 id="question-3">Question</h4>
<p>
  In this question, we would like to examine the frequency of CD8s vs. overall T
  cell population, in a collection of pre-identified samples.
</p>
<h4 id="models-3">Models</h4>
<p>
  From the  <a href="#fig:ipi_map">IPI Map view</a>, we can see that there is a
  model called <code>population</code>, so this seems like a good place to
  start. You’ll note that this model is two steps away from the Sample model
  (Sample -&gt; Flow -&gt; Population), where we have our pre-identified sample
  names, so we’ll want to account for that when constructing our query.
</p>
<p>
  Like the previous example, we could think about this model in "down the tree"
  or "up the tree" ways. Because we know that the flow data does not include any
  duplicate stains (each Sample has at most one flow record per stain, unlike
  with the Bulk RNA Seq data), we could go in either direction and extract the
  same data. However, because going down the tree will involve a lot of column
  slices, we will focus on the up the tree direction. Where relevant, we will
  discuss the down the tree query, as different users may find a specific
  approach more intuitive.
</p>
<p>
  Constructing this query does assume some knowledge about how cell populations
  are labelled during the flow gating process. You can use the Search page and
  examine the Population model to see what names were used. For the sake of this
  example, it appears that <code>CD8a+,CD4-</code> and
  <code>CD8a-,CD4+</code> may be useful population names.
</p>
<p>
  To calculate total frequency of T cells, we will look for populations named
  <code>CD8a+,CD4-</code> OR <code>CD8a-,CD4+</code>. We want to compare those
  against the CD8 T cells, which would be populations named
  <code>CD8a+,CD4-</code>.
</p>
<p>
  Up to this point, the populations we are interested in apply regardless if we
  go down the tree or up the tree. If we go down the tree, we will need to add
  more columns and more slices for each column, to grab a single flow stain per
  column. If we are interested in specific stains, this could be manageable. If
  we are interested in all stains, this would require us to create fifteen
  columns for the query!! Definitely a situation where we would recommend an
  up-the-tree approach, with Population as the root model.
</p>
<p>
  For the sake of the example, we will examine samples
  <code>IPIADR001.T1</code> and <code>IPIGYN070.T1</code>.
</p>
<p>
  Looking at the above criteria and keeping to an up-the-tree query, it seems
  like our research question might be formulated along the lines of:
</p>
<pre><code>    From the population model, I want the counts for the &quot;CD8a-,CD4+&quot; and &quot;CD8a+,CD4-&quot; populations, for samples IPIADR001.T1 and IPIGYN070.T1.</code></pre>
<h4 id="ui-input-2">UI Input</h4>
<p>First, let’s translate the question we’ve formulated into the Query UI.</p>
<pre><code>    From the population model</code></pre>
<p>indicates that the root model should be <code>population</code>.</p>
<pre><code>    I want the counts for the &quot;CD8a-,CD4+&quot; and &quot;CD8a+,CD4-&quot; populations,</code></pre>
<p>
  Since Magma itself will not perform calculations for us, we just extract the
  raw counts data and will have to do frequency calculations in downstream
  analysis.
</p>
<p>
  At this point, it might seem like we should add two columns to get the two
  different count values, and then perform some column slices to make sure our
  data frame is flat. However, because we are using Population as the root
  model, we won’t do that! Instead, we will add two filters to only pull out the
  population records with those names – remember to join them with an OR clause
  by checking the OR boxes. Because the population names themselves have commas,
  we cannot use a single <code>In</code> operator in this scenario.
</p>
<div id="table:filters-for-population-query">
  <table>
    <caption>
      Population filters
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">population</td>
        <td style="text-align: center">population</td>
        <td style="text-align: center">name</td>
        <td style="text-align: center">Contains</td>
        <td style="text-align: center">CD8a-,CD4+</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">population</td>
        <td style="text-align: center">population</td>
        <td style="text-align: center">name</td>
        <td style="text-align: center">Contains</td>
        <td style="text-align: center">CD8a+,CD4-</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  We will then add columns for <code>name</code> and <code>count</code> so that
  we can separate the data and run frequency calculations in downstream
  analysis. You may also decide to include <code>ancestry</code> and
  <code>flow</code> <code>stain_name</code> if you want to gather more
  information.
</p>
<div id="table:columns-for-population-query">
  <table>
    <caption>
      Population columns
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Model</th>
        <th style="text-align: center">Attribute</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">population</td>
        <td style="text-align: center">name</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">population</td>
        <td style="text-align: center">count</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">population</td>
        <td style="text-align: center">ancestry</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">flow</td>
        <td style="text-align: center">stain_name</td>
      </tr>
    </tbody>
  </table>
</div>
<p>Lastly,</p>
<pre><code>    for samples IPIADR001.T1 and IPIGYN070.T1</code></pre>
<p>
  will be an additional filter on the Sample model, as in  <a
    href="#table:sample-filter-for-bulk-rna-seq"
    data-reference-type="ref"
    data-reference="table:sample-filter-for-bulk-rna-seq"
    >2.5</a
  >.
</p>
<p>
  As in the previous example, we could also use multiple operators joined by an
  OR clause, for this filter instead. However, with a lot of target samples,
  adding many filters manually can be tedious, and a single filter with an
  <code>In</code> operator will be simpler.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_population_counts"
    data-reference-type="ref"
    data-reference="fig:query_population_counts"
    >2.7</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser-2">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/ipi/query?rootModel=population&amp;recordFilters=%5B%7B%22modelName%22%3A%22sample%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22sample_name%22%2C%22operator%22%3A%22%3A%3Ain%22%2C%22operand%22%3A%22IPIADR001.T1%2CIPIGYN070.T1%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sample%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22population%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22name%22%2C%22operator%22%3A%22%3A%3Amatches%22%2C%22operand%22%3A%22CD8a%2B%2CCD4-%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22population%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22population%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22name%22%2C%22operator%22%3A%22%3A%3Amatches%22%2C%22operand%22%3A%22CD8a-%2CCD4%2B%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22population%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B1%2C2%5D&amp;columns=%5B%7B%22model_name%22%3A%22population%22%2C%22attribute_name%22%3A%22id%22%2C%22display_label%22%3A%22population.id%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22flow%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22stain_name%22%2C%22display_label%22%3A%22flow.stain_name%22%7D%2C%7B%22model_name%22%3A%22population%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22population.name%22%7D%2C%7B%22model_name%22%3A%22population%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22count%22%2C%22display_label%22%3A%22population.count%22%7D%2C%7B%22model_name%22%3A%22population%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22ancestry%22%2C%22display_label%22%3A%22population.ancestry%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame-2">Data Frame</h4>
<p>
  You should see the first page of data in your browser, and you can check out
  other pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_PopulationCounts.png"
    id="fig:query_population_counts"
    alt="Population Counts"
  />
  <figcaption aria-hidden="true">Population Counts</figcaption>
</figure>
<h3 id="gene-expression-with-high-t-cell-count">
  Gene expression with high T cell count
</h3>
<h4 id="question-4">Question</h4>
<p>
  In this question, we would like to extract the gene expression data in the
  myeloid compartment, for a given set of genes, but only for samples with a
  high T cell count, as assessed by flow.
</p>
<h4 id="models-4">Models</h4>
<p>
  From the  <a href="#fig:ipi_map">IPI Map view</a>, we can see that both the
  <code>population</code> (to find T cell counts) and <code>rna_seq</code> (to
  find gene expression data) models are leaf models, and the path between them
  goes through <code>sample</code>. However, we can note that we mainly want to
  use flow population data as a filter constraint, and so perhaps we can start
  with the <code>rna_seq</code> model as the root model, and include some Any /
  Every filters.
</p>
<p>
  For the sake of this example, we’ll say that we are interested in this set of
  genes:
</p>
<pre><code>    ENSG00000000003,ENSG00000000005</code></pre>
<p>
  Also, in order to construct our population filter, we will have to make some
  assumptions about how populations are gated. Inspecting the Population table,
  we may decide to focus on all populations named <code>CD8a-,CD4+</code> OR
  <code>CD8a+,CD4-</code>.
</p>
<p>
  Looking at the above criteria, it seems like our research question might be
  formulated along the lines of:
</p>
<pre><code>    From the rna_seq model, I want the gene_tpm data for ENSG00000000003 and ENSG00000000005 in the myeloid compartment, but only for samples where every flow stain has a T cell population named CD8a-,CD4+ OR CD8a+,CD4- with count greater than X.</code></pre>
<p>
  We can play around with different values of X to see what might make a good
  cutoff.
</p>
<h4 id="ui-input-3">UI Input</h4>
<p>First, let’s translate the question we’ve formulated into the Query UI.</p>
<pre><code>    From the rna_seq model</code></pre>
<p>indicates that the root model should be <code>rna_seq</code>.</p>
<pre><code>    I want the gene_tpm data for ENSG00000000003 and ENSG00000000005</code></pre>
<p>
  So here we’ll add a column for gene expression data, and include a matrix
  slice.
</p>
<div id="table:bulk-rna-seq-gene-expression-column-flow-example">
  <table>
    <caption>
      Bulk RNA Seq gene expression column
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">gene_tpm</td>
        <td style="text-align: center">Slice</td>
        <td style="text-align: center">ENSG00000000003,ENSG00000000005</td>
      </tr>
    </tbody>
  </table>
</div>
<pre><code>    in the myeloid compartment</code></pre>
<p>Sounds like a filter on <code>rna_seq</code> <code>compartment</code>!</p>
<pre><code>    but only for samples where every flow stain has a T cell population named CD8a-,CD4+ OR CD8a+,CD4- with count greater than X</code></pre>
<p>
  Here it definitely seems like we’ll need some additional fitlers. However, we
  have hit a current limitation of the Query UI, in its lack of flexibility
  around nesting AND and OR clauses in the filters. Ideally we would add a set
  of filters like
</p>
<pre><code>    every flow stain has any population with count greater than 100 AND (name Contains CD8a-,CD4+ OR name Contains CD8a+,CD4-)</code></pre>
<p>
  Since we cannot currently do that, we’ll break this research question up into
  two distinct queries. First, we should follow the
  <a href="#section:cell-populations-example">Cell Populations example</a> and
  extract all T cell populations without any sample filters. Doing our
  downstream analysis, we can determine which samples qualify as having "high T
  cell count", and extract their sample IDs. At that point, we would return to
  this query and add a filter on <code>sample</code> <code>sample_name</code> to
  be in our set of high T cell samples (i.e. <code>IPIADR001.T1</code> and
  <code>IPIGYN070.T1</code>). This is now the same as the
  <a href="#section:gene-expression-specific-samples"
    >Gene expression for specific samples, example</a
  >! So we’ve broken up a complex query into two simpler queries. Our filters
  now look like:
</p>
<div id="table:gene-expression-for-high-tcell-count-samples">
  <table>
    <caption>
      Gene expression for high T cell count samples
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">rna_seq</td>
        <td style="text-align: center">compartment</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">myeloid</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="2">sample</td>
        <td style="text-align: center" rowspan="2">sample</td>
        <td style="text-align: center" rowspan="2">sample_name</td>
        <td style="text-align: center" rowspan="2">In</td>
        <td style="text-align: center">IPIADR001.T1,</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">IPIGYN070.T1</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_rna_seq_high_t_cell_counts"
    data-reference-type="ref"
    data-reference="fig:query_rna_seq_high_t_cell_counts"
    >2.8</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser-3">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/ipi/query?rootModel=rna_seq&amp;recordFilters=%5B%7B%22modelName%22%3A%22rna_seq%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22compartment%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22myeloid%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22sample%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22sample_name%22%2C%22operator%22%3A%22%3A%3Ain%22%2C%22operand%22%3A%22IPIADR001.T1%2CIPIGYN070.T1%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sample%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22rna_seq%22%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22rna_seq.tube_name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22rna_seq%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22gene_tpm%22%2C%22operator%22%3A%22%3A%3Aslice%22%2C%22operand%22%3A%22ENSG00000000003%2CENSG00000000005%22%2C%22attributeType%22%3A%22matrix%22%2C%22modelName%22%3A%22rna_seq%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22gene_tpm%22%2C%22display_label%22%3A%22rna_seq.gene_tpm%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame-3">Data Frame</h4>
<p>
  You should see the first page of data in your browser, and you can check out
  other pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_RnaSeq_HighTCellCount.png"
    id="fig:query_rna_seq_high_t_cell_counts"
    alt="RNA Seq with high T cell counts"
  />
  <figcaption aria-hidden="true">RNA Seq with high T cell counts</figcaption>
</figure>
<h2 id="humu">HuMu</h2>
<h3 id="subjects-with-specific-assay-data-available">
  Subjects with specific assay data available
</h3>
<h4 id="question-5">Question</h4>
<p>
  Many times we want to know what data is available in the Data Library. In this
  case, we want to know which subjects have single cell RNA seq data available.
</p>
<h4 id="models-5">Models</h4>
<p>
  From the HuMu Map view, we can see that there are models for
  <code>subject</code> and <code>sc_seq</code>.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_HuMu_Map.png"
    id="fig:query_humu_map"
    alt="HuMu Map"
  />
  <figcaption aria-hidden="true">HuMu Map</figcaption>
</figure>
<p>We might formulate our research query into:</p>
<pre><code>    I want to know all subject records with sc_seq data.</code></pre>
<h4 id="ui-input-4">UI Input</h4>
<p>
  Now let’s translate the question we’ve formulated for this example into the
  Query UI.
</p>
<pre><code>    I want to know all subject records</code></pre>
<p>indicates that the root model should be <code>subject</code>.</p>
<pre><code>    with sc_seq data</code></pre>
<p>
  indicates a filter of some sort. Initially, we might be tempted to just add
  one like
</p>
<div id="table:invalid-filter-for-sc-seq-data">
  <table>
    <caption>
      Invalid filter for sc-seq data
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">operator</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">biospecimen_group</td>
        <td style="text-align: center">biospecimen_group</td>
        <td style="text-align: center">sc_seq</td>
        <td style="text-align: center">Is present</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  But we’ll soon realize that we can’t construct a filter like that – the
  <code>biospecimen_group</code> model doesn’t have an attribute option called
  <code>sc_seq</code>. Recall that to check model data existence, we have to
  perform a <code>Is present</code> filter on the identifier of the model
  itself. So we need to add a filter like:
</p>
<div id="table:valid-filter-for-sc-seq-data">
  <table>
    <caption>
      Valid filter for sc-seq data
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">operator</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">sc_seq</td>
        <td style="text-align: center">sc_seq</td>
        <td style="text-align: center">tube_name</td>
        <td style="text-align: center">Is present</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Because we want all sample records with any sort of single cell seq data, we
  will leave the Any / Every toggles at <code>Any biospecimen</code> and
  <code>Any sc_seq</code>.
</p>
<p>
  To check our data, we’ll also add a column for the <code>sc_seq</code>
  <code>tube_name</code>, so that we can more easily find the assay data.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_humu_subjects_with_sc_seq_data"
    data-reference-type="ref"
    data-reference="fig:query_humu_subjects_with_sc_seq_data"
    >2.10</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser-4">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/xcrs1/query?rootModel=subject&amp;recordFilters=%5B%7B%22modelName%22%3A%22sc_seq%22%2C%22anyMap%22%3A%7B%22biospecimen%22%3Atrue%2C%22sc_seq%22%3Atrue%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22tube_name%22%2C%22operator%22%3A%22%3A%3Ahas%22%2C%22operand%22%3A%22%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sc_seq%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22subject%22%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22subject.name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22sc_seq%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22sc_seq.tube_name%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame-4">Data Frame</h4>
<figure>
  <img
    src="/assets/images/timur/query/Query_HuMu_SubjectRecords.png"
    id="fig:query_humu_subjects_with_sc_seq_data"
    alt="Subjects with ScSeq data"
  />
  <figcaption aria-hidden="true">Subjects with ScSeq data</figcaption>
</figure>
<p>
  Hm, it looks like the query finished, but some cells for the
  <code>tube_name</code>s are blank. What happened??!!?? How could the filter be
  working (only fetch subjects with single-cell RNA seq data), yet result in no
  <code>tube_name</code>s for some subjects?
</p>
<p>
  If you take a peek back at  <a
    href="#fig:query_humu_map"
    data-reference-type="ref"
    data-reference="fig:query_humu_map"
    >2.9</a
  >, we can see that the path from <code>subject</code> to
  <code>sc_seq</code> has two one-to-many relationships. Since we did not add
  any column slices, our data may not be flat! If we toggle the "Nested" vs
  "Flattened" option and re-run the query, we can see that there are now data
  entries in every cell.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_HuMu_SubjectRecords_Nested.png"
    id="fig:query_humu_subjects_with_nested_sc_seq_data"
    alt="Subjects with nested ScSeq data"
  />
  <figcaption aria-hidden="true">Subjects with nested ScSeq data</figcaption>
</figure>
<p>
   <a
    href="#fig:query_humu_subjects_with_sc_seq_data"
    data-reference-type="ref"
    data-reference="fig:query_humu_subjects_with_sc_seq_data"
    >2.10</a
  >
  showed blank cells because the "first" biospecimen record attached to each
  subject may not have had single cell data available. By nesting, we can see
  all the entries and determine if that is okay for our purposes (to gut-check
  that these subjects have single cell data), or if we need to further refine
  our query to get flattened data. One strategy to get flattened data might be
  to set <code>sc_seq</code> as the root model, instead.
</p>
<h2 id="comet">COMET</h2>
<h3 id="analyte-counts-for-a-subset-of-patients">
  Analyte counts for a subset of patients
</h3>
<h4 id="question-6">Question</h4>
<p>
  For this research question, we are interested in looking at IL-6 levels for
  patients who have been diagnosed with COVID.
</p>
<h4 id="models-6">Models</h4>
<p>
  From the COMET view, it may not be very intuitive where to find the IL-6 data.
  However, with some digging around, we may realize that it is stored in the
  <code>analyte</code> model.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_COMET_Map.png"
    id="fig:query_comet_map"
    alt="COMET Map"
  />
  <figcaption aria-hidden="true">COMET Map</figcaption>
</figure>
<p>
  <code>analyte</code> has both <code>value</code> and
  <code>analyte_name</code> attributes which seem like they might be useful. We
  will have to examine the <code>analyte</code> table to know that IL-6 is
  formatted as <code>IL-6</code> and not something else like <code>IL6</code>.
</p>
<p>
  Also, it looks like the <code>patient</code> model has an attribute that suits
  our purpose, perfectly! The <code>covid_pos</code> boolean flag indicates
  patients who have been diagnosed with COVID.
</p>
<p>
  This query can be formulated in an up-the-tree fashion or down-the-tree. Let’s
  explore both and see what the final data frames look like. Regardless of path
  direction, the general research question might appear like:
</p>
<pre><code>    I want to know analyte value when the analyte_name is IL-6, but only for patients whose covid_pos flag is true.</code></pre>
<h4 id="ui-input---up-the-tree-1">UI Input - up the tree</h4>
<p>
  Now let’s translate the question we’ve formulated for this example into the
  Query UI. Since we want to go up the tree, we’ll set the root model to
  <code>analyte</code>.
</p>
<pre><code>    I want to know analyte value</code></pre>
<p>
  This is one of the output data points we want, so we’ll add a column for
  <code>analyte</code> and attribute <code>value</code>.
</p>
<pre><code>    when the analyte_name is IL-6</code></pre>
<p>
  Because we are using <code>analyte</code> as the root model, this constraint
  gets translated as a filter.
</p>
<pre><code>    but only for patients whose covid_pos flag is true</code></pre>
<p>Would be an additional filter, on the <code>patient</code> model.</p>
<div id="table:filters-for-analyte-up-tree">
  <table>
    <caption>
      Filters for analyte, up the tree
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">analyte</td>
        <td style="text-align: center">analyte</td>
        <td style="text-align: center">analyte_name</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">IL-6</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">patient</td>
        <td style="text-align: center">patient</td>
        <td style="text-align: center">covid_pos</td>
        <td style="text-align: center">Is true</td>
        <td style="text-align: center"></td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  To check our query and give us a bit more useful context, we can also include
  columns for <code>analyte</code> with attribute <code>analyte_name</code>,
  <code>patient</code> with attribute <code>name</code>, and
  <code>patient</code> with attribute <code>covid_pos</code>.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_analyte_up_tree_example"
    data-reference-type="ref"
    data-reference="fig:query_analyte_up_tree_example"
    >2.13</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser---up-the-tree-1">View in browser - up the tree</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/mvir1/query?rootModel=analyte&amp;recordFilters=%5B%7B%22modelName%22%3A%22analyte%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22analyte_name%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22IL-6%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22analyte%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22patient%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22covid_pos%22%2C%22operator%22%3A%22%3A%3Atrue%22%2C%22operand%22%3A%22%22%2C%22attributeType%22%3A%22boolean%22%2C%22modelName%22%3A%22patient%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22analyte%22%2C%22attribute_name%22%3A%22id%22%2C%22display_label%22%3A%22analyte.id%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22patient.name%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22covid_pos%22%2C%22display_label%22%3A%22patient.covid_pos%22%7D%2C%7B%22model_name%22%3A%22analyte%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22analyte_name%22%2C%22display_label%22%3A%22analyte.analyte_name%22%7D%2C%7B%22model_name%22%3A%22analyte%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22value%22%2C%22display_label%22%3A%22analyte.value%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame---up-the-tree-1">Data Frame - up the tree</h4>
<p>
  You should see the first page of data in your browser, and you can check out
  other pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_Analyte_UpTree.png"
    id="fig:query_analyte_up_tree_example"
    alt="Analyte data for subset of patients, up tree"
  />
  <figcaption aria-hidden="true">
    Analyte data for subset of patients, up tree
  </figcaption>
</figure>
<h4 id="ui-input---down-the-tree-1">UI Input - down the tree</h4>
<p>
  Now let’s translate the question we’ve formulated for this example into the
  Query UI. Since we want to go down the tree, we’ll set the root model to
  <code>patient</code>.
</p>
<pre><code>    I want to know analyte value</code></pre>
<p>
  This is one of the output data points we want, so we’ll add a column for
  <code>analyte</code> and attribute <code>value</code>.
</p>
<pre><code>    when the analyte_name is IL-6</code></pre>
<p>
  Because we are using <code>patient</code> as the root model, this constraint
  gets translated as a column slice.
</p>
<div id="table:slice-for-analyte-down-tree">
  <table>
    <caption>
      Slice for analyte value, down the tree
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">analyte</td>
        <td style="text-align: center">analyte_name</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">IL-6</td>
      </tr>
    </tbody>
  </table>
</div>
<pre><code>    but only for patients whose covid_pos flag is true</code></pre>
<p>Would be a filter on the <code>patient</code> model.</p>
<div id="table:filter-for-analyte-down-tree">
  <table>
    <caption>
      Filter for analyte, down the tree
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">patient</td>
        <td style="text-align: center">patient</td>
        <td style="text-align: center">covid_pos</td>
        <td style="text-align: center">Is true</td>
        <td style="text-align: center"></td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  To check our query, we can also include columns for <code>analyte</code> with
  attribute <code>analyte_name</code> (also slicing on
  <code>analyte_name</code> Equals <code>IL-6</code>) and
  <code>patient</code> with attribute <code>covid_pos</code>. The
  <code>analyte_name</code> column with the slice is a bit superfluous, but lets
  the resulting data frame look like our up-the-tree output.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_analyte_down_tree_error"
    data-reference-type="ref"
    data-reference="fig:query_analyte_down_tree_error"
    >2.14</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser---down-the-tree-1">View in browser - down the tree</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/mvir1/query?rootModel=patient&amp;recordFilters=%5B%7B%22modelName%22%3A%22patient%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22covid_pos%22%2C%22operator%22%3A%22%3A%3Atrue%22%2C%22operand%22%3A%22%22%2C%22attributeType%22%3A%22boolean%22%2C%22modelName%22%3A%22patient%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22patient%22%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22patient.name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22analyte%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22analyte_name%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22IL-6%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22analyte%22%2C%22any%22%3Atrue%7D%7D%2C%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22day%22%2C%22operator%22%3A%22%3A%3A%3D%22%2C%22operand%22%3A%220%22%2C%22attributeType%22%3A%22number%22%2C%22modelName%22%3A%22timepoint%22%2C%22any%22%3Atrue%7D%7D%2C%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22assay_type%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22Cytokine%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22immunoassay%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22value%22%2C%22display_label%22%3A%22analyte.value%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22covid_pos%22%2C%22display_label%22%3A%22patient.covid_pos%22%7D%2C%7B%22model_name%22%3A%22analyte%22%2C%22slices%22%3A%5B%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22analyte_name%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22IL-6%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22analyte%22%2C%22any%22%3Atrue%7D%7D%2C%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22day%22%2C%22operator%22%3A%22%3A%3A%3D%22%2C%22operand%22%3A%220%22%2C%22attributeType%22%3A%22number%22%2C%22modelName%22%3A%22timepoint%22%2C%22any%22%3Atrue%7D%7D%2C%7B%22modelName%22%3A%22%22%2C%22clause%22%3A%7B%22attributeName%22%3A%22assay_type%22%2C%22operator%22%3A%22%3A%3Aequals%22%2C%22operand%22%3A%22Cytokine%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22immunoassay%22%2C%22any%22%3Atrue%7D%7D%5D%2C%22attribute_name%22%3A%22analyte_name%22%2C%22display_label%22%3A%22analyte.analyte_name%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame---down-the-tree-1">Data Frame - down the tree</h4>
<figure>
  <img
    src="/assets/images/timur/query/Query_Analyte_DownTree_Error.png"
    id="fig:query_analyte_down_tree_error"
    alt="Weird analyte data for subset of patients, down tree"
  />
  <figcaption aria-hidden="true">
    Weird analyte data for subset of patients, down tree
  </figcaption>
</figure>
<p>
  Hm...this data frame looks spectacularly blank, and doesn’t match our
  up-the-tree example. Let’s examine what we’ve got so far, and think about what
  might cause the mismatch.
</p>
<p>
  Recall that down the tree paths may require column slicing, which we did for
  the <code>analyte</code> model already. However, looking at  <a
    href="#fig:query_comet_map"
    data-reference-type="ref"
    data-reference="fig:query_comet_map"
    >2.12</a
  >, we can see that actually there are several one-to-many relationships along
  the path of Patient -&gt; Timepoint -&gt; Immunoassay -&gt; Analyte. We also
  can see this by the dropdown options available in the column slice we
  constructed for Analyte. Perhaps we need to add more column slices?
</p>
<p>
  Adding in the right slices depends on us better understanding the COMET data
  structure. Looking at the data, we may realize that each Patient has multiple
  Timepoints, but cytokine data (IL-6 is a cytokine marker) was only generated
  for day 0. So we should add slices for our two Analyte columns for Timepoint
  where day equals 0. Also, Immunoassay appears to be fairly generic, and
  different kinds of Immunoassays may have been performed on day 0 – for
  example, cytokine, phip-seq, olink, etc. So we will want to also slice out
  only the Immunoassay records where the <code>assay_type</code> equals
  <code>Cytokine</code>. For both of our <code>analyte</code> columns, their
  slices will then look like  <a
    href="#table:slices-for-analyte-down-tree"
    data-reference-type="ref"
    data-reference="table:slices-for-analyte-down-tree"
    >2.17</a
  >.
</p>
<div id="table:slices-for-analyte-down-tree">
  <table>
    <caption>
      Slices for analyte value, down the tree
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">analyte</td>
        <td style="text-align: center">analyte_name</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">IL-6</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">timepoint</td>
        <td style="text-align: center">day</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">0</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">immunoassay</td>
        <td style="text-align: center">assay_type</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Cytokine</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Querying again, we then get a data frame that better matches our up-the-tree
  data!
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_Analyte_DownTree.png"
    id="fig:query_analyte_down_tree"
    alt="Analyte data for subset of patients, down tree"
  />
  <figcaption aria-hidden="true">
    Analyte data for subset of patients, down tree
  </figcaption>
</figure>
<h4 id="expanding-on-this-query-1">Expanding on this query</h4>
<p>
  If we wanted to filter out the empty rows and get exactly the same results as
  up-the-tree, we could add another filter to remove those records:
</p>
<div id="table:filter-remove-empty-analyte-records">
  <table>
    <caption>
      Filter to remove records with no cytokine data
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">analyte</td>
        <td style="text-align: center">analyte</td>
        <td style="text-align: center">analyte_name</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">IL-6</td>
      </tr>
    </tbody>
  </table>
</div>
<h3 id="section:patients-with-data-available">
  Patients with specific assay data available
</h3>
<h4 id="question-7">Question</h4>
<p>
  Sometimes when we are doing quality control or just want to know what is in
  the Data Library, we may want to do a query like, "Give me the list of
  patients who have X type of data". So in this example, we’ll figure out how to
  find the list of patients with day 0 PBMC single-cell RNA seq data as well as
  CyTOF data.
</p>
<h4 id="models-7">Models</h4>
<p>
  From  <a
    href="#fig:query_comet_map"
    data-reference-type="ref"
    data-reference="fig:query_comet_map"
    >2.12</a
  >, we can see that both <code>sc_rna_seq</code> and <code>cytof</code> both
  hang off of the <code>timepoint</code> model, so perhaps we should start with
  <code>timepoint</code> as our root model.
</p>
<pre><code>    I want a list of patient names where the timepoint on day 0 has some PBMC sc_rna_seq data, as well as cytof data.</code></pre>
<h4 id="ui-input-5">UI Input</h4>
<p>
  Now let’s translate the question we’ve formulated for this example into the
  Query UI.
</p>
<pre><code>    I want a list of patient names</code></pre>
<p>
  To avoid having to slice columns for Timepoint day equals 0, we’ll start with
  <code>timepoint</code> as the root model. That means to extract the patient
  name, we’ll add a column for <code>patient</code>, attribute
  <code>name</code>.
</p>
<pre><code>    where the timepoint on day 0</code></pre>
<p>indicates we need a filter.</p>
<pre><code>    has some PBMC sc_rna_seq data</code></pre>
<p>And</p>
<pre><code>    as well as cytof data</code></pre>
<p>
  Recall that to see if data exists in an assay or child collection, we add
  filters that the identifier value exists for the assay model. We can find
  those attributes for both models of interest from the map, and realize that
  both model use <code>tube_name</code> as the identifier attribute.
</p>
<p>
  Note that for the <code>sc_rna_seq</code> model, we won’t do an "exists" check
  on the <code>tube_name</code>. Since we also want to verify that PBMC data
  exists, and we know that the tube names encode the assay type, we use that
  inside information to use a single filter to check for record existence plus
  filter only PBMC data, instead of using two distinct filters (existence on
  <code>tube_name</code> and match on <code>biospecimen</code> attribute).
</p>
<p>So we wind up with three filters, like:</p>
<div id="table:assay-existence-filters">
  <table>
    <caption>
      Assay existence filters
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">timepoint</td>
        <td style="text-align: center">timepoint</td>
        <td style="text-align: center">day</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">0</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">sc_rna_seq</td>
        <td style="text-align: center">sc_rna_seq</td>
        <td style="text-align: center">tube_name</td>
        <td style="text-align: center">Contains</td>
        <td style="text-align: center">PBMC</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">cytof</td>
        <td style="text-align: center">cytof</td>
        <td style="text-align: center">tube_name</td>
        <td style="text-align: center">Is present</td>
        <td style="text-align: center"></td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  Because we only care that some data exists, we can leave both toggles at
  <code>Any timepoint</code>. We can also check our query by including columns
  for <code>sc_rna_seq</code> <code>tube_name</code> and <code>cytof</code>
  <code>tube_name</code>.
</p>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_comet_patients_with_data"
    data-reference-type="ref"
    data-reference="fig:query_comet_patients_with_data"
    >2.16</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser-5">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/mvir1/query?rootModel=timepoint&amp;recordFilters=%5B%7B%22modelName%22%3A%22timepoint%22%2C%22anyMap%22%3A%7B%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22day%22%2C%22operator%22%3A%22%3A%3A%3D%22%2C%22operand%22%3A%220%22%2C%22attributeType%22%3A%22number%22%2C%22modelName%22%3A%22timepoint%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22cytof%22%2C%22anyMap%22%3A%7B%22cytof%22%3Atrue%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22tube_name%22%2C%22operator%22%3A%22%3A%3Ahas%22%2C%22operand%22%3A%22%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22cytof%22%2C%22any%22%3Atrue%7D%5D%7D%2C%7B%22modelName%22%3A%22sc_rna_seq%22%2C%22anyMap%22%3A%7B%22sc_rna_seq%22%3Atrue%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22tube_name%22%2C%22operator%22%3A%22%3A%3Amatches%22%2C%22operand%22%3A%22PBMC%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sc_rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22timepoint%22%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22timepoint.name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22cytof%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22cytof.tube_name%22%7D%2C%7B%22model_name%22%3A%22sc_rna_seq%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22tube_name%22%2C%22display_label%22%3A%22sc_rna_seq.tube_name%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22patient.name%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame-5">Data Frame</h4>
<p>
  You should see the first page of data in your browser, and you can check out
  other pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_COMET_PatientsWithData.png"
    id="fig:query_comet_patients_with_data"
    alt="Patients with Day 0 PBMC and Cytof data"
  />
  <figcaption aria-hidden="true">
    Patients with Day 0 PBMC and Cytof data
  </figcaption>
</figure>
<h3 id="patient-data-when-assay-data-is-available">
  Patient data when assay data is available
</h3>
<h4 id="question-8">Question</h4>
<p>
  We may want to know more than just when patients have data available, like in
  <a href="#section:patients-with-data-available">the previous example</a>.
  Sometimes we may also want to collect specific data points from a model, when
  assay data is available. So in this example, let’s explore a research question
  about grabbing a set of patient data (COVID status, IMPACC status, day
  enrolled, day mechanical vent, and day since symptom onset) when they have ETA
  single-cell RNA seq data.
</p>
<h4 id="models-8">Models</h4>
<p>
  From  <a
    href="#fig:query_comet_map"
    data-reference-type="ref"
    data-reference="fig:query_comet_map"
    >2.12</a
  >, we can see that the Patient model includes the following attributes, which
  seem to provide the information we want:
</p>
<ul>
  <li>
    <p><code>covid_status</code></p>
  </li>
  <li>
    <p><code>impacc_enrollment</code></p>
  </li>
  <li>
    <p><code>admission_date</code></p>
  </li>
  <li>
    <p><code>vent_duration</code></p>
  </li>
  <li>
    <p><code>symptom_date</code></p>
  </li>
</ul>
<p>So we wind up with a research question that looks like:</p>
<pre><code>    From the patient model, I want the covid_status, impacc_enrollment, admission_date, vent_duration, and symptom_date, but only when the patient has ETA sc_rna_seq data.</code></pre>
<h4 id="ui-input-6">UI Input</h4>
<p>
  Now let’s translate the question we’ve formulated for this example into the
  Query UI.
</p>
<pre><code>    From the patient model</code></pre>
<p>means we’ll set the root model as <code>patient</code>.</p>
<pre><code>    I want the covid_status, impacc\enrollment, admission_date, vent_duration, and symptom_date</code></pre>
<p>indicates the set of columns that we’ll add.</p>
<pre><code>    but only when the patient has ETA sc_rna_seq data</code></pre>
<p>
  will be a filter. We’ll use the same technique as in
  <a href="#section:patients-with-data-available">the previous example</a>,
  where we can check that the <code>sc_rna_seq</code>
  <code>tube_name</code> includes <code>ETA</code> to do both a data-existence
  check as well as an ETA biospecimen check.
</p>
<div id="table:eta-existence-filter">
  <table>
    <caption>
      ETA existence filter
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Filter Model</th>
        <th style="text-align: center">Clause Model</th>
        <th style="text-align: center">Attribute</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center">sc_rna_seq</td>
        <td style="text-align: center">sc_rna_seq</td>
        <td style="text-align: center">tube_name</td>
        <td style="text-align: center">Contains</td>
        <td style="text-align: center">ETA</td>
      </tr>
    </tbody>
  </table>
</div>
<p>
  The entire query configuration can be seen in  <a
    href="#fig:query_comet_patients_with_eta_data"
    data-reference-type="ref"
    data-reference="fig:query_comet_patients_with_eta_data"
    >2.17</a
  >. Now hit <code>Query</code>!
</p>
<h4 id="view-in-browser-6">View in browser</h4>
<p>
  You can click  <a
    href="https://timur.ucsf.edu/mvir1/query?rootModel=patient&amp;recordFilters=%5B%7B%22modelName%22%3A%22sc_rna_seq%22%2C%22anyMap%22%3A%7B%22timepoint%22%3Atrue%2C%22sc_rna_seq%22%3Atrue%7D%2C%22clauses%22%3A%5B%7B%22attributeName%22%3A%22tube_name%22%2C%22operator%22%3A%22%3A%3Amatches%22%2C%22operand%22%3A%22ETA%22%2C%22attributeType%22%3A%22text%22%2C%22modelName%22%3A%22sc_rna_seq%22%2C%22any%22%3Atrue%7D%5D%7D%5D&amp;orRecordFilterIndices=%5B%5D&amp;columns=%5B%7B%22model_name%22%3A%22patient%22%2C%22attribute_name%22%3A%22name%22%2C%22display_label%22%3A%22patient.name%22%2C%22slices%22%3A%5B%5D%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22admission_date%22%2C%22display_label%22%3A%22patient.admission_date%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22covid_status%22%2C%22display_label%22%3A%22patient.covid_status%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22impacc_enrollment%22%2C%22display_label%22%3A%22patient.impacc_enrollment%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22vent_duration%22%2C%22display_label%22%3A%22patient.vent_duration%22%7D%2C%7B%22model_name%22%3A%22patient%22%2C%22slices%22%3A%5B%5D%2C%22attribute_name%22%3A%22symptom_date%22%2C%22display_label%22%3A%22patient.symptom_date%22%7D%5D"
    >here to open this query in your browser</a
  >, if you have access to this project.
</p>
<h4 id="data-frame-6">Data Frame</h4>
<p>
  You should see the first page of data in your browser, and you can check out
  other pages or download the full data set.
</p>
<figure>
  <img
    src="/assets/images/timur/query/Query_COMET_PatientAttributes.png"
    id="fig:query_comet_patients_with_eta_data"
    alt="Patients with ETA data"
  />
  <figcaption aria-hidden="true">Patients with ETA data</figcaption>
</figure>
<h1 id="appendix">Appendix</h1>
<h2 id="filter-operators">Filter operators</h2>
<div id="table:appendix-where-filter-operators">
  <table>
    <caption>
      Where filter operators
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Attribute Type</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center" rowspan="11">String</td>
        <td style="text-align: center">Contains</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Greater than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">In</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Less than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Less than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Not</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Not in</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="8">Date</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Greater than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Less than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Less than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Not equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="10">Number</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Greater than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">In</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Less than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Less than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Not equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Not in</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="5">Boolean</td>
        <td style="text-align: center">Is false</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is true</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is untrue</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center" rowspan="2">Matrix</td>
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
    </tbody>
  </table>
</div>
<h2 id="column-slice-operators">Column slice operators</h2>
<div id="table:appendix-column-slice-operators">
  <table>
    <caption>
      Column slice operators
    </caption>
    <thead>
      <tr class="header">
        <th style="text-align: center">Attribute Type</th>
        <th style="text-align: center">Operator</th>
        <th style="text-align: center">Operand</th>
      </tr>
    </thead>
    <tbody>
      <tr class="odd">
        <td style="text-align: center" rowspan="11">String</td>
        <td style="text-align: center">Contains</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Greater than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">In</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Less than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Less than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Not</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Not in</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="8">Date</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Greater than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Less than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Less than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Not equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="10">Number</td>
        <td style="text-align: center">Equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Greater than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Greater than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">In</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Less than</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Less than or equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Not equals</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Not in</td>
        <td style="text-align: center">Yes</td>
      </tr>
      <tr class="even">
        <td style="text-align: center" rowspan="5">Boolean</td>
        <td style="text-align: center">Is false</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is missing</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is present</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Is true</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="even">
        <td style="text-align: center">Is untrue</td>
        <td style="text-align: center">No</td>
      </tr>
      <tr class="odd">
        <td style="text-align: center">Matrix</td>
        <td style="text-align: center">Slice</td>
        <td style="text-align: center">Yes</td>
      </tr>
    </tbody>
  </table>
</div>