Merge 20240301 US changes into master

.gitignore

@@ -214,3 +214,8 @@ pip-log.txt
 #Mr Developer
 .mr.developer.cfg
 /target
+
+#################
+## IntelliJ
+#################
+.idea

README.md

@@ -1,31 +1,37 @@
 SNOMED CT DATABASE LOAD SCRIPTS
 ===============================
-Scripts to create and populate a MySQL, Postgres, or Oracle database with a 
-SNOMED CT terminology release in RF2.
+Scripts to build a transitive closure table from an RF2, snapshot,
+inferred SNOMED CT relationships file, create and populate a MySQL,
+Postgres, or Oracle database with a SNOMED CT terminology release in RF2.
 
 To download a package built for the latest US or International edition, visit
-http://www.westcoastinformatics.com/resources.html
+https://www.westcoastinformatics.com/products/db-load-scripts
+
+*New feature: a "depth" flag is now computed as well where 0 means "self",
+1 means "child", and >1 means "non-child descendant".  This enables asking
+queries about "child of", "descendant of", and "descendant or self of".
 
 Minimum Specification
 ---------------------
-- MYSQL v5.6.+
-- Oracle v11.+
-- Postgres 9.+
+- MySQL v8.4
+  - MySQL v8 requires the server to run with "--local_infile=ON"
+- MariaDB v10.+, v11.+, v12.+
+- Oracle v19.+, v21.+, v26.+
+- Postgres v14.+, v15.+, v16.+, v17.+, v18.+
 
 Installation
 ------------
-* See HOWTO.txt file to understand how to load the rf2 files into the database
-
+* See [HOWTO.md](src/main/resources/HOWTO.md) file to understand how to load the rf2 files into the database
 
 See Also
 --------
-* src/main/resources/LICENSE.txt
-* src/main/resources/README.txt
-* src/main/resources/HOWTO.txt
-   
+* [src/main/resources/LICENSE.txt](src/main/resources/LICENSE.txt)
+* [src/main/resources/README.md](src/main/resources/README.md)
+* [src/main/resources/HOWTO.md](src/main/resources/HOWTO.md)
+
 TODO
 ----
 Next Steps:
 * Better automated QA to validate file sizes against table sizes
 * Better automated QA to validate views have same row count as driving tables
-* Support additional SNOMED files
+

pom.xml

@@ -28,15 +28,15 @@
         <activeByDefault>true</activeByDefault>
       </activation>
       <properties>
-        <editionVersion>20190731</editionVersion>
+        <editionVersion>20260701</editionVersion>
         <editionType>Core</editionType>
         <editionLabel>INT</editionLabel>
       </properties>
     </profile>
     <profile>
       <id>US</id>
       <properties>
-        <editionVersion>20190901</editionVersion>
+        <editionVersion>20260301</editionVersion>
         <editionType>National</editionType>
         <editionLabel>US1000124</editionLabel>
       </properties>

src/main/assembly/oracle.xml

@@ -11,6 +11,7 @@
       <excludes>
         <exclude>rf1/**</exclude>
         <exclude>**/*mysql*</exclude>
+        <exclude>**/*maria*</exclude>
         <exclude>**/*postgres*</exclude>
         <exclude>**/*psql*</exclude>
       </excludes>

src/main/assembly/postgres.xml

@@ -11,6 +11,7 @@
       <excludes>
         <exclude>rf1/**</exclude>
         <exclude>**/*oracle*</exclude>
+        <exclude>**/*maria*</exclude>
         <exclude>**/*.ctl</exclude>
         <exclude>**/*mysql*</exclude>
       </excludes>

src/main/assembly/tc.xml

@@ -0,0 +1,39 @@
+<assembly>
+  <id>tc</id>
+  <baseDirectory>/</baseDirectory>
+  <formats>
+    <format>zip</format>
+  </formats>
+  <fileSets>
+    <fileSet>
+      <directory>src/main/resources</directory>
+      <outputDirectory>/</outputDirectory>
+      <excludes>
+        <exclude>rf1/**</exclude>
+        <exclude>**/*mysql*</exclude>
+        <exclude>**/*oracle*</exclude>
+        <exclude>**/*.ctl</exclude>
+      </excludes>
+      <filtered>true</filtered>
+      <lineEnding>keep</lineEnding>
+    </fileSet>
+
+    <!-- Copy code to RF1 directory <fileSet> <directory>target/classes</directory> 
+      <outputDirectory>/rf1</outputDirectory> <includes> <include>**/*class</include> <include>**/*class</include> 
+      </includes> <filtered>false</filtered> </fileSet> <fileSet> <directory>src/main/java</directory> 
+      <outputDirectory>/rf1</outputDirectory> </fileSet> -->
+    <!-- Copy code to RF2 directory -->
+    <fileSet>
+      <directory>target/classes</directory>
+      <outputDirectory>/rf2</outputDirectory>
+      <includes>
+        <include>**/*class</include>
+      </includes>
+      <filtered>false</filtered>
+    </fileSet>
+    <fileSet>
+      <directory>src/main/java</directory>
+      <outputDirectory>/rf2</outputDirectory>
+    </fileSet>
+  </fileSets>
+</assembly>

src/main/resources/HOWTO.md

@@ -0,0 +1,75 @@
+# How To Use the SNOMED CT Database Loader
+This HOWTO describes how to use the installer scripts to load SNOMEDCT into a database.
+
+## Watch the Demo Video
+
+See [https://youtu.be/917T6D3b5ro](https://youtu.be/917T6D3b5ro)
+
+## Installation
+1.  Download the database loader zip file from the https://www.westcoastinformatics.com/products/db-load-scripts website. Select the appropriate database-version pair (i.e. oracle-US Edition). Depending on your database and edition, identify the appropriate target's zip file.
+
+2.  Unzip the contents of the zip file into a 'data' folder
+
+3.  Download the SNOMED CT Release files in ZIP format. US Users can find the files here: 
+	* [SNOMED CT® US Edition Release Files](https://www.nlm.nih.gov/healthit/snomedct/us_edition.html)
+    * [SNOMED CT® International Edition Release Files](https://www.nlm.nih.gov/healthit/snomedct/international.html)
+
+4.  Open the desired zip file and extract the 'Snapshot' folder into the 'data/rf2' folder. Once completed:
+	* The 'data' folder's content will include:
+		- 'rf2' folder
+		- HOWTO.md
+		- LICENSE.txt
+		- README.md
+	* The 'data/rf2' folder's contents will include:
+		- 'Snapshot' folder
+		- *.sql files
+		- *.ctl files (for oracle only)
+		- one *.sh file
+
+5.	In a text editor, open the temp-folder's appropriate "populate" script as defined by your database. 
+   * For example:
+       - For MySQL (or MariaDB < 11), use the "populate_mysql_db.sh" script
+       - For MariaDB 11+, use the "populate_maria_db.sh" script
+       - For Postgres, use the "populate_postgres_db.sh" script
+       - For Oracle, use the "populate_oracle_db.sh" script
+   * Configure settings at the top of the "populate" script for your environment
+       - use your host's IP address or hostname
+       - user is your database's username
+       - password is your database's password
+       - tns_name or db_name is the database's schema which is to be populated
+
+6.  Create the database schema as defined in step #5
+
+    * NOTE: postgres supports only db-level character encoding, so make sure to create your database using: `WITH ENCODING 'UTF-8'`
+
+
+7.	Execute the "populate" script
+	* Note: a complete log file will appear as `mysql.log`, `mariadb.log`, `postgres.log`, or `oracle.log`
+
+## Oracle: VARCHAR2 vs CLOB for description and text definition (4096 characters)
+
+SNOMED CT allows description and text definition terms up to 4096 characters. Oracle's default `VARCHAR2` limit is 4000 characters, so this package uses **CLOB** for:
+
+- `description.term`
+- `textdefinition.term`
+
+**Writing queries**
+
+- **Simple SELECT:** Use the column as usual. Oracle converts CLOB to character data for display.
+  ```sql
+  SELECT term FROM description WHERE id = 12345;
+  SELECT term FROM textdefinitionwithnames WHERE conceptId = 12345;
+  ```
+
+- **GROUP BY, ORDER BY, or DISTINCT** on these columns: Use `DBMS_LOB.SUBSTR` so Oracle can compare/sort. Use the same expression in both SELECT and GROUP BY when grouping by the term.
+  ```sql
+  SELECT DBMS_LOB.SUBSTR(term, 4000, 1) AS term, COUNT(*)
+  FROM description
+  WHERE active = 1
+  GROUP BY DBMS_LOB.SUBSTR(term, 4000, 1);
+  ```
+
+- **Views:** Views that expose these columns (e.g. `descriptionwithnames`, `textdefinitionwithnames`, `conceptwithnames` / preferred name) return CLOB. Same rules: simple SELECT is fine; for GROUP BY/ORDER BY on the term column, wrap in `DBMS_LOB.SUBSTR(column, 4000, 1)`.
+
+**Why CLOB and not VARCHAR2(4096)?**  
+`VARCHAR2(4096)` requires Oracle's `MAX_STRING_SIZE=EXTENDED`, which is not default and is not assumed for this package. Using CLOB works on all supported Oracle versions without that setting.

src/main/resources/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright 2019 West Coast Informatics, LLC
+Copyright 2026 West Coast Informatics, LLC
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

src/main/resources/README.md

@@ -0,0 +1,24 @@
+# Welcome to DB Load Scripts
+Welcome to SNOMED CT MySQL, Postgres, and Oracle Database Load Scripts
+https://www.westcoastinformatics.com/products/db-load-scripts
+
+Go to the above link for documentation, and additional downloads.
+
+## Important Notes
+* These database load scripts are configured to work with the ${editionVersion}
+  SNOMEDCT ${editionLabel} Edition.
+
+## Key Features
+
+* Support for Oracle, Postgres, and MySQL (and MariaDB)
+* Support for RF2 data format
+* Denormalization of concept/description ids for contextualized lookup
+* Builds transitive closure table from RF2 snapshot, inferred, SNOMEDCT file.
+* Views with denormalized concept preferred names
+
+## Watch the Demo Video
+
+See [https://youtu.be/917T6D3b5ro](https://youtu.be/917T6D3b5ro)
+
+### Developer Notes
+https://github.com/WestCoastInformatics/SNOMED-DB-Load-Scripts