diff --git a/.gitignore b/.gitignore index e39c14a..818c15f 100644 --- a/.gitignore +++ b/.gitignore @@ -214,3 +214,8 @@ pip-log.txt #Mr Developer .mr.developer.cfg /target + +################# +## IntelliJ +################# +.idea \ No newline at end of file diff --git a/README.md b/README.md index 2d25ba0..2d2d7ae 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,37 @@ SNOMED CT DATABASE LOAD SCRIPTS =============================== -Scripts to create and populate a MySQL, Postgres, or Oracle database with a -SNOMED CT terminology release in RF2. +Scripts to build a transitive closure table from an RF2, snapshot, +inferred SNOMED CT relationships file, create and populate a MySQL, +Postgres, or Oracle database with a SNOMED CT terminology release in RF2. To download a package built for the latest US or International edition, visit -http://www.westcoastinformatics.com/resources.html +https://www.westcoastinformatics.com/products/db-load-scripts + +*New feature: a "depth" flag is now computed as well where 0 means "self", +1 means "child", and >1 means "non-child descendant". This enables asking +queries about "child of", "descendant of", and "descendant or self of". Minimum Specification --------------------- -- MYSQL v5.6.+ -- Oracle v11.+ -- Postgres 9.+ +- MySQL v8.4 + - MySQL v8 requires the server to run with "--local_infile=ON" +- MariaDB v10.+, v11.+, v12.+ +- Oracle v19.+, v21.+, v26.+ +- Postgres v14.+, v15.+, v16.+, v17.+, v18.+ Installation ------------ -* See HOWTO.txt file to understand how to load the rf2 files into the database - +* See [HOWTO.md](src/main/resources/HOWTO.md) file to understand how to load the rf2 files into the database See Also -------- -* src/main/resources/LICENSE.txt -* src/main/resources/README.txt -* src/main/resources/HOWTO.txt - +* [src/main/resources/LICENSE.txt](src/main/resources/LICENSE.txt) +* [src/main/resources/README.md](src/main/resources/README.md) +* [src/main/resources/HOWTO.md](src/main/resources/HOWTO.md) + TODO ---- Next Steps: * Better automated QA to validate file sizes against table sizes * Better automated QA to validate views have same row count as driving tables -* Support additional SNOMED files + diff --git a/pom.xml b/pom.xml index d7e61aa..684d1d4 100644 --- a/pom.xml +++ b/pom.xml @@ -28,7 +28,7 @@ true - 20190731 + 20260701 Core INT @@ -36,7 +36,7 @@ US - 20190901 + 20260301 National US1000124 diff --git a/src/main/assembly/oracle.xml b/src/main/assembly/oracle.xml index acad967..93be388 100644 --- a/src/main/assembly/oracle.xml +++ b/src/main/assembly/oracle.xml @@ -11,6 +11,7 @@ rf1/** **/*mysql* + **/*maria* **/*postgres* **/*psql* diff --git a/src/main/assembly/postgres.xml b/src/main/assembly/postgres.xml index 3cb5007..6d97931 100644 --- a/src/main/assembly/postgres.xml +++ b/src/main/assembly/postgres.xml @@ -11,6 +11,7 @@ rf1/** **/*oracle* + **/*maria* **/*.ctl **/*mysql* diff --git a/src/main/assembly/tc.xml b/src/main/assembly/tc.xml new file mode 100644 index 0000000..683dc1f --- /dev/null +++ b/src/main/assembly/tc.xml @@ -0,0 +1,39 @@ + + tc + / + + zip + + + + src/main/resources + / + + rf1/** + **/*mysql* + **/*oracle* + **/*.ctl + + true + keep + + + + + + target/classes + /rf2 + + **/*class + + false + + + src/main/java + /rf2 + + + diff --git a/src/main/resources/HOWTO.md b/src/main/resources/HOWTO.md new file mode 100644 index 0000000..fb311f2 --- /dev/null +++ b/src/main/resources/HOWTO.md @@ -0,0 +1,75 @@ +# How To Use the SNOMED CT Database Loader +This HOWTO describes how to use the installer scripts to load SNOMEDCT into a database. + +## Watch the Demo Video + +See [https://youtu.be/917T6D3b5ro](https://youtu.be/917T6D3b5ro) + +## Installation +1. Download the database loader zip file from the https://www.westcoastinformatics.com/products/db-load-scripts website. Select the appropriate database-version pair (i.e. oracle-US Edition). Depending on your database and edition, identify the appropriate target's zip file. + +2. Unzip the contents of the zip file into a 'data' folder + +3. Download the SNOMED CT Release files in ZIP format. US Users can find the files here: + * [SNOMED CT® US Edition Release Files](https://www.nlm.nih.gov/healthit/snomedct/us_edition.html) + * [SNOMED CT® International Edition Release Files](https://www.nlm.nih.gov/healthit/snomedct/international.html) + +4. Open the desired zip file and extract the 'Snapshot' folder into the 'data/rf2' folder. Once completed: + * The 'data' folder's content will include: + - 'rf2' folder + - HOWTO.md + - LICENSE.txt + - README.md + * The 'data/rf2' folder's contents will include: + - 'Snapshot' folder + - *.sql files + - *.ctl files (for oracle only) + - one *.sh file + +5. In a text editor, open the temp-folder's appropriate "populate" script as defined by your database. + * For example: + - For MySQL (or MariaDB < 11), use the "populate_mysql_db.sh" script + - For MariaDB 11+, use the "populate_maria_db.sh" script + - For Postgres, use the "populate_postgres_db.sh" script + - For Oracle, use the "populate_oracle_db.sh" script + * Configure settings at the top of the "populate" script for your environment + - use your host's IP address or hostname + - user is your database's username + - password is your database's password + - tns_name or db_name is the database's schema which is to be populated + +6. Create the database schema as defined in step #5 + + * NOTE: postgres supports only db-level character encoding, so make sure to create your database using: `WITH ENCODING 'UTF-8'` + + +7. Execute the "populate" script + * Note: a complete log file will appear as `mysql.log`, `mariadb.log`, `postgres.log`, or `oracle.log` + +## Oracle: VARCHAR2 vs CLOB for description and text definition (4096 characters) + +SNOMED CT allows description and text definition terms up to 4096 characters. Oracle's default `VARCHAR2` limit is 4000 characters, so this package uses **CLOB** for: + +- `description.term` +- `textdefinition.term` + +**Writing queries** + +- **Simple SELECT:** Use the column as usual. Oracle converts CLOB to character data for display. + ```sql + SELECT term FROM description WHERE id = 12345; + SELECT term FROM textdefinitionwithnames WHERE conceptId = 12345; + ``` + +- **GROUP BY, ORDER BY, or DISTINCT** on these columns: Use `DBMS_LOB.SUBSTR` so Oracle can compare/sort. Use the same expression in both SELECT and GROUP BY when grouping by the term. + ```sql + SELECT DBMS_LOB.SUBSTR(term, 4000, 1) AS term, COUNT(*) + FROM description + WHERE active = 1 + GROUP BY DBMS_LOB.SUBSTR(term, 4000, 1); + ``` + +- **Views:** Views that expose these columns (e.g. `descriptionwithnames`, `textdefinitionwithnames`, `conceptwithnames` / preferred name) return CLOB. Same rules: simple SELECT is fine; for GROUP BY/ORDER BY on the term column, wrap in `DBMS_LOB.SUBSTR(column, 4000, 1)`. + +**Why CLOB and not VARCHAR2(4096)?** +`VARCHAR2(4096)` requires Oracle's `MAX_STRING_SIZE=EXTENDED`, which is not default and is not assumed for this package. Using CLOB works on all supported Oracle versions without that setting. diff --git a/src/main/resources/HOWTO.txt b/src/main/resources/HOWTO.txt deleted file mode 100644 index ca9a9ba..0000000 --- a/src/main/resources/HOWTO.txt +++ /dev/null @@ -1,52 +0,0 @@ -This HOWTO describes how to use the installer scripts to load SNOMED into a database. - -Installation ------------- -1. Download the database loader zip file from the http://www.westcoastinformatics.com/resources.html website. Select the appropriate database-version pair (i.e. oracle-US Edition). Depending on your database and edition, identify the appropriate target's zip file. - For example: - * For MySQL SNOMED-CORE, use the "snomed-db-scripts-mysql.20190731.zip" file - * For MySQL US-Edition, use the "snomed-db-scripts-mysql.20190901.zip" file - * For Postgres SNOMED-CORE, use the "snomed-db-scripts-postgres.20190731.zip" file - * For Postgres US-Edition, use the "snomed-db-scripts-postgres.20190901.zip" file - * For Oracle SNOMED-CORE, use the "snomed-db-scripts-oracle.20190731.zip" file - * For Oracle US-Edition, use the "snomed-db-scripts-oracle.20190901.zip" file - -2. Unzip the contents of the zip file into a 'data' folder - -3. Download the SNOMED CT Release files in ZIP format. US Users can find the files here: - * SNOMED CT® US Edition Release Files: https://www.nlm.nih.gov/healthit/snomedct/us_edition.html - * SNOMED CT® International Edition Release Files: https://www.nlm.nih.gov/healthit/snomedct/international.html - -4. Open the desired zip file and extract the 'Snapshot' folder into the 'data/rf2' folder. Once completed: - * The 'data' folder's content will include: - - 'rf2' folder - - HOWTO.txt - - LICENSE.txt - - README.txt - * The 'data/rf2' folder's contents will include: - - 'Snapshot' folder - - *.sql files - - *.ctl files (for oracle only) - - one *.bat file - - one *.sh file - -5. In a text editor, open the temp-folder's appropriate "populate" script in a text editor as defined by your platform and database. - * For example: - - For MySQL on Windows, use the "populate_mysql_db.bat" script - - For Postgres on Windows, use the "populate_postgres_db.bat" script - - For Oracle on Windows, use the "populate_oracle_db.bat" script - - For MySQL on Linux/Unix/MacOS, use the "populate_mysql_db.sh" script - - For Postgres on Linux/Unix/MacOS, use the "populate_postgres_db.sh" script - - For Oracle on Linux/Unix/MacOS, use the "populate_oracle_db.sh" script - * Configure settings at the top of the "populate" script for your environment - - user is your database's username - - password is your database's password - - tns_name or db_name is the database's schema which is to be populated - -6. Create the database schema as defined in step #5.b.iii - - * NOTE: postgres supports only db-level character encoding, so make sure to create your database using: WITH ENCODING 'UTF-8' - - -7. Execute the "populate" script - * Note: a complete log file will appear as "mysql.log", "postgres.log", or "oracle.log" diff --git a/src/main/resources/LICENSE.txt b/src/main/resources/LICENSE.txt index 61a2fa8..bf84c2d 100644 --- a/src/main/resources/LICENSE.txt +++ b/src/main/resources/LICENSE.txt @@ -1,4 +1,4 @@ -Copyright 2019 West Coast Informatics, LLC +Copyright 2026 West Coast Informatics, LLC Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/src/main/resources/README.md b/src/main/resources/README.md new file mode 100644 index 0000000..f564e4b --- /dev/null +++ b/src/main/resources/README.md @@ -0,0 +1,24 @@ +# Welcome to DB Load Scripts +Welcome to SNOMED CT MySQL, Postgres, and Oracle Database Load Scripts +https://www.westcoastinformatics.com/products/db-load-scripts + +Go to the above link for documentation, and additional downloads. + +## Important Notes +* These database load scripts are configured to work with the ${editionVersion} + SNOMEDCT ${editionLabel} Edition. + +## Key Features + +* Support for Oracle, Postgres, and MySQL (and MariaDB) +* Support for RF2 data format +* Denormalization of concept/description ids for contextualized lookup +* Builds transitive closure table from RF2 snapshot, inferred, SNOMEDCT file. +* Views with denormalized concept preferred names + +## Watch the Demo Video + +See [https://youtu.be/917T6D3b5ro](https://youtu.be/917T6D3b5ro) + +### Developer Notes +https://github.com/WestCoastInformatics/SNOMED-DB-Load-Scripts diff --git a/src/main/resources/README.txt b/src/main/resources/README.txt deleted file mode 100644 index 2883fbd..0000000 --- a/src/main/resources/README.txt +++ /dev/null @@ -1,20 +0,0 @@ -Welcome to SNOMED CT MySQL, Postgres, and Oracle Database Load Scripts -http://www.wcinformatics.com/resources.html - -Go to the above link for documentation, and additional downloads. - -Important Notes ---------------- -* These database load scripts are configured to work with the ${editionVersion} - SNOMEDCT ${editionLabel} Edition. - -Key Features ------------- - -* Support for Oracle, Postgres, and MySQL -* Support for RF2 data format -* Denormalization of concept/description ids for contextualized lookup - -Developer Notes ---------------- -https://github.com/bcarlsenca/SNOMED-DB-Load-Scripts.git diff --git a/src/main/resources/rf2/compute_transitive_closure.pl b/src/main/resources/rf2/compute_transitive_closure.pl new file mode 100644 index 0000000..313701b --- /dev/null +++ b/src/main/resources/rf2/compute_transitive_closure.pl @@ -0,0 +1,313 @@ +#!/usr/bin/perl +# Copyright 2026 West Coast Informatics, Inc +# +# Computes transitive closure from a snapshot inferred RF2 relationships file. +# +use strict vars; + +# +# Set Defaults & Environment +# +our $badargs; +our $badvalue; +our $isaRel = "116680003"; +our $root = "138875005"; +our $history = ""; +our $force = 0; +our $self = 1; + +# +# Check options +# +our @ARGS=(); +while (@ARGV) { + my $arg = shift(@ARGV); + if ($arg !~ /^-/) { + push @ARGS, $arg; + next; + } + if ($arg eq "--history-min") { + $history = "min"; + next; + } elsif ($arg eq "--history-mod") { + $history = "mod"; + next; + } elsif ($arg eq "--history-max") { + $history = "max"; + next; + } elsif ($arg eq "--noself") { + $self = 0; + next; + } elsif ($arg eq "--force") { + $force = 1; + next; + } elsif ($arg eq "--help") { + &PrintHelp; + exit(0); + } else { + $badargs = 1; + $badvalue = $arg; + } +} + +# +# Handle parameters +# +if (scalar(@ARGS) != 1) { + $badargs = 3; + $badvalue = scalar(@ARGS); +} +our ($relsFile) = @ARGS; +our $outputFile = "$relsFile"; +$outputFile =~ s/_Relationship_/_TransitiveClosure_/; + +# Check output file +if (-e "$outputFile") { + if ($force) { + unlink $outputFile; + } else { + die "Output file already exists: $outputFile. Check this is not an old file\n"; + } +} + +# Check history file +our $historyFile = ""; +if ($history) { + # Attempt to find the Association refset + $historyFile = "$relsFile"; + $historyFile =~ s/sct2_Relationship_/der2_cRefset_Association/; + if ($historyFile =~ /^der2/) { + $historyFile =~ s/^/..\/Refset\/Content\//; + } elsif ($historyFile =~ /Terminology/) { + $historyFile =~ s/Terminology/Refset\/Content/; + } else { + die "Unable to determine path to Association refset file from relationships file = $relsFile\n"; + } + if (! -e $historyFile) { + die "Computed path to Association refset file does not exist = $historyFile\n"; + } +} + + +# +# Write any arg errors +# +our %errors = (1 => "Illegal switch: $badvalue", + 2 => "Illegal service: $badvalue", + 3 => "Bad number of arguments: $badvalue", + 4 => "$badvalue must be set"); +if ($badargs) { + &PrintUsage; + print "\n$errors{$badargs}\n"; + exit(1); +} + +# +# Begin +# +print "------------------------------------------------------------\n"; +print "Starting ...", scalar(localtime),"\n"; +print "------------------------------------------------------------\n"; +print "Isa rel : $isaRel\n"; +print "Rels file : $relsFile\n"; +print "Self : $self\n"; +print "Output file : $outputFile\n"; +if ($history) { + print "History : $history\n"; + print "History file : $historyFile\n"; +} +print "\n"; + +# +# Load PAR/CHD relationships map. +# +print " Load PAR/CHD rels ...".(scalar(localtime))."\n"; +our %codes = (); +our %parChd = (); +our %seen = (); +our $eol = "\n"; +initRelationships($relsFile, $historyFile); + +# +# Write transitive closure file +# +print " Write transitive closure table ...".(scalar(localtime))."\n"; +our $OUT; +open ($OUT, ">$outputFile") || die "Could not open $outputFile: $! $?\n"; +print $OUT "superTypeId\tsubTypeId\tdepth$eol"; +our $code; +our $ct = 0; +foreach $code (keys %codes) { + # skip root + next if $code eq $root; + $ct++; + + # short circuit for leaf nodes + if (!$parChd{$code} && $self) { + print $OUT "$code\t$code\t0$eol"; + } + + # otherwise, gather descendants + else { + %seen = (); + my %descs = getDescendants($code, 1); + my $desc; + foreach $desc (keys %descs) { + if (!$self && !($descs{$desc}-1)) { + next; + } + print $OUT "$code\t$desc\t".($descs{$desc}-1)."$eol"; + } + } + + if ($ct % 10000 == 0) { + print " $ct codes processed ...", scalar(localtime),"\n"; + } +} +close($OUT); + +# +# Finish +# +print "------------------------------------------------------------\n"; +print "finished ...", scalar(localtime),"\n"; +print "------------------------------------------------------------\n"; + +exit 0; + + +######################### LOCAL PROCEDURES ####################### + +# +# Print the usage +# +sub PrintUsage { + + print qq{ +Usage: compute-transitive-closure.pl [--help] [--force] [--history-{min,mod,max}] + e.g. compute-transitive-closure.pl --history-min --noself +}; +} + +# +# Print help info +# +sub PrintHelp { + &PrintUsage; + print qq{ + Parameters + relsFile: A snapshot inferred RF2 relationships file + Options: + --force : If output file already exists, remove it and proceed + --history-min: Add historical relationships to support ECL HISTORY-MIN profile + --history-mod: Add historical relationships to support ECL HISTORY-MOD profile + --history-max: Add historical relationships to support ECL HISTORY-MAX profile + --help: On-line help + +}; +} + +# +# Initialize relationships hashes +# +sub initRelationships { + my($relsFile, $historyFile) = @_; + my $IN; + open ($IN, "<$relsFile") || die "Failed to open $relsFile: $! $?\n"; + my $ct = 1; + # iterate through rels file + while (<$IN>) { + chop; + if (/\r$/) { + s/\r//; + $eol = "\r\n"; + } + my ($id, $effectiveTime, $active, $moduleId, $sourceId, $destinationId, + $relationshipGroup, $typeId, $characteristicTypeId, $modifierId) = split /\t/; + + # skip inactive or non-isa rels + next unless $typeId eq $isaRel; + next unless $active; + $ct++; + + # push new child for parent + push @{$parChd{$destinationId}}, $sourceId; + $codes{$sourceId} = 1; + $codes{$destinationId} = 1; + } + print " $ct relationships loaded\n"; + close($IN); + + # Load appropriate history relationships + if ($history) { + + $ct = 0; + our %qual = (); + if ($history eq "min") { + $qual{"900000000000527005"} = 1; + } elsif ($history eq "mod") { + $qual{"900000000000527005"} = 1; + $qual{"900000000000526001"} = 1; + $qual{"900000000000528000"} = 1; + $qual{"1186924009"} = 1; + } + open ($IN, "<$historyFile") || die "Failed to open $historyFile: $! $?\n"; + my $ct = 0; + # iterate through history file + while (<$IN>) { + chop; s/\r//; + + my ($id, $effectiveTime, $active, $moduleId, $refsetId, + $referencedComponentId, $targetComponentId) = split /\t/; + + # skip unless the refsetId represents a qualifying historical association + next unless $history eq "max" || $qual{$refsetId}; + # skip inactive + next unless $active; + # skip if the targetComponentId isn't one of the codes (to avoid moved_to/from stuff) + next unless $codes{$targetComponentId}; + + $ct++; + + # push new child (inactive concept) for parent (active concept) + push @{$parChd{$targetComponentId}}, $referencedComponentId; + $codes{$referencedComponentId} = 1; + } + print " $ct historical relationships loaded\n"; + close($IN); + } +} + +# +# Get descendants of a root +# Input: root id +# Output: array of descendant ids +# +sub getDescendants { + my($par, $depth) = @_; + + # if we've seen this node already, children are accounted for - bail + if ($seen{$par}) { return (); } + $seen{$par} = 1; + + # Add an entry for this concept + my %descendants = (); + $descendants{$par} = $depth; + + # Get Children of this node + my @chd = @{$parChd{$par}}; + + # If this is a leaf node, bail + return %descendants if scalar(@chd) == 0; + + # Iterate through children, mark as descendant and recursively call + foreach my $chd (@chd) { + my %chddesc = getDescendants($chd, $depth+1); + # merge child descendants with the current descendants list. + foreach my $desc ( keys %chddesc) { + $descendants{"$desc"} = $chddesc{$desc}; + } + } + + return %descendants; +} \ No newline at end of file diff --git a/src/main/resources/rf2/compute_transitive_closure.py b/src/main/resources/rf2/compute_transitive_closure.py new file mode 100644 index 0000000..3993331 --- /dev/null +++ b/src/main/resources/rf2/compute_transitive_closure.py @@ -0,0 +1,251 @@ +#!/usr/bin/python +# Copyright 2026 West Coast Informatics, Inc +# +# Computes transitive closure from a snapshot inferred RF2 relationships file. +# + +import sys +import os +import csv +import glob +import argparse +import time +import logging +# Remove type hints for Python 2.7 compatibility + +# +# Set Defaults & Environment +# +isaRel = "116680003" +root = "138875005" +history = "" +force = False +self = True +codes = {} +parChd = {} +seen = {} + +# Set up logging +logging.basicConfig(level=logging.DEBUG) + +# Define the errors dictionary globally +errors = { + 1: "Illegal switch: ", + 2: "Illegal service: ", + 3: "Bad number of arguments: ", + 4: "{} must be set" +} + +## Custom help action for handling --help argparse option +class HelpAction(argparse.Action): + def __init__(self, option_strings, dest, nargs=0, **kwargs): + super(HelpAction, self).__init__(option_strings, dest, nargs=nargs, **kwargs) + + def __call__(self, parser, namespace, values, option_string=None): + print_help() + parser.exit() + +## Function to parse arguments +def parse_args(): + parser = argparse.ArgumentParser(add_help=False) + parser.add_argument('--history-min', action='store_const', const='min', dest='history') + parser.add_argument('--history-mod', action='store_const', const='mod', dest='history') + parser.add_argument('--history-max', action='store_const', const='max', dest='history') + parser.add_argument('--force', action='store_true') + parser.add_argument('--noself', action='store_true') + parser.add_argument('relsFile', help='RF2 relationships file') + parser.add_argument('--help', action=HelpAction) + + args = parser.parse_args() + return args + +## Function to initialize relationships +def initRelationships(relsFile, historyFile): + global codes, parChd + try: + # Read relationships + with open(relsFile, 'r') as f: + reader = csv.reader(f, delimiter='\t') + next(reader) # Skip header row + ct = 1 + # iterate through rels file + for row in reader: + if len(row) < 10: # Add validation + continue + id, effectiveTime, active, moduleId, sourceId, destinationId, relationshipGroup, typeId, characteristicTypeId, modifierId = row + # skip inactive or non-isa relationships + if typeId != isaRel or active != '1': # Check for '1' instead of boolean + continue + ct += 1 + + # push new child for parent + if destinationId not in parChd: + parChd[destinationId] = [] + parChd[destinationId].append(sourceId) + codes[sourceId] = 1 + codes[destinationId] = 1 + + logging.info(" {} relationships loaded".format(ct)) + except IOError as e: + logging.error("Failed to open {}: {}".format(relsFile, e)) + exit(1) + + # Load appropriate history relationships + if history: + try: + ct = 0 + qual = {} + if history == "min": + qual = {"900000000000527005": 1} + elif history == "mod": + qual = {"900000000000527005": 1, "900000000000526001": 1, + "900000000000528000": 1, "1186924009": 1} + + with open(historyFile, 'r') as IN: + reader = csv.reader(IN, delimiter='\t') + next(reader) # Skip header + for row in reader: + id, effectiveTime, active, moduleId, refsetId, referencedComponentId, targetComponentId = row + + # skip unless qualifying historical association + if not (history == "max" or refsetId in qual): + continue + # skip inactive + if active != '1': # Check for string '1' instead of boolean + continue + # skip if targetComponentId not in codes + if targetComponentId not in codes: + continue + + ct += 1 + + # push new child for parent + if targetComponentId not in parChd: + parChd[targetComponentId] = [] + parChd[targetComponentId].append(referencedComponentId) + codes[referencedComponentId] = 1 + + logging.info(" {} historical relationships loaded".format(ct)) + except Exception as e: + logging.error("Error processing history file: {}".format(e)) + raise + +## Function to get descendants +def getDescendants(par, depth): + global seen + # If parent has been seen, return empty dictionary + if par in seen: + return {} + + seen[par] = 1 + descendants = {par: depth} + + # If parent has children, get their descendants + if par in parChd: + for chd in parChd[par]: + descendants.update(getDescendants(chd, depth+1)) + return descendants + +## Function to print help +def print_help(parser=None, namespace=None, values=None, option_strings=None, dest=None): + print_usage() + + help_text = """ + Parameters: + relsFile: A snapshot inferred RF2 relationships file + Options: + --force: If output file already exists, remove it and proceed + --history-{min,mod,max}: Add historical relationships to support ECL HISTORY-{MIN,MOD,MAX} profile + --help: On-line help + """ + print(help_text) + sys.exit(0) # Exit the program + +## Function to print usage +def print_usage(): + print(""" +Usage: compute_transitive_closure.py [--help] [--force] [--history-{min,mod,max}] + e.g. compute_transitive_closure.py --history-min --noself +""") + +## Main function +def main(): + args = parse_args() + global force, self, history + force = args.force + self = not args.noself + history = args.history + relsFile = args.relsFile + outputFile = relsFile.replace("_Relationship_", "_TransitiveClosure_") + + # Check output file + if os.path.exists(outputFile): + if force: + os.remove(outputFile) + else: + raise Exception("Output file already exists: {}. Check this is not an old file".format(outputFile)) + + # Determine history file path + historyFile = "" + if history: + historyFile = relsFile.replace("sct2_Relationship_", "der2_cRefset_Association") + if historyFile.startswith("der2"): + historyFile = os.path.join("..", "Refset", "Content", historyFile) + elif "Terminology" in historyFile: + historyFile = historyFile.replace("Terminology", "Refset/Content") + else: + raise Exception("Unable to determine path to Association refset file from relationships file = {}".format(relsFile)) + if not os.path.exists(historyFile): + raise Exception("Computed path to Association refset file does not exist = {}".format(historyFile)) + + # Print configuration + print("------------------------------------------------------------") + print("Starting ... {}".format(time.asctime())) + print("------------------------------------------------------------") + print("Isa rel : {}".format(isaRel)) + print("Rels file : {}".format(relsFile)) + print("Self : {}".format(self)) + print("Output file : {}".format(outputFile)) + if history: + print("History : {}".format(history)) + print("History file : {}".format(historyFile)) + print("\n") + + # Initialize relationships + print(" Load PAR/CHD rels ... {}".format(time.asctime())) + initRelationships(relsFile, historyFile) + + # Write transitive closure to output file + print(" Write transitive closure table ... {}".format(time.asctime())) + ct = 0 # Initialize counter here + with open(outputFile, 'w') as f: + writer = csv.writer(f, delimiter='\t') + writer.writerow(["superTypeId", "subTypeId", "depth"]) + for code in codes.keys(): + ct += 1 # Increment counter for each code + # skip root + if code == root: + continue + # short circuit for leaf nodes + if code not in parChd and self: + writer.writerow([code, code, 0]) + # gather descendants + else: + seen.clear() # Clear seen dict for each new code + descendants = getDescendants(code, 1) + for desc, depth in descendants.items(): + if not self and not (depth-1): + continue + writer.writerow([code, desc, depth-1]) + + if ct % 10000 == 0: + logging.info(" {} codes processed ... {}".format(ct, time.asctime())) + + print("------------------------------------------------------------") + print("finished ... {}".format(time.asctime())) + print("------------------------------------------------------------\n") + + +## Run main function +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/src/main/resources/rf2/description.ctl b/src/main/resources/rf2/description.ctl index b0b2b74..156b8fe 100644 --- a/src/main/resources/rf2/description.ctl +++ b/src/main/resources/rf2/description.ctl @@ -17,6 +17,6 @@ trailing nullcols conceptId INTEGER EXTERNAL, languageCode CHAR(2), typeId INTEGER EXTERNAL, - term CHAR(255), + term CHAR(4096), caseSignificanceId INTEGER EXTERNAL ) \ No newline at end of file diff --git a/src/main/resources/rf2/identifier.ctl b/src/main/resources/rf2/identifier.ctl index 132673a..6231ec1 100644 --- a/src/main/resources/rf2/identifier.ctl +++ b/src/main/resources/rf2/identifier.ctl @@ -11,7 +11,7 @@ fields terminated by X'09' trailing nullcols ( identifierSchemeId INTEGER EXTERNAL, - alternateIdentifier CHAR(255), + alternateIdentifier CHAR(4096), effectiveTime DATE "YYYYMMDD", active INTEGER EXTERNAL, moduleId INTEGER EXTERNAL, diff --git a/src/main/resources/rf2/mrcmattributedomain.ctl b/src/main/resources/rf2/mrcmattributedomain.ctl index b702dae..c05c577 100644 --- a/src/main/resources/rf2/mrcmattributedomain.ctl +++ b/src/main/resources/rf2/mrcmattributedomain.ctl @@ -1,7 +1,7 @@ options (skip=1,direct=true) load data characterset UTF8 length semantics char -infile 'Snapshot/Refset/Metadata/der2_ssRefset_MRCMAttributeDomainSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +infile 'Snapshot/Refset/Metadata/der2_cissccRefset_MRCMAttributeDomainSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" badfile 'mrcmattributedomain.bad' discardfile 'mrcmattributedomain.dsc' insert diff --git a/src/main/resources/rf2/mrcmattributerange.ctl b/src/main/resources/rf2/mrcmattributerange.ctl index fd722b4..dee9cab 100644 --- a/src/main/resources/rf2/mrcmattributerange.ctl +++ b/src/main/resources/rf2/mrcmattributerange.ctl @@ -1,7 +1,7 @@ options (skip=1,direct=true) load data characterset UTF8 length semantics char -infile 'Snapshot/Refset/Metadata/der2_ssRefset_MRCMAttributeRangeSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +infile 'Snapshot/Refset/Metadata/der2_ssccRefset_MRCMAttributeRangeSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" badfile 'mrcmattributerange.bad' discardfile 'mrcmattributerange.dsc' insert diff --git a/src/main/resources/rf2/mrcmdomain.ctl b/src/main/resources/rf2/mrcmdomain.ctl index 34fdf23..3da70a4 100644 --- a/src/main/resources/rf2/mrcmdomain.ctl +++ b/src/main/resources/rf2/mrcmdomain.ctl @@ -1,20 +1,26 @@ options (skip=1,direct=true) load data characterset UTF8 length semantics char -infile 'Snapshot/Refset/Metadata/der2_ssRefset_MRCMModuleScopeSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" -badfile 'mrcmmodulescope.bad' -discardfile 'mrcmmodulescope.dsc' +infile 'Snapshot/Refset/Metadata/der2_sssssssRefset_MRCMDomainSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +badfile 'mrcmdomain.bad' +discardfile 'mrcmdomain.dsc' insert -into table mrcmmodulescope +into table mrcmdomain reenable disabled_constraints fields terminated by X'09' trailing nullcols ( - id CHAR(52), + id CHAR(52), effectiveTime DATE "YYYYMMDD", active INTEGER EXTERNAL, moduleId INTEGER EXTERNAL, refsetId INTEGER EXTERNAL, referencedComponentId INTEGER EXTERNAL, - mrcmRuleRefsetId INTEGER EXTERNAL + domainConstraint CHAR(4000), + parentDomain CHAR(4000), + proximalPrimitiveConstraint CHAR(4000), + proximalPrimitiveRefinement CHAR(4000), + domainTemplateForPrecoordination VARCHAR2(5000), + domainTemplateForPostcoordination CHAR(4000), + guideURL CHAR(4000) ) diff --git a/src/main/resources/rf2/mrcmmodulescope.ctl b/src/main/resources/rf2/mrcmmodulescope.ctl index cb3d1b3..353c9f6 100644 --- a/src/main/resources/rf2/mrcmmodulescope.ctl +++ b/src/main/resources/rf2/mrcmmodulescope.ctl @@ -1,11 +1,11 @@ options (skip=1,direct=true) load data characterset UTF8 length semantics char -infile 'Snapshot/Refset/Metadata/der2_ssRefset_MRCMDomainSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" -badfile 'mrcmdomain.bad' -discardfile 'mrcmdomain.dsc' +infile 'Snapshot/Refset/Metadata/der2_cRefset_MRCMModuleScopeSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +badfile 'mrcmmodulescope.bad' +discardfile 'mrcmmodulescope.dsc' insert -into table mrcmdomain +into table mrcmmodulescope reenable disabled_constraints fields terminated by X'09' trailing nullcols @@ -16,11 +16,5 @@ trailing nullcols moduleId INTEGER EXTERNAL, refsetId INTEGER EXTERNAL, referencedComponentId INTEGER EXTERNAL, - domainConstraint CHAR(4000), - parentDomain CHAR(4000), - proximalPrimitiveConstraint CHAR(4000) - proximalPrimitiveRefinement CHAR(4000), - domainTemplateForPrecoordination CHAR(4000), - domainTemplateForPostcoordination CHAR(4000), - guideURL CHAR(4000) + mrcmRuleRefsetId INTEGER EXTERNAL ) diff --git a/src/main/resources/rf2/mysql_tables.sql b/src/main/resources/rf2/mysql_tables.sql index 52cc990..f70f3b1 100644 --- a/src/main/resources/rf2/mysql_tables.sql +++ b/src/main/resources/rf2/mysql_tables.sql @@ -7,17 +7,19 @@ SET FOREIGN_KEY_CHECKS = 0; -- Concept file. DROP TABLE IF EXISTS concept; -CREATE TABLE concept ( - id NUMERIC(18) UNSIGNED NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - definitionStatusId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (definitionStatusId) REFERENCES concept(id) +CREATE TABLE concept +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + definitionStatusId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (definitionStatusId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Concept_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE concept LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Concept_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE concept LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@definitionStatusId) SET id = @id, effectiveTime = @effectiveTime, @@ -28,23 +30,25 @@ definitionStatusId = @definitionStatusId; -- Description file. DROP TABLE IF EXISTS description; -CREATE TABLE description ( - id NUMERIC(18) UNSIGNED NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - conceptId NUMERIC(18) UNSIGNED NOT NULL, - languageCode CHAR(2) NOT NULL, - typeId NUMERIC(18) UNSIGNED NOT NULL, - term VARCHAR(255) NOT NULL, - caseSignificanceId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (conceptId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (caseSignificanceId) REFERENCES concept(id) +CREATE TABLE description +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + conceptId NUMERIC(20) NOT NULL, + languageCode CHAR(2) NOT NULL, + typeId NUMERIC(20) NOT NULL, + term VARCHAR(4096) NOT NULL, + caseSignificanceId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (conceptId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (caseSignificanceId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Description_Snapshot-en_${editionLabel}_${editionVersion}.txt' INTO TABLE description LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Description_Snapshot-en_${editionLabel}_${editionVersion}.txt' INTO TABLE description LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@conceptId,@languageCode,@typeId,@term,@caseSignificanceId) SET id = @id, effectiveTime = @effectiveTime, @@ -59,17 +63,19 @@ caseSignificanceId=@caseSignificanceId; -- Identifier file. DROP TABLE IF EXISTS identifier; -CREATE TABLE identifier ( - identifierSchemeId NUMERIC(18) UNSIGNED NOT NULL, - alternateIdentifier VARCHAR(255) NOT NULL, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id) +CREATE TABLE identifier +( + identifierSchemeId NUMERIC(20) NOT NULL, + alternateIdentifier VARCHAR(4096) NOT NULL, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Identifier_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE identifier LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Identifier_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE identifier LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@identifierSchemeId,@alternateIdentifier,@effectiveTime,@active,@moduleId,@referencedComponentId) SET identifierSchemeId = @identifierSchemeId, alternateIdentifier = @alternateIdentifier, @@ -81,26 +87,28 @@ referencedComponentId = @referencedComponentId; -- Relationship file. DROP TABLE IF EXISTS relationship; -CREATE TABLE relationship ( - id NUMERIC(18) UNSIGNED NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - sourceId NUMERIC(18) UNSIGNED NOT NULL, - destinationId NUMERIC(18) UNSIGNED NOT NULL, - relationshipGroup INT UNSIGNED NOT NULL, - typeId NUMERIC(18) UNSIGNED NOT NULL, - characteristicTypeId NUMERIC(18) UNSIGNED NOT NULL, - modifierId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (sourceId) REFERENCES concept(id), - FOREIGN KEY (destinationId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (characteristicTypeId) REFERENCES concept(id), - FOREIGN KEY (modifierId) REFERENCES concept(id) +CREATE TABLE relationship +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + destinationId NUMERIC(20) NOT NULL, + relationshipGroup INT UNSIGNED NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (sourceId) REFERENCES concept (id), + FOREIGN KEY (destinationId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept (id), + FOREIGN KEY (modifierId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Relationship_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE relationship LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_Relationship_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE relationship LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@sourceId,@destinationId,@relationshipGroup,@typeId,@characteristicTypeId,@modifierId) SET id = @id, effectiveTime = @effectiveTime, @@ -113,22 +121,59 @@ typeId = @typeId, characteristicTypeId = @characteristicTypeId, modifierId = @modifierId; +-- Relationship concrete values file. +DROP TABLE IF EXISTS relationshipconcretevalues; +CREATE TABLE relationshipconcretevalues +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + value VARCHAR(256) NOT NULL, + relationshipGroup INT UNSIGNED NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (sourceId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept (id), + FOREIGN KEY (modifierId) REFERENCES concept (id) +) CHARACTER SET utf8; + +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_RelationshipConcreteValues_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE relationshipconcretevalues LINES TERMINATED BY '\r\n' IGNORE 1 LINES +(@id,@effectiveTime,@active,@moduleId,@sourceId,@value,@relationshipGroup,@typeId,@characteristicTypeId,@modifierId) +SET id = @id, +effectiveTime = @effectiveTime, +active = @active, +moduleId = @moduleId, +sourceId = @sourceId, +value = @value, +relationshipGroup = @relationshipGroup, +typeId = @typeId, +characteristicTypeId = @characteristicTypeId, +modifierId = @modifierId; + -- OWL Expression file. DROP TABLE IF EXISTS owlexpression; -CREATE TABLE owlexpression ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - owlExpression VARCHAR(10000) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (referencedComponentId) REFERENCES concept(id) +CREATE TABLE owlexpression +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + owlExpression TEXT NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (referencedComponentId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_sRefset_OWLExpressionSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE owlexpression LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_sRefset_OWLExpressionSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE owlexpression LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@owlExpression) SET id = @id, effectiveTime = @effectiveTime, @@ -141,26 +186,28 @@ owlExpression = @owlExpression; -- Stated Relationship file. DROP TABLE IF EXISTS statedrelationship; -CREATE TABLE statedrelationship ( - id NUMERIC(18) UNSIGNED NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - sourceId NUMERIC(18) UNSIGNED NOT NULL, - destinationId NUMERIC(18) UNSIGNED NOT NULL, - relationshipGroup INT UNSIGNED NOT NULL, - typeId NUMERIC(18) UNSIGNED NOT NULL, - characteristicTypeId NUMERIC(18) UNSIGNED NOT NULL, - modifierId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (sourceId) REFERENCES concept(id), - FOREIGN KEY (destinationId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (characteristicTypeId) REFERENCES concept(id), - FOREIGN KEY (modifierId) REFERENCES concept(id) +CREATE TABLE statedrelationship +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + destinationId NUMERIC(20) NOT NULL, + relationshipGroup INT UNSIGNED NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (sourceId) REFERENCES concept (id), + FOREIGN KEY (destinationId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept (id), + FOREIGN KEY (modifierId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_StatedRelationship_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE statedrelationship LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_StatedRelationship_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE statedrelationship LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@sourceId,@destinationId,@relationshipGroup,@typeId,@characteristicTypeId,@modifierId) SET id = @id, effectiveTime = @effectiveTime, @@ -175,23 +222,25 @@ modifierId = @modifierId; -- Text Definition file. DROP TABLE IF EXISTS textdefinition; -CREATE TABLE textdefinition ( - id NUMERIC(18) UNSIGNED NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - conceptId NUMERIC(18) UNSIGNED NOT NULL, - languageCode CHAR(2) NOT NULL, - typeId NUMERIC(18) UNSIGNED NOT NULL, - term VARCHAR(2048) NOT NULL, - caseSignificanceId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (conceptId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (caseSignificanceId) REFERENCES concept(id) +CREATE TABLE textdefinition +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + conceptId NUMERIC(20) NOT NULL, + languageCode CHAR(2) NOT NULL, + typeId NUMERIC(20) NOT NULL, + term VARCHAR(2048) NOT NULL, + caseSignificanceId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (conceptId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (caseSignificanceId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Terminology/sct2_TextDefinition_Snapshot-en_${editionLabel}_${editionVersion}.txt' INTO TABLE textdefinition LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_TextDefinition_Snapshot-en_${editionLabel}_${editionVersion}.txt' INTO TABLE textdefinition LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@conceptId,@languageCode,@typeId,@term,@caseSignificanceId) SET id = @id, effectiveTime = @effectiveTime, @@ -206,20 +255,22 @@ caseSignificanceId = @caseSignificanceId; -- Association Reference refset file. DROP TABLE IF EXISTS association; -CREATE TABLE association ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - targetComponent NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (targetComponent) REFERENCES concept(id) +CREATE TABLE association +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + targetComponent NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (targetComponent) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Content/der2_cRefset_AssociationSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE association LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Content/der2_cRefset_AssociationSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE association LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@targetComponent) SET id = @id, effectiveTime = @effectiveTime, @@ -232,20 +283,22 @@ targetComponent = @targetComponent; -- Attribute Value refset file. DROP TABLE IF EXISTS attributevalue; -CREATE TABLE attributevalue ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - valueId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (valueId) REFERENCES concept(id) +CREATE TABLE attributevalue +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + valueId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (valueId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Content/der2_cRefset_AttributeValueSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE attributevalue LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Content/der2_cRefset_AttributeValueSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE attributevalue LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@valueId) SET id = @id, effectiveTime = @effectiveTime, @@ -258,18 +311,20 @@ valueId = @valueId; -- Simple refset file. DROP TABLE IF EXISTS simple; -CREATE TABLE simple ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE simple +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Content/der2_Refset_SimpleSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE simple LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Content/der2_Refset_SimpleSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE simple LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId) SET id = @id, effectiveTime = @effectiveTime, @@ -286,15 +341,15 @@ referencedComponentId = @referencedComponentId; -- id CHAR(52) NOT NULL PRIMARY KEY, -- effectiveTime DATE NOT NULL, -- active BOOLEAN NOT NULL, --- moduleId NUMERIC(18) UNSIGNED NOT NULL, --- refsetId NUMERIC(18) UNSIGNED NOT NULL, --- referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, +-- moduleId NUMERIC(20) NOT NULL, +-- refsetId NUMERIC(20) NOT NULL, +-- referencedComponentId NUMERIC(20) NOT NULL, -- mapGroup INT UNSIGNED NOT NULL, -- mapPriority INT UNSIGNED NOT NULL, -- mapRule VARCHAR(4000) NOT NULL, -- mapAdvice VARCHAR(4000) NOT NULL, -- mapTarget VARCHAR(100) NOT NULL, --- correlationId NUMERIC(18) UNSIGNED NOT NULL, +-- correlationId NUMERIC(20) NOT NULL, -- FOREIGN KEY (moduleId) REFERENCES concept(id), -- FOREIGN KEY (refsetId) REFERENCES concept(id), -- FOREIGN KEY (correlationId) REFERENCES concept(id) @@ -318,27 +373,29 @@ referencedComponentId = @referencedComponentId; -- Extended Map refset file. DROP TABLE IF EXISTS extendedmap; -CREATE TABLE extendedmap ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - mapGroup INT UNSIGNED NOT NULL, - mapPriority INT UNSIGNED NOT NULL, - mapRule VARCHAR(4000) NOT NULL, - mapAdvice VARCHAR(4000) NOT NULL, - mapTarget VARCHAR(100) NOT NULL, - correlationId NUMERIC(18) UNSIGNED NOT NULL, - mapCategoryId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (correlationId) REFERENCES concept(id), - FOREIGN KEY (mapCategoryId) REFERENCES concept(id) +CREATE TABLE extendedmap +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mapGroup INT UNSIGNED NOT NULL, + mapPriority INT UNSIGNED NOT NULL, + mapRule VARCHAR(4000) NOT NULL, + mapAdvice VARCHAR(4000) NOT NULL, + mapTarget VARCHAR(100) NOT NULL, + correlationId NUMERIC(20) NOT NULL, + mapCategoryId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (correlationId) REFERENCES concept (id), + FOREIGN KEY (mapCategoryId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Map/der2_iisssccRefset_ExtendedMapSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE extendedmap LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Map/der2_iisssccRefset_ExtendedMapSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE extendedmap LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@mapGroup,@mapPriority,@mapRule,@mapAdvice,@mapTarget,@correlationId,@mapCategoryId) SET id = @id, effectiveTime = @effectiveTime, @@ -357,19 +414,21 @@ mapCategoryId = @mapCategoryId; -- Simple Map refset file. DROP TABLE IF EXISTS simplemap; -CREATE TABLE simplemap ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - mapTarget VARCHAR(100) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE simplemap +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mapTarget VARCHAR(100) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Map/der2_sRefset_SimpleMapSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE simplemap LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Map/der2_sRefset_SimpleMapSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE simplemap LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@mapTarget) SET id = @id, effectiveTime = @effectiveTime, @@ -382,20 +441,22 @@ mapTarget = @mapTarget; -- Language refset file. DROP TABLE IF EXISTS language; -CREATE TABLE language ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - acceptabilityId NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (acceptabilityId) REFERENCES concept(id) +CREATE TABLE language +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + acceptabilityId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (acceptabilityId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Language/der2_cRefset_LanguageSnapshot-en_${editionLabel}_${editionVersion}.txt' INTO TABLE language LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Language/der2_cRefset_LanguageSnapshot-en_${editionLabel}_${editionVersion}.txt' INTO TABLE language LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@acceptabilityId) SET id = @id, effectiveTime = @effectiveTime, @@ -408,21 +469,23 @@ acceptabilityId = @acceptabilityId; -- Refset Descriptor refset file. DROP TABLE IF EXISTS refsetdescriptor; -CREATE TABLE refsetdescriptor ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - attributeDescription NUMERIC(18) UNSIGNED NOT NULL, - attributeType NUMERIC(18) UNSIGNED NOT NULL, - attributeOrder NUMERIC(18) UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE refsetdescriptor +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + attributeDescription NUMERIC(20) NOT NULL, + attributeType NUMERIC(20) NOT NULL, + attributeOrder NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cciRefset_RefsetDescriptorSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE refsetdescriptor LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cciRefset_RefsetDescriptorSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE refsetdescriptor LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@attributeDescription,@attributeType,@attributeOrder) SET id = @id, effectiveTime = @effectiveTime, @@ -437,20 +500,22 @@ attributeOrder = @attributeOrder; -- Description Type refset file. DROP TABLE IF EXISTS descriptiontype; -CREATE TABLE descriptiontype ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - descriptionFormat NUMERIC(18) UNSIGNED NOT NULL, - descriptionLength INT UNSIGNED NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE descriptiontype +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + descriptionFormat NUMERIC(20) NOT NULL, + descriptionLength INT UNSIGNED NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_ciRefset_DescriptionTypeSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE descriptiontype LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_ciRefset_DescriptionTypeSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE descriptiontype LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@descriptionFormat,@descriptionLength) SET id = @id, effectiveTime = @effectiveTime, @@ -464,27 +529,29 @@ descriptionLength = @descriptionLength; -- MRCM Attribute Domain refset file. DROP TABLE IF EXISTS mrcmattributedomain; -CREATE TABLE mrcmattributedomain ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - domainId NUMERIC(18) NOT NULL, - grouped INT NOT NULL, - attributeCardinality VARCHAR(20) NOT NULL, +CREATE TABLE mrcmattributedomain +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + domainId NUMERIC(20) NOT NULL, + grouped INT NOT NULL, + attributeCardinality VARCHAR(20) NOT NULL, attributeInGroupCardinality VARCHAR(20) NOT NULL, - ruleStrengthId NUMERIC(18) NOT NULL, - contentTypeId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (domainId) REFERENCES concept(id), - FOREIGN KEY (ruleStrengthId) REFERENCES concept(id), - FOREIGN KEY (contentTypeId) REFERENCES concept(id) + ruleStrengthId NUMERIC(20) NOT NULL, + contentTypeId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (domainId) REFERENCES concept (id), + FOREIGN KEY (ruleStrengthId) REFERENCES concept (id), + FOREIGN KEY (contentTypeId) REFERENCES concept (id) ); -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cissccRefset_MRCMAttributeDomainSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmattributedomain LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cissccRefset_MRCMAttributeDomainSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmattributedomain LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@domainId,@grouped,@attributeCardinality,@attributeInGroupCardinality,@ruleStrengthId,@contentTypeId) SET id = @id, effectiveTime = @effectiveTime, @@ -502,20 +569,22 @@ contentTypeId = @contentTypeId; -- MRCM Module Scope refset file. DROP TABLE IF EXISTS mrcmmodulescope; -CREATE TABLE mrcmmodulescope ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - mrcmRuleRefsetId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (mrcmRuleRefsetId) REFERENCES concept(id) +CREATE TABLE mrcmmodulescope +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mrcmRuleRefsetId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (mrcmRuleRefsetId) REFERENCES concept (id) ); -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cRefset_MRCMModuleScopeSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmmodulescope LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cRefset_MRCMModuleScopeSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmmodulescope LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@mrcmRuleRefsetId) SET id = @id, effectiveTime = @effectiveTime, @@ -528,24 +597,26 @@ mrcmRuleRefsetId = @mrcmRuleRefsetId; -- MRCM Attribute Range refset file. DROP TABLE IF EXISTS mrcmattributerange; -CREATE TABLE mrcmattributerange ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - rangeConstraint VARCHAR(4000) NOT NULL, - attributeRule VARCHAR(4000) NOT NULL, - ruleStrengthId NUMERIC(18) NOT NULL, - contentTypeId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (ruleStrengthId) REFERENCES concept(id), - FOREIGN KEY (contentTypeId) REFERENCES concept(id) +CREATE TABLE mrcmattributerange +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + rangeConstraint VARCHAR(4000) NOT NULL, + attributeRule VARCHAR(4000) NOT NULL, + ruleStrengthId NUMERIC(20) NOT NULL, + contentTypeId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (ruleStrengthId) REFERENCES concept (id), + FOREIGN KEY (contentTypeId) REFERENCES concept (id) ); -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_ssccRefset_MRCMAttributeRangeSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmattributerange LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_ssccRefset_MRCMAttributeRangeSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmattributerange LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@rangeConstraint,@attributeRule,@ruleStrengthId,@contentTypeId) SET id = @id, effectiveTime = @effectiveTime, @@ -561,25 +632,27 @@ contentTypeId = @contentTypeId; -- MRCM Domain refset file. DROP TABLE IF EXISTS mrcmdomain; -CREATE TABLE mrcmdomain ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - domainConstraint VARCHAR(4000) NOT NULL, - parentDomain VARCHAR(4000), - proximalPrimitiveConstraint VARCHAR(4000) NOT NULL, - proximalPrimitiveRefinement VARCHAR(4000), - domainTemplateForPrecoordination VARCHAR(40000) NOT NULL, - domainTemplateForPostcoordination VARCHAR(40000) NOT NULL, - guideURL VARCHAR(4000) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE mrcmdomain +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + domainConstraint VARCHAR(4000) NOT NULL, + parentDomain VARCHAR(4000), + proximalPrimitiveConstraint VARCHAR(4000) NOT NULL, + proximalPrimitiveRefinement VARCHAR(4000), + domainTemplateForPrecoordination TEXT NOT NULL, + domainTemplateForPostcoordination TEXT NOT NULL, + guideURL VARCHAR(4000), + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_cissccRefset_MRCMAttributeDomainSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmdomain LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_sssssssRefset_MRCMDomainSnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE mrcmdomain LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@domainConstraint,@parentDomain,@proximalPrimitiveConstraint,@proximalPrimitiveRefinement,@domainTemplateForPrecoordination,@domainTemplateForPostcoordination,@guideURL) SET id = @id, effectiveTime = @effectiveTime, @@ -597,20 +670,22 @@ guideURL = @guideURL; -- Module Dependency refset file. DROP TABLE IF EXISTS moduledependency; -CREATE TABLE moduledependency ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - sourceEffectiveTime DATE NOT NULL, - targetEffectiveTime DATE NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE moduledependency +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + sourceEffectiveTime DATE NOT NULL, + targetEffectiveTime DATE NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) ) CHARACTER SET utf8; -LOAD DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE moduledependency LINES TERMINATED BY '\r\n' IGNORE 1 LINES +LOAD +DATA LOCAL INFILE 'Snapshot/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE moduledependency LINES TERMINATED BY '\r\n' IGNORE 1 LINES (@id,@effectiveTime,@active,@moduleId,@refsetId,@referencedComponentId,@sourceEffectiveTime,@targetEffectiveTime) SET id = @id, effectiveTime = @effectiveTime, @@ -622,5 +697,25 @@ sourceEffectiveTime = @sourceEffectiveTime, targetEffectiveTime = @targetEffectiveTime; +-- Transitive Closure table. +DROP TABLE IF EXISTS transitiveclosure; +CREATE TABLE transitiveclosure +( + superTypeId NUMERIC(20) NOT NULL, + subTypeId NUMERIC(20) NOT NULL, + depth INT NOT NULL, + PRIMARY KEY (superTypeId, subTypeId), + FOREIGN KEY (superTypeId) REFERENCES concept (id), + FOREIGN KEY (subTypeId) REFERENCES concept (id) +) CHARACTER SET utf8; + +LOAD +DATA LOCAL INFILE 'Snapshot/Terminology/sct2_TransitiveClosure_Snapshot_${editionLabel}_${editionVersion}.txt' INTO TABLE transitiveclosure LINES TERMINATED BY '\r\n' IGNORE 1 LINES +(@superTypeId,@subTypeId,@depth) +SET superTypeId = @superTypeId, +subTypeId = @subTypeId, +depth = @depth; + -- Restore foreign key checks to enforce referential integrity. -SET FOREIGN_KEY_CHECKS = 1; +SET +FOREIGN_KEY_CHECKS = 1; \ No newline at end of file diff --git a/src/main/resources/rf2/mysql_views.sql b/src/main/resources/rf2/mysql_views.sql index 24236b7..dda19c4 100644 --- a/src/main/resources/rf2/mysql_views.sql +++ b/src/main/resources/rf2/mysql_views.sql @@ -92,6 +92,27 @@ AND typeId = cpn4.conceptId AND characteristicTypeId = cpn5.conceptId AND modifierId = cpn6.conceptId; +-- Relationship concrete values table. +DROP VIEW IF EXISTS relationshipconcretevalueswithnames; +CREATE VIEW relationshipconcretevalueswithnames AS +SELECT id, effectiveTime, active, + moduleId, cpn1.preferredName moduleIdName, + sourceId, cpn2.preferredName sourceIdName, + value, relationshipGroup, + typeId, cpn4.preferredName typeIdName, + characteristicTypeId, cpn5.preferredName characteristicTypeIdName, + modifierId, cpn6.preferredName modifierIdName +from relationshipconcretevalues, + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 +WHERE moduleId = cpn1.conceptId +AND sourceId = cpn2.conceptId +AND typeId = cpn4.conceptId +AND characteristicTypeId = cpn5.conceptId +AND modifierId = cpn6.conceptId; -- OWL Expression table DROP VIEW IF EXISTS owlexpressionwithnames; @@ -432,3 +453,15 @@ WHERE moduleId = cpn1.conceptId AND refsetId = cpn2.conceptId AND referencedComponentId = cpn3.conceptId; +-- transitive closure table +-- NOTE: this assumes conceptpreferredname has already been created +-- via the standard RF2 load scripts +DROP VIEW IF EXISTS transitiveclosurewithnames; +CREATE VIEW transitiveclosurewithnames AS +SELECT superTypeId, cpn1.preferredName superTypeName, + subTypeId, cpn2.preferredName subTypeName, a.depth +FROM transitiveclosure a, + conceptpreferredname cpn1, + conceptpreferredname cpn2 +WHERE a.superTypeId = cpn1.conceptId + AND a.subTypeId = cpn2.conceptId; \ No newline at end of file diff --git a/src/main/resources/rf2/oracle_tables.sql b/src/main/resources/rf2/oracle_tables.sql index c4c8394..e8f69c2 100644 --- a/src/main/resources/rf2/oracle_tables.sql +++ b/src/main/resources/rf2/oracle_tables.sql @@ -3,226 +3,233 @@ SET ECHO ON -- Session settings. -ALTER SESSION SET NLS_LENGTH_SEMANTICS='CHAR'; +ALTER +SESSION SET NLS_LENGTH_SEMANTICS='CHAR'; -- Define helper procedure to drop tables. BEGIN - EXECUTE IMMEDIATE 'DROP PROCEDURE drop_table'; +EXECUTE IMMEDIATE 'DROP PROCEDURE drop_table'; EXCEPTION WHEN OTHERS THEN -- If procedure doesn't exist, it can't be dropped, -- so an exception will be thrown. Ignore it. IF SQLCODE != -4043 THEN RAISE; - END IF; +END IF; END; / -CREATE PROCEDURE drop_table (table_name IN VARCHAR2) IS +CREATE PROCEDURE drop_table(table_name IN VARCHAR2) IS BEGIN - EXECUTE IMMEDIATE 'DROP TABLE ' || table_name || ' CASCADE CONSTRAINTS'; +EXECUTE IMMEDIATE 'DROP TABLE ' || table_name || ' CASCADE CONSTRAINTS'; EXCEPTION -- If table doesn't exist, it can't be dropped, -- so an exception will be thrown. Ignore it. WHEN OTHERS THEN IF SQLCODE != -942 THEN RAISE; - END IF; +END IF; END; / -- Concept table. EXECUTE drop_table('concept'); -CREATE TABLE concept ( - id NUMERIC(18) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - definitionStatusId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (definitionStatusId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE concept +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + definitionStatusId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (definitionStatusId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Description table. EXECUTE drop_table('description'); -CREATE TABLE description ( - id NUMERIC(18) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - conceptId NUMERIC(18) NOT NULL, - languageCode CHAR(2) NOT NULL, - typeId NUMERIC(18) NOT NULL, - term VARCHAR2(255) NOT NULL, - caseSignificanceId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (conceptId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (caseSignificanceId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE description +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + conceptId NUMERIC(20) NOT NULL, + languageCode CHAR(2) NOT NULL, + typeId NUMERIC(20) NOT NULL, + term CLOB NOT NULL, + caseSignificanceId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (conceptId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (caseSignificanceId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Identifier table. EXECUTE drop_table('identifier'); -CREATE TABLE identifier ( - identifierSchemeId NUMERIC(18) NOT NULL, - alternateIdentifier VARCHAR2(255) NOT NULL, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE identifier +( + identifierSchemeId NUMERIC(20) NOT NULL, + alternateIdentifier CLOB NOT NULL, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Relationship table. EXECUTE drop_table('relationship'); -CREATE TABLE relationship ( - id NUMERIC(18) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - sourceId NUMERIC(18) NOT NULL, - destinationId NUMERIC(18) NOT NULL, - relationshipGroup INT NOT NULL, - typeId NUMERIC(18) NOT NULL, - characteristicTypeId NUMERIC(18) NOT NULL, - modifierId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (sourceId) REFERENCES concept(id), - FOREIGN KEY (destinationId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (characteristicTypeId) REFERENCES concept(id), - FOREIGN KEY (modifierId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; - -DROP TABLE IF EXISTS owlexpression; -CREATE TABLE owlexpression ( - id NUMERIC(18) UNSIGNED NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active BOOLEAN NOT NULL, - moduleId NUMERIC(18) UNSIGNED NOT NULL, - refsetId NUMERIC(18) UNSIGNED NOT NULL, - referencedComponentId NUMERIC(18) UNSIGNED NOT NULL, - owlExpression VARCHAR(10000) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (sourceId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (referencedComponentId) REFERENCES concept(id) -) CHARACTER SET utf8; +CREATE TABLE relationship +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + destinationId NUMERIC(20) NOT NULL, + relationshipGroup INT NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (sourceId) REFERENCES concept (id), + FOREIGN KEY (destinationId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept (id), + FOREIGN KEY (modifierId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; + +-- Relationship concrete values table. +EXECUTE drop_table('relationshipconcretevalues'); +CREATE TABLE relationshipconcretevalues +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + value VARCHAR2(256) NOT NULL, + relationshipGroup INT NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (sourceId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept (id), + FOREIGN KEY (modifierId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- OWL Expression table. EXECUTE drop_table('owlexpression'); -CREATE TABLE owlexpression ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - owlExpression VARCHAR2(4000) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (referencedComponentId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE owlexpression +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + owlExpression VARCHAR2(4000) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (referencedComponentId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Stated Relationship table. EXECUTE drop_table('statedrelationship'); -CREATE TABLE statedrelationship ( - id NUMERIC(18) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - sourceId NUMERIC(18) NOT NULL, - destinationId NUMERIC(18) NOT NULL, - relationshipGroup INT NOT NULL, - typeId NUMERIC(18) NOT NULL, - characteristicTypeId NUMERIC(18) NOT NULL, - modifierId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (sourceId) REFERENCES concept(id), - FOREIGN KEY (destinationId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (characteristicTypeId) REFERENCES concept(id), - FOREIGN KEY (modifierId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE statedrelationship +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + destinationId NUMERIC(20) NOT NULL, + relationshipGroup INT NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (sourceId) REFERENCES concept (id), + FOREIGN KEY (destinationId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept (id), + FOREIGN KEY (modifierId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Text Definition table. EXECUTE drop_table('textdefinition'); -CREATE TABLE textdefinition ( - id NUMERIC(18) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - conceptId NUMERIC(18) NOT NULL, - languageCode CHAR(2) NOT NULL, - typeId NUMERIC(18) NOT NULL, - term VARCHAR2(2048) NOT NULL, - caseSignificanceId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (conceptId) REFERENCES concept(id), - FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (caseSignificanceId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE textdefinition +( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + conceptId NUMERIC(20) NOT NULL, + languageCode CHAR(2) NOT NULL, + typeId NUMERIC(20) NOT NULL, + term VARCHAR2(2048) NOT NULL, + caseSignificanceId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (conceptId) REFERENCES concept (id), + FOREIGN KEY (typeId) REFERENCES concept (id), + FOREIGN KEY (caseSignificanceId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Association refset table. EXECUTE drop_table('association'); -CREATE TABLE association ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - targetComponent NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE association +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + targetComponent NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) -- TODO: FOREIGN KEY targetComponent? -) -PCTFREE 10 PCTUSED 80; +) PCTFREE 10 PCTUSED 80; -- Attribute Value refset table. EXECUTE drop_table('attributevalue'); -CREATE TABLE attributevalue ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - valueId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (valueId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE attributevalue +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + valueId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (valueId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Simple refset table. EXECUTE drop_table('simple'); -CREATE TABLE simple ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE simple +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- No more complex map, ICD9CM maps deprecated -- Complex Map refset table. @@ -231,15 +238,15 @@ PCTFREE 10 PCTUSED 80; -- id CHAR(52) NOT NULL PRIMARY KEY, -- effectiveTime DATE NOT NULL, -- active NUMERIC(1) NOT NULL, --- moduleId NUMERIC(18) NOT NULL, --- refsetId NUMERIC(18) NOT NULL, --- referencedComponentId NUMERIC(18) NOT NULL, +-- moduleId NUMERIC(20) NOT NULL, +-- refsetId NUMERIC(20) NOT NULL, +-- referencedComponentId NUMERIC(20) NOT NULL, -- mapGroup INT NOT NULL, -- mapPriority INT NOT NULL, -- mapRule VARCHAR2(4000), -- mapAdvice VARCHAR2(4000), -- mapTarget VARCHAR2(100), --- correlationId NUMERIC(18) NOT NULL, +-- correlationId NUMERIC(20) NOT NULL, -- FOREIGN KEY (moduleId) REFERENCES concept(id), -- FOREIGN KEY (refsetId) REFERENCES concept(id), -- FOREIGN KEY (correlationId) REFERENCES concept(id) @@ -249,196 +256,207 @@ PCTFREE 10 PCTUSED 80; -- Extended Map refset table. EXECUTE drop_table('extendedmap'); -CREATE TABLE extendedmap ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - mapGroup INT NOT NULL, - mapPriority INT NOT NULL, - mapRule VARCHAR2(4000), - mapAdvice VARCHAR2(4000), - mapTarget VARCHAR2(100), - correlationId NUMERIC(18) NOT NULL, - mapCategoryId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (correlationId) REFERENCES concept(id), - FOREIGN KEY (mapCategoryId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE extendedmap +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mapGroup INT NOT NULL, + mapPriority INT NOT NULL, + mapRule VARCHAR2(4000), + mapAdvice VARCHAR2(4000), + mapTarget VARCHAR2(100), + correlationId NUMERIC(20) NOT NULL, + mapCategoryId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (correlationId) REFERENCES concept (id), + FOREIGN KEY (mapCategoryId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Simple Map refset table. EXECUTE drop_table('simplemap'); -CREATE TABLE simplemap ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - mapTarget VARCHAR2(100) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE simplemap +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mapTarget VARCHAR2(100) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Lanuage refset table. EXECUTE drop_table('language'); -CREATE TABLE language ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - acceptabilityId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (acceptabilityId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE language +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + acceptabilityId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (acceptabilityId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Refset Descriptor refset table. EXECUTE drop_table('refsetdescriptor'); -CREATE TABLE refsetdescriptor ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - attributeDescription NUMERIC(18) NOT NULL, - attributeType NUMERIC(18) NOT NULL, - attributeOrder NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE refsetdescriptor +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + attributeDescription NUMERIC(20) NOT NULL, + attributeType NUMERIC(20) NOT NULL, + attributeOrder NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Description Type refset table. EXECUTE drop_table('descriptiontype'); -CREATE TABLE descriptiontype ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - descriptionFormat NUMERIC(18) NOT NULL, - descriptionLength INT NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE descriptiontype +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + descriptionFormat NUMERIC(20) NOT NULL, + descriptionLength INT NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- MRCM Attribute Domain refset file. EXECUTE drop_table('mrcmattributedomain'); -CREATE TABLE mrcmattributedomain ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - domainId NUMERIC(18) NOT NULL, - grouped INT NOT NULL, - attributeCardinality VARCHAR2(20) NOT NULL, +CREATE TABLE mrcmattributedomain +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + domainId NUMERIC(20) NOT NULL, + grouped INT NOT NULL, + attributeCardinality VARCHAR2(20) NOT NULL, attributeInGroupCardinality VARCHAR2(20) NOT NULL, - ruleStrengthId NUMERIC(18) NOT NULL, - contentTypeId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (domainId) REFERENCES concept(id), - FOREIGN KEY (ruleStrengthId) REFERENCES concept(id), - FOREIGN KEY (contentTypeId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; + ruleStrengthId NUMERIC(20) NOT NULL, + contentTypeId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (domainId) REFERENCES concept (id), + FOREIGN KEY (ruleStrengthId) REFERENCES concept (id), + FOREIGN KEY (contentTypeId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- MRCM Module Scope refset file. EXECUTE drop_table('mrcmmodulescope'); -CREATE TABLE mrcmmodulescope ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - mrcmRuleRefsetId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (mrcmRuleRefsetId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE mrcmmodulescope +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mrcmRuleRefsetId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (mrcmRuleRefsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- MRCM Attribute Range refset file. EXECUTE drop_table('mrcmattributerange'); -CREATE TABLE mrcmattributerange ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - rangeConstraint VARCHAR2(4000) NOT NULL, - attributeRule VARCHAR2(4000) NOT NULL, - ruleStrengthId NUMERIC(18) NOT NULL, - contentTypeId NUMERIC(18) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id), - FOREIGN KEY (ruleStrengthId) REFERENCES concept(id), - FOREIGN KEY (contentTypeId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; +CREATE TABLE mrcmattributerange +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + rangeConstraint VARCHAR2(4000) NOT NULL, + attributeRule VARCHAR2(4000) NOT NULL, + ruleStrengthId NUMERIC(20) NOT NULL, + contentTypeId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id), + FOREIGN KEY (ruleStrengthId) REFERENCES concept (id), + FOREIGN KEY (contentTypeId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- MRCM Domain refset file. EXECUTE drop_table('mrcmdomain'); -CREATE TABLE mrcmdomain ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - domainConstraint VARCHAR2(4000) NOT NULL, - parentDomain VARCHAR2(4000), - proximalPrimitiveConstraint VARCHAR2(4000) NOT NULL, - proximalPrimitiveRefinement VARCHAR2(4000), - domainTemplateForPrecoordination VARCHAR2(4000) NOT NULL, +CREATE TABLE mrcmdomain +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + domainConstraint VARCHAR2(4000) NOT NULL, + parentDomain VARCHAR2(4000), + proximalPrimitiveConstraint VARCHAR2(4000) NOT NULL, + proximalPrimitiveRefinement VARCHAR2(4000), + domainTemplateForPrecoordination VARCHAR2(4000) NOT NULL, domainTemplateForPostcoordination VARCHAR2(4000) NOT NULL, - guideURL VARCHAR2(4000) NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) -) -PCTFREE 10 PCTUSED 80; + guideURL VARCHAR2(4000), + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; -- Module Dependency refset table. EXECUTE drop_table('moduledependency'); -CREATE TABLE moduledependency ( - id CHAR(52) NOT NULL PRIMARY KEY, - effectiveTime DATE NOT NULL, - active NUMERIC(1) NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - sourceEffectiveTime DATE NOT NULL, - targetEffectiveTime DATE NOT NULL, - FOREIGN KEY (moduleId) REFERENCES concept(id), - FOREIGN KEY (refsetId) REFERENCES concept(id) +CREATE TABLE moduledependency +( + id CHAR(52) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active NUMERIC(1) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + sourceEffectiveTime DATE NOT NULL, + targetEffectiveTime DATE NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept (id), + FOREIGN KEY (refsetId) REFERENCES concept (id) +) PCTFREE 10 PCTUSED 80; + +-- Transitive closure table +EXECUTE drop_table('transitiveclosure'); +CREATE TABLE transitiveclosure ( + superTypeId NUMERIC(20) NOT NULL, + subTypeId NUMERIC(20) NOT NULL, + depth INTEGER NOT NULL, + PRIMARY KEY (superTypeId, subTypeId), + FOREIGN KEY (superTypeId) REFERENCES concept(id), + FOREIGN KEY (subTypeId) REFERENCES concept(id) ) PCTFREE 10 PCTUSED 80; - -- Clean up helper procedures. DROP PROCEDURE drop_table; diff --git a/src/main/resources/rf2/oracle_views.sql b/src/main/resources/rf2/oracle_views.sql index cab450e..69626dc 100644 --- a/src/main/resources/rf2/oracle_views.sql +++ b/src/main/resources/rf2/oracle_views.sql @@ -3,31 +3,32 @@ SET ECHO ON -- Session settings. -ALTER SESSION SET NLS_LENGTH_SEMANTICS='CHAR'; +ALTER +SESSION SET NLS_LENGTH_SEMANTICS='CHAR'; -- Define helper procedure to drop views. BEGIN - EXECUTE IMMEDIATE 'DROP PROCEDURE drop_view'; +EXECUTE IMMEDIATE 'DROP PROCEDURE drop_view'; EXCEPTION WHEN OTHERS THEN -- If procedure doesn't exist, it can't be dropped, -- so an exception will be thrown. Ignore it. IF SQLCODE != -4043 THEN RAISE; - END IF; +END IF; END; / -CREATE PROCEDURE drop_view (view_name IN VARCHAR2) IS +CREATE PROCEDURE drop_view(view_name IN VARCHAR2) IS BEGIN - EXECUTE IMMEDIATE 'DROP VIEW ' || view_name || ' CASCADE CONSTRAINTS'; +EXECUTE IMMEDIATE 'DROP VIEW ' || view_name || ' CASCADE CONSTRAINTS'; EXCEPTION -- If view doesn't exist, it can't be dropped, -- so an exception will be thrown. Ignore it. WHEN OTHERS THEN IF SQLCODE != -942 THEN RAISE; - END IF; +END IF; END; / @@ -40,190 +41,280 @@ EXECUTE drop_view('conceptpreferredname'); CREATE VIEW conceptpreferredname AS SELECT c.id conceptId, NVL(d.term, 'no active preferred synonym') preferredName, d.id descriptionId FROM concept c -LEFT OUTER JOIN description d -ON c.id = d.conceptId -AND d.active = 1 -AND d.typeId = 900000000000013009 -LEFT OUTER JOIN language l -ON d.id = l.referencedComponentId -AND l.active = 1 -AND l.acceptabilityId = 900000000000548007 -AND l.refSetId = 900000000000509007 + LEFT OUTER JOIN description d + ON c.id = d.conceptId + AND d.active = 1 + AND d.typeId = 900000000000013009 + LEFT OUTER JOIN language l + ON d.id = l.referencedComponentId + AND l.active = 1 + AND l.acceptabilityId = 900000000000548007 + AND l.refSetId = 900000000000509007 WHERE (l.acceptabilityId IS NOT null OR d.TERM IS null); -- Concept table with names. EXECUTE drop_view('conceptwithnames'); CREATE VIEW conceptwithnames AS -SELECT id, cpn1.preferredName idName, effectiveTime, active, - moduleId, cpn2.preferredName moduleIdName, - definitionStatusId, cpn3.preferredName definitionStatusIdName +SELECT id, + cpn1.preferredName idName, + effectiveTime, + active, + moduleId, + cpn2.preferredName moduleIdName, + definitionStatusId, + cpn3.preferredName definitionStatusIdName FROM concept, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE id = cpn1.conceptId -AND moduleId = cpn2.conceptId -AND definitionStatusId = cpn3.conceptId; + AND moduleId = cpn2.conceptId + AND definitionStatusId = cpn3.conceptId; -- Description table. EXECUTE drop_view('descriptionwithnames'); CREATE VIEW descriptionwithnames AS -SELECT id, term, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - d.conceptId, cpn2.preferredName conceptIdName, - languageCode, - typeId, cpn3.preferredName typeIdName, - caseSignificanceId, cpn4.preferredName caseSignificanceIdName +SELECT id, + term, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + d.conceptId, + cpn2.preferredName conceptIdName, + languageCode, + typeId, + cpn3.preferredName typeIdName, + caseSignificanceId, + cpn4.preferredName caseSignificanceIdName FROM description d, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 -WHERE moduleId = cpn1.conceptId AND d.conceptId = cpn2.conceptId AND typeId = cpn3.conceptId AND caseSignificanceId = cpn4.conceptId; + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 +WHERE moduleId = cpn1.conceptId + AND d.conceptId = cpn2.conceptId + AND typeId = cpn3.conceptId + AND caseSignificanceId = cpn4.conceptId; -- Identifier table. EXECUTE drop_view('identifierwithnames'); CREATE VIEW identifierwithnames AS -SELECT identifierSchemeId, cpn1.preferredName identifierSchemeIdName, - alternateIdentifier, effectiveTime, active, - moduleId, cpn2.preferredName moduleIdName, - referencedComponentId +SELECT identifierSchemeId, + cpn1.preferredName identifierSchemeIdName, + alternateIdentifier, + effectiveTime, + active, + moduleId, + cpn2.preferredName moduleIdName, + referencedComponentId FROM identifier, - conceptpreferredname cpn1, - conceptpreferredname cpn2 + conceptpreferredname cpn1, + conceptpreferredname cpn2 WHERE identifierSchemeId = cpn1.conceptId -AND moduleId = cpn2.conceptId; + AND moduleId = cpn2.conceptId; -- Relationship table. EXECUTE drop_view('relationshipwithnames'); CREATE VIEW relationshipwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - sourceId, cpn2.preferredName sourceIdName, - destinationId, cpn3.preferredName destinationIdName, - relationshipGroup, - typeId, cpn4.preferredName typeIdName, - characteristicTypeId, cpn5.preferredName characteristicTypeIdName, - modifierId, cpn6.preferredName modifierIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + sourceId, + cpn2.preferredName sourceIdName, + destinationId, + cpn3.preferredName destinationIdName, + relationshipGroup, + typeId, + cpn4.preferredName typeIdName, + characteristicTypeId, + cpn5.preferredName characteristicTypeIdName, + modifierId, + cpn6.preferredName modifierIdName from relationship, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5, - conceptpreferredname cpn6 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 WHERE moduleId = cpn1.conceptId -AND sourceId = cpn2.conceptId -AND destinationId = cpn3.conceptId -AND typeId = cpn4.conceptId -AND characteristicTypeId = cpn5.conceptId -AND modifierId = cpn6.conceptId; - + AND sourceId = cpn2.conceptId + AND destinationId = cpn3.conceptId + AND typeId = cpn4.conceptId + AND characteristicTypeId = cpn5.conceptId + AND modifierId = cpn6.conceptId; + +xxx-- Relationship concrete values table. +EXECUTE drop_view('relationshipconcretevalueswithnames'); +CREATE VIEW relationshipconcretevalueswithnames AS +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + sourceId, + cpn2.preferredName sourceIdName, + value, + relationshipGroup, + typeId, + cpn4.preferredName typeIdName, + characteristicTypeId, + cpn5.preferredName characteristicTypeIdName, + modifierId, + cpn6.preferredName modifierIdName +from relationshipconcretevalues, + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 +WHERE moduleId = cpn1.conceptId + AND sourceId = cpn2.conceptId + AND typeId = cpn4.conceptId + AND characteristicTypeId = cpn5.conceptId + AND modifierId = cpn6.conceptId; -- Stated Relationship table. EXECUTE drop_view('statedrelationshipwithnames'); CREATE VIEW statedrelationshipwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - sourceId, cpn2.preferredName sourceIdName, - destinationId, cpn3.preferredName destinationIdName, - relationshipGroup, - typeId, cpn4.preferredName typeIdName, - characteristicTypeId, cpn5.preferredName characteristicTypeIdName, - modifierId, cpn6.preferredName modifierIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + sourceId, + cpn2.preferredName sourceIdName, + destinationId, + cpn3.preferredName destinationIdName, + relationshipGroup, + typeId, + cpn4.preferredName typeIdName, + characteristicTypeId, + cpn5.preferredName characteristicTypeIdName, + modifierId, + cpn6.preferredName modifierIdName from statedrelationship, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5, - conceptpreferredname cpn6 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 WHERE moduleId = cpn1.conceptId -AND sourceId = cpn2.conceptId -AND destinationId = cpn3.conceptId -AND typeId = cpn4.conceptId -AND characteristicTypeId = cpn5.conceptId -AND modifierId = cpn6.conceptId; + AND sourceId = cpn2.conceptId + AND destinationId = cpn3.conceptId + AND typeId = cpn4.conceptId + AND characteristicTypeId = cpn5.conceptId + AND modifierId = cpn6.conceptId; -- Text Definition table. EXECUTE drop_view('textdefinitionwithnames'); CREATE VIEW textdefinitionwithnames AS -SELECT id, term, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - td.conceptId, cpn2.preferredName conceptIdName, - languageCode, - typeId, cpn3.preferredName typeIdName, - caseSignificanceId, cpn4.preferredName caseSignificanceIdName +SELECT id, + term, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + td.conceptId, + cpn2.preferredName conceptIdName, + languageCode, + typeId, + cpn3.preferredName typeIdName, + caseSignificanceId, + cpn4.preferredName caseSignificanceIdName from textdefinition td, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND td.conceptId = cpn2.conceptId -AND typeId = cpn3.conceptId -AND caseSignificanceId = cpn4.conceptId; + AND td.conceptId = cpn2.conceptId + AND typeId = cpn3.conceptId + AND caseSignificanceId = cpn4.conceptId; -- Association refset table. EXECUTE drop_view('associationwithnames'); CREATE VIEW associationwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - targetComponent, cpn4.preferredName targetComponentName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + targetComponent, + cpn4.preferredName targetComponentName from association, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND targetComponent = cpn4.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND targetComponent = cpn4.conceptId; -- Attribute Value refset table. EXECUTE drop_view('attributevaluewithnames'); CREATE VIEW attributevaluewithnames AS -SELECT attributevalue.id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - valueId, cpn4.preferredName valueIdName +SELECT attributevalue.id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + valueId, + cpn4.preferredName valueIdName FROM attributevalue, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - (SELECT conceptId id, preferredName FROM conceptpreferredname - UNION ALL - SELECT d.id, term from description d) cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + (SELECT conceptId id, preferredName + FROM conceptpreferredname + UNION ALL + SELECT d.id, term + from description d) cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.id -AND valueId = cpn4.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.id + AND valueId = cpn4.conceptId; -- Simple refset table. EXECUTE drop_view('simplewithnames'); CREATE VIEW simplewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName FROM simple, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- No more complex map, ICD9CM maps deprecated @@ -254,116 +345,165 @@ AND referencedComponentId = cpn3.conceptId; -- Extended Map refset table. EXECUTE drop_view('extendedmapwithnames'); CREATE VIEW extendedmapwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - mapGroup, - mapPriority, - mapRule, - mapAdvice, - mapTarget, - correlationId, cpn4.preferredName correlationIdName, - mapCategoryId, cpn5.preferredName mapCategoryIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + mapGroup, + mapPriority, + mapRule, + mapAdvice, + mapTarget, + correlationId, + cpn4.preferredName correlationIdName, + mapCategoryId, + cpn5.preferredName mapCategoryIdName FROM extendedmap, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND correlationId = cpn4.conceptId -AND mapCategoryId = cpn5.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND correlationId = cpn4.conceptId + AND mapCategoryId = cpn5.conceptId; -- Simple Map refset table. EXECUTE drop_view('simplemapwithnames'); CREATE VIEW simplemapwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - mapTarget +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + mapTarget FROM simplemap, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Language refset table. EXECUTE drop_view('languagewithnames'); CREATE VIEW languagewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - acceptabilityId, cpn4.preferredName acceptabilityIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + acceptabilityId, + cpn4.preferredName acceptabilityIdName FROM language, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.descriptionId -AND acceptabilityId = cpn4.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.descriptionId + AND acceptabilityId = cpn4.conceptId; -- Refset Descriptor refset table. EXECUTE drop_view('refsetdescriptorwithnames'); CREATE VIEW refsetdescriptorwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - attributeDescription, attributeType, attributeOrder +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + attributeDescription, + attributeType, + attributeOrder FROM refsetdescriptor, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Description Type refset table. EXECUTE drop_view('descriptiontypewithnames'); CREATE VIEW descriptiontypewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - descriptionFormat, descriptionLength +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + descriptionFormat, + descriptionLength FROM descriptiontype, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Module Dependency refset table. EXECUTE drop_view('moduledependencywithnames'); CREATE VIEW moduledependencywithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - sourceEffectiveTime, targetEffectiveTime +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + sourceEffectiveTime, + targetEffectiveTime FROM moduledependency, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; - + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; + +-- transitive closure table +-- NOTE: this assumes conceptpreferredname has already been created +-- via the standard RF2 load scripts +EXECUTE drop_view('transitiveclosurewithnames'); +CREATE VIEW transitiveclosurewithnames AS +SELECT superTypeId, cpn1.preferredName superTypeName, + subTypeId, cpn2.preferredName subTypeName, a.depth +FROM transitiveclosure a, + conceptpreferredname cpn1, + conceptpreferredname cpn2 +WHERE a.superTypeId = cpn1.conceptId + AND a.subTypeId = cpn2.conceptId; -- Clean up helper procedures. DROP PROCEDURE drop_view; diff --git a/src/main/resources/rf2/owlexpression.ctl b/src/main/resources/rf2/owlexpression.ctl index ab3baa3..0ef9ae1 100644 --- a/src/main/resources/rf2/owlexpression.ctl +++ b/src/main/resources/rf2/owlexpression.ctl @@ -1,7 +1,7 @@ options (skip=1,direct=true) load data characterset UTF8 length semantics char -infile 'Snapshot/Terminology/sct2_sRefset_OWLExpression_Snapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +infile 'Snapshot/Terminology/sct2_sRefset_OWLExpressionSnapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" badfile 'owlexpression.bad' discardfile 'owlexpression.dsc' insert diff --git a/src/main/resources/rf2/populate_maria_db.sh b/src/main/resources/rf2/populate_maria_db.sh new file mode 100644 index 0000000..d3e7763 --- /dev/null +++ b/src/main/resources/rf2/populate_maria_db.sh @@ -0,0 +1,79 @@ +#!/bin/bash -f +# +# Database connection parameters +# PLEASE EDIT THESE VARIABLES TO REFLECT YOUR ENVIRONMENT +host=[UPDATE] +user=root +password=admin +db_name=snomed + +LOG_FILE=maria_db.log + +/bin/rm -f $LOG_FILE +touch $LOG_FILE +ef=0 + +echo "See $LOG_FILE for detailed output" + +echo "----------------------------------------" | tee -a $LOG_FILE +echo "Starting ... $(/bin/date)" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE +echo "user = $user" | tee -a $LOG_FILE +echo "db_name = $db_name" | tee -a $LOG_FILE +echo "host = $host" | tee -a $LOG_FILE + +if [ "${password}" != "" ]; then + password="-p${password}" +fi +if [ "${host}" != "" ]; then + host="-h ${host}" +fi + +DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) + +echo " Compute transitive closure relationship file ... $(/bin/date)" | tee -a $LOG_FILE +relFile=$(find $DIR/Snapshot/Terminology/ -name "*_Relationship_Snapshot_*.txt" -print -quit) +#check if system has python or perl, run the corresponding script +if command -v perl &>/dev/null; then + echo "perl found, running perl script" >>$LOG_FILE 2>&1 + perl $DIR/compute_transitive_closure.pl --force --noself $relFile >>$LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +elif command -v python &>/dev/null; then + echo "python found, running python script" >>$LOG_FILE 2>&1 + python $DIR/compute_transitive_closure.py --force --noself $relFile >>$LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +# if none are present, print error message +else + echo "No perl or python found on your machine. Please install one of them to proceed." | tee -a $LOG_FILE + ef=1 +fi + +if [ $ef -ne 1 ]; then + echo " Create and load tables ... $(/bin/date)" | tee -a $LOG_FILE + mariadb -vvv $host -u $user $password --local-infile $db_name >$LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +fi + +if [ $ef -ne 1 ]; then + echo " Create indexes ... $(/bin/date)" | tee -a $LOG_FILE + mariadb -vvv $host -u $user $password --local-infile $db_name >$LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +fi + +if [ $ef -ne 1 ]; then + echo " Create views ... $(/bin/date)" | tee -a $LOG_FILE + mariadb -vvv $host -u $user $password --local-infile $db_name >$LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +fi + +echo "----------------------------------------" | tee -a $LOG_FILE +if [ $ef -eq 1 ]; then + echo "There were one or more errors." | tee -a $LOG_FILE + retval=-1 +else + echo "Completed without errors." | tee - a $LOG_FILE + retval=0 +fi +echo "Finished ... $(/bin/date)" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE +exit $retval diff --git a/src/main/resources/rf2/populate_mysql_db.bat b/src/main/resources/rf2/populate_mysql_db.bat deleted file mode 100644 index 5c256a8..0000000 --- a/src/main/resources/rf2/populate_mysql_db.bat +++ /dev/null @@ -1,51 +0,0 @@ -:: -:: Database connection parameters -:: Please edit these variables to reflect your environment -:: -set mysql=C:/mysql/mysql-5.6.46-winx64/bin/mysql.exe -set user=root -set password=admin -set db_name=snomed - -del mysql.log -echo. > mysql.log -echo ---------------------------------------- >> mysql.log 2>&1 -echo Starting ... >> mysql.log 2>&1 -date /T >> mysql.log 2>&1 -time /T >> mysql.log 2>&1 -echo ---------------------------------------- >> mysql.log 2>&1 -echo MYSQL_HOME = %MYSQL_HOME% >> mysql.log 2>&1 -echo user = %user% >> mysql.log 2>&1 -echo db_name = %db_name% >> mysql.log 2>&1 -set error=0 - -echo Create and load tables >> mysql.log 2>&1 -"%mysql%" -vvv -u %user% -p%password% --local-infile=1 %db_name% < mysql_tables.sql >> mysql.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -echo Create indexes >> mysql.log 2>&1 -"%mysql%" -vvv -u %user% -p%password% --local-infile=1 %db_name% < mysql_indexes.sql >> mysql.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -echo Create views >> mysql.log 2>&1 -"%mysql%" -vvv -u %user% -p%password% --local-infile=1 %db_name% < mysql_views.sql >> mysql.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -:trailer -echo ---------------------------------------- >> mysql.log 2>&1 -IF %error% NEQ 0 ( -echo There were one or more errors. Please reference the mysql.log file for details. >> mysql.log 2>&1 -set retval=-1 -) else ( -echo Completed without errors. >> mysql.log 2>&1 -set retval=0 -) -echo Finished ... >> mysql.log 2>&1 -date /T >> mysql.log 2>&1 -time /T >> mysql.log 2>&1 -echo ---------------------------------------- >> mysql.log 2>&1 -pause -exit %retval% diff --git a/src/main/resources/rf2/populate_mysql_db.sh b/src/main/resources/rf2/populate_mysql_db.sh index 58617b0..e228008 100644 --- a/src/main/resources/rf2/populate_mysql_db.sh +++ b/src/main/resources/rf2/populate_mysql_db.sh @@ -1,27 +1,26 @@ #!/bin/bash -f # # Database connection parameters -# Please edit these variables to reflect your environment -# - Tested with docker mysql 5.6, 5.7 (using docker mysql server and client) -# - host=host.docker.internal -# -host= +# PLEASE EDIT THESE VARIABLES TO REFLECT YOUR ENVIRONMENT +host=[UPDATE] user=root password=admin db_name=snomed -/bin/rm -f mysql.log -touch mysql.log +LOG_FILE=mysql.log + +/bin/rm -f $LOG_FILE +touch $LOG_FILE ef=0 -echo "See mysql.log for output" +echo "See $LOG_FILE for detailed output" -echo "----------------------------------------" >> mysql.log 2>&1 -echo "Starting ... `/bin/date`" >> mysql.log 2>&1 -echo "----------------------------------------" >> mysql.log 2>&1 -echo "user = $user" >> mysql.log 2>&1 -echo "db_name = $db_name" >> mysql.log 2>&1 -echo "host = $host" >> mysql.log 2>&1 +echo "----------------------------------------" | tee -a $LOG_FILE +echo "Starting ... `/bin/date`" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE +echo "user = $user" | tee -a $LOG_FILE +echo "db_name = $db_name" | tee -a $LOG_FILE +echo "host = $host" | tee -a $LOG_FILE if [ "${password}" != "" ]; then password="-p${password}" @@ -30,31 +29,54 @@ if [ "${host}" != "" ]; then host="-h ${host}" fi -echo " Create and load tables ... `/bin/date`" >> mysql.log 2>&1 -mysql -vvv $host -u $user $password --local-infile $db_name < mysql_tables.sql >> mysql.log 2>&1 +DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +echo " Compute transitive closure relationship file ... `/bin/date`" | tee -a $LOG_FILE +relFile=$(find $DIR/Snapshot/Terminology/ -name "*_Relationship_Snapshot_*.txt" -print -quit) +#check if system has python or perl, run the corresponding script +if command -v perl &> /dev/null +then + echo "perl found, running perl script" >> $LOG_FILE 2>&1 + perl $DIR/compute_transitive_closure.pl --force --noself $relFile >> $LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +elif command -v python &> /dev/null +then + echo "python found, running python script" >> $LOG_FILE 2>&1 + python $DIR/compute_transitive_closure.py --force --noself $relFile >> $LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +# if none are present, print error message +else + echo "No perl or python found on your machine. Please install one of them to proceed." | tee -a $LOG_FILE + ef=1 +fi + +if [ $ef -ne 1 ]; then +echo " Create and load tables ... `/bin/date`" | tee -a $LOG_FILE +mysql -vvv $host -u $user $password --local-infile $db_name < mysql_tables.sql >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi +fi if [ $ef -ne 1 ]; then -echo " Create indexes ... `/bin/date`" >> mysql.log 2>&1 -mysql -vvv $host -u $user $password --local-infile $db_name < mysql_indexes.sql >> mysql.log 2>&1 +echo " Create indexes ... `/bin/date`" | tee -a $LOG_FILE +mysql -vvv $host -u $user $password --local-infile $db_name < mysql_indexes.sql >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi fi if [ $ef -ne 1 ]; then -echo " Create views ... `/bin/date`" >> mysql.log 2>&1 -mysql -vvv $host -u $user $password --local-infile $db_name < mysql_views.sql >> mysql.log 2>&1 +echo " Create views ... `/bin/date`" | tee -a $LOG_FILE +mysql -vvv $host -u $user $password --local-infile $db_name < mysql_views.sql >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi fi -echo "----------------------------------------" >> mysql.log 2>&1 +echo "----------------------------------------" | tee -a $LOG_FILE if [ $ef -eq 1 ] then - echo "There were one or more errors." >> mysql.log 2>&1 + echo "There were one or more errors." | tee -a $LOG_FILE retval=-1 else - echo "Completed without errors." >> mysql.log 2>&1 + echo "Completed without errors." | tee - a $LOG_FILE retval=0 fi -echo "Finished ... `/bin/date`" >> mysql.log 2>&1 -echo "----------------------------------------" >> mysql.log 2>&1 +echo "Finished ... `/bin/date`" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE exit $retval diff --git a/src/main/resources/rf2/populate_oracle_db.bat b/src/main/resources/rf2/populate_oracle_db.bat deleted file mode 100644 index b999065..0000000 --- a/src/main/resources/rf2/populate_oracle_db.bat +++ /dev/null @@ -1,158 +0,0 @@ -:: -:: Database connection parameters -:: Please edit these variables to reflect your environment -:: -set ORACLE_HOME=C:\app\oracle\product\12.1.0\dbhome_1 -set user=snomed -set password=snomed -set tns_name=global -set NLS_LANG=AMERICAN_AMERICA.UTF8 - -del oracle.log -echo. > oracle.log -echo ---------------------------------------- >> oracle.log 2>&1 -echo Starting ... >> oracle.log 2>&1 -date /T >> oracle.log 2>&1 -time /T >> oracle.log 2>&1 -echo ---------------------------------------- >> oracle.log 2>&1 -echo ORACLE_HOME = %ORACLE_HOME% >> oracle.log 2>&1 -echo user = %user% >> oracle.log 2>&1 -echo tns_name = %tns_name% >> oracle.log 2>&1 -set error=0 - -echo Create tables >> oracle.log 2>&1 -echo @oracle_tables.sql|%ORACLE_HOME%\bin\sqlplus %user%/%password%@%tns_name% >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -echo Load concept table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="concept.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type concept.log >> oracle.log - -echo Load description table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="description.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type description.log >> oracle.log - -echo Load identifier table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="identifier.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type identifier.log >> oracle.log - -echo Load relationship table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="relationship.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type relationship.log >> oracle.log - -echo Load owl expression table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="owlexpression.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type owlexpression.log >> oracle.log - -echo Load stated relationship table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="statedrelationship.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type statedrelationship.log >> oracle.log - -echo Load text definition table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="textdefinition.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type textdefinition.log >> oracle.log - -echo Load association table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="association.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type association.log >> oracle.log - -echo Load attribute value table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="attributevalue.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type attributevalue.log >> oracle.log - -echo Load simple refset table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="simple.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type simple.log >> oracle.log - -:: No more complex map, ICD9CM maps deprecated -:: echo Load complex map refset table data >> oracle.log 2>&1 -:: %ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="complexmap.ctl" >> oracle.log 2>&1 -:: IF %ERRORLEVEL% NEQ 0 (set error=1) -:: type complexmap.log >> oracle.log - -echo Load extended map table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="extendedmap.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type extendedmap.log >> oracle.log - -echo Load simple map table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="simplemap.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type simplemap.log >> oracle.log - -echo Load language table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="language.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type language.log >> oracle.log - -echo Load refset descriptor table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="refsetdescriptor.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type refsetdescriptor.log >> oracle.log - -echo Load description type table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="descriptiontype.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type descriptiontype.log >> oracle.log - -echo Load mrcm attribute domain table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="mrcmattributedomain.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type mrcmattributedomain.log >> oracle.log - -echo Load mrcm module scope table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="mrcmmodulescope.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type mrcmmodulescope.log >> oracle.log - -echo Load mrcm attribute range table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="mrcmattributerange.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type mrcmattributerange.log >> oracle.log - -echo Load mrcm domain table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="mrcmdomain.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type mrcmdomain.log >> oracle.log - -echo Load module dependency table data >> oracle.log 2>&1 -%ORACLE_HOME%\bin\sqlldr %user%/%password%@%tns_name% control="moduledependency.ctl" >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1) -type moduledependency.log >> oracle.log - -echo Create indexes >> oracle.log 2>&1 -echo @oracle_indexes.sql|%ORACLE_HOME%\bin\sqlplus %user%/%password%@%tns_name% >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -echo Create views >> oracle.log 2>&1 -echo @oracle_views.sql|%ORACLE_HOME%\bin\sqlplus %user%/%password%@%tns_name% >> oracle.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -:trailer -echo ---------------------------------------- >> oracle.log 2>&1 -IF %error% NEQ 0 ( -echo There were one or more errors. >> oracle.log 2>&1 -set retval=-1 -) else ( -echo Completed without errors. >> oracle.log 2>&1 -set retval=0 -) -echo Finished ... >> oracle.log 2>&1 -date /T >> oracle.log 2>&1 -time /T >> oracle.log 2>&1 -echo ---------------------------------------- >> oracle.log 2>&1 -pause -exit %retval% diff --git a/src/main/resources/rf2/populate_oracle_db.sh b/src/main/resources/rf2/populate_oracle_db.sh index bc5080f..c705577 100644 --- a/src/main/resources/rf2/populate_oracle_db.sh +++ b/src/main/resources/rf2/populate_oracle_db.sh @@ -2,174 +2,213 @@ # # Database connection parameters -# Please edit these variables to reflect your environment -# - Tested on Windows with "Git Bash" as a shell -# -ORACLE_HOME=/app/oracle/product/12.1.0/dbhome_1 -export ORACLE_HOME +# Set if necessary +#export ORACLE_HOME=/opt/oracle/product/19c/dbhome_1 +# PLEASE EDIT THESE VARIABLES TO REFLECT YOUR ENVIRONMENT user=snomed password=snomed -tns_name=global -NLS_LANG=AMERICAN_AMERICA.UTF8 -export NLS_LANG +tns_name=ORCLCDB +export NLS_LANG=AMERICAN_AMERICA.UTF8 + +LOG_FILE=oracle.log -/bin/rm -f oracle.log -touch oracle.log +# Clear the log file. If it doesn't exist, create it. +> $LOG_FILE ef=0 -echo "See oracle.log for output" -echo "----------------------------------------" >> oracle.log 2>&1 -echo "Starting ... `/bin/date`" >> oracle.log 2>&1 -echo "----------------------------------------" >> oracle.log 2>&1 -echo "ORACLE_HOME = $ORACLE_HOME" >> oracle.log 2>&1 -echo "user = $user" >> oracle.log 2>&1 -echo "tns_name = $tns_name" >> oracle.log 2>&1 +echo "See $LOG_FILE for detailed output" +echo "----------------------------------------" | tee -a $LOG_FILE +echo "Starting ... `/bin/date`" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE +echo "ORACLE_HOME = $ORACLE_HOME" | tee -a $LOG_FILE +echo "user = $user" | tee -a $LOG_FILE +echo "tns_name = $tns_name" | tee -a $LOG_FILE + + +DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +echo " Compute transitive closure relationship file ... `/bin/date`" | tee -a $LOG_FILE +relFile=$(find $DIR/Snapshot/Terminology/ -name "*_Relationship_Snapshot_*.txt" -print -quit) +#check if system has python or perl, run the corresponding script +if command -v perl &> /dev/null +then + echo "perl found, running perl script" >> $LOG_FILE 2>&1 + perl $DIR/compute_transitive_closure.pl --force --noself $relFile >> $LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +elif command -v python &> /dev/null +then + echo "python found, running python script" >> $LOG_FILE 2>&1 + python $DIR/compute_transitive_closure.py --force --noself $relFile >> $LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +# if none are present, print error message +else + echo "No perl or python found on your machine. Please install one of them to proceed." | tee -a $LOG_FILE + ef=1 +fi + +if [ $ef -ne 1 ]; then +echo " Create tables ... `/bin/date`" | tee -a $LOG_FILE +echo "@oracle_tables.sql" | $ORACLE_HOME/bin/sqlplus $user/$password@$tns_name >> $LOG_FILE 2>&1 +if [ $? -ne 0 ]; then ef=1; fi +fi + +## Load RF2 tables +if [ $ef -ne 1 ]; then +echo " Load concept table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="concept.ctl" >> $LOG_FILE 2>&1 +if [ $? -ne 0 ]; then ef=1; fi +cat concept.log >> $LOG_FILE +fi -echo " Create tables ... `/bin/date`" >> oracle.log 2>&1 -echo "@oracle_tables.sql"|$ORACLE_HOME/bin/sqlplus $user/$password@$tns_name >> oracle.log 2>&1 +if [ $ef -ne 1 ]; then +echo " Load description table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="description.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi - +cat description.log >> $LOG_FILE +fi + if [ $ef -ne 1 ]; then -echo " Load concept table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="concept.ctl" >> oracle.log 2>&1 +echo " Load identifier table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="identifier.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat concept.log >> oracle.log +cat identifier.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load description table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="description.ctl" >> oracle.log 2>&1 +echo " Load relationship table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="relationship.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat description.log >> oracle.log +cat relationship.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load identifier table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="identifier.ctl" >> oracle.log 2>&1 +echo " Load relationship concrete values table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="relationshipconcretevalues.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat identifier.log >> oracle.log +cat relationshipconcretevalues.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load relationship table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="relationship.ctl" >> oracle.log 2>&1 +echo " Load owl expression table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="owlexpression.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat relationship.log >> oracle.log +cat owlexpression.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load owl expression table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="owlexpression.ctl" >> oracle.log 2>&1 +echo " Load stated relationship table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="statedrelationship.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat owlexpression.log >> oracle.log +cat statedrelationship.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load stated relationship table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="statedrelationship.ctl" >> oracle.log 2>&1 +echo " Load text definition table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="textdefinition.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat statedrelationship.log >> oracle.log +cat textdefinition.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load text definition table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="textdefinition.ctl" >> oracle.log 2>&1 +echo " Load association table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="association.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat textdefinition.log >> oracle.log +cat association.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load association table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="association.ctl" >> oracle.log 2>&1 +echo " Load attribute value table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="attributevalue.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat association.log >> oracle.log +cat attributevalue.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load attribute value table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="attributevalue.ctl" >> oracle.log 2>&1 +echo " Load simple refset table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="simple.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat attributevalue.log >> oracle.log +cat simple.log >> $LOG_FILE fi +## Load TC table if [ $ef -ne 1 ]; then -echo " Load simple refset table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="simple.ctl" >> oracle.log 2>&1 +echo " Load transitive closure table ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="transitiveclosure.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat simple.log >> oracle.log +cat concept.log >> $LOG_FILE fi # No more complex maps # if [ $ef -ne 1 ]; then -# echo " Load complex map table data ... `/bin/date`" >> oracle.log 2>&1 -# $ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="complexmap.ctl" >> oracle.log 2>&1 +# echo " Load complex map table data ... `/bin/date`" >> $LOG_FILE 2>&1 +# $ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="complexmap.ctl" >> $LOG_FILE 2>&1 # if [ $? -ne 0 ]; then ef=1; fi -# cat complexmap.log >> oracle.log +# cat complexmap.log >> $LOG_FILE # fi if [ $ef -ne 1 ]; then -echo " Load extended map table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="extendedmap.ctl" >> oracle.log 2>&1 +echo " Load extended map table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="extendedmap.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat extendedmap.log >> oracle.log +cat extendedmap.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load simple map table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="simplemap.ctl" >> oracle.log 2>&1 +echo " Load simple map table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="simplemap.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat simplemap.log >> oracle.log +cat simplemap.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load language table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="language.ctl" >> oracle.log 2>&1 +echo " Load language table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="language.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat language.log >> oracle.log +cat language.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load refset descriptor table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="refsetdescriptor.ctl" >> oracle.log 2>&1 +echo " Load refset descriptor table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="refsetdescriptor.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat refsetdescriptor.log >> oracle.log +cat refsetdescriptor.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load description type table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="descriptiontype.ctl" >> oracle.log 2>&1 +echo " Load description type table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="descriptiontype.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat descriptiontype.log >> oracle.log +cat descriptiontype.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Load module dependency table data ... `/bin/date`" >> oracle.log 2>&1 -$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="moduledependency.ctl" >> oracle.log 2>&1 +echo " Load module dependency table data ... `/bin/date`" | tee -a $LOG_FILE +$ORACLE_HOME/bin/sqlldr $user/$password@$tns_name control="moduledependency.ctl" >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi -cat moduledependency.log >> oracle.log +cat moduledependency.log >> $LOG_FILE fi if [ $ef -ne 1 ]; then -echo " Create indexes ... `/bin/date`" >> oracle.log 2>&1 -echo "@oracle_indexes.sql"|$ORACLE_HOME/bin/sqlplus $user/$password@$tns_name >> oracle.log 2>&1 +echo " Create indexes ... `/bin/date`" | tee -a $LOG_FILE +echo "@oracle_indexes.sql"|$ORACLE_HOME/bin/sqlplus $user/$password@$tns_name >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi fi if [ $ef -ne 1 ]; then -echo " Create views ... `/bin/date`" >> oracle.log 2>&1 -echo "@oracle_views.sql"|$ORACLE_HOME/bin/sqlplus $user/$password@$tns_name >> oracle.log 2>&1 +echo " Create views ... `/bin/date`" | tee -a $LOG_FILE +echo "@oracle_views.sql"|$ORACLE_HOME/bin/sqlplus $user/$password@$tns_name >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi fi -echo "----------------------------------------" >> oracle.log 2>&1 +echo "----------------------------------------" | tee -a $LOG_FILE if [ $ef -eq 1 ] then - echo "There were one or more errors." >> oracle.log 2>&1 + echo "There were one or more errors." | tee -a $LOG_FILE retval=-1 else - echo "Completed without errors." >> oracle.log 2>&1 + echo "Completed without errors." | tee -a $LOG_FILE retval=0 fi -echo "Finished ... `/bin/date`" >> oracle.log 2>&1 -echo "----------------------------------------" >> oracle.log 2>&1 +echo "Finished ... `/bin/date`" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE exit $retval diff --git a/src/main/resources/rf2/populate_postgres_db.bat b/src/main/resources/rf2/populate_postgres_db.bat deleted file mode 100644 index ee52500..0000000 --- a/src/main/resources/rf2/populate_postgres_db.bat +++ /dev/null @@ -1,53 +0,0 @@ -:: -:: Database connection parameters -:: Please edit these variables to reflect your environment -:: -set psql=C:/PostgreSQL/pg96/bin/psql.exe -set PGUSER=postgres -set PGPASSWORD= -set PGDATABASE=snomed - -del postgres.log -echo. > postgres.log -echo ---------------------------------------- >> postgres.log 2>&1 -echo Starting ... >> postgres.log 2>&1 -date /T >> postgres.log 2>&1 -time /T >> postgres.log 2>&1 -echo ---------------------------------------- >> postgres.log 2>&1 -echo psql = %psql% >> postgres.log 2>&1 -echo user = %PGUSER% >> postgres.log 2>&1 -echo db_name = %PGDATABASE% >> postgres.log 2>&1 -set error=0 - -echo Create and load tables >> postgres.log 2>&1 -"%psql%" < psql_tables.sql >> postgres.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -echo Create indexes >> postgres.log 2>&1 -"%psql%" < psql_indexes.sql >> postgres.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -echo Create views >> postgres.log 2>&1 -"%psql%" < psql_views.sql >> postgres.log 2>&1 -IF %ERRORLEVEL% NEQ 0 (set error=1 -goto trailer) - -:trailer -echo ---------------------------------------- >> postgres.log 2>&1 -IF %error% NEQ 0 ( -echo There were one or more errors. Please reference the postgres.log file for details. >> postgres.log 2>&1 -set retval=-1 -) else ( -echo Completed without errors. >> postgres.log 2>&1 -set retval=0 -) -echo Finished ... >> postgres.log 2>&1 -date /T >> postgres.log 2>&1 -time /T >> postgres.log 2>&1 -echo ---------------------------------------- >> postgres.log 2>&1 - -pause -exit %retval% - diff --git a/src/main/resources/rf2/populate_postgres_db.sh b/src/main/resources/rf2/populate_postgres_db.sh index 5549270..0066689 100644 --- a/src/main/resources/rf2/populate_postgres_db.sh +++ b/src/main/resources/rf2/populate_postgres_db.sh @@ -2,56 +2,74 @@ # # Database connection parameters -# Please edit these variables to reflect your environment -# - Tested on Windows with "Git Bash" as a shell -# - and psql in the path, use the alias if needed -# -#alias psql="C:/PostgreSQL/pg96/bin/psql.exe" +# PLEASE EDIT THESE VARIABLES TO REFLECT YOUR ENVIRONMENT +export PGHOST=[UPDATE] export PGUSER=postgres export PGPASSWORD= export PGDATABASE=snomed -/bin/rm -f postgres.log -touch postgres.log +LOG_FILE=postgres.log + +/bin/rm -f $LOG_FILE +touch $LOG_FILE ef=0 -echo "See postgres.log for output" +echo "See $LOG_FILE for output" -echo "----------------------------------------" >> postgres.log 2>&1 -echo "Starting ... `/bin/date`" >> postgres.log 2>&1 -echo "----------------------------------------" >> postgres.log 2>&1 -echo "user = $PGUSER" >> postgres.log 2>&1 -echo "db_name = $PGDATABASE" >> postgres.log 2>&1 +echo "----------------------------------------" | tee -a $LOG_FILE +echo "Starting ... `/bin/date`" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE +echo "user = $PGUSER" | tee -a $LOG_FILE +echo "db_name = $PGDATABASE" | tee -a $LOG_FILE -if [ "${password}" != "" ]; then - password="-p${password}" +DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +echo " Compute transitive closure relationship file ... `/bin/date`" | tee -a $LOG_FILE +relFile=$(find $DIR/Snapshot/Terminology/ -name "*_Relationship_Snapshot_*.txt" -print -quit) +#check if system has perl first, then python, run the corresponding script +if command -v perl &> /dev/null +then + echo "perl found, running perl script" >> $LOG_FILE 2>&1 + perl $DIR/compute_transitive_closure.pl --force --noself $relFile >> $LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +elif command -v python &> /dev/null +then + echo "python found, running python script" >> $LOG_FILE 2>&1 + python $DIR/compute_transitive_closure.py --force --noself $relFile >> $LOG_FILE 2>&1 + if [ $? -ne 0 ]; then ef=1; fi +# if none are present, print error message +else + echo "No perl or python found on your machine. Please install one of them to proceed." | tee -a $LOG_FILE + ef=1 fi -echo " Create and load tables ... `/bin/date`" >> postgres.log 2>&1 -psql < psql_tables.sql >> postgres.log 2>&1 +if [ $ef -ne 1 ]; then +echo " Create and load tables ... `/bin/date`" | tee -a $LOG_FILE +psql < psql_tables.sql >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi +fi if [ $ef -ne 1 ]; then -echo " Create indexes ... `/bin/date`" >> postgres.log 2>&1 -psql < psql_indexes.sql >> postgres.log 2>&1 +echo " Create indexes ... `/bin/date`" | tee -a $LOG_FILE +psql < psql_indexes.sql >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi fi if [ $ef -ne 1 ]; then -echo " Create views ... `/bin/date`" >> postgres.log 2>&1 -psql < psql_views.sql >> postgres.log 2>&1 +echo " Create views ... `/bin/date`" | tee -a $LOG_FILE +psql < psql_views.sql >> $LOG_FILE 2>&1 if [ $? -ne 0 ]; then ef=1; fi fi -echo "----------------------------------------" >> postgres.log 2>&1 +echo "----------------------------------------" | tee -a $LOG_FILE if [ $ef -eq 1 ] then - echo "There were one or more errors." >> postgres.log 2>&1 + echo "There were one or more errors." | tee -a $LOG_FILE retval=-1 else - echo "Completed without errors." >> postgres.log 2>&1 + echo "Completed without errors." | tee -a $LOG_FILE retval=0 fi -echo "Finished ... `/bin/date`" >> postgres.log 2>&1 -echo "----------------------------------------" >> postgres.log 2>&1 +echo "Finished ... `/bin/date`" | tee -a $LOG_FILE +echo "----------------------------------------" | tee -a $LOG_FILE exit $retval diff --git a/src/main/resources/rf2/psql_tables.sql b/src/main/resources/rf2/psql_tables.sql index d4eff6f..643671f 100644 --- a/src/main/resources/rf2/psql_tables.sql +++ b/src/main/resources/rf2/psql_tables.sql @@ -1,13 +1,12 @@ -\ -- Concept file. DROP TABLE IF EXISTS concept CASCADE; CREATE TABLE concept ( - id NUMERIC(18) NOT NULL PRIMARY KEY, + id NUMERIC(20) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - definitionStatusId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + definitionStatusId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (definitionStatusId) REFERENCES concept(id) ); @@ -17,15 +16,15 @@ CREATE TABLE concept ( -- Description file. DROP TABLE IF EXISTS description CASCADE; CREATE TABLE description ( - id NUMERIC(18) NOT NULL PRIMARY KEY, + id NUMERIC(20) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - conceptId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + conceptId NUMERIC(20) NOT NULL, languageCode CHAR(2) NOT NULL, - typeId NUMERIC(18) NOT NULL, - term VARCHAR(255) NOT NULL, - caseSignificanceId NUMERIC(18) NOT NULL, + typeId NUMERIC(20) NOT NULL, + term VARCHAR(4096) NOT NULL, + caseSignificanceId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (conceptId) REFERENCES concept(id), FOREIGN KEY (typeId) REFERENCES concept(id), @@ -37,12 +36,12 @@ CREATE TABLE description ( -- Identifier file. DROP TABLE IF EXISTS identifier CASCADE; CREATE TABLE identifier ( - identifierSchemeId NUMERIC(18) NOT NULL, - alternateIdentifier VARCHAR(255) NOT NULL, + identifierSchemeId NUMERIC(20) NOT NULL, + alternateIdentifier VARCHAR(4096) NOT NULL, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id) ); @@ -52,16 +51,16 @@ CREATE TABLE identifier ( -- Relationship file. DROP TABLE IF EXISTS relationship CASCADE; CREATE TABLE relationship ( - id NUMERIC(18) NOT NULL PRIMARY KEY, + id NUMERIC(20) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - sourceId NUMERIC(18) NOT NULL, - destinationId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + destinationId NUMERIC(20) NOT NULL, relationshipGroup INT NOT NULL, - typeId NUMERIC(18) NOT NULL, - characteristicTypeId NUMERIC(18) NOT NULL, - modifierId NUMERIC(18) NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (sourceId) REFERENCES concept(id), FOREIGN KEY (destinationId) REFERENCES concept(id), @@ -72,6 +71,28 @@ CREATE TABLE relationship ( \copy relationship FROM 'Snapshot/Terminology/sct2_Relationship_Snapshot_${editionLabel}_${editionVersion}.txt' WITH DELIMITER E'\t' QUOTE E'\\' ENCODING 'UTF8' CSV HEADER; +-- Relationship concrete values file. +DROP TABLE IF EXISTS relationshipconcretevalues CASCADE; +CREATE TABLE relationshipconcretevalues ( + id NUMERIC(20) NOT NULL PRIMARY KEY, + effectiveTime DATE NOT NULL, + active BOOLEAN NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + value VARCHAR(256) NOT NULL, + relationshipGroup INT NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, + FOREIGN KEY (moduleId) REFERENCES concept(id), + FOREIGN KEY (sourceId) REFERENCES concept(id), + FOREIGN KEY (typeId) REFERENCES concept(id), + FOREIGN KEY (characteristicTypeId) REFERENCES concept(id), + FOREIGN KEY (modifierId) REFERENCES concept(id) +); + +\copy relationshipconcretevalues FROM 'Snapshot/Terminology/sct2_RelationshipConcreteValues_Snapshot_${editionLabel}_${editionVersion}.txt' WITH DELIMITER E'\t' QUOTE E'\\' ENCODING 'UTF8' CSV HEADER; + -- OWL Expression file. DROP TABLE IF EXISTS owlexpression CASCADE; @@ -79,9 +100,9 @@ CREATE TABLE owlexpression ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, owlExpression VARCHAR(10000) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), @@ -94,16 +115,16 @@ CREATE TABLE owlexpression ( -- Stated Relationship file. DROP TABLE IF EXISTS statedrelationship CASCADE; CREATE TABLE statedrelationship ( - id NUMERIC(18) NOT NULL PRIMARY KEY, + id NUMERIC(20) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - sourceId NUMERIC(18) NOT NULL, - destinationId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + sourceId NUMERIC(20) NOT NULL, + destinationId NUMERIC(20) NOT NULL, relationshipGroup INT NOT NULL, - typeId NUMERIC(18) NOT NULL, - characteristicTypeId NUMERIC(18) NOT NULL, - modifierId NUMERIC(18) NOT NULL, + typeId NUMERIC(20) NOT NULL, + characteristicTypeId NUMERIC(20) NOT NULL, + modifierId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (sourceId) REFERENCES concept(id), FOREIGN KEY (destinationId) REFERENCES concept(id), @@ -118,19 +139,19 @@ CREATE TABLE statedrelationship ( -- Text Definition file. DROP TABLE IF EXISTS textdefinition CASCADE; CREATE TABLE textdefinition ( - id NUMERIC(18) NOT NULL PRIMARY KEY, + id NUMERIC(20) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - conceptId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + conceptId NUMERIC(20) NOT NULL, languageCode CHAR(2) NOT NULL, - typeId NUMERIC(18) NOT NULL, + typeId NUMERIC(20) NOT NULL, term VARCHAR(2048) NOT NULL, - caseSignificanceId NUMERIC(18) NOT NULL, + caseSignificanceId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (conceptId) REFERENCES concept(id), FOREIGN KEY (typeId) REFERENCES concept(id), - FOREIGN KEY (caseSignificanceId) REFERENCES concept(id) + FOREIGN KEY (caseSignificanceId) REFERENCES concept(id) ); \copy textdefinition FROM 'Snapshot/Terminology/sct2_TextDefinition_Snapshot-en_${editionLabel}_${editionVersion}.txt' WITH DELIMITER E'\t' QUOTE E'\\' ENCODING 'UTF8' CSV HEADER; @@ -142,10 +163,10 @@ CREATE TABLE association ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - targetComponent NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + targetComponent NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (targetComponent) REFERENCES concept(id) @@ -160,10 +181,10 @@ CREATE TABLE attributevalue ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - valueId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + valueId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (valueId) REFERENCES concept(id) @@ -178,9 +199,9 @@ CREATE TABLE simple ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id) ); @@ -195,15 +216,15 @@ CREATE TABLE simple ( -- id CHAR(52) NOT NULL PRIMARY KEY, -- effectiveTime DATE NOT NULL, -- active BOOLEAN NOT NULL, --- moduleId NUMERIC(18) NOT NULL, --- refsetId NUMERIC(18) NOT NULL, --- referencedComponentId NUMERIC(18) NOT NULL, +-- moduleId NUMERIC(20) NOT NULL, +-- refsetId NUMERIC(20) NOT NULL, +-- referencedComponentId NUMERIC(20) NOT NULL, -- mapGroup INT NOT NULL, -- mapPriority INT NOT NULL, -- mapRule VARCHAR(4000) NOT NULL, -- mapAdvice VARCHAR(4000) NOT NULL, -- mapTarget VARCHAR(100) NOT NULL, --- correlationId NUMERIC(18) NOT NULL, +-- correlationId NUMERIC(20) NOT NULL, -- FOREIGN KEY (moduleId) REFERENCES concept(id), -- FOREIGN KEY (refsetId) REFERENCES concept(id), -- FOREIGN KEY (correlationId) REFERENCES concept(id) @@ -231,16 +252,16 @@ CREATE TABLE extendedmap ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, mapGroup INT NOT NULL, mapPriority INT NOT NULL, mapRule VARCHAR(4000) NOT NULL, mapAdvice VARCHAR(4000) NOT NULL, mapTarget VARCHAR(100), - correlationId NUMERIC(18) NOT NULL, - mapCategoryId NUMERIC(18) NOT NULL, + correlationId NUMERIC(20) NOT NULL, + mapCategoryId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (correlationId) REFERENCES concept(id), @@ -256,9 +277,9 @@ CREATE TABLE simplemap ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, mapTarget VARCHAR(100) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id) @@ -273,10 +294,10 @@ CREATE TABLE language ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - acceptabilityId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + acceptabilityId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (acceptabilityId) REFERENCES concept(id) @@ -291,12 +312,12 @@ CREATE TABLE refsetdescriptor ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - attributeDescription NUMERIC(18) NOT NULL, - attributeType NUMERIC(18) NOT NULL, - attributeOrder NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + attributeDescription NUMERIC(20) NOT NULL, + attributeType NUMERIC(20) NOT NULL, + attributeOrder NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id) ); @@ -310,10 +331,10 @@ CREATE TABLE descriptiontype ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - descriptionFormat NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + descriptionFormat NUMERIC(20) NOT NULL, descriptionLength INT NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id) @@ -328,15 +349,15 @@ CREATE TABLE mrcmattributedomain ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - domainId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + domainId NUMERIC(20) NOT NULL, grouped INT NOT NULL, attributeCardinality VARCHAR(20) NOT NULL, attributeInGroupCardinality VARCHAR(20) NOT NULL, - ruleStrengthId NUMERIC(18) NOT NULL, - contentTypeId NUMERIC(18) NOT NULL, + ruleStrengthId NUMERIC(20) NOT NULL, + contentTypeId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (domainId) REFERENCES concept(id), @@ -353,10 +374,10 @@ CREATE TABLE mrcmmodulescope ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, - mrcmRuleRefsetId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, + mrcmRuleRefsetId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (mrcmRuleRefsetId) REFERENCES concept(id) @@ -371,13 +392,13 @@ CREATE TABLE mrcmattributerange ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, rangeConstraint VARCHAR(4000) NOT NULL, attributeRule VARCHAR(4000) NOT NULL, - ruleStrengthId NUMERIC(18) NOT NULL, - contentTypeId NUMERIC(18) NOT NULL, + ruleStrengthId NUMERIC(20) NOT NULL, + contentTypeId NUMERIC(20) NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id), FOREIGN KEY (ruleStrengthId) REFERENCES concept(id), @@ -393,16 +414,16 @@ CREATE TABLE mrcmdomain ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, domainConstraint VARCHAR(4000) NOT NULL, parentDomain VARCHAR(4000), proximalPrimitiveConstraint VARCHAR(4000) NOT NULL, proximalPrimitiveRefinement VARCHAR(4000), domainTemplateForPrecoordination VARCHAR(40000) NOT NULL, domainTemplateForPostcoordination VARCHAR(40000) NOT NULL, - guideURL VARCHAR(4000) NOT NULL, + guideURL VARCHAR(4000), FOREIGN KEY (moduleId) REFERENCES concept(id), FOREIGN KEY (refsetId) REFERENCES concept(id) ); @@ -416,9 +437,9 @@ CREATE TABLE moduledependency ( id CHAR(52) NOT NULL PRIMARY KEY, effectiveTime DATE NOT NULL, active BOOLEAN NOT NULL, - moduleId NUMERIC(18) NOT NULL, - refsetId NUMERIC(18) NOT NULL, - referencedComponentId NUMERIC(18) NOT NULL, + moduleId NUMERIC(20) NOT NULL, + refsetId NUMERIC(20) NOT NULL, + referencedComponentId NUMERIC(20) NOT NULL, sourceEffectiveTime DATE NOT NULL, targetEffectiveTime DATE NOT NULL, FOREIGN KEY (moduleId) REFERENCES concept(id), @@ -426,3 +447,16 @@ CREATE TABLE moduledependency ( ); \copy moduledependency FROM 'Snapshot/Refset/Metadata/der2_ssRefset_ModuleDependencySnapshot_${editionLabel}_${editionVersion}.txt' WITH DELIMITER E'\t' QUOTE E'\\' ENCODING 'UTF8' CSV HEADER; + +-- Transitive closure table. +DROP TABLE IF EXISTS transitiveclosure CASCADE; +CREATE TABLE transitiveclosure ( + superTypeId NUMERIC(20) NOT NULL, + subTypeId NUMERIC(20) NOT NULL, + depth INT NOT NULL, + PRIMARY KEY (superTypeId, subTypeId), + FOREIGN KEY (superTypeId) REFERENCES concept(id), + FOREIGN KEY (subTypeId) REFERENCES concept(id) +); + +\copy transitiveclosure FROM 'Snapshot/Terminology/sct2_TransitiveClosure_Snapshot_${editionLabel}_${editionVersion}.txt' WITH DELIMITER E'\t' QUOTE E'\\' ENCODING 'UTF8' CSV HEADER; diff --git a/src/main/resources/rf2/psql_views.sql b/src/main/resources/rf2/psql_views.sql index 69559da..066ade4 100644 --- a/src/main/resources/rf2/psql_views.sql +++ b/src/main/resources/rf2/psql_views.sql @@ -1,4 +1,3 @@ - -- Concept Preferred Name view, used by other views. -- 900000000000013009 = Synonym description type -- 900000000000548007 = Preferred @@ -7,200 +6,294 @@ DROP VIEW IF EXISTS conceptpreferredname; CREATE VIEW conceptpreferredname AS SELECT c.id conceptId, NULLIF(d.term, 'no active preferred synonym') preferredName, d.id descriptionId FROM concept c -LEFT OUTER JOIN description d -ON c.id = d.conceptId -AND d.active = TRUE -AND d.typeId = 900000000000013009 -LEFT OUTER JOIN language l -ON d.id = l.referencedComponentId -AND l.active = TRUE -AND l.acceptabilityId = 900000000000548007 -AND l.refSetId = 900000000000509007 + LEFT OUTER JOIN description d + ON c.id = d.conceptId + AND d.active = TRUE + AND d.typeId = 900000000000013009 + LEFT OUTER JOIN language l + ON d.id = l.referencedComponentId + AND l.active = TRUE + AND l.acceptabilityId = 900000000000548007 + AND l.refSetId = 900000000000509007 WHERE (l.acceptabilityId IS NOT null OR d.TERM IS null); -- Concept table with names. DROP VIEW IF EXISTS conceptwithnames; CREATE VIEW conceptwithnames AS -SELECT id, cpn1.preferredName idName, effectiveTime, active, - moduleId, cpn2.preferredName moduleIdName, - definitionStatusId, cpn3.preferredName definitionStatusIdName +SELECT id, + cpn1.preferredName idName, + effectiveTime, + active, + moduleId, + cpn2.preferredName moduleIdName, + definitionStatusId, + cpn3.preferredName definitionStatusIdName FROM concept, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE id = cpn1.conceptId -AND moduleId = cpn2.conceptId -AND definitionStatusId = cpn3.conceptId; + AND moduleId = cpn2.conceptId + AND definitionStatusId = cpn3.conceptId; -- Description table. DROP VIEW IF EXISTS descriptionwithnames; CREATE VIEW descriptionwithnames AS -SELECT id, term, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - d.conceptId, cpn2.preferredName conceptIdName, - languageCode, - typeId, cpn3.preferredName typeIdName, - caseSignificanceId, cpn4.preferredName caseSignificanceIdName +SELECT id, + term, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + d.conceptId, + cpn2.preferredName conceptIdName, + languageCode, + typeId, + cpn3.preferredName typeIdName, + caseSignificanceId, + cpn4.preferredName caseSignificanceIdName FROM description d, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 -WHERE moduleId = cpn1.conceptId AND d.conceptId = cpn2.conceptId AND typeId = cpn3.conceptId AND caseSignificanceId = cpn4.conceptId; + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 +WHERE moduleId = cpn1.conceptId + AND d.conceptId = cpn2.conceptId + AND typeId = cpn3.conceptId + AND caseSignificanceId = cpn4.conceptId; -- Identifier table. DROP VIEW IF EXISTS identifierwithnames; CREATE VIEW identifierwithnames AS -SELECT identifierSchemeId, cpn1.preferredName identifierSchemeIdName, - alternateIdentifier, effectiveTime, active, - moduleId, cpn2.preferredName moduleIdName, - referencedComponentId +SELECT identifierSchemeId, + cpn1.preferredName identifierSchemeIdName, + alternateIdentifier, + effectiveTime, + active, + moduleId, + cpn2.preferredName moduleIdName, + referencedComponentId FROM identifier, - conceptpreferredname cpn1, - conceptpreferredname cpn2 + conceptpreferredname cpn1, + conceptpreferredname cpn2 WHERE identifierSchemeId = cpn1.conceptId -AND moduleId = cpn2.conceptId; + AND moduleId = cpn2.conceptId; -- Relationship table. DROP VIEW IF EXISTS relationshipwithnames; CREATE VIEW relationshipwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - sourceId, cpn2.preferredName sourceIdName, - destinationId, cpn3.preferredName destinationIdName, - relationshipGroup, - typeId, cpn4.preferredName typeIdName, - characteristicTypeId, cpn5.preferredName characteristicTypeIdName, - modifierId, cpn6.preferredName modifierIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + sourceId, + cpn2.preferredName sourceIdName, + destinationId, + cpn3.preferredName destinationIdName, + relationshipGroup, + typeId, + cpn4.preferredName typeIdName, + characteristicTypeId, + cpn5.preferredName characteristicTypeIdName, + modifierId, + cpn6.preferredName modifierIdName from relationship, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5, - conceptpreferredname cpn6 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 WHERE moduleId = cpn1.conceptId -AND sourceId = cpn2.conceptId -AND destinationId = cpn3.conceptId -AND typeId = cpn4.conceptId -AND characteristicTypeId = cpn5.conceptId -AND modifierId = cpn6.conceptId; + AND sourceId = cpn2.conceptId + AND destinationId = cpn3.conceptId + AND typeId = cpn4.conceptId + AND characteristicTypeId = cpn5.conceptId + AND modifierId = cpn6.conceptId; + +-- Relationship concrete values table. +DROP VIEW IF EXISTS relationshipconcretevalueswithnames; +CREATE VIEW relationshipconcretevalueswithnames AS +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + sourceId, + cpn2.preferredName sourceIdName, + value, + relationshipGroup, + typeId, + cpn4.preferredName typeIdName, + characteristicTypeId, + cpn5.preferredName characteristicTypeIdName, + modifierId, + cpn6.preferredName modifierIdName +from relationshipconcretevalues, + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 +WHERE moduleId = cpn1.conceptId + AND sourceId = cpn2.conceptId + AND typeId = cpn4.conceptId + AND characteristicTypeId = cpn5.conceptId + AND modifierId = cpn6.conceptId; -- OWL Expression table DROP VIEW IF EXISTS owlexpressionwithnames; CREATE VIEW owlexpressionwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - owlExpression +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + owlExpression from owlexpression, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Stated Relationship table. DROP VIEW IF EXISTS statedrelationshipwithnames; CREATE VIEW statedrelationshipwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - sourceId, cpn2.preferredName sourceIdName, - destinationId, cpn3.preferredName destinationIdName, - relationshipGroup, - typeId, cpn4.preferredName typeIdName, - characteristicTypeId, cpn5.preferredName characteristicTypeIdName, - modifierId, cpn6.preferredName modifierIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + sourceId, + cpn2.preferredName sourceIdName, + destinationId, + cpn3.preferredName destinationIdName, + relationshipGroup, + typeId, + cpn4.preferredName typeIdName, + characteristicTypeId, + cpn5.preferredName characteristicTypeIdName, + modifierId, + cpn6.preferredName modifierIdName from statedrelationship, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5, - conceptpreferredname cpn6 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 WHERE moduleId = cpn1.conceptId -AND sourceId = cpn2.conceptId -AND destinationId = cpn3.conceptId -AND typeId = cpn4.conceptId -AND characteristicTypeId = cpn5.conceptId -AND modifierId = cpn6.conceptId; + AND sourceId = cpn2.conceptId + AND destinationId = cpn3.conceptId + AND typeId = cpn4.conceptId + AND characteristicTypeId = cpn5.conceptId + AND modifierId = cpn6.conceptId; -- Text Definition table. DROP VIEW IF EXISTS textdefinitionwithnames; CREATE VIEW textdefinitionwithnames AS -SELECT id, term, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - td.conceptId, cpn2.preferredName conceptIdName, - languageCode, - typeId, cpn3.preferredName typeIdName, - caseSignificanceId, cpn4.preferredName caseSignificanceIdName +SELECT id, + term, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + td.conceptId, + cpn2.preferredName conceptIdName, + languageCode, + typeId, + cpn3.preferredName typeIdName, + caseSignificanceId, + cpn4.preferredName caseSignificanceIdName from textdefinition td, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND td.conceptId = cpn2.conceptId -AND typeId = cpn3.conceptId -AND caseSignificanceId = cpn4.conceptId; + AND td.conceptId = cpn2.conceptId + AND typeId = cpn3.conceptId + AND caseSignificanceId = cpn4.conceptId; -- Association refset table. DROP VIEW IF EXISTS associationwithnames; CREATE VIEW associationwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - targetComponent, cpn4.preferredName targetComponentName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + targetComponent, + cpn4.preferredName targetComponentName from association, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND targetComponent = cpn4.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND targetComponent = cpn4.conceptId; -- Attribute Value refset table. DROP VIEW IF EXISTS attributevaluewithnames; CREATE VIEW attributevaluewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, REPLACE(cpn3.preferredName, 'no active preferred synonym', 'no such concept') referencedComponentIdName, - valueId, cpn4.preferredName valueIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + REPLACE(cpn3.preferredName, 'no active preferred synonym', 'no such concept') referencedComponentIdName, + valueId, + cpn4.preferredName valueIdName FROM attributevalue -LEFT OUTER JOIN conceptpreferredname cpn3 ON referencedComponentId = cpn3.conceptId -INNER JOIN conceptpreferredname cpn1 ON moduleId = cpn1.conceptId -INNER JOIN conceptpreferredname cpn2 ON refsetId = cpn2.conceptId -INNER JOIN conceptpreferredname cpn4 ON valueId = cpn4.conceptId; + LEFT OUTER JOIN conceptpreferredname cpn3 ON referencedComponentId = cpn3.conceptId + INNER JOIN conceptpreferredname cpn1 ON moduleId = cpn1.conceptId + INNER JOIN conceptpreferredname cpn2 ON refsetId = cpn2.conceptId + INNER JOIN conceptpreferredname cpn4 ON valueId = cpn4.conceptId; -- Simple refset table. DROP VIEW IF EXISTS simplewithnames; CREATE VIEW simplewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName FROM simple, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- No more complex map, ICD9CM maps deprecated @@ -231,202 +324,282 @@ AND referencedComponentId = cpn3.conceptId; -- Extended Map refset table. DROP VIEW IF EXISTS extendedmapwithnames; CREATE VIEW extendedmapwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - mapGroup, - mapPriority, - mapRule, - mapAdvice, - mapTarget, - correlationId, cpn4.preferredName correlationIdName, - mapCategoryId, cpn5.preferredName mapCategoryIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + mapGroup, + mapPriority, + mapRule, + mapAdvice, + mapTarget, + correlationId, + cpn4.preferredName correlationIdName, + mapCategoryId, + cpn5.preferredName mapCategoryIdName FROM extendedmap, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND correlationId = cpn4.conceptId -AND mapCategoryId = cpn5.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND correlationId = cpn4.conceptId + AND mapCategoryId = cpn5.conceptId; -- Simple Map refset table. DROP VIEW IF EXISTS simplemapwithnames; CREATE VIEW simplemapwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - mapTarget +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + mapTarget FROM simplemap, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Language refset table. DROP VIEW IF EXISTS languagewithnames; CREATE VIEW languagewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - acceptabilityId, cpn4.preferredName acceptabilityIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + acceptabilityId, + cpn4.preferredName acceptabilityIdName FROM language, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.descriptionId -AND acceptabilityId = cpn4.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.descriptionId + AND acceptabilityId = cpn4.conceptId; -- Refset Descriptor refset table. DROP VIEW IF EXISTS refsetdescriptorwithnames; CREATE VIEW refsetdescriptorwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - attributeDescription, attributeType, attributeOrder +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + attributeDescription, + attributeType, + attributeOrder FROM refsetdescriptor, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Description Type refset table. DROP VIEW IF EXISTS descriptiontypewithnames; CREATE VIEW descriptiontypewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - descriptionFormat, descriptionLength +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + descriptionFormat, + descriptionLength FROM descriptiontype, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- MRCM Attribute Domain refset table. DROP VIEW IF EXISTS mrcmattributedomainwithnames; CREATE VIEW mrcmattributedomainwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - domainId, cpn4.preferredName domainIdName, - grouped, attributeCardinality, attributeInGroupCardinality, - ruleStrengthId, cpn4.preferredName ruleStrengthIdName, - contentTypeId, cpn4.preferredName contentTypeIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + domainId, + cpn4.preferredName domainIdName, + grouped, + attributeCardinality, + attributeInGroupCardinality, + ruleStrengthId, + cpn4.preferredName ruleStrengthIdName, + contentTypeId, + cpn4.preferredName contentTypeIdName FROM mrcmattributedomain, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5, - conceptpreferredname cpn6 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5, + conceptpreferredname cpn6 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND domainId = cpn4.conceptId -AND ruleStrengthId = cpn5.conceptId -AND contentTypeId = cpn6.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND domainId = cpn4.conceptId + AND ruleStrengthId = cpn5.conceptId + AND contentTypeId = cpn6.conceptId; -- MRCM Module Scope refset table. DROP VIEW IF EXISTS mrcmmodulescopewithnames; CREATE VIEW mrcmmodulescopewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - mrcmRuleRefsetId, cpn4.preferredName mrcmRuleRefsetIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + mrcmRuleRefsetId, + cpn4.preferredName mrcmRuleRefsetIdName FROM mrcmmodulescope, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND mrcmRuleRefsetId = cpn4.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND mrcmRuleRefsetId = cpn4.conceptId; -- MRCM Attribute Range refset table. DROP VIEW IF EXISTS mrcmattributerangewithnames; CREATE VIEW mrcmattributerangewithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - rangeConstraint, attributeRule, - ruleStrengthId, cpn4.preferredName ruleStrengthIdName, - contentTypeId, cpn4.preferredName contentTypeIdName +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + rangeConstraint, + attributeRule, + ruleStrengthId, + cpn4.preferredName ruleStrengthIdName, + contentTypeId, + cpn4.preferredName contentTypeIdName FROM mrcmattributerange, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3, - conceptpreferredname cpn4, - conceptpreferredname cpn5 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3, + conceptpreferredname cpn4, + conceptpreferredname cpn5 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId -AND ruleStrengthId = cpn4.conceptId -AND contentTypeId = cpn5.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId + AND ruleStrengthId = cpn4.conceptId + AND contentTypeId = cpn5.conceptId; + - -- MRCM Domain refset table. DROP VIEW IF EXISTS mrcmdomainwithnames; CREATE VIEW mrcmdomainwithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - domainConstraint, parentDomain, - proximalPrimitiveConstraint, proximalPrimitiveRefinement, - domainTemplateForPrecoordination, - domainTemplateForPostcoordination, - guideURL +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + domainConstraint, + parentDomain, + proximalPrimitiveConstraint, + proximalPrimitiveRefinement, + domainTemplateForPrecoordination, + domainTemplateForPostcoordination, + guideURL FROM mrcmdomain, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; -- Module Dependency refset table. DROP VIEW IF EXISTS moduledependencywithnames; CREATE VIEW moduledependencywithnames AS -SELECT id, effectiveTime, active, - moduleId, cpn1.preferredName moduleIdName, - refsetId, cpn2.preferredName refsetIdName, - referencedComponentId, cpn3.preferredName referencedComponentIdName, - sourceEffectiveTime, targetEffectiveTime +SELECT id, + effectiveTime, + active, + moduleId, + cpn1.preferredName moduleIdName, + refsetId, + cpn2.preferredName refsetIdName, + referencedComponentId, + cpn3.preferredName referencedComponentIdName, + sourceEffectiveTime, + targetEffectiveTime FROM moduledependency, - conceptpreferredname cpn1, - conceptpreferredname cpn2, - conceptpreferredname cpn3 + conceptpreferredname cpn1, + conceptpreferredname cpn2, + conceptpreferredname cpn3 WHERE moduleId = cpn1.conceptId -AND refsetId = cpn2.conceptId -AND referencedComponentId = cpn3.conceptId; - + AND refsetId = cpn2.conceptId + AND referencedComponentId = cpn3.conceptId; + +-- transitive closure table with names +-- NOTE: this assumes conceptpreferredname has already been created +-- via the standard RF2 load scripts +DROP VIEW IF EXISTS transitiveclosurewithnames; +CREATE VIEW transitiveclosurewithnames AS +SELECT superTypeId, cpn1.preferredName superTypeName, + subTypeId, cpn2.preferredName subTypeName, a.depth +FROM transitiveclosure a, + conceptpreferredname cpn1, + conceptpreferredname cpn2 +WHERE a.superTypeId = cpn1.conceptId + AND a.subTypeId = cpn2.conceptId; diff --git a/src/main/resources/rf2/relationshipconcretevalues.ctl b/src/main/resources/rf2/relationshipconcretevalues.ctl new file mode 100644 index 0000000..421e687 --- /dev/null +++ b/src/main/resources/rf2/relationshipconcretevalues.ctl @@ -0,0 +1,23 @@ +options (skip=1,direct=true) +load data +characterset UTF8 length semantics char +infile 'Snapshot/Terminology/sct2_RelationshipConcreteValues_Snapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +badfile 'relationshipconcretevalues.bad' +discardfile 'relationshipconcretevalues.dsc' +insert +into table relationshipconcretevalues +reenable disabled_constraints +fields terminated by X'09' +trailing nullcols +( + id INTEGER EXTERNAL, + effectiveTime DATE "YYYYMMDD", + active INTEGER EXTERNAL, + moduleId INTEGER EXTERNAL, + sourceId INTEGER EXTERNAL, + value CHAR(256), + relationshipGroup INTEGER EXTERNAL, + typeId INTEGER EXTERNAL, + characteristicTypeId INTEGER EXTERNAL, + modifierId INTEGER EXTERNAL +) \ No newline at end of file diff --git a/src/main/resources/rf2/transitiveclosure.ctl b/src/main/resources/rf2/transitiveclosure.ctl new file mode 100644 index 0000000..9f587d5 --- /dev/null +++ b/src/main/resources/rf2/transitiveclosure.ctl @@ -0,0 +1,16 @@ +options (skip=1,direct=true) +load data +characterset UTF8 length semantics char +infile 'Snapshot/Terminology/sct2_TransitiveClosure_Snapshot_${editionLabel}_${editionVersion}.txt' "str X'0d0a'" +badfile 'transitiveclosure.bad' +discardfile 'transitiveclosure.dsc' +insert +into table transitiveclosure +reenable disabled_constraints +fields terminated by X'09' +trailing nullcols +( + superTypeId INTEGER EXTERNAL, + subTypeId INTEGER EXTERNAL, + depth INTEGER EXTERNAL +) \ No newline at end of file diff --git a/src/test/resources/README-announcements.txt b/src/test/resources/README-announcements.txt new file mode 100644 index 0000000..196be50 --- /dev/null +++ b/src/test/resources/README-announcements.txt @@ -0,0 +1,60 @@ +## Sample texts for announcing new releases + +### Twitter + +-- SNOMED International July 1: + +Is it July already?! Announcing #SNOMEDCT DB load and transitive +closure scripts for the 20260701 International Edition! Out-of-the-box +support for #Postgres, #Oracle, #MySQL, and #MariaDB. +#MakingTerminologyEasy #MakingSnomedEasy #snomed +https://www.westcoastinformatics.com/products/db-load-scripts + +-- US Edition March 1: + +Spring is upon us! Announcing #SNOMEDCT DB load and transitive +closure scripts for the 20260301 US Edition! Out-of-the-box +support for #Postgres, #Oracle, #MySQL, and #MariaDB. +#MakingTerminologyEasy #snomedct #termhub +https://www.westcoastinformatics.com/products/db-load-scripts + +---- + +The new year is here! Announcing #SNOMEDCT DB load and transitive closure scripts for the 20260701 International Edition! Out-of-the-box support for #Postgres, #Oracle, #MySQL, and #MariaDB. +See https://westcoastinformatics.com/products/db-load-scripts #MakingTerminologyEasy #MakingSnomedEasy #snomedct + +At West Coast Informatics, we are experts in loading healthcare terminology data +into standard databases. Announcing updates to our Database Load Scripts for the +20260301 US Edition of SNOMEDCT. +Download now at https://www.westcoastinformatics.com/products/db-load-scripts + +These tools are designed to provide an easy mechanism to load SNOMEDCT into Postgres, MySQL (or MariaDB), or Oracle databases. These resources are free to use and will assist you with loading core SNOMED CT data and transitive closure tables into supported environments. These projects are hosted on Github and we welcome any feedback or community contributions. +#MakingTerminologyEasy #MakingSnomedEasy #snomedct + +### UMLS Users Mailing List +Subject: 20260701 SNOMEDCT DB load and transitive closure scripts + +Hi all, + +NLM recently posted the July 01, 2026 update of the SNOMEDCT Edition. + +For those interested in interacting with SNOMEDCT data through a relational database, +we have a utility to make it easy to load the data into a Postgres, Oracle, MySQL (or MariaDB) database. +We offer support for both International Edition and the US Edition. + +This utility is freely available through our website: + +* Information and download: https://www.westcoastinformatics.com/products/db-load-scripts + - See the "DB Load Scripts" section + - These scripts have been updated to automatically compute a transitive closure table and load it also + +* Github project: + - https://github.com/WestCoastInformatics/SNOMED-DB-Load-Scripts + +* Updated Training video: + - https://youtu.be/917T6D3b5ro + + +Feel free to share this announcement with others or retweet the announcement - +https://twitter.com/wcinformatics/status/ ! +https://twitter.com/wcinformatics/status/1766236868088386039 diff --git a/src/test/resources/README-check-files.txt b/src/test/resources/README-check-files.md similarity index 59% rename from src/test/resources/README-check-files.txt rename to src/test/resources/README-check-files.md index 870acf2..5f1ef76 100644 --- a/src/test/resources/README-check-files.txt +++ b/src/test/resources/README-check-files.md @@ -1,17 +1,25 @@ -# With a new download of SNOMED international edition or US edition, check that the file list -# still matches what we are loading. +# Check Files +With a new download of SNOMED international edition or US edition, check that the file list still matches what we are loading. -# Look for extra files (this should return zero rows) +## Look for extra files +- International Edition will return a 2, but these two files only contain 1 header file +- US Edition should return 0 + +``` find . -name "*txt" | grep -v Refset_Association | grep -v Refset_AttributeValue | grep -v Refset_SimpleSn |\ grep -v Refset_Language | grep -v Refset_ExtendedMap | grep -v Refset_SimpleMap |\ grep -v Refset_RefsetDescriptor | grep -v Refset_DescriptionType |\ grep -v Refset_MRCMAttributeDomain | grep -v Refset_MRCMModuleScope |\ grep -v Refset_MRCMAttributeRange | grep -v Refset_ModuleDependency |\ grep -v Refset_MRCMDomain | grep -v sct2_Concept | grep -v sct2_Description |\ - grep -v sct2_Identifier | grep -v sct2_Relationship | grep -v Refset_OWLExpression |\ - grep -v sct2_StatedRelationship | grep -v sct2_TextDefinition | wc -l + grep -v sct2_Identifier | grep -v sct2_Relationship_ | grep -v Refset_OWLExpression |\ + grep -v sct2_StatedRelationship | grep -v sct2_TextDefinition |\ + grep -v sct2_RelationshipConcreteValues | wc -l +``` -# Look that all files are there (this should equal 20) +## Look that all files are there +- Both editions should return 21 +``` find . -name "*txt" |\ perl -ne 'print if /Refset_Association|AttributeValue|Refset_SimpleSn/; \ print if /Refset_Language|Refset_ExtendedMap|Refset_SimpleMap/; \ @@ -19,6 +27,6 @@ find . -name "*txt" |\ print if /Refset_MRCMAttributeDomain|Refset_MRCMModuleScope/; \ print if /Refset_MRCMAttributeRange|Refset_ModuleDependency/; \ print if /Refset_MRCMDomain|sct2_Concept|sct2_Description/; \ - print if /sct2_Identifier|sct2_Relationship|Refset_OWLExpression/; \ - print if /sct2_StatedRelationship|sct2_TextDefinition/; \ - print if /sct2_TransitiveClosure/' | wc -l + print if /sct2_Identifier|sct2_Relationship(ConcreteValues)?_|Refset_OWLExpression/; \ + print if /sct2_StatedRelationship|sct2_TextDefinition/' | wc -l +``` \ No newline at end of file diff --git a/src/test/resources/README-mysql-docker.md b/src/test/resources/README-mysql-docker.md new file mode 100644 index 0000000..698e12d --- /dev/null +++ b/src/test/resources/README-mysql-docker.md @@ -0,0 +1,86 @@ +# BUILD INSTRUCTIONS + +## Pre-requisites + +Before proceeding, make sure you updated the hostname in the load script. + +Check current versions: https://endoflife.software/applications/databases/mysql +Add new versions, remove versions whose final release is the past and, bump the minor version numbers. + +## File Setup + +``` + cd /wci/data + unzip -o /wci/projects/SNOMED-DB-Load-Scripts/target/snomed-db-scripts-mysql.*.zip + + sudo chmod +x rf2/populate_mysql_db.sh + sudo chmod +x rf2/compute_transitive_closure.pl + sudo chmod +x rf2/compute_transitive_closure.py + +``` + +Follow the instructions for the version you are using below. + +### MySQL 8.4 + +1. Launch the container
+ `docker run --name snomed-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -e MYSQL_DATABASE=snomed -d --rm mysql:8.4 --local_infile=ON` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v $dir:/data mysql:8.4 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_mysql_db.sh +``` + +### MariaDB - 10.11 + +1. Launch the container
+ `docker run --name snomed-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -e MYSQL_DATABASE=snomed -d --rm mariadb:10.11` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v $dir:/data mariadb:10.11 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_mysql_db.sh +``` + +### MariaDB - 11.4, 11.8 + +1. Launch the container
+ `docker run --name snomed-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -e MYSQL_DATABASE=snomed -d --rm mariadb:11.4` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v $dir:/data mariadb:11.4 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_maria_db.sh +``` + +### MariaDB - 12.3 + +1. Launch the container
+ `docker run --name snomed-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -e MYSQL_DATABASE=snomed -d --rm mariadb:11.4` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v $dir:/data mariadb:11.4 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_maria_db.sh +``` diff --git a/src/test/resources/README-mysql-docker.txt b/src/test/resources/README-mysql-docker.txt deleted file mode 100644 index 2f8db98..0000000 --- a/src/test/resources/README-mysql-docker.txt +++ /dev/null @@ -1,42 +0,0 @@ - -##### MySQL 5.6 - -# -# For testing mysql, run a docker mysql instance -# - https://hub.docker.com/_/mysql - -# -# Launch the container -# -docker run --name snomed-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -e MYSQL_DATABASE=snomed -d mysql:5.6 - -# -# populate_mysql_db.sh -# host = host.docker.internal (**make sure to edit this setting before proceeding) -# -dir=c:/data/SnomedCt_International -docker run -it -v $dir:/data mysql:5.6 /bin/bash -root@842bfb3da1f1:/# cd /data -root@842bfb3da1f1:/data# ./populate_mysql_db.sh & - - -##### MySQL 5.7 - -# -# For testing mysql, run a docker mysql instance -# - https://hub.docker.com/_/mysql - -# -# Launch the container -# -docker run --name snomed-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -e MYSQL_DATABASE=snomed -d mysql:5.7 - -# -# populate_mysql_db.sh -# host = host.docker.internal (**make sure to edit this setting before proceeding) -# -dir=c:/data/SnomedCt_International -docker run -it -v $dir:/data mysql:5.7 /bin/bash -root@842bfb3da1f1:/# cd /data -root@842bfb3da1f1:/data# ./populate_mysql_db.sh & - diff --git a/src/test/resources/README-oracle-docker.md b/src/test/resources/README-oracle-docker.md new file mode 100644 index 0000000..30c057e --- /dev/null +++ b/src/test/resources/README-oracle-docker.md @@ -0,0 +1,67 @@ +# Oracle 19c, 21c, 26ai BUILD INSTRUCTIONS +## Prerequisites +1. Log into DockerHub and agree to Oracle's Terms of Service + - [Oracle DB Enterprise Repository](https://container-registry.oracle.com/ords/f?p=113:1:103153009327673::::FSP_LANGUAGE_PREFERENCE:&cs=3vvrLolDIFGAxz-ReKVoNrPBwhCCJtGRffMagNzTnvafG3AeKT2KLWKQSclqKpS-c6TF2uprvjENxXOETZOmgfQ) + - Log in with your account and then click on "Continue" on the line enterprise. This will allow you to pull the Oracle Database Enterprise Edition image from the Oracle Container Registry. + - Pull the image from the Oracle Container Registry + - `docker login container-registry.oracle.com` and enter your hub.docker.com credentials + - `docker pull container-registry.oracle.com/database/enterprise:19.3.0.0` + - `docker pull container-registry.oracle.com/database/enterprise:21.3.0.0` + - `docker pull container-registry.oracle.com/database/enterprise:23.26.1.0` + +2. For testing oracle, run a docker oracle instance + - https://dzone.com/articles/oracle-19c-image-installation-in-docker + +3. If you're running this locally, you will use `localhost` as the host. If you're running this on a remote server, + you will use the server's IP address as the host. + +4. If running an Oracle Database Docker Container on Linux use docker login and enter your hub.docker.com + credentials for image to download. + +# Instructions + +## File Setup +``` + cd /wci/data + unzip -o /wci/projects/SNOMED-DB-Load-Scripts/target/snomed-db-scripts-oracle.*.zip + sudo chmod +x rf2/populate_oracle_db.sh + sudo chmod +x rf2/compute_transitive_closure.py + + # To avoid permission issues, make sure to grant write perms to the all folders + chmod -R 777 rf2 +``` + +## If not running Docker as root +``` + cd $dir/rf2 + touch oracle.log concept.log description.log identifier.log relationship.log owlexpression.log statedrelationship.log textdefinition.log association.log attributevalue.log simple.log complexmap.log extendedmap.log simplemap.log language.log refsetdescriptor.log descriptiontype.log moduledependency.log relationshipconcretevalues.log + chmod a=rw oracle.log concept.log description.log identifier.log relationship.log owlexpression.log statedrelationship.log textdefinition.log association.log attributevalue.log simple.log complexmap.log extendedmap.log simplemap.log language.log refsetdescriptor.log descriptiontype.log moduledependency.log relationshipconcretevalues.log +``` + +## Launch the container +Create log files and allow read/write to all users. Use the appropriate image for the version you are using for 19c, 21c, or 26ai. +``` + export dir=/wci/data/ + cd $dir + + #For container-registry.oracle.com/database/enterprise:19.3.0.0 + sudo docker run --name snomed-oracle -v $dir:/data -d --rm -p 8080:8080 -p 1521:1521 container-registry.oracle.com/database/enterprise:19.3.0.0 +``` +## Populate DB + +1. Launch the container + +`sudo docker exec -it snomed-oracle /bin/bash` + +2. Populate the database +``` + root@842bfb3da1f1:/# . /home/oracle/.bashrc + root@842bfb3da1f1:/# sqlplus sys/Oradoc_db1 as sysdba + alter session set "_ORACLE_SCRIPT"=true; + create user snomed identified by snomed; + GRANT CONNECT, RESOURCE, DBA TO snomed; + exit + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_oracle_db.sh +``` diff --git a/src/test/resources/README-postgres-docker.md b/src/test/resources/README-postgres-docker.md new file mode 100644 index 0000000..6543a1c --- /dev/null +++ b/src/test/resources/README-postgres-docker.md @@ -0,0 +1,99 @@ +# BUILD INSTRUCTIONS + +## Pre-requisites + +Before proceeding, make sure you updated the PGHOST in the load script. + +Check current versions: https://www.postgresql.org/support/versioning/ +Add new versions, remove versions whose final release is the past and, bump the minor version numbers. + +## File Setup + +``` + cd /wci/data + unzip -o /wci/projects/SNOMED-DB-Load-Scripts/target/snomed-db-scripts-postgres.*.zip + sudo chmod +x rf2/populate_postgres_db.sh + sudo chmod +x rf2/compute_transitive_closure.pl +``` + +- For testing postgres, run a docker postgres instance - https://hub.docker.com/_/postgres + +### Postgres 14.23 + +1. Launch the container
+ `docker run --name snomed-postgres -p 5432:5432 -e POSTGRES_PASSWORD=admin -e POSTGRES_DB=snomed -d --rm postgres:14.23` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v "$dir":/data postgres:14.23 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_postgres_db.sh +``` + +### Postgres 15.18 + +1. Launch the container
+ `docker run --name snomed-postgres -p 5432:5432 -e POSTGRES_PASSWORD=admin -e POSTGRES_DB=snomed -d --rm postgres:15.1812` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v "$dir":/data postgres:15.18 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_postgres_db.sh +``` + +### Postgres 16.14 + +1. Launch the container
+ `docker run --name snomed-postgres -p 5432:5432 -e POSTGRES_PASSWORD=admin -e POSTGRES_DB=snomed -d --rm postgres:16.14` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v "$dir":/data postgres:16.14 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_postgres_db.sh +``` + +### Postgres 17.10 + +1. Launch the container
+ `docker run --name snomed-postgres -p 5432:5432 -e POSTGRES_PASSWORD=admin -e POSTGRES_DB=snomed -d --rm postgres:17.10` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v "$dir":/data postgres:17.10 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_postgres_db.sh +``` + +### Postgres 18.4 + +1. Launch the container
+ `docker run --name snomed-postgres -p 5432:5432 -e POSTGRES_PASSWORD=admin -e POSTGRES_DB=snomed -d --rm postgres:18.4` + +2. Build Server + +``` + export dir=/wci/data/ + cd $dir + docker run -it -v "$dir":/data postgres:18.4 /bin/bash + + root@842bfb3da1f1:/# cd /data/rf2 + root@842bfb3da1f1:/data/rf2# ./populate_postgres_db.sh +```