RSD creation hanging on 'checking for skill similarity'

[mstaal1974]

I have had my tech team deploy an instance of the OSMT but are not familiar with the format.

When trying to add a single RSD or batch upload the system does not get passed "checking for skill statement similarity". i.e continuously searches.

any assistance is appreciated.

[JohnKallies]

Hello, thanks for your question. I hope we can help. I have a couple questions...

• Did you do a "Quickstart" deployment, using osmt_cli.sh
• If not, can you briefly describe your deployment environment? Are you using docker-compose/containers?
• Have you imported the BLS and O*NET job code data? Have you reindexed your ElasticSearch instance?
• What kind of log output do you get from the Spring app when this happens? And the ElasticSearch instance?

I look forward to your feedback.

[mstaal1974]

Hi John, See answers from my tech team * Did you do a "Quickstart" deployment, using osmt_cli.sh Yes * If not, can you briefly describe your deployment environment? Are you using docker-compose/containers? This is contained in the .yml file * Have you imported the BLS and O*NET job code data? By default this is imported when running quick start Have you reindexed your ElasticSearch instance? Yes, by default included in the env file. * What kind of log output do you get from the Spring app when this happens? "this happens" are you referring to which: install or transaction install - tons of logs transaction - depends on transaction And the ElasticSearch instance? yes there is ElasticSearch log From: John Kallies ***@***.***> Sent: Wednesday, March 2, 2022 2:36 AM To: wgu-opensource/osmt ***@***.***> Cc: Michael Staal ***@***.***>; Author ***@***.***> Subject: Re: [wgu-opensource/osmt] RSD creation hanging on 'checking for skill similarity' (Discussion #172) Hello, thanks for your question. I hope we can help. I have a couple questions... * Did you do a "Quickstart" deployment, using osmt_cli.sh * If not, can you briefly describe your deployment environment? Are you using docker-compose/containers? * Have you imported the BLS and O*NET job code data? Have you reindexed your ElasticSearch instance? * What kind of log output do you get from the Spring app when this happens? And the ElasticSearch instance? I look forward to your feedback. — Reply to this email directly, view it on GitHub<#172 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AU4QDAJFN2IQERZSN6DQVZLU5ZBNZANCNFSM5PUM3OYQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[JohnKallies]

Thank you for additional details. This helps me narrow the focus to log output for the Spring app container and ElasticSearch container.

I don't understand the distinction between install and transaction above. I'm asking for log output for when you are having a problem. I understand this problem to be "OSMT spins on similarity searching while inputing or importing an RSD". I guess let's make a checkpoint here. Do I understand your problem correctly?

Assuming we are on the same page...

Just for fun, would you please run ./osmt_cli.sh -v from the OSMT git repo directory? This command collects information about the machine in question. There shouldn't be anything sensitive in that output, so please paste that output here. You don't need to stop a Quickstart instance to do this.

From the machine where your Quickstart instance is running, if you run docker ps, you will see command output that should look like this (it's a problem if it's very different...):

/Users/john.kallies  > docker ps
CONTAINER ID   IMAGE                                                  COMMAND                  CREATED       STATUS        PORTS                                            NAMES
539e66db1b19   wguopensource/osmt-app:latest                          "/opt/osmt/bin/docke…"   2 weeks ago   Up 18 hours   0.0.0.0:8080->8080/tcp                           osmt_quickstart_app_1
ca786f1ac748   docker.elastic.co/elasticsearch/elasticsearch:7.16.2   "/bin/tini -- /usr/l…"   2 weeks ago   Up 18 hours   0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   osmt_quickstart_elasticsearch_1
77adf2325033   mysql:8.0                                              "docker-entrypoint.s…"   2 weeks ago   Up 18 hours   0.0.0.0:3306->3306/tcp, 33060/tcp                osmt_quickstart_db_1
d47fca3272da   redis:6.0.6                                            "docker-entrypoint.s…"   2 weeks ago   Up 18 hours   0.0.0.0:6379->6379/tcp                           osmt_quickstart_redis_1

The far right column shows the Docker container names (osmt_quickstart_app_1, etc). You can see the combined log output of the docker-compose stack in the terminal where you started the Quickstart; each container prefixes the log entry, but that can be a little overwhelming. You access the log output of each individual container with docker log <container_name>. See this for a little more about the docker log command -- https://devconnected.com/docker-logs-complete-guide/#Docker_Container_logs.

I would recommend starting or restarting your Quickstart instance, and give it a few minutes to get started (wait until there is no more activity in the main docker-compose log).

• At that point, before starting RSD input or imports, take a look at the logs and see if any issues are reported (connection issues between services/containers, storage or memory issues, or anything else).
• Then take the small step of trying to add 1 RSD. You can watch the main docker-compose log output at that point, or watch the individual container's logs. I would look first at the Spring app container, then the MySQL DB container, then the ElasticSearch container.

Please let me know what turns up? Hopefully we can resolve this soon. Thank you.

[mstaal1974]

HI John, |Sorry for the delay in getting back to you. This is the error report they get. [image.png] I am arranging Server access for you so you can have see exactly what they have done. This is the only thing we have on that server so no worries about security. From: John Kallies ***@***.***> Sent: Thursday, March 3, 2022 2:35 AM To: wgu-opensource/osmt ***@***.***> Cc: Michael Staal ***@***.***>; Author ***@***.***> Subject: Re: [wgu-opensource/osmt] RSD creation hanging on 'checking for skill similarity' (Discussion #172) Thank you for additional details. This helps me narrow the focus to log output for the Spring app container and ElasticSearch container. I don't understand the distinction between install and transaction above. I'm asking for log output for when you are having a problem. I understand this problem to be OSMT spins on similarity searching while inputing or importing an RSD. I guess let's make a checkpoint here. Do I understand your problem correctly? Assuming we are on the same page... Just for fun, would you please run ./osmt_cli.sh -v from the OSMT git repo directory? This command collects information about the machine in question. There shouldn't be anything sensitive in that output, so please paste that output here. You don't need to stop a Quickstart instance to do this. From the machine where your Quickstart instance is running, if you run docker ps, you will see command output that should look like this (it's a problem if it's very different...): /Users/john.kallies > docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 539e66db1b19 wguopensource/osmt-app:latest "/opt/osmt/bin/docke…" 2 weeks ago Up 18 hours 0.0.0.0:8080->8080/tcp osmt_quickstart_app_1 ca786f1ac748 docker.elastic.co/elasticsearch/elasticsearch:7.16.2 "/bin/tini -- /usr/l…" 2 weeks ago Up 18 hours 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp osmt_quickstart_elasticsearch_1 77adf2325033 mysql:8.0 "docker-entrypoint.s…" 2 weeks ago Up 18 hours 0.0.0.0:3306->3306/tcp, 33060/tcp osmt_quickstart_db_1 d47fca3272da redis:6.0.6 "docker-entrypoint.s…" 2 weeks ago Up 18 hours 0.0.0.0:6379->6379/tcp osmt_quickstart_redis_1 The far right column shows the Docker container names (osmt_quickstart_app_1, etc). You can see the combined log output of the docker-compose stack in the terminal where you started the Quickstart; each container prefixes the log entry, but that can be a little overwhelming. You access the log output of each individual container with docker log <container_name>. See this for a little more about the docker log command -- https://devconnected.com/docker-logs-complete-guide/#Docker_Container_logs. I would recommend starting or restarting your Quickstart instance, and give it a few minutes to get started. Before starting RSD input or imports, take a look at the logs and see if any issues are reported (connection issues between services/containers, storage or memory issues, or anything else). Then take the small step of trying to add 1 RSD. You can watch the main docker-compose log output at that point, or watch the individual container's logs. I would start with the Spring app container, then the MySQL DB container, then the ElasticSearch container. Please let me know what turns up? Hopefully we can resolve this soon. Thank you. — Reply to this email directly, view it on GitHub<#172 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AU4QDANMMDHOYL37GST7URTU56KCJANCNFSM5PUM3OYQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[mstaal1974]

P.S. I am happy to pay for your time. From: John Kallies ***@***.***> Sent: Thursday, March 3, 2022 2:35 AM To: wgu-opensource/osmt ***@***.***> Cc: Michael Staal ***@***.***>; Author ***@***.***> Subject: Re: [wgu-opensource/osmt] RSD creation hanging on 'checking for skill similarity' (Discussion #172) Thank you for additional details. This helps me narrow the focus to log output for the Spring app container and ElasticSearch container. I don't understand the distinction between install and transaction above. I'm asking for log output for when you are having a problem. I understand this problem to be OSMT spins on similarity searching while inputing or importing an RSD. I guess let's make a checkpoint here. Do I understand your problem correctly? Assuming we are on the same page... Just for fun, would you please run ./osmt_cli.sh -v from the OSMT git repo directory? This command collects information about the machine in question. There shouldn't be anything sensitive in that output, so please paste that output here. You don't need to stop a Quickstart instance to do this. From the machine where your Quickstart instance is running, if you run docker ps, you will see command output that should look like this (it's a problem if it's very different...): /Users/john.kallies > docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 539e66db1b19 wguopensource/osmt-app:latest "/opt/osmt/bin/docke…" 2 weeks ago Up 18 hours 0.0.0.0:8080->8080/tcp osmt_quickstart_app_1 ca786f1ac748 docker.elastic.co/elasticsearch/elasticsearch:7.16.2 "/bin/tini -- /usr/l…" 2 weeks ago Up 18 hours 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp osmt_quickstart_elasticsearch_1 77adf2325033 mysql:8.0 "docker-entrypoint.s…" 2 weeks ago Up 18 hours 0.0.0.0:3306->3306/tcp, 33060/tcp osmt_quickstart_db_1 d47fca3272da redis:6.0.6 "docker-entrypoint.s…" 2 weeks ago Up 18 hours 0.0.0.0:6379->6379/tcp osmt_quickstart_redis_1 The far right column shows the Docker container names (osmt_quickstart_app_1, etc). You can see the combined log output of the docker-compose stack in the terminal where you started the Quickstart; each container prefixes the log entry, but that can be a little overwhelming. You access the log output of each individual container with docker log <container_name>. See this for a little more about the docker log command -- https://devconnected.com/docker-logs-complete-guide/#Docker_Container_logs. I would recommend starting or restarting your Quickstart instance, and give it a few minutes to get started. Before starting RSD input or imports, take a look at the logs and see if any issues are reported (connection issues between services/containers, storage or memory issues, or anything else). Then take the small step of trying to add 1 RSD. You can watch the main docker-compose log output at that point, or watch the individual container's logs. I would start with the Spring app container, then the MySQL DB container, then the ElasticSearch container. Please let me know what turns up? Hopefully we can resolve this soon. Thank you. — Reply to this email directly, view it on GitHub<#172 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AU4QDANMMDHOYL37GST7URTU56KCJANCNFSM5PUM3OYQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[JohnKallies]

Hello Michael (@mstaal1974),

If you look at our conversation in GitHub (#172), many things are being lost in translation, and the message thread is being duplicated (email clients are smart enough to fold a thread history closed, but GitHub seems to just keep stacking messages...) Most importantly, I can not see the PNG attachment. Would you please try replying here instead of via email?

Thank you,
John