Fix some OPL script issues

[drgrice1]

The $dbpass variable in the OPL scripts should not be shell quoted anymore. It is not used in the shell command anymore. This fixes an issue discovered in https://webwork.maa.org/moodle/mod/forum/discuss.php?d=5018 when the database password contains characters like "*".

Also fix a long standing typo in OPL-update that causes the update-OPL-statistics.pl script not to be run as advertised.

I also have a question regarding the logic of in the OPL-update and when the two scripts update-OPL-statistics.pl and load-OPL-global-statistics.pl are run. Shouldn't the first script only be run if $ce->{problemLibrary}{showLibraryLocalStats} is true, and the second script only be run if $ce->{problemLibrary}{showLibraryGlobalStats} is true, instead of both scripts being run if either is true?

[Alex-Jordan]

This did not work for me, but it is something different than special characters in the database password.

On a server I am using to conduct other testing, I hit the issue reported at https://webwork.maa.org/moodle/mod/forum/discuss.php?d=5018. I assumed this server also had special characters in its password. So I merged this. Then I ran ./load-OPL-global-statistics.pl but I got the same error message:

ERROR 1045 (28000): Access denied for user 'webworkDevWrite'@'localhost' (using password: YES)

Now I checked the DB password and it turns out it is entirely alphanumeric.

Then I reverted the changes introduced by 00d4f30, and I can successfully run ./load-OPL-global-statistics.pl`. I'm not sure what is going on.

[drgrice1]

Do you by chance have a .my.cnf file that is interfering? If the password is set and set incorrectly in that file it would cause this issue.

[drgrice1]

Prior to #985 the .my.cnf file contents would have been ignored.

[Alex-Jordan]

I don't find a .my.cnf in user home folders. Should I look other places?

There is a my.cnf in /etc/. It does not say much:

#
# This group is read both both by the client and the server
# use it for options that affect everything
#
[client-server]

#
# include *.cnf from the config directory
#
!includedir /etc/my.cnf.d

I looked at the files inside /etc/my.cnf.d:
enable_encryption.preset mysql-clients.cnf server.cnf

None of them have anything in them that appears to set a password.

[drgrice1]

It would have to be in your home folder. The files in /etc wouldn't (or at least shoudn't) have this. So that is probably not the problem.

[drgrice1]

Can you share the odd characters in your password without giving to much information about your password away?

Nevermind. I just saw that you said your password is entirely alphanumeric.

[Alex-Jordan]

An unusual thing about this server (one of the AIM servers) is perhaps irrelevant, but I'll mention it. WeBWorK components are not in /opt/webwork/. Rather, everything is in /home/webwork/www/webwork-dev.aimath.org/webwork/. (And there is no /my.cnf in /home/webwork, nor in /home/alexjordan.)

[drgrice1]

The location of the files doesn't matter for this. The database does not use those locations. Can you try running

MYSQL_PWD=yourpassword mysql --host=localhost --port=3306 --user=webworkDevWrite webwork < /home/webwork/www/webwork-dev.aimath.org/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/OPL_global_statistics.sql

Where yourpassword is obviously the webworkDevWrite password.

You may need to adjust the path to the OPL_global_statistics.sql file. I am guessing a bit on that based on what you have said.

[Alex-Jordan]

Same output:

# MYSQL_PWD=**************** mysql --host=localhost --port=3306 --user=webworkDevWrite webwork < /home/webwork/www/webwork-dev.aimath.org/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/OPL_global_statistics.sql
ERROR 1045 (28000): Access denied for user 'webworkDevWrite'@'localhost' (using password: YES)

Who is webworkDevWrite? That is indeed the database user in this server's site.conf, but I've never seen that before. Just webworkWrite, which is what I see in site.conf.dist. Should I be looking closely and updating my conf files as I move to the development branch and test new branches? Could that matter here?

[Alex-Jordan]

Now I think webworkDevWrite is some peculiarlity introduced by the person who set up this server. And I think I get that the name of this user does not matter.

[drgrice1]

What do you have in site.conf for the $database_username?

[drgrice1]

Also try:
mysql --host=localhost --port=3306 --user=webworkDevWrite --password=yourdbpassword webwork < /home/webwork/www/webwork-dev.aimath.org/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/OPL_global_statistics.sql

[Alex-Jordan]

Sorry, now I understand that webworkDevWrite is just the value of $database_username that was used when this server was set up. The person who set up this server simultaneously set up a production server, and used a different name (even though they are separate servers using separate databases.)

With your last command (but changing the "webwork" to "webwork_dev" as specified in this server's site.conf):

mysql --host=localhost --port=3306 --user=webworkDevWrite --password=**************** webwork_dev < /home/webwork/www/webwork-dev.aimath.org/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/OPL_global_statistics.sql

There is no output or error message. This made me realize to rerun what you asked earlier, but with webwork_dev:

# MYSQL_PWD=**************** mysql --host=localhost --port=3306 --user=webworkDevWrite webwork_dev < /home/webwork/www/webwork-dev.aimath.org/webwork/libraries/webwork-open-problem-library/OpenProblemLibrary/OPL_global_statistics.sql
ERROR 1045 (28000): Access denied for user 'webworkDevWrite'@'localhost' (using password: YES)

So still getting that error. In case it is usefl, here is what is in site.conf:

$database_dsn = "dbi:mysql:webwork_dev:localhost";
$database_storage_engine = 'myisam';

Just for a sanity check, I tried submitting an answer in an assignment using htis server and that worked.

[drgrice1]

That is odd. If one of those commands works, then the other should work as well. This server is running Ubuntu 18.04 at least isn't it?

Let's try things one more time with simplified commands to see that there isn't anything else going on in the commands.
If you run
MYSQL_PWD=***** mysql -u webworkDevWrite webwork_dev
do you get access denied, and if you run
mysql -u webworkDevWrite --password=***** webwork_dev
are you granted command line access to the database?

What do you get if you run mysql --version?

[Alex-Jordan]

Some progress. First, trying your suggestions didn't change things. The first command did not give access, and the second one did. I had a nagging feeling that something else about this server's configuration could be a problem, and I just confirmed it is relevant. I am following the instructions of the AIM IT person who originally set up this server (who no longer works at AIM). For this server, I log in as alexjordan. But to do any work related to webwork, I was told to run 'sudo bash' before moving to the webwork root folder and proceed from there. So I'm basically always acting as root on this server. I exited and returned to being alexjordan, and then I can run the first command you recently gave and it works. I'm sure my inexperience with linux is showing, and maybe now this all makes sense to you. And now that I'm actively thinking about this instead of just following old instructions, I think I should follow up with current AIM IT staff and find a different workflow.

…

On Thu, Feb 25, 2021 at 3:25 AM Glenn Rice ***@***.***> wrote: That is odd. If one of those commands works, then the other should work as well. This server is running Ubuntu 18.04 at least isn't it? Let's try things one more time with simplified commands to see that there isn't anything else going on in the commands. If you run MYSQL_PWD=***** mysql -u webworkDevWrite webwork_dev do you get access denied, and if you run mysql -u webworkDevWrite --password=***** webwork_dev are you granted command line access to the database? What do you get if you run mysql --version? — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#1189 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEDOACAWXCK5BP5UQKTGOLTAYXRJANCNFSM4XI53L3Q> .

-- Alex Jordan Mathematics Instructor Portland Community College

[drgrice1]

Running everything as root via something like sudo bash is really bad advice. Only commands that properly need root access should be run as root, and generally that is done by prefixing the command with sudo, and not logging in as root via sudo bash.

Database access is something that should certainly not be run as root, unless you are dealing with database administration like creating a user or changing the permissions for a user and such.

In the old days (more than 20 years ago), logging in as root for administrative tasks was how it was done. But that is strongly discouraged now.

[drgrice1]

Although, I just tested on my server, and both commands work for me even if I am logged in as root (via sudo bash or sudo -s).

[dlglin]

If you are running commands as root, you should also check if there is a .my.cnf file in root's home directory, i.e. /root/.my.cnf.

[Alex-Jordan]

That's probably it. In /, there is a .my.cnf file with password = ******* that does not match the database password.

Does this only mean I'm doing things wrong the way I work in this server? I know it's wrong, and have a meeting with the AIM IT person to put in place permissions that make things work for me without using root.

But is there more here to think about wrt the pull request? Maybe only to note in the release notes about what a .my.cnf might do?

[drgrice1]

Well, you are doing thing wrong in terms of using root. However, that is not the problem in relation to this pull request. The problem is that the .my.cnf file is the first thing that the mysql command checks for credentials. Next is the MYSQL_PWD environment variable. Then it uses a password insecurely passed via the --password option (which is what the script used to do). The .my.cnf file or the MYSQL_PWD approaches are secure (except if MYSQL_PWD is passed on the command line the way I told you to do in testing). Delete the .my.cnf file or correct the password in that file, and everything will work the way you are accustomed. Then you can delay learning how to work without root.

[Alex-Jordan]

How can I know it is OK to delete or edit the .my.cnf? Could it be set the way it is for other purposes on this server? There is more going on with this server than just webwork, which may have to do with why it was set up in a nonstandard way in the first place.

…

On Fri, Feb 26, 2021 at 9:39 AM Glenn Rice ***@***.***> wrote: Well, you are doing thing wrong in terms of using root. However, that is not the problem in relation to this pull request. The problem is that the .my.cnf file is the first thing that the mysql command checks for credentials. Next is the MYSQL_PWD environment variable. Then it uses a password insecurely passed via the --password option (which is what the script used to do). The .my.cnf file or the MYSQL_PWD approaches are secure (except if MYSQL_PWD is passed on the command line the way I told you to do in testing). Delete the .my.cnf file or correct the password in that file, and everything will work the way you are accustomed. Then you can delay learning how to work without root. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#1189 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEDOAD5ZTX6OCOC72CWLNLTA7MFRANCNFSM4XI53L3Q> .

-- Alex Jordan Mathematics Instructor Portland Community College

[drgrice1]

If all that you see in the file is

[client]
user=webworkDevWrite
password=*******

with the incorrect password, then delete the file. The only thing it is used for is to access mysql with the webworkDevWrite user and that clearly doesn't work because the password is incorrect.
If you decide to keep the file, then just fix the password. That is still only used for the purpose mentioned above, and still won't work.

[Alex-Jordan]

All I see in that file is: [client] password = ******** with no line declaring a user. This will not be an issue for me with this server (which is not a production server). Soon I'll be able to do things without acting as root. My question at this point is if we need to do anything proactively for some future person who has a .my.cnf that sets a password like this. Even if it's not for root. And even if all we do is document that .my.cnf may need changes.

…

On Fri, Feb 26, 2021 at 9:54 AM Glenn Rice ***@***.***> wrote: If all that you see in the file is [client] user=webworkDevWrite password=******* with the incorrect password, then delete the file. The only thing it is used for is to access mysql with the webworkDevWrite user and that clearly doesn't work because the password is incorrect. If you decide to keep the file, then just fix the password. That is still only used for the purpose mentioned above, and still won't work. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub, or unsubscribe.

-- Alex Jordan Mathematics Instructor Portland Community College

[drgrice1]

The safest thing to do here then, is to just add user=root above the line password=*****. Then the file won't conflict if you try to use another mysql user. What is happening is that without the user line in there, it is trying to use that password for all database users. Although, really that file shouldn't exist in / or in /root. root access to the database doesn't work that way anymore. At least not for Ubuntu.

[drdrew42]

I also have a question regarding the logic of in the OPL-update and when the two scripts update-OPL-statistics.pl and load-OPL-global-statistics.pl are run. Shouldn't the first script only be run if $ce->{problemLibrary}{showLibraryLocalStats} is true, and the second script only be run if $ce->{problemLibrary}{showLibraryGlobalStats} is true, instead of both scripts being run if either is true?

I think that the statistics scripts should run regardless, as the config settings are for displaying only -- if this data is not loaded into the db by the scripts, then toggling these configs will not work.

[drgrice1]

Perhaps that would be better. It is probably a moot point though, as both config settings are true by default in any case. Most system administrators are just going to leave those at the default values.

[drdrew42]

Worked for me.

[Alex-Jordan]

We discussed this today and Iiirc decided to go forward with merging it. But I think the discussion moved on without it actually happening.