Skip to content

WeBWorK-2.15-beta#948

Merged
mgage merged 378 commits into
masterfrom
WeBWorK-2.15
Oct 6, 2019
Merged

WeBWorK-2.15-beta#948
mgage merged 378 commits into
masterfrom
WeBWorK-2.15

Conversation

@mgage

@mgage mgage commented May 25, 2019

Copy link
Copy Markdown
Member

This is a version containing multilingual features (utf8 4 byte characters). There are database changes so test cautiously (backup up the database just in case). If $ENABLE_UTF8MB4=0 you should be able to use this on existing sites, even with mysql databases earlier than 5.3. If you set $ENABLE_UTF8MB4=1 then you will get a "wide character" error message from the database if it is too old. If it is new enough you will be able to use all characters including Chinese, emoji, Arabic and so forth. (Download the companion PG-2.15-beta as well.)

taniwallach and others added 30 commits March 6, 2019 17:33
To allow the 1000 byte limit for keys, the OPL_local_statistics was
changed to use ENGINE=MyISAM. That change allowed reducing the
source_file column from varchar(255) to varchar(245), while the
767 bytes key limit of ENGINE=InnoDB as was used by this table and
by OPL_global_statistics.sql (at present) would have limited this to
about 191 characters.

The OPL_problem_user table had 3 keys changed from using 100 characters
each to using only 80 characters each. The same changes were already
made to lib/WeBWorK/DB/Record/PastAnswer.pm for the same reason. Only
the first 245 characters of source_file are used as a key. Finally,
the CHARACTER SET of OPL_problem_user was explicitly changed from ascii
to utf8mb4.

The saved version of OPL_global_statistics.sql needs to be hand modified
as follows, until the GitHub master branch has these changes made. Only
after those changes can the file be loaded into a utf8mb4 database.
        utf8    => utf8mb4
        255     => 245
  ENGINE=InnoDB => ENGINE=MyISAM
=================================

10c10
< /*!40101 SET NAMES utf8mb4 */;
---
> /*!40101 SET NAMES utf8 */;
24c24
< /*!40101 SET character_set_client = utf8mb4 */;
---
> /*!40101 SET character_set_client = utf8 */;
26c26
<   `source_file` varchar(245) NOT NULL,
---
>   `source_file` varchar(255) NOT NULL,
31c31
< ) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4;
---
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Note: When I tried to include XML::Simple near the start of the first
"cpanm install" line, there was an error:
   Building and testing XMLRPC-Lite-0.717 ... ! Installing XMLRPC::Lite failed.
   See /root/.cpanm/work/1551887935.125/build.log for details.
   Retry with --force to force install it.
so it was put into a second "cpanm install" line.
…2019_utf8mb4

Tani develop candidate 02 24 2019 utf8mb4
Fix the union toolbar button to show the union, rather than the letter
U.  It will still put the letter U into the actual webwork answer box.
Add a subscript toolbar button.
Clean up the appearance of the toolbar a bit.
…le. (Not currently possible in read_whole_file inside the Safe compartment.
…3_10_2019

# Conflicts:
#	lib/Apache/WeBWorK.pm
Change FileManager to be compatible with UTF-8
implement handlers that notify when a TextBlock gains and loses focus.
These handlers are then used to disable the toolbar when a TextBlock is
entered, and re-enable the toolbar when a TextBlock is exited.  With
this we can switch back to using the 'cmd' method of MathQuill rather
than the 'typedText' method.  The 'cmd' method works better on latex
commands.  Particularly if there is a selection in the MathField.

Remove the subscript toolbar button.  At this point it seems there
isn't anything in webwork that works with.

Finally, remove the charsThatBreakOutOfSupSub and autoSubscriptNumerals
options.
When set to zero this can be used with mysql before 5.3 (they cannot handle utf8mb4), when set to 1
OPL scripts will build a utf8mb4 version of the OPL.

$ENABLE_UTF8MB4 =1;
Removed localOverrides.conf.dist section on setting ENABLE_UTF8MB4.  There was potential for confusion
because changing the value in site.conf would not change behavior.
…ocess from dying when htdocs/tmp directory doesn't exist.
…ocess from dying when htdocs/tmp directory doesn't exist.

install XMLRPC::Lite from ubuntu source rather than CPAN
…ides.conf in order to over ride the default in default.config

The configuration in site.conf is always overridden by the one in default.config
mgage and others added 14 commits September 26, 2019 17:16
Code to support adding additional CSS files requested by problems
Check for proper action (non-capitalized) in getFile().
intended for use in an LTI course where automatic LTI account
creation is not desired, ex. after the end of a semester if new
accounts may potentially be added to the LMS. The setting
	$block_lti_create_user = 1;
can be added to the course.conf file.
…dXMLRPC

Webwork 2.15 missing fixes to send xmlrpc
…er_setting

Add handling for a new LTI setting: $block_lti_create_user
	encodeAnswers() and decodeAnswers()
which are used to encode/decode the last_answer field in the
COURSEID_problem_user table to use JSON instead of
	Storable::nfreeze() and Storable::thaw()
which failed to properly handle UTF-8 data in answers, or
many answers with over 127 characters (due to the binary length
code appearing to be an invalid UTF-8 character).

The decodeAnswers() falls back to the OLD Storable::thaw()
based code for backwards compatibility when the value returned
from the database does not seem to be a JSON array (currently
detected by checking for a leading '[' and a trailing ']')
while the Storable::nfreeze() seems to always start with
several control characters.

The new code has been tested, and stores long answer strings,
answer strings with UTF-8 characters, and prior last_answer data
from the old format seems to load properly in all cases tested.
Change the serialization/deserialization code for the last_answer database field
@taniwallach

Copy link
Copy Markdown
Member

As far as I can tell there are no pending PRs or major issues for either pg or webwork2 remaining to hold off the release of 2.15.

The following "minor" issues may need to be addressed with hotfixes:

@taniwallach taniwallach added essentially ready to merge and removed NeedsTesting Tentatively fixed bug or implemented feature labels Oct 4, 2019
@mgage

mgage commented Oct 4, 2019

Copy link
Copy Markdown
Member Author

@drgrice1 @taniwallach @Alex-Jordan @heiderich @drdrew42 @drjt @dpvc @dlglin
I think this and the corresponding pg repos are ready to merge. Are there any objections?

  • Mike

@Alex-Jordan

Copy link
Copy Markdown
Contributor

I'm not aware of any reason to hold back. I can think of a few loose threads that are waiting now for 2.16.

Thank you to all of you who have been working so hard on this release. I've only been involved with WW since about edition 2.7, but this seems to me to have been the heaviest lift so far, with some new features that have been years in the making, plus lots and lots of testing and debugging. Congratulations on reaching the end.

Comment thread PG_VERSION Outdated
$PG_VERSION ='2.15'; #beta!!!
$PG_COPYRIGHT_YEARS = '1996-2019';

# this file is not being used (I believe)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a comment here that this file is not being used. IIRC, because of something related to Docker, something is in place that assumes the PG and WW versions match. But that is not always true. I hope something is in place that faithfully displays which versions of PG and WW are being used on a given server.

@drgrice1 drgrice1 Oct 5, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am pretty certain that comment is correct. Also, see line 33-34 of webwork2/conf/defaults.config and lines 220-243 of webwork2/lib/WeBWorK/CourseEnvironment.pm. The PG version is now obtained from pg/VERSION.

@mgage

mgage commented Oct 6, 2019 via email

Copy link
Copy Markdown
Member Author

taniwallach and others added 6 commits October 6, 2019 18:43
…ide changes from working if not problem specific css file load was in use
$ce->{PG_VERSION} is now obtained directly from the file pg/VERSION via a special exception which
allows CourseEnvironment to read that file.  (Normally CourseEnvironment only has access to files in webwork2. )
Remove !!beta
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

essentially ready to merge Multilingual Part of the Multilingual (localization) project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants