Add an option to get all versions of rows in the /diff API

[kervel]

By default, the /diff API only returns the latest version of a row in one fetch block. When the full history is needed, now one can pass getFullLog=true to get ALL changes

see https://forum.opendatakit.org/t/retrieving-the-full-history-of-an-odk-x-table-from-the-server/21314 for more context

[wbrunette]

Thanks for the pull request @kervel! This will be helpful for some users.

This change will require some discussion at the TSC as the get "all" changes was purposefully left out because of information leakage possibilities. However, some other things have changed in sync-endpoint so a review of the threat model should be done before review and merging to see if the same concerns are still valid or if there is a better way to fix it.

[kervel]

Hello @wbrunette

Thanks for the feedback. I'm trying to give some extra context that might be relevant to the discussion on the TSC:

• our usecase is a migration tool (which we plan to release as part of odkxpy) that - i think - should be quite useful for other people outside our team: think of migrating to a new server, disaster recovery, or transferring to a new version of the form with some schema changes (so it will be possible to specify column mappings as part of a migration)

• i don't really think this change exposes more information than it already did: if you set the batchSize to 1, you will get exactly the same (albeit extremely slow, having to retrieve row by row), since the version deduplication is only done per batch (and not globally).

If needed, we can join the relevant meeting and explain our goals further.

Frank

[kervel]

in the mean time, we put the code of our migrator tool online (we still need to improve the docs a bit)
https://github.com/Kapernikov/odkxpy

for us it is a very convenient way to do migrations to new versions of forms: we can do any transformation we need, we can rewrite history if needed, ...

[wbrunette]

The ODK-X TSC will be discussing this issue at the Open Data Kit Convening in October. If you would like to discuss the issue before the convening with the TSC please join the next TSC meeting:
https://forum.opendatakit.org/t/odk-x-community-call-2019-10-08/22547