Release new version

[ftomassetti]

I know there are many refactorings still going on but perhaps we could release a new version because there are already a lot of improvements (over 60 commits since v0.18.0 v0.18.0...master).

What do you think @psi29a ?

[tcld]

Even though I wasn't explicitly asked:
I think it would be good to get at least two more things out of the way first.

1. Removing pickle
This might simplify installation and, if done at a later point, could break compatibility with old saves. Better get it done as soon as possible (or rule it out completely).
I think that pickle-saves would not transfer well between Python 2 and 3, by the way.

2. More changes to seeding different algorithms
As stated here #133 the current solution does not seem 100% deterministically stable. Therefore changes might come up at some point that make old worlds "unregeneratable". I don't know yet how, but if I could polish that branch so that those concerns will never crop up again, that would be really nice. (Help is appreciated since so far I have not come up with a clean solution.)

Aside from those things a new release would be great! Especially for users under Windows it is probably quite painful to get the latest version running. Giving them an easy-to-use release with some incredible speed-boosts (50-66% less time needed than before) on top would certainly be really good. :)

[psi29a]

We can re-write the pickle tests to only run based on python version. That way we can keep pickle and compare apples to apples as a result. Not that I like pickle, I'm just not 100% sold on protobuf yet. :)

[tcld]

protobuf is a Google-thing, isn't it? They tend to radically change things without hesitation, so I am kind of with you on that.^^

Do you know what would have to be changed to make pickle work properly?

[psi29a]

Nothing really, just that world files created with python2 should only be compared on python2. Same with python3, the resulting file should only be read by python3. I suggest in our current tests to add a check on python version:

    @unittest.skipIf(mylib.__version__ < (1, 3),
                     "not supported in this library version")
    def test_format(self):
        # Tests that work for only a certain version of the library.
        pass

https://docs.python.org/2/library/unittest.html#skipping-tests-and-expected-failures

That is an example of course, the @blahblah are decorators that we can use.

We can then write tests that are specific to a version.

We can nuke pickle and the tests when we are comfortable with our new 'stable' format.

In the next release, we can make protobuf the default and see what breaks for people. :)

The release after that, we can consider removing pickle.

[tcld]

That is a great solution. Once Google releases a proper version 3.0 of protobuf (not an alpha) we will at least have that as a fallback, should they ever decide to completely change directions on it. But until then supporting both formats sounds like a very good idea.
(How about a custom format, by the way? I don't even know what things are really saved, but I loved to come up with custom file-formats in C++.^^)

[ftomassetti]

Probably next version should be 0.19.0: there are a lot of improvements!

[tcld]

To come back to this:

 @unittest.skipIf(mylib.__version__ < (1, 3),
                     "not supported in this library version")
    def test_format(self):
        # Tests that work for only a certain version of the library.
        pass

Would this not be equally good, maybe even better? This way the functionality that is the same under Python 2 and 3 can easily be reused.

diff --git a/tests/blessed_images/generated_blessed_images.py b/tests/blessed_images/generated_blessed_images.py
index 913ccbe..0d070b2 100644
--- a/tests/blessed_images/generated_blessed_images.py
+++ b/tests/blessed_images/generated_blessed_images.py
@@ -8,6 +8,7 @@ A script, generate_blessed_images, can be used to regenerate blessed images
 """

 import os
+import platform
 from worldengine.world import *
 from worldengine.draw import *

@@ -26,7 +27,10 @@ def main(blessed_images_dir, tests_data_dir):
     draw_temperature_levels_on_file(w, "%s/temperature_28070.png" % blessed_images_dir)
     draw_biome_on_file(w, "%s/biome_28070.png" % blessed_images_dir)

-    w_large = World.from_pickle_file("%s/seed_48956.world" % tests_data_dir)
+    if platform.python_version_tuple()[0] < 3:
+        w_large = World.from_pickle_file("%s/py2_seed_48956.world" % tests_data_dir)
+    else:
+        w_large = World.from_pickle_file("%s/py3_seed_48956.world" % tests_data_dir)
     draw_ancientmap_on_file(w, "%s/ancientmap_28070_factor3.png" % blessed_images_dir, resize_factor=3)
     draw_ancientmap_on_file(w_large, "%s/ancientmap_48956.png" % blessed_images_dir, resize_factor=1)

diff --git a/tests/data/data_generator.py b/tests/data/data_generator.py
index a47dcee..5793ad0 100644
--- a/tests/data/data_generator.py
+++ b/tests/data/data_generator.py
@@ -8,12 +8,16 @@ because the plate simulation steps do not provide the same results on all the pl
 """

 import os
+import platform
 from worldengine.plates import _plates_simulation


 def main(tests_data_dir):
     w = _plates_simulation("Foo", 300, 200, 279)
-    w.to_pickle_file("%s/plates_279.world" % tests_data_dir)
+    if platform.python_version_tuple()[0] < 3:
+        w.to_pickle_file("%s/py2_plates_279.world" % tests_data_dir)
+    else:
+        w.to_pickle_file("%s/py3_plates_279.world" % tests_data_dir)


 if __name__ == '__main__':
diff --git a/tests/drawing_functions_test.py b/tests/drawing_functions_test.py
index 1a6775d..a18e676 100644
--- a/tests/drawing_functions_test.py
+++ b/tests/drawing_functions_test.py
@@ -1,4 +1,5 @@
 import unittest
+import platform

 from worldengine.drawing_functions import draw_ancientmap, gradient, draw_rivers_on_image
 from worldengine.world import World
@@ -12,7 +13,10 @@ class TestDrawingFunctions(TestBase):
         self.w = World.open_protobuf("%s/seed_28070.world" % self.tests_data_dir)

     def test_draw_ancient_map_factor1(self):
-        w_large = World.from_pickle_file("%s/seed_48956.world" % self.tests_data_dir)
+        if platform.python_version_tuple()[0] < 3:
+            w_large = World.from_pickle_file("%s/py2_seed_48956.world" % self.tests_data_dir)
+        else:
+            w_large = World.from_pickle_file("%s/py3_seed_48956.world" % self.tests_data_dir)
         target = PixelCollector(w_large.width, w_large.height)
         draw_ancientmap(w_large, target, resize_factor=1)
         self._assert_img_equal("ancientmap_48956", target)
diff --git a/tests/generation_test.py b/tests/generation_test.py
index 8ad27d3..2142f9c 100644
--- a/tests/generation_test.py
+++ b/tests/generation_test.py
@@ -1,4 +1,5 @@
 import unittest
+import platform
 from worldengine.plates import Step, center_land, world_gen
 from worldengine.world import World

@@ -29,7 +30,10 @@ class TestGeneration(TestBase):
         return borders_total_elevation / n_cells_on_border

     def test_center_land(self):
-        w = World.from_pickle_file("%s/plates_279.world" % self.tests_data_dir)
+        if platform.python_version_tuple()[0] < 3:
+            w = World.from_pickle_file("%s/py2_plates_279.world" % self.tests_data_dir)
+        else:
+            w = World.from_pickle_file("%s/py3_plates_279.world" % self.tests_data_dir)

         # We want to have less land than before at the borders
         el_before = TestGeneration._mean_elevation_at_borders(w)

If that was ok, I would immediately generated the Python 3 file, rename the Python 2 file, create a PR for this patch and all pickle-related problems might be delayed for a bit. :P

[tcld]

Here is a different version, prettier in my opinion.

diff --git a/tests/blessed_images/generated_blessed_images.py b/tests/blessed_images/generated_blessed_images.py
index 913ccbe..d3d03b6 100644
--- a/tests/blessed_images/generated_blessed_images.py
+++ b/tests/blessed_images/generated_blessed_images.py
@@ -8,6 +8,7 @@ A script, generate_blessed_images, can be used to regenerate blessed images
 """

 import os
+import platform
 from worldengine.world import *
 from worldengine.draw import *

@@ -26,7 +27,7 @@ def main(blessed_images_dir, tests_data_dir):
     draw_temperature_levels_on_file(w, "%s/temperature_28070.png" % blessed_images_dir)
     draw_biome_on_file(w, "%s/biome_28070.png" % blessed_images_dir)

-    w_large = World.from_pickle_file("%s/seed_48956.world" % tests_data_dir)
+    w_large = World.from_pickle_file("%s/py%s_seed_48956.world" % (tests_data_dir, platform.python_version_tuple()[0]))
     draw_ancientmap_on_file(w, "%s/ancientmap_28070_factor3.png" % blessed_images_dir, resize_factor=3)
     draw_ancientmap_on_file(w_large, "%s/ancientmap_48956.png" % blessed_images_dir, resize_factor=1)

diff --git a/tests/data/data_generator.py b/tests/data/data_generator.py
index a47dcee..2d627c3 100644
--- a/tests/data/data_generator.py
+++ b/tests/data/data_generator.py
@@ -8,12 +8,13 @@ because the plate simulation steps do not provide the same results on all the pl
 """

 import os
+import platform
 from worldengine.plates import _plates_simulation


 def main(tests_data_dir):
     w = _plates_simulation("Foo", 300, 200, 279)
-    w.to_pickle_file("%s/plates_279.world" % tests_data_dir)
+    w.to_pickle_file("%s/py%s_plates_279.world" % (tests_data_dir, platform.python_version_tuple()[0]))


 if __name__ == '__main__':
diff --git a/tests/drawing_functions_test.py b/tests/drawing_functions_test.py
index 1a6775d..91600a5 100644
--- a/tests/drawing_functions_test.py
+++ b/tests/drawing_functions_test.py
@@ -1,4 +1,5 @@
 import unittest
+import platform

 from worldengine.drawing_functions import draw_ancientmap, gradient, draw_rivers_on_image
 from worldengine.world import World
@@ -12,7 +13,7 @@ class TestDrawingFunctions(TestBase):
         self.w = World.open_protobuf("%s/seed_28070.world" % self.tests_data_dir)

     def test_draw_ancient_map_factor1(self):
-        w_large = World.from_pickle_file("%s/seed_48956.world" % self.tests_data_dir)
+        w_large = World.from_pickle_file("%s/py%s_seed_48956.world" % (self.tests_data_dir, platform.python_version_tuple()[0]))
         target = PixelCollector(w_large.width, w_large.height)
         draw_ancientmap(w_large, target, resize_factor=1)
         self._assert_img_equal("ancientmap_48956", target)
diff --git a/tests/generation_test.py b/tests/generation_test.py
index 8ad27d3..0a16b4a 100644
--- a/tests/generation_test.py
+++ b/tests/generation_test.py
@@ -1,4 +1,5 @@
 import unittest
+import platform
 from worldengine.plates import Step, center_land, world_gen
 from worldengine.world import World

@@ -29,7 +30,7 @@ class TestGeneration(TestBase):
         return borders_total_elevation / n_cells_on_border

     def test_center_land(self):
-        w = World.from_pickle_file("%s/plates_279.world" % self.tests_data_dir)
+        w = World.from_pickle_file("%s/py%s_plates_279.world" % (self.tests_data_dir, platform.python_version_tuple()[0]))

         # We want to have less land than before at the borders
         el_before = TestGeneration._mean_elevation_at_borders(w)

[psi29a]

If the pickle can be written and read by both Py2 and Py3, then that is also great IMHO.

@ftomassetti we ok with replacing the blessed images? Did you want to eyeball them first?

[tcld]

I hope you noticed this:

"%s/py%s_plates_279.world" % (self.tests_data_dir, platform.python_version_tuple()[0])

Regarding the "new" blessed images: They are probably so different because I fed different parameters to platec.

[psi29a]

The blessed images are just the output created from the world files... 'blessed' is all data that goes into worldengine-data. ;)

Ah, now that you mentioned it... building it into the name is rather handy. :)

[ftomassetti]

The idea of the "blessed" images is that any time you change them you look at them and approve them to be used in tests after "visual inspection". In other words you have to check they looks good and give to them your blessing.

[ftomassetti]

86 commits and 43 files changed so far. I would definitely go for v 0.19.0: this is a huge improvement over v 0.18.0!
I would love to release what has been done throught these great contributions (thanks @tcld and @esampson !) and share it.

[tcld]

What kind of timeframe are you thinking about?