-
Notifications
You must be signed in to change notification settings - Fork 5
History
This release adds some much-needed documentation to the entire project while changing the contract of many methods. The end result is an API targeted by the Cascading stack at Etsy. In particular, use of *args is minimized, named options are well documented, but there are still some obvious warts like global registries and the tendency of job code to leak into the global namespace.
This release removes the code associated with running jobs on a Hadoop cluster as that functionality is now moved to Jading. This separates the DSL library (a gem) from the repository required to build and package job jars (Jading). Additionally, this release removes a lot of the auto-generated Rake garbage inherited from the original version and greatly streamlines the packaged gem. Finally, it starts the precedent for making the contract of operations idiomatic Ruby, removing the unnecessary Operations module, and eventually eliminating global cascade and flow registries. A non-backwards compatible change was made to the contract of complete and draw which allowed the mechnism for propagating properties through a job to be righted and made more Hadoop-idiomatic.
This release upgrades to Cascading 2.0.0 (final) and introduces Cascading local mode. Ambiguous node names are now prohibited and the insert helper correctly supports constants.
This release upgrades to Cascading 2.0.0 wip-286, but again does not implement local mode. It does, however, replace group_by/union which contain only count, sum, and average with the corresponding composite AggregateBy. Various refactorings were required to support this, including the addition of limited support for SubAssemblies.
This release upgrades to Cascading 2.0.0 wip-255, but implements neither local mode nor aggregations through AggregateBy. Local jobs continue to run in Hadoop mode, but we intend to release local mode support soon.
The primary key feature was a source of great confusion at Etsy, so it's been removed. This release also closes several issues, warns about potential node name conflicts, and updates the tests to work under JRuby 1.6.5.
This release expands upon the ExprStub class adding composition time compilation and validation of Janino expressions. This causes faulty Janino expressions to fail early rather than hours into a job.
This release merges in a good deal of work from Etsy and addresses many internal structural issues with cascading.jruby node hierarchies. Greatly expands unit tests and makes samples easily executable as a form of functional testing.
First major commit of work from Etsy that introduces scope propagation to cascading.jruby scripts in support of field propagation and primary keys. Mainly a cleanup and testing release.