@markusk: the current snapshot version works with spark 2.0
which is to say, the tests pass
@sorenmacbeth: Thanks! Does that mean they don't for 0.7.2? How stable is the current snapshot?
0.7.2 isn’t compatible with spark 2.0
@markusk: It’s stable insofar as I can test it. I don’t have a 2.0 cluster yet. If you want to give 0.8.0-SNAPSHOT
a whirl and file any issues you run into that would be wizard
@sorenmacbeth: We'll give it a shot.
@sorenmacbeth: Thanks for the updates! We might have some pull requests for you if we run into issues.
@markusk: that would be much appreciated
@sorenmacbeth: Btw, as you might know, EMR 5.0.0 on AWS supports Spark 2.0, so if you'd like your own test environment, you can get one pretty cheap and easy. https://aws.amazon.com/blogs/aws/amazon-emr-5-0-0-major-app-updates-ui-improvements-better-debugging-and-more/
@markusk: I can set one up on some of yieldbot’s mesos clusters, it’s just a matter of having time to do it 😕
@sorenmacbeth I hear you.
sooner or later I will migrate our production cluster to 2.0.0
waiting for 2.0.1 probably
I expect it to be a fair bit faster with kryo 3
@sorenmacbeth: Eagerly awaiting next release, we'll experiment with the snapshot in the meantime.