flambo

sorenmacbeth 2016-08-20T17:48:59.000007Z

@markusk: the current snapshot version works with spark 2.0

sorenmacbeth 2016-08-20T17:51:12.000008Z

which is to say, the tests pass

markusk 2016-08-20T18:07:53.000009Z

@sorenmacbeth: Thanks! Does that mean they don't for 0.7.2? How stable is the current snapshot?

sorenmacbeth 2016-08-20T18:13:58.000010Z

0.7.2 isn’t compatible with spark 2.0

sorenmacbeth 2016-08-20T18:16:23.000011Z

@markusk: It’s stable insofar as I can test it. I don’t have a 2.0 cluster yet. If you want to give 0.8.0-SNAPSHOT a whirl and file any issues you run into that would be wizard

markusk 2016-08-20T18:57:56.000013Z

@sorenmacbeth: We'll give it a shot.

markusk 2016-08-20T18:58:31.000014Z

@sorenmacbeth: Thanks for the updates! We might have some pull requests for you if we run into issues.

sorenmacbeth 2016-08-20T18:58:52.000015Z

@markusk: that would be much appreciated

markusk 2016-08-20T19:00:11.000016Z

@sorenmacbeth: Btw, as you might know, EMR 5.0.0 on AWS supports Spark 2.0, so if you'd like your own test environment, you can get one pretty cheap and easy. https://aws.amazon.com/blogs/aws/amazon-emr-5-0-0-major-app-updates-ui-improvements-better-debugging-and-more/

sorenmacbeth 2016-08-20T19:06:33.000017Z

@markusk: I can set one up on some of yieldbot’s mesos clusters, it’s just a matter of having time to do it 😕

markusk 2016-08-20T19:06:57.000018Z

@sorenmacbeth I hear you.

sorenmacbeth 2016-08-20T19:07:02.000019Z

sooner or later I will migrate our production cluster to 2.0.0

sorenmacbeth 2016-08-20T19:07:40.000021Z

waiting for 2.0.1 probably

sorenmacbeth 2016-08-20T19:09:38.000022Z

I expect it to be a fair bit faster with kryo 3

markusk 2016-08-20T19:10:40.000023Z

@sorenmacbeth: Eagerly awaiting next release, we'll experiment with the snapshot in the meantime.

👍 2