Clojurians Log v2

everything merged into master!

spark2 by default; use with-profile spark1.5 to revert (hopefully any version in between is supported since we are doing “feature detection” (cough) on classes rather than relying on versions)

👍

Spark2 down, now tests and datasets

CI would be neat for running tests, personally I like CircleCI :)

If you are volunteering, you choose the solution 🙂

hum, CircleCI works with github organizations

what do you mean? don’t work for plain users?

members of github organizations can add projects in that organization to CircleCI

one signs into CircleCI through github, so they rely on github organization for access to stuff like passwords/deployment keys

In the meantime can’t you try to setup Circle CI for your fork?

yep, can do that

travis seems to have similar github project sourcing, might need to think about organization structure a bit

https://circleci.com/gh/viesti/powderkeg/1

all green 🙂

now add some dockers and figure out networking 🙂

me on docker == trying to put legos onto duplos

well, learned that Docker for mac still uses a virtualization thingy in the background so suggestions to use host networking (http://stackoverflow.com/a/32737662) don’t work

but might have figured out list of switches

what is neat in docker is it’s packaging

otherwise I’d think that one could just run some “fecth spark if not exists and run bin/start-all.sh” in circle.yml and rely in CirleCI caching (by telling it to cache the download directory) to skip downloading subsequent builds

https://github.com/viesti/powderkeg/commit/1f95eae6d6a0ab422f5c2eb37bf0ab43169ca2e4

took some configuration from https://github.com/gettyimages/docker-spark

for some reason, docker-compose up is slow on my mac so used just clojure.java.shell/sh instead

hum, maybe looking into this more in the weekend: https://circleci.com/gh/viesti/powderkeg/3