(map increase-clojure-adoption belgian-programmers)
raymcdermott 2016-05-24T05:49:36.000269Z


raymcdermott 2016-05-24T11:54:42.000270Z

I have question for the group: whatโ€™s the smoothest track into production

raymcdermott 2016-05-24T11:55:30.000271Z

best options for [ cmd line | web app | event based service | โ€ฆ. others ] ...

stijn 2016-05-24T11:56:26.000272Z

web app: heroku

raymcdermott 2016-05-24T11:57:14.000273Z


stijn 2016-05-24T11:57:24.000274Z

what do you mean by cmd line / event based service?

raymcdermott 2016-05-24T11:57:41.000275Z

cmd line โ€ฆ some batch or scheduled task

stijn 2016-05-24T11:57:42.000276Z

batch: heroku worker I guess.

raymcdermott 2016-05-24T11:58:16.000277Z

event based service โ€ฆ. something sitting behind a queue / kafka | kinesis style ingestion system

stijn 2016-05-24T11:58:30.000278Z

aws lambda? ๐Ÿ™‚

stijn 2016-05-24T11:58:35.000279Z

it runs clojure

raymcdermott 2016-05-24T11:59:02.000280Z

I read the Lambda Clojure thing โ€ฆ seems awkward ๐Ÿ˜ž

raymcdermott 2016-05-24T11:59:14.000281Z

or do you have a better experience?

stijn 2016-05-24T11:59:22.000282Z

no experience

stijn 2016-05-24T11:59:33.000283Z

since we have so many components in our backend (queues, db's, web api, batch) we are running everything in docker containers on google container engine

stijn 2016-05-24T12:00:14.000284Z

before we were running on VMs, with the necessary ceremony with ansible and supervisorctl to keep stuff running

stijn 2016-05-24T12:01:30.000285Z

but I wouldn't say it's the smoothest track into production ๐Ÿ™‚

stijn 2016-05-24T12:02:00.000286Z

although it's a lot better than the VM setup

raymcdermott 2016-05-24T12:04:29.000287Z

You have a lot going on .... But let's think hello world in each of these options and then I think it's easy to see how the infra dominates the code

raymcdermott 2016-05-24T12:04:58.000288Z

Assuming you want to say hello to everybody at once ;-)

stijn 2016-05-24T12:06:35.000289Z


raymcdermott 2016-05-24T12:07:07.000290Z

How do you keep your docker resources up to date?

stijn 2016-05-24T12:07:35.000291Z

well, event based, if you don't want to use lambda, you're going to need some compute infrastructure that watches for changes

stijn 2016-05-24T12:08:01.000292Z

docker resources as in, the VMs, docker, kubernetes?

stijn 2016-05-24T12:08:22.000293Z

or as in 'you have nginx running, now upgrade to the next version'?

raymcdermott 2016-05-24T12:09:07.000294Z

The latter (nginx, JVM, ...)

stijn 2016-05-24T12:09:24.000295Z

yes, that is still a manual work of course

stijn 2016-05-24T12:09:46.000296Z

as for the JVM, everytime a component is built it will use the latest version of a major release (now 1.8)

raymcdermott 2016-05-24T12:09:48.000297Z

I guess the platform should look after you for the other stuff

stijn 2016-05-24T12:10:16.000298Z

other components will get updated manually and go through dev > staging > prod path

raymcdermott 2016-05-24T12:10:42.000300Z

So you have some base images with common infra?

stijn 2016-05-24T12:10:46.000301Z

obviously there still is configuration management

stijn 2016-05-24T12:10:48.000302Z


stijn 2016-05-24T12:11:00.000303Z

and try to use as much as possible the official images

raymcdermott 2016-05-24T12:11:08.000304Z

That everyone 'uses' in docker speak

stijn 2016-05-24T12:11:17.000305Z


raymcdermott 2016-05-24T12:11:32.000306Z

Makes a ton of sense

stijn 2016-05-24T12:12:00.000307Z

so, e.g. our app docker container is

FROM java:openjdk-8-jre

stijn 2016-05-24T12:12:34.000308Z

and then a few lines to copy the uberjar and

ENTRYPOINT exec java $JVM_OPTS -jar /opt/your.jar

stijn 2016-05-24T12:12:46.000309Z

docker and kubernetes will watch that it keeps running

raymcdermott 2016-05-24T12:13:17.000310Z

Ok so you use openjdk - easiest by far too!

stijn 2016-05-24T12:13:26.000311Z

indeed ๐Ÿ™‚

stijn 2016-05-24T12:14:21.000312Z

and then for some stuff like queues I intend to use Google pub/sub

stijn 2016-05-24T12:14:49.000313Z

I wish datomic ran on Google Cloud Datastore

stijn 2016-05-24T12:14:56.000314Z

no need to run your own cassandra and stuff

raymcdermott 2016-05-24T12:16:08.000315Z

yeah - and itโ€™s annoying that they donโ€™t enable storage extension points

stijn 2016-05-24T12:16:13.000316Z


raymcdermott 2016-05-24T12:16:25.000317Z

cos it seems quite simple

stijn 2016-05-24T12:16:48.000318Z

I don't think the requirements for the underlying storage are complex indeed

stijn 2016-05-24T12:17:01.000319Z

you just need some transactionality to store the pointer to the indexes

raymcdermott 2016-05-24T12:17:21.000320Z

I would love them to provide a protocol that could be implemented

raymcdermott 2016-05-24T12:18:14.000321Z

anyway, off-topic ๐Ÿ™‚

raymcdermott 2016-05-24T12:18:58.000322Z

how do you determine the size of the compute resource on the Google engine?

stijn 2016-05-24T12:19:19.000323Z

container engine?

stijn 2016-05-24T12:19:48.000324Z

we knew what our app was consuming in terms of resources on a VM

stijn 2016-05-24T12:20:13.000325Z

so, we took that, increased it a bit and spun up 4 hosts ๐Ÿ™‚

stijn 2016-05-24T12:20:33.000326Z

you can increase the resources by updating the spec of the 'instance group'

stijn 2016-05-24T12:20:49.000327Z

so, you shouldn't be tied to what you have chosen

stijn 2016-05-24T12:21:14.000328Z

I think they are also working on a 'mixed' cluster with e.g. CPU heavy and MEM heavy VMs

stijn 2016-05-24T12:21:31.000329Z

and you can tie certain components of your app to a specific type of host

raymcdermott 2016-05-24T13:02:08.000330Z

thatโ€™s nice

raymcdermott 2016-05-24T13:03:21.000331Z

and I guess it attaches the equivalent of EBS to the instances so that compute is swappable?

stijn 2016-05-24T13:05:16.000332Z


stijn 2016-05-24T13:05:39.000333Z

well, the thing is, you should not use the default disks attached to the instances

stijn 2016-05-24T13:06:18.000334Z

kubernetes will manage volumes for you. you just specify which container needs which volume mounted where and it will do that on the right host where the container is running

stijn 2016-05-24T13:06:44.000335Z

you can use all kind of volumes but google persistent disk is a good choice for not losing data on cluster upgrades

raymcdermott 2016-05-24T13:17:51.000337Z

side question - on something else Iโ€™m looking at - do you know how easy it is to resize the volumes (itโ€™s a pain on EBS)

raymcdermott 2016-05-24T13:30:45.000338Z

I have found a decent example from the uswitch guys to implement AWS Lambda

raymcdermott 2016-05-24T13:30:56.000339Z

raymcdermott 2016-05-24T13:31:26.000341Z

But when you see their example you see the yuckiness of the AWS Lambda API

raymcdermott 2016-05-24T13:33:03.000345Z

I mean holy crap, you have to return the status via this bullshit JSON

raymcdermott 2016-05-24T13:34:24.000346Z

I think this example could be wrapped nicer (I will make a PR for that but it doesnโ€™t hide the fact)

stijn 2016-05-24T14:12:42.000347Z

resizing a disk is easy, but only possible when the resulting disk is bigger than the old one