Clojurians Log v2

We use gitlab for our build process, which includes the container build and deploy to kube

we have a CI container that has all the deps already built into it that looks like this:

FROM clojure
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY project.clj /usr/src/app/
RUN lein deps

and then a Base container that contains any other things you might need to bundle in, starting with: FROM java:openjdk-8-jre This is then stored at: <http://registry.gitlab.com/projectname/appname:base-image|registry.gitlab.com/projectname/appname:base-image>

and then the actual Docker file that builds off those is very simple, like this:

FROM <http://registry.gitlab.com/projectname/appname:base-image|registry.gitlab.com/projectname/appname:base-image>

COPY target/standalone.jar standalone.jar

ENV PORT 8080

EXPOSE 8080

CMD ["java", "-jar", "standalone.jar"]

then build is literally just lein uberjar from inside the CI container.

you can similarly run lein test.

@mobileink ☝️ Sorry I didn’t see the question before…

yikes. i am so 2012 i have no docker clue. how does this all work in the dev cycle? can you do something like a repl?

So the dev cycle is all as normal. For us it’s still simple in that we dev locally, so the repl is local too.

basically, repl, code, test, then when you commit, it kicks this build off.

then we have a deploy step that builds the container and deployes it to app engine.

fwiw i'm the https://github.com/migae/boot-gae guy. the problem with gae is interactive dev. how do you solve that?

yeah, I was looking at that while doing this. I nearly went there, but went with this pattern that we knew.

so, because we’re just bundling an uberjar, I don’t even bother with app engine until deployment.

so it’s just like interactive dev cycles with any ring app

(running the server locally)

we use a model similar to component for hot reloading of backends, and figwheel for the cljs bits)

so you run your own server for dev, rather than the gae dev server? i'm confused.

you use gae services (e.g. datastore, memcache, etc.)?

@domparry can you post your code?

ah.

sure. For this service, it’s very simple… We’re kicking off Cloud dataflow jobs.

we have a simple handler like this:

(ns fulliautomatix.handler
  (:require [fulliautomatix
             [datastore-etl :as be]
             [callcentre-leads :as cl]]
            [compojure.core :as cc]
            [compojure.handler :as handler]
            [compojure.route :as route]
            [ring.middleware.keyword-params :as rparams]
            [clojure.tools.logging :as log]
            [ring.adapter.jetty :as jetty])
  (:gen-class))

(defn wrap-result [result]
  (str "Dataflow Job Submitted."))

(defn run-datastore-etl [args headers]
 (let [{:keys [namespace kind filterField filterValue]} args
       result (be/run-main namespace kind filterField filterValue)]
   (wrap-result result)))

(defn run-callcentre-leads [_ _]
  (wrap-result (cl/run-main nil)))

(defn secure-route [function params headers]
  (if (get headers "x-appengine-cron")
    (function params headers)
    (str "Request not from App Engine Cron service")))

(cc/defroutes app-routes
 (cc/GET "/" []
         (str "healthy"))
 (cc/GET "/jobs/etl" {params :params headers :headers}
      (secure-route run-datastore-etl params headers))
 (cc/GET "/_ah/warmup" []
         (str "healthy"))
 (cc/GET "/_ah/health" []
         (str "healthy"))
 (cc/GET "/jobs/callcentre-leads" {params :params headers :headers}
      (secure-route run-callcentre-leads params headers))
 (route/not-found "Not Found"))

(def app
  (->
   (handler/site app-routes)
   (rparams/wrap-keyword-params)))

(defn -main [& [port]]
  (let [port (Integer. (or port
                           (System/getenv "PORT")
                           5000))]
    (jetty/run-jetty #'app {:port  port
                            :join? false})))

this then calls some code that set’s up the dataflow pipeline and runs it.

you're using flexible env, not standard env?

in std env,

(jetty/run-jetty #'app {:port port :join? false})))

yes, we’re in flexible env.

we run all our other stuff in Container Engine, and build it the same way.

hence not really worth redoing the build pipeline just for the simple app to work in Standard Env