Clojurians Log v2

$ lein ring server-headless
2020-10-06 12:36:28.980:INFO::main: Logging initialized @1474ms
Exception in thread "main" Syntax error compiling at (/private/var/folders/9d/3zkfrpnj15j6j2qnr53jqsmw0000gn/T/form-init2589101478955306490.clj:1:125).
	at clojure.lang.Compiler.load(Compiler.java:7647)
	at clojure.lang.Compiler.loadFile(Compiler.java:7573)
	at clojure.main$load_script.invokeStatic(main.clj:452)
	at clojure.main$init_opt.invokeStatic(main.clj:454)
	at clojure.main$init_opt.invoke(main.clj:454)
	at clojure.main$initialize.invokeStatic(main.clj:485)
	at clojure.main$null_opt.invokeStatic(main.clj:519)
	at clojure.main$null_opt.invoke(main.clj:516)
	at clojure.main$main.invokeStatic(main.clj:598)
	at clojure.main$main.doInvoke(main.clj:561)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:37)
Caused by: java.lang.AssertionError: Assert failed: self-referential dependency
(not (= x dep))
	at ns_tracker.dependency$depend.invokeStatic(dependency.clj:54)
	at ns_tracker.dependency$depend.invoke(dependency.clj:48)
	at ns_tracker.dependency$depend$fn__1319.invoke(dependency.clj:59)
	at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
	at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
	at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169)
	at clojure.core.protocols$fn__8144.invoke(protocols.clj:124)
	at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19)
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
	at clojure.core.protocols$fn__8129.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__8129.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6828)
	at clojure.core$reduce.invoke(core.clj:6810)
	at ns_tracker.dependency$depend.invokeStatic(dependency.clj:59)
	at ns_tracker.dependency$depend.doInvoke(dependency.clj:48)
	at clojure.lang.RestFn.applyTo(RestFn.java:146)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$apply.invoke(core.clj:660)
	at ns_tracker.core$add_to_dep_graph$fn__1784.invoke(core.clj:53)
	at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
	at clojure.core$reduce.invokeStatic(core.clj:6827)
	at clojure.core$reduce.invoke(core.clj:6810)
	at ns_tracker.core$add_to_dep_graph.invokeStatic(core.clj:50)
	at ns_tracker.core$add_to_dep_graph.invoke(core.clj:49)
	at ns_tracker.core$update_dependency_graph.invokeStatic(core.clj:62)
	at ns_tracker.core$update_dependency_graph.invoke(core.clj:59)
	at ns_tracker.core$ns_tracker.invokeStatic(core.clj:90)
	at ns_tracker.core$ns_tracker.invoke(core.clj:79)
	at ns_tracker.core$ns_tracker.invokeStatic(core.clj:84)
	at ns_tracker.core$ns_tracker.invoke(core.clj:79)
	at ring.middleware.reload$reloader.invokeStatic(reload.clj:8)
	at ring.middleware.reload$reloader.invoke(reload.clj:7)
	at ring.middleware.reload$wrap_reload.invokeStatic(reload.clj:34)
	at ring.middleware.reload$wrap_reload.invoke(reload.clj:21)
	at ring.server.standalone$add_auto_reload.invokeStatic(standalone.clj:61)
	at ring.server.standalone$add_auto_reload.invoke(standalone.clj:59)
	at ring.server.standalone$add_middleware.invokeStatic(standalone.clj:72)
	at ring.server.standalone$add_middleware.invoke(standalone.clj:69)
	at ring.server.standalone$serve.invokeStatic(standalone.clj:95)
	at ring.server.standalone$serve.doInvoke(standalone.clj:75)
	at clojure.lang.RestFn.invoke(RestFn.java:423)
	at ring.server.leiningen$serve.invokeStatic(leiningen.clj:19)
	at ring.server.leiningen$serve.invoke(leiningen.clj:16)
	at user$eval4205.invokeStatic(form-init2589101478955306490.clj:1)
	at user$eval4205.invoke(form-init2589101478955306490.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:7176)
	at clojure.lang.Compiler.eval(Compiler.java:7166)
	at clojure.lang.Compiler.load(Compiler.java:7635)
	... 12 more
Subprocess failed (exit code: 1)

I am clueless as this stack trace points no where in my code. Not sure whats wrong. Any help is appreciated. Thanks

Answering my own question in the hope it will be helpful to others. Turns out two of my clojure files were having save namespace. Apparently ring checks all the clojure files to make sure everything is alright.

For future reference, you can see that in the stacktrace:

...
Caused by: java.lang.AssertionError: Assert failed: self-referential dependency
(not (= x dep))
	at ns_tracker.dependency$depend.invokeStatic(dependency.clj:54)
...
	at ns_tracker.core$ns_tracker.invoke(core.clj:79)
	at ring.middleware.reload$reloader.invokeStatic(reload.clj:8)

My recommendation would be: 1) do not use the ring plugin for lein -- learn how to start Jetty from your -main and use lein run to start it 2) do not use reloading middleware -- learn to develop a "REPL-driven" workflow so you don't need any of these auto-reload/refresh plugins etc Both of these will make you a better Clojure developer and you'll be relying less on "magic" -- when those "easy" things go wrong, you won't know how to fix them; if you can do this without the "magic" pieces, you'll have more control and more insight into the process.

In particular, if you had a tight REPL-driven workflow, you would probably have caught the namespace problem immediately, in your editor, via the REPL -- before even trying to run the code from the command-line.

I also concur with Sean about the lein-ring plugin, it's quite simple to define a sever that can be managed by the repl in just a few lines of simple code https://practicalli.github.io/clojure-webapps/app-servers/simple-restart.html#code-example

Using this kind approach approach should make it easier to understand libraries like mount, integrant for managing the lifecycle of services (database, app server, etc)

And if you do want to "look ahead" at some simple web apps that avoid lein-ring (in fact, avoid Leiningen altogether), that use Component, Compojure or Integrant, Reitit, along with some database access and and HTML templating, take a look at https://github.com/seancorfield/usermanager-example and https://github.com/PrestanceDesign/usermanager-reitit-integrant-example

Is it normal to see a DevTools message i prod?

Is it normal to see a DevTools message i prod? How can I resolve this?

DevTools failed to load SourceMap: Could not parse content 
for 
<https://xxx/js/Loading.js.map>: 
Unexpected token &lt; in JSON at position 0

I also have this error. I am stuck here.

Warning: React.createElement: type is invalid -- expected a 
string (for built-in components) or a class/function (for 
composite components) but got: undefined. You likely forgot 
to export your component from the file it's defined in, or 
you might have mixed up default and named imports.

Check the render method of `lba`.
in lba (created by mba)
in Suspense (created by M4)
in M4 (created by mba)
in mba (created by $$)
in $$
in Unknown (created by class_1)
in class_1

@flora.makgaba "Unexpected token < in JSON" sounds like the HTTP request for the source map is returning an HTML page, but I don't know what "DevTools" you're referring to. If the source map isn't available, I'd expect a 404 Not Found error so perhaps your apps is not set up correctly? No idea about React. Depending on what specific tech you are using, there may be a more relevant channel to ask than here?

we're discussing in this thread: https://app.slack.com/client/T03RZGPFR/C03S1L9DN/thread/C03S1L9DN-1601960028.287200

in #clojurescript

Thanks. They also posted on StackOverflow (but no one has answered yet).

Thanks guys I will keep these in mind.

Hi all. I am looking into configuration management - blending environmental variables, defined configuration files and system property files, perhaps command-line stuff - both properties (-D) and maybe arguments. There are many libraries here, which is fine for a single application, but what about when building libraries themselves? My initial thought on this, as a clojure beginner, is to build some kind of facade config pattern to decouple getting configuration data from the way it is set when used in a specific application? The facade simply presents configuration data as a map, obviously… but how that map is initialised would then depend on runtime - and not whether one uses mount or component or a specific config library or something else… A bit like tools.logging / slf4j facade and then a concrete implementation? Or is this a crazy thing and am I overthinking this?

@mark354 you are correct to want a map as config. I would decouple the config data from the components themselves. The components should never "reach outwards" to discover config, config should be provided as an argument to the component's constructors

Thanks. That’s what I’ve done so far but even things like security credentials - feels messy once you have more than one or two configuration details for a specific function call.

elaborate?

another useful pattern is making config a first class system component

Well… I have multiple modules that might have 4-5 configuration parameters that, in java, I’d use system properties to define. URLs for production, security credentials, etc.. Here’s some code as example:

(defn resolve!
  "Performs an EMPI fetch using the identifier as defined by `system` and `value` and
  the defined configuration.

  - url : the URL of the EMPI endpoint
  - processing-id : one of P (production) U (testing) or T (development) for type of server.
  - proxy-host : if required, proxy hostname as per clj-http
  - proxy-port : if required, proxy port as per clj-http"
  [system value {:keys [url processing-id proxy-host proxy-port] :as opts}]
  (-> (make-identifier-request system value opts)
      (do-post!)
      (parse-pdq)))

(deftype EmpiService [url processing-id opts]
  Resolver
  (resolve-id [this system value] 
    (resolve! system value (merge {:url url :processing-id processing-id} opts))))

and providing the config component to your other components

heh, I used to work with EMPIs back in the day

you shouldn't care about whether your config comes from Java properties, environment variables, files, or S3

something should aggregate the application's config, and perform validation -- then provide the checked config map to things that need it

That’s a good idea. Thanks.

So I can isolate the config implementation, draw it together, and then inject into modules as necessary…

yes

the config is a module itself

So you used clojure in healthcare, or was that in a past life? 😉

yes and yes

i first started using Clojure as an analyst at an academic medical institution

All my work in java until 3 years ago… halved the codesize and doubled my speed of development using golang… and now have done the same again moving to clojure, and yet get to use all my old libraries/ecosystem!

Thanks for your help.

my pleasure

Hey folks.. I have a beginner question about maps. I have a api request that I do using clj-http.client and then parse the json with chesire So I get something like

where get-itemsis just a wrapper around (:data (json/parse-string (:body response) true))

the issue comes after get-items this returns an array like structure with multiple items that have things like:

and I need to access all of that nested attribute.. what is the clojure way of doing it?

I know that I need to do something like (map parse-items items) and define parse-items

I was looking at select-keys but not sure how to used in this particular case

look at get-in

or destructuring https://clojure.org/guides/destructuring

Hey @matiasfh nested data access can be done with clojure and there are also a couple of libraries invented to make this easier, one called Specter and another the name escapes me. Anyway, if I understand your question right, you want to traverse this nested structure? You can do it in pure clojure, other hats here can provide code, but you may want to check out Specter. There's a HN discussion about it from recently here: https://news.ycombinator.com/item?id=18578920

Yeah I want to traverse the array-like structure to get a few of the attributes of it so then i can use it for format a string that will be pushed back to output

will check the thread and Specter library

There's a really great comment in that thread that says "Specter solves a problem you should aim to avoid" x) but if you have no choice about the data coming in you may want it

I'm not a fan of giving beginners Specter

the marketing around Specter is hyperbolic

for a #beginner, focus on learning the core library

Yeah.. in fact I want to learn the clojure way. And @sova yes.. I can't fix the way the data is returned by the api is a 3rd party server

The goal of this is to retrieve some of the attributes of this structure to create a set of strings based on that attributes

so you already know how to map. and your question is how to define a function parse-item which works on the following map:

{:attributes {:title "title"
              :company {:data {:attributes {:name "company name"}}}}
 :links {:public-url "public url"}}

given this map, what would you like parse-item to return?

the map is bigger.. will have more items inside the attributes level so what I want is to map an get a easier structure like

I use to do this with JS by just mapping over the array of objects to create a new array of objects with the new shape.. but that is the JS way.. i figure that is somehow similar with Clojure in terms of the approach. map the map

`

(let [data {:attributes {:title "title"
                         :company {:data {:attributes {:name "company name"}}}}
            :links {:public-url "public url"}}]
  {:title (get-in data [:attributes :title])
   :company-name (get-in data [:attributes :company :data :attributes :name])
   :link (get-in data [:links :public-url])})

shows how to get this data

so you could do

(defn parse-item [data]
  {:title (get-in data [:attributes :title])
   :company-name (get-in data [:attributes :company :data :attributes :name])
   :link (get-in data [:links :public-url])})

(map parse-item response)

you know what.. I had that in mind but was unsure of the idea of how that is written.. Thanks!

Looking good. and as @ghadi says it's sage advice to learn your way around core 😃 many many useful scissors, snips, and pocket knives in there.

In which situations would (:key coll) return nil if coll is a map? and :key is def not nil because (vals coll) returns the non-nil values and (keys coll) returns :key

If anyone is interested this is my full script. I’m scratching my head as to why the final lines don’t give me the value of :comments for each map in the vector. (map map?) is true, (map vals) gives me the correct values.. 🤷

#!/usr/bin/env bb

(ns jcpsantiago.controltower.airportsscript
  (:require [babashka.curl :as curl]
            [clojure.string :as s]
            [babashka.pods :as pods]))


(pods/load-pod "bootleg")
(require '[pod.retrogradeorbit.bootleg.utils :refer [convert-to]]
         '[pod.retrogradeorbit.hickory.select :as hs])


;; Airline data ;;
(def wiki-airlines 
  (-&gt; "<https://en.wikipedia.org/wiki/List_of_airline_codes>"
      curl/get
      :body))


(defn deepest-text
  "Drill down to the deepest text node(s) and return them as a string."
  [node]
  (cond (vector? node) (apply str (mapcat deepest-text node))
        (map? node) (deepest-text (:content node))
        :else node))


(defn extract-tables [html]
  "Takes an html page and extracts a table element."
  (mapv (fn [table]
          (mapv #(mapv deepest-text
                       (hs/select (hs/or (hs/tag :th) (hs/tag :td)) %))
                (hs/select (hs/tag :tr) table)))
        (-&gt;&gt; (convert-to html :hickory) 
             (hs/select (hs/tag :table)))))


(defn to-keyword
  "This takes a string and returns a normalized keyword."
  [input]
  (-&gt; input
    s/lower-case
    (s/replace \space \-)
    (s/replace "/.+" "")
    keyword))


(defn html-&gt;map 
  "Extracts an html table and turns it into a map."
  [wiki-airlines] 
  (let [table (first (extract-tables wiki-airlines))
        headers (-&gt;&gt; table
                     first
                     (map to-keyword)
                     vec)
        rows (-&gt;&gt; table
                  (drop 1)
                  (mapv #(mapv s/trim-newline %)))]
    (println "Joining rows and headers...")
    (map #(zipmap headers %) rows)))


(defn complete?
  [k coll]
  (every? true? (map not [(= (k coll) "\n")
                          (= (k coll) "n/a")])))

(defn not-defunct?
  [coll]
  (let [c (s/lower-case (:comments coll))]
    (println c)
    (not (s/includes? c "defunct"))))


(def complete-icao? (partial complete? :icao))
(def complete-iata? (partial complete? :iata))


(-&gt;&gt; (html-&gt;map wiki-airlines)
     (filter (every-pred complete-iata? complete-icao?)) 
     (map :comments))

does it return :key: (with a trailing colon)?

no sorry, that’s a typo:sweat_smile:

Hi, what tool can be used to automate creation and update of openAPI documentation in clojure?

what you can do is (doseq [[k _] some-map :when (= (vec (.getBytes (name k))) (vec (.getBytes (name :key))))] (prn k))

which will print out true if any keyword key in the map has a name that is byte for byte the same as the name of your :key

er, print out the key

if it doesn't the keyword you are using as a key in your map (by calling keyword on data you pull from the internet) contains characters that aren't printing, either because of your terminal setup, or because they are some kind of funky whitespace

you are stripping out the space character, but there are many other kinds of whitespace

interesting… saving to a text file as edn and reading it back works fine. Wouldn’t this also carry over any whitespace characters?

I saved it using pr-str and spit

(->> (html->map wiki-airlines)
     pr-str
     edn/read-string
     (map :icao))

no idea what works find means

ah, I see, no, my guess is there is some other kind of whitespace character that is not a space

so your :icao keywords all contain an extra character that is not visible when printed, which means it doesn't round trip

great question! 😃 bump.

if you’re using https://github.com/metosin/reitit, it comes with “built it” swagger support. See an example: https://github.com/metosin/reitit/tree/master/examples/http-swagger

hello, why is the output of this:

(defn create-es-bulk-writer []
  (let [es-write-chan (async/chan)]
    (println 1)
    (async/go
      (println 2))
    (println 3)
    es-write-chan))

1
3

default thread pool is full (and blocked)?

put a do around the printlns:

(defn create-es-bulk-writer []
  (let [es-write-chan (async/chan)]
    (do 
     (println 1)
     (async/go (println 2))
     (println 3) )
    es-write-chan))

user=> (create-es-bulk-writer )
1
3
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x6c26e588 "clojure.core.as2y
nc.impl.channels.ManyToManyChannel@6c26e588"]
user=>2

i think the problem was threads

i restarted the repl and now it works

that said, you’re not actually using the channel you make

i know

ok

i changed the code before posting

but left some stuff

the go block was waiting on the channel

and I was using "stop execution" on the repl but I guess the go threads were still running waiting for messages on the channel

Hello, I have more than 100 files in my lein project that describe the tasks, the data for them and their solution to make sure that the solution meets the described data I am testing the call of the solve function in another test file. For example, I have namespace task-one and task-two and files task_one.clj and task_two.clj, to test them I need to create a separate task_one_test.clj and task_two_test.clj inside test directory, inside test files the same test function is launched, only test data and solution functions differ. Since the task files are more than 100, the test files are also more than 100, but I want to run tests for all the tasks inside one common file, without having to create additional files. I have understood that the deftest macro is tied to the namespace in which it runs, so the test will run only once for this common file, which does not solve my problem with a large number of files describing the same tests. How this problem can be solved?

seems like that ws the issue.. string/trim fixed it geez.. thanks!

I am not sure I followed all of your description, but you can put all of the tests for a project into a single file of code if you want, in a single namespace, with as many separate deftest forms as you want (or as few as one, but having separate deftest forms can make it easier later to run only a subset of the tests.

No matter how many namespaces your production code is spread across, a single test file can call functions in all of them. There is no requirement that task_one.clj is tested by code in a file named task_one_test.clj

My main goal is to get rid of the heap of files and avoid copying the deftest to each task, I would like to run tests dynamically for each task, for example by getting a list of namespaces and run a function like this for every task:

(defn run-test [result data solution task-name]
  (deftest task-name
    (is (= result (apply solution data)))))