Clojurians Log v2

Is there a recommended tool for compiling *.java files in a deps.edn based project?

@smith.adriane I tend to just shell out and run javac 🙂

I guess I haven't run javac in a really long time. I just need clj to generate a class path to compile? To package the jar, I guess there's an option to tell depstar where to find the *.class files?

clj -Spath will give you the cp

I've found an interesting example of a macro-generating macro in Joy of Clojure, 2nd ed (p. 360). I'm not completely sure why unit-of-distance has to be a macro and not just a function. Is the primary reason here performance? https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/books/joy_of_clojure/ch14_data_oriented_programming.clj#L397-L465

(defmacro defunits-of [name base-unit & conversions]
  (let [magnitude (gensym)
        unit (gensym)
        units-map (into `{~base-unit 1}
                        (map vec (partition 2 conversions)))]
    `(defmacro ~(symbol (str "unit-of-" name))
       [~magnitude ~unit]
       `(* ~~magnitude
           ~(case ~unit
              ~@(mapcat (fn [[u# & r#]]
                          `[~u# ~(relative-units units-map u#)])
                        units-map))))))

(defunits-of distance :m
  :km 1000
  :cm 1/100
  :mm [1/10 :cm]
  :ft 0.3048
  :mile [5280 :ft])
;; => macroexpands to:
#_(defmacro unit-of-distance [G__21971 G__21972]
  (seq
   (concat
    (list '*)
    (list G__21971)
    (list
     (case
         G__21972
       :mm
       1/1000
       :m
       1
       :cm
       1/100
       :ft
       0.3048
       :km
       1000
       :mile
       1609.344)))))

(unit-of-distance 1 :m)
;; => 1

;; look how simple the expansion is!
(macroexpand '(unit-of-distance 1 :cm))
;; => (clojure.core/* 1 1/100)

Where can I find more examples of macro-generating macros and what other use cases are there apart from moving the computation to compile time to achieve faster execution?

macros generating macros is sometime used in cljs to work around not being able to programmatically add definitions to namespaces

Spec 2 has some macro generating macros for creating custom spec ops (which are macros)

Hmm. More articles should use phrases like "custom spec ops". :D

Are there any general rules relating to defrecords vs deftypes when the fields are mutable? I have a thing that is a grouping together of a bunch of promises representing potential future states. Should I expose the promises with a protocol and make the thing via a deftype? Or use a record and expose the promises via it's keys (and potentially also a protocol?). A third option is to use a map, but since I do need this to participate in other protocols, I'm thinking that might be ruled out first.

defrecord will also implement the protocols that enable assoc .based off your description, an assoced value might break some invariances. it seems like deftype might be preferable so you can limit which protocols are implemented to the ones that make sense

that makes sense, thanks!

I have another really odd bug where it works at the REPL but fails when compiled.

❯ java -jar target/uberjar/enki-master-SNAPSHOT-standalone.jar
Using database: /home/dsp/.enki.db
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.IFn.invoke(Object)" is null
        at enki.db$create_all_tables$fn__15909.invoke(db.clj:50)
        at clojure.core$run_BANG_$fn__8790.invoke(core.clj:7715)
        at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
        at clojure.core.protocols$fn__8140.invokeStatic(protocols.clj:75)
        at clojure.core.protocols$fn__8140.invoke(protocols.clj:75)
        at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
        at clojure.core$reduce.invokeStatic(core.clj:6828)
        at clojure.core$run_BANG_.invokeStatic(core.clj:7710)
        at clojure.core$run_BANG_.invoke(core.clj:7710)
        at enki.db$create_all_tables.invokeStatic(db.clj:50)
        at enki.db$create_all_tables.invoke(db.clj:46)
        at enki.db$delete_and_recreate_database.invokeStatic(db.clj:63)
        at enki.db$delete_and_recreate_database.invoke(db.clj:58)
        at enki.core$_main.invokeStatic(core.clj:47)
        at enki.core$_main.doInvoke(core.clj:43)
        at clojure.lang.RestFn.invoke(RestFn.java:397)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at enki.core.main(Unknown Source)
❯ lein repl
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
nREPL server started on port 44969 on host 127.0.0.1 - <nrepl://127.0.0.1:44969>
REPL-y 0.4.4, nREPL 0.8.3
Clojure 1.10.1
OpenJDK 64-Bit Server VM 15+36-Ubuntu-1
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

enki.core=&gt; (-main)
Using database: /home/dsp/.enki.db
true
enki.core=&gt; 

(defn create-all-tables
  "Create database tables. Expects each table listed in the `tables` set
  to have a corresponding create-`table`-table function defined."
  [x]
  (run! #((-> (str "create-" % "-table") symbol resolve) x) tables))

(defn -main
  [& args]

  (println "Using database:" (:subname db))
  (delete-and-recreate-database))

(defn delete-and-recreate-database
  []
  (let [database-file (io/file (:subname db))]
    (when (.exists database-file)
      (.delete database-file)))
  (create-all-tables db)
  (create-all-indexes db)
  true)

@dsp Are all of those create-*-table functions in the same ns as create-all-tables?

Yes, but not in the same ns as core

I have [enki.db :refer :all] at the top.

Prior to the create-all-tables defn, I do:

(def tables #{"nodekeys"
              "contactkeys"
              "mykeys"
              "incoming"
              "outgoing"
              "authtokens"
              "data"
              })

(def indexes #{"data"})

(def tables-loaded (into #{} (map #(let [p (str "enki/db/sql/" % ".sql")]
                                     (hugsql/def-db-fns p) %)
                                  tables)))

I know, kinda esoteric. I want to programmatically define the hugsql functions just by crawling the dir.

Ah, I think I found the culprit.

create-all-tables is in enki.db -- and so are all the create-*-table functions, yes?

Map again. Doh.

I probably need to force that one. I keep tripping up on these old bits of seq code around the place...

Likely the def-db-fns isn't being evaluated except at REPL.

Thanks for pointing my head in the right direction.

So, apparently Clojure is not a "real lisp" because I can't metaprogram at run-time and the code is recompiled on the fly. Is this because of the jvm as a base? https://softwareengineering.stackexchange.com/questions/210274/does-lisp-still-have-any-special-feature-which-has-not-been-adopted-by-other-pro#comment414013_210276

into with map should be eager -- but it is kind of dangerous to have a top-level def with side-effects since that will run when the ns is loaded which will also happen during compilation (likely when you build the uberjar).

Yep, doing doall didn't solve it.

The only effect should be function definition I think? Is that dangerous?

Clojure is what it is. There's no universal definition for what a lisp is and then people argue based on their own definitions. I've seen someone claim Clojure isn't a real lisp because symbols are globally interned. Some people claim that if you can't evaluate code from 1962 you aren't a real lisp.

It seems like the assertion that clojure can't be meta-programmed at runtime is wrong. Am I missing something?

> you never stop the running program, the running program itself is a set of objects in memory that you manipulate the same way you manipulate any other object, while it is running. working on my program while it's running is a big reason I enjoy clojure

I have no idea what they are talking about

i've never seen one of these discussions benefit anyone in any way. arguing what is a lisp on the internet is just a dumpster fire

This is very much one of those "how many angels...?" type questions (and belongs in #off-topic at best since it is not a technical question that can be answered as asked).

What I am essentially trying to accomplish is to programmatically do what the hugsql docs give as an example:

;; The path is relative to the classpath (not proj dir!),
;; so "src" is not included in the path.
;; The same would apply if the sql was under "resources/..."
;; Also, notice the under_scored path compliant with
;; Clojure file paths for hyphenated namespaces
(hugsql/def-db-fns "princess_bride/db/sql/characters.sql")

@dsp "at the REPL, but not when compiled": what exactly do you do outside the REPL to kick this off?

I had no idea it is an idealogical thing. I was surprised to read that clojure can't be changed at run-time because it has to be recompiled. At the moment I take the recompilation as an "inbetween" step to be back again and then it's run time again. when it's quick it doesn't make a difference to me.

Literally just

❯ java -jar target/uberjar/enki-master-SNAPSHOT-standalone.jar

Effects like this should probably kicked off from your -main and not from the top level of your namespace

I had wanted to keep db funcs in the db namespace for cleanliness, but maybe there is a way for me to do definitions there from core.

It'll be exposed via HTTP API, so has to be accessible through the regular execution chain.

main functions need to return an integer right?

not really

(System/exit n) is what is the return code

or an exception will also trigger a non-zero one

Am just trying to figure out exactly what's going wrong and why. When expanding -main like so:

(defn -main
  [& args]

  (println "Using database:" (:subname db))
  (let [database-file (<http://clojure.java.io/file|clojure.java.io/file> (:subname db))]
     (when (.exists database-file)
       (.delete database-file)))
  (println tables-loaded)
  (println 'enki.db/create-data-table)
  (create-all-tables db))

❯ java -jar /home/dsp/Development/enki/target/uberjar/enki-master-SNAPSHOT-standalone.jar
Using database: /home/dsp/.enki.db
#{contactkeys authtokens incoming outgoing data nodekeys mykeys}
enki.db/create-data-table
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.IFn.invoke(Object)" is
 null
       at enki.db$create_all_tables$fn__15912.invoke(db.clj:50)
       at clojure.core$map$fn__5866.invoke(core.clj:2755)
       at clojure.lang.LazySeq.sval(LazySeq.java:42)
       at clojure.lang.LazySeq.seq(LazySeq.java:51)
       at clojure.lang.RT.seq(RT.java:535)
       at clojure.core$seq__5402.invokeStatic(core.clj:137)
       at clojure.core$dorun.invokeStatic(core.clj:3133)
       at clojure.core$doall.invokeStatic(core.clj:3148)
       at clojure.core$doall.invoke(core.clj:3148)
       at enki.db$create_all_tables.invokeStatic(db.clj:50)
       at enki.db$create_all_tables.invoke(db.clj:46)
       at enki.core$_main.invokeStatic(core.clj:52)
       at enki.core$_main.doInvoke(core.clj:43)
       at clojure.lang.RestFn.invoke(RestFn.java:397)
       at clojure.lang.AFn.applyToHelper(AFn.java:152)
       at clojure.lang.RestFn.applyTo(RestFn.java:132)
       at enki.core.main(Unknown Source)

So it looks as though the functions are indeed being defined in the right namespace.

As others have noted, the question and most of the answers there are... "opinions"... and many of those folks choose their definitions in such as a way as to support the points they want to make so their definitions are also... "opinions"...

Indeed, a direct call in -main to that create-data-table function that is defined programmatically works. So I suspect it's related to the symbol resolve?

(most "opinion-based" questions on StackOverflow get shut down pretty quickly by the moderators so I'm a bit surprised something like that was allowed to "live" in the first place... but that probably says more about the Stack Exchange moderators than it does about the merit of the question 🙂 )

That's why I asked which functions are in which namespaces... but you would likely be safer to specify the namespace explicit in the symbol before calling resolve.

Yep, that is definitely the issue.

Strange that it works at the REPL. In db namespace I did:

(defn show-all-resolved-symbols
  []
  (map #(-> (str "create-" % "-table") symbol resolve) tables))

(println (into #{} (show-all-resolved-symbols)))

enki.core> (show-all-resolved-symbols)
(#'enki.db/create-contactkeys-table
 #'enki.db/create-authtokens-table
 #'enki.db/create-incoming-table
 #'enki.db/create-outgoing-table
 #'enki.db/create-data-table
 #'enki.db/create-nodekeys-table
 #'enki.db/create-mykeys-table)

dev=> (doc resolve)
-------------------------
clojure.core/resolve
([sym] [env sym])
  same as (ns-resolve *ns* symbol) or (ns-resolve *ns* &env symbol)

If you do (-> (str "enki.db/create-" % "-table") symbol resolve) I think it'll solve your problem @dsp

or resolve from the other ns using ns-resolve - should yield the same (`(ns-resolve 'enki.db %)`)

(defmacro create-all-tables
  "Create database tables. Expects each table listed in the `tables` set
  to have a corresponding create-`table`-table function defined."
  [x]
  `(run! #((ns-resolve (find-ns 'enki.db) (-> (str "create-" % "-table") symbol)) ~x) ~tables))

This seems to work. Thanks so much! I hope one day I more fully understand the pitfalls of relying on REPL behaviour as being akin to the endproduct.

Gone midnight, bedtime for me. But glad to squash this bug that's had me pulling at my hair for the last hour or two 🙂 Appreciate the help folks.

I didn't try to dig into that StackOverflow discussion in depth, but at least from a first reading of what they were claiming about Clojure, it appears some of the opinions expressed about Clojure were factually incorrect. You definitely can build a Clojure program from the ground up in a running JVM, function by function, without ever having to start the JVM over again -- exactly as many other Lisps allow you to do in a REPL.

All Lisps allow you to stop that running program and start a new one using the same source code (if you have saved that source code somewhere accessible to the next running process).