Clojurians Log v2

We do a check on the file by creating a hash and if it matches the hash that is stored in an atom, then we skip the load and just use the model that is stored in a registry. We dont need have any state in the models so we always use the same model. But this does help me understand whats going on a little better

Hey team, say I have a function like this:

(def b (fn [a] a))

(def x (fn [a] (b a))) 

(expand '(x 1))
; => 
'((fn [a] ((fn [a] a) a))

(defmacro fn# [args & body]
  (with-meta
    `(fn ~args ~@body)
    {:args '~args
     :body '~body}))

(meta (fn# [a] (+ 1 a)))

Syntax error compiling at (/private/var/folders/sz/rnrd6cv509x2t8nbm6k4mzsh0000gn/T/form-init16854188530357342528.clj:1:7).
Unable to resolve symbol: args in this context

{:args '~args
 :body '~body}

I think ~'args might be what you want

~ inside of a syntax-quoted expression means sort of "don't syntax-quote the next subexpression". If you use ~args, the thing that gets inserted into the syntax-quoted expression is args, but evaluated at compile time. If you use ~'args, then 'args is evaluated at compile time, which is just the symbol args

Did some experimenting, because I always get this kind of stuff wrong on first guess without trying it out:

(defmacro fn1 [args & body]
  (with-meta
    `(fn ~args ~@body)
    `{:args '~args
      :body '~body}))

Ah, amazing. Thanks @andy.fingerhut!

Ah, I see, the syntax-quote was necessary around our meta map, because otherwise there would be no way to use ~

Okay, final question for the day team. Say I have a namespace: I want to “monkey-patch” def with this:

(ns church-factorial
  (:refer-clojure :exclude [def]))

(defmacro def [name v]
  `(do
    (clojure.core/def ~name ~v)
    (alter-meta! (var ~name) assoc :source {:name '~name :v '~v})
    (var ~name)))

I don't think that is possible except via modifying Java code in the Clojure compiler.

def is one of the handful of special forms that are not implemented as functions or macros.

The :exclude [def] does not give an error, but it does not actually mean that def is undefined within the namespace, either:

$ clj
Clojure 1.10.1
user=> (ns foo (:refer-clojure :exclude [def]))
nil
foo=> (def bar 5)
#'foo/bar
foo=> bar
5

As opposed to +, say, which you can actually exclude, and it will not be defined in that namespace:

$ clj
Clojure 1.10.1
user=> (ns foo (:refer-clojure :exclude [+]))
nil
foo=> (+ 1 2)
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: + in this context

You can find a list of special forms here, and I would guess that for any of them, you won't get an error if you exclude them, but they will be like def in their behavior -- still defined if excluded, and cannot be monkey-patched from within Clojure itself: https://clojure.org/reference/special_forms

fn and let might be exceptions in that list, since they actually are implemented as macros, with fn* and let* being the underlying special forms they are based upon.

also, I think if you just use def there the compiler doesn't attempt to look special forms up in the environment

❯❯❯ clj
Clojure 1.10.1
(defmacro def [name v]
  `(do
     (def ~name ~v)
     (alter-meta! (var ~name) assoc :source {:name '~name :v '~v})
     (println "i defined " ~name)
     (var ~name)))
#'user/def
user=> (def x 3) ;; refers to clojure def
#'user/x
user=> (user/def x 5) ;; must always have a namespace or alias to refer to def so it doesn't resolve to clojure's
i defined  5
#'user/x
user=>

Async profiler is great and it basically monitors the whole vm, not particular group of functions; although you can, I think, limit it only to certain threads? Anyway, it’s very useful for cpu profiling and great first tool to reach for. Beaware that if your program spends lot of time “off cpu” you won’t see such bottlenecks in the profiling output

For OOM: async profiler has “allocation profiling” but I think it does not work on Mac os; yourkit can do that too but it tends to have huge overhead. You can also dump heap on OOM (there’s jvm arg for that) and load the dump in yourkit or VisualVM

Why you don’t include the models as standard dependency? I don’t see any benefit in having them in the bucket and extracting then into the resources folder

Hello, I try to run -main function in a fresh Clojure CLI project but I got this error.

➜  contsscraper.app git:(main) ✗ clj -A:run        
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate constsscraper/core__init.class, constsscraper/core.clj or constsscraper/core.cljc on classpath.

Full report at:
/var/folders/gv/ry9mmtjj0vx87_ppthlrc4nr0000gn/T/clojure-4702191647831092797.edn

Hello, is there any good plotting lib which can generate vector graphics, for scientific papers. Thanks.

@ampersanda

You're missing a s

https://hypirion.github.io/clj-xchart/ will generate SVG (vector graphics) I have use Oz for data visualisation, I didnt check to see if it creates vector graphics. The hiccup syntax can be used to create SVG graphics, which is just a data structure. https://clojurebridgelondon.github.io/workshop/introducing-clojure/clojure-svg-graphics.html I havent tried these libraries yet, but they may be of use https://keminglabs.com/c2/ http://liebke.github.io/analemma/

That looks right to me, unless there’s some spelling error in the namespaces somewhere that I didn’t catch

Might be worth trying clj -Sforce -A:run to make sure your classpath isn’t cached wrong somehow

Then I’d back out of the alias and just do clj -m constsscraper.core

Something doesn’t line up here, just need to jiggle it enough to see what

ah thanks, I didn’t see that

Thank you for the replies, I missed the spelling of the namespace https://clojurians.slack.com/archives/C053AK3F9/p1602930866127900

Hey team, one more wacky question:

(defn make-code [x] 
  ...)
(make-code '(foo 1 2)) 

(
  ; bar
  foo 
  1 2)

@stopachka Not if you're producing symbolic code (you'd have to produce a string instead).

I see, this makes sense. Thanks Sean!

(because comments are handled by the reader, and the reader turns strings into data structures/code and removes the comments)

You could always produce (do (comment "bar") (foo 1 2)) -- the comment form is executable and returns nil 🙂

There are some libraries written with the goal of enabling you to read text representing Clojure code/data, and return not what clojure.core/read or clojure.edn/read would return, but instead a custom data structure that represents all white space, comments, etc. and let you manipulate that. e.g. https://github.com/xsc/rewrite-clj I have not used them, so can't comment on all of the possible gotchas and limitations they might have.

I saw this thing from borkdude just the other day, seems to fit the bill perfectly! https://github.com/borkdude/rewrite-edn

For creating / manipulating code while preserving whitespace / comments rewrite-clj is my goto library. I recommend using https://github.com/lread/rewrite-cljc-playground since that's going to be the maintained future of both rewrite-clj(s)(c)

It will be transferred to clj-commons later on

rewrite-edn is a lib on top of rewrite-cljc that can help with editing config files while preserving comments

Is there a way to watch activity on an async chan without taking from it? I tried to do something like (tap (mult channel-to-watch) put-on-this-channel) and then but it seemed to mess with the activity on the channel-to-watch

This works (though is not good because of the sleep)

(let [program [3,26,1001,26,-4,26,3,27,1002,27,2,27,1,27,26
                 27,4,27,1001,28,-1,28,1005,28,6,99,0,0,5]
        c1-in (async/chan)
        c1-out (run-async program c1-in)
        c2-out (run-async program c1-out)
        c3-out (run-async program c2-out)
        c4-out (run-async program c3-out)
        c5-out (run-async program c4-out)]
    (async/pipe c5-out c1-in)
    (>!! c1-in 9)
    (>!! c1-out 8)
    (>!! c2-out 7)
    (>!! c3-out 6)
    (>!! c4-out 5)
    (>!! c1-in 0)
    (Thread/sleep 5000)
    (<!! c1-in))

what about poll!?

What I'm trying to do is set up a tap on ch1-in (or I suppose ch5-out) so I can see what's getting put on there - the final value before the channel closes is what I want. This doesn't work - it throw a null pointer somewhere much further up in the code, which makes me think that the tap is impeding the pipe flow between c1-in and c5-out

(let [program [3,26,1001,26,-4,26,3,27,1002,27,2,27,1,27,26
                 27,4,27,1001,28,-1,28,1005,28,6,99,0,0,5]
        c1-in (async/chan)
        c1-out (run-async program c1-in)
        c2-out (run-async program c1-out)
        c3-out (run-async program c2-out)
        c4-out (run-async program c3-out)
        c5-out (run-async program c4-out)
        read (async/chan)]
    (async/pipe c5-out c1-in)
    (async/tap (mult c1-in) read)
    (>!! c1-in 9)
    (>!! c1-out 8)
    (>!! c2-out 7)
    (>!! c3-out 6)
    (>!! c4-out 5)
    (>!! c1-in 0)
    (async/<!! (async/go-loop [outputs []]
                 (let [val (async/<! read)]
                   (println "read got val" val)
                   (if val (recur (conj outputs val))
                       outputs)))))

> what about poll!? Won't that take the value off the chan? I'm trying to not impact the machine that's running in the background, just look at the traffic

https://clojuredocs.org/clojure.core.async/poll!

> Takes a val from port if it's possible to do so immediately.

My worry would be that if I take the val while polling, then it won't get taken by the actual thing that's meant to be consuming it

yea, I don't think poll! is what you want here

in general, it's easier to test each individual process and then put everything together than to try to test everything at once

you could "spy" on values flowing through by wrapping each link in the chain with a process that just prints the value passing through before piping it to the next channel

something like:

(defn spy-process [prefix ch1 ch2]
  (go-loop [v (<! ch1)]
    (when v
      (prn prefix v)
      (>! ch2 v)
      (recur))))

;; eg.
(spy-process "ch1->ch2" ch1 ch2)

> in general, it's easier to test each individual process and then put everything together than to try to test everything at once Thanks, this is good advice. I'm getting closer I think, by simplifying the machines a bit:

(defn simple-run [inchan name]
    (let [outchan (async/chan)]
      (async/go-loop [counter 0]
        (if (> counter 10) (do (async/close! outchan))
            (do (println name counter)
                (async/>! outchan (inc (async/<! inchan)))
                (recur (inc counter)))))
      outchan))
  
  (let [in (async/chan)
        out1 (simple-run in :machine1)
        out2 (simple-run out1 :machine2)]
    (async/pipe out2 in)
    (>!! in 100)
    (Thread/sleep 1000)
    (<!! in))
  
  (let [in (async/chan)
        out1 (simple-run in :machine1)
        out2 (simple-run out1 :machine2)
        read (async/chan 1000)
        m (async/mult out2)]
    (async/tap m in)
    (async/tap m read)
    (>!! in 100)
    (<!! (async/go-loop [outputs []]
           (let [val (async/<! read)]
             (if val (recur (conj outputs val))
                 outputs)))))

I think trying to mix and match pipes and mults in the original was a bad idea. The last one still isn't working - It breaks my repl, I think it's not terminating or something, but I can't figure out why

OK, I guess the read channel isn't closing on it's own properly, since this works OK

(def read (async/chan 1000))

  (let [in (async/chan)
        out1 (simple-run in :machine1)
        out2 (simple-run out1 :machine2)
        m (async/mult out2)]
    (async/tap m in)
    (async/tap m read)
    (>!! in 100))

  (async/close! read)

  (<!! read)

if there's an exception, I don't think simple-run will close the channel which might be causing your last block to hang

you'll probably want to wrap the go-loop in a try/catch and close the channel in the catch clause

I don't think there is any exception being thrown, because I can run it corectly 'by hand', just pulling values of the read chan until it nils. I don't see anything in the docs for mult about it automatically closing when its out2 closes. Maybe that's it?

I would think it would though...

OK, works fine when I increase the buffer

(defn simple-run [inchan name]
    (let [outchan (async/chan 100)]
      (async/go-loop [counter 0]
        (if (> counter 10) (do (async/close! outchan))
            (do (println name counter)
                (async/>! outchan (inc (async/<! inchan)))
                (recur (inc counter)))))
      outchan))

  (let [in (async/chan 100)
        out1 (simple-run in :machine1)
        out2 (simple-run out1 :machine2)
        m (async/mult out2)
        read (async/chan 100)]
    (async/tap m in)
    (async/tap m read)
    (>!! in 100)
    (async/<!! (async/go-loop [outputs []]
                 (let [val (async/<! read)]
                   (println "read got val" val)
                   (if val (recur (conj outputs val))
                       outputs)))))

can you use a chan transducer for spying?