Clojurians Log v2

Closing the loop on this. Was able to get it working! Thanks to a combo of • Heroku docs - https://devcenter.heroku.com/articles/debugging-clojure • ~1yr blog post updated for newer version of drawbridge: https://blog.jakubholy.net/nrepl-over-http-with-drwabridge-in-2020/ • This GH issue which was updated ~ 3 weeks ago repro-ing the issue with the latest lein but having it work in 2.9.3: https://github.com/nrepl/drawbridge/issues/40

hello there, fellow clojurians. I’m trying to find a tracing library where I can provide a fn to customize the trace output. More specifically I'd like to use tap>

I looked at tools.trace but there doesn't seem to be a facility to get ahold of the output

i was looking at that the other day. seems like there should be a *trace-out* that is by default *out* but able to be rerouted. i think it directly uses *out*. Thus if there's any io in your code you get all or none. wouldn't necessarily solve your issue with tap but would give another way to redirect at least

Interestingly, it says of tracer "may be rebound to do anything you like" but it isn't marked dynamic so I guess... alter-var-root?

thanks for the suggestions

i must be thinking of something else. its a different library. i thought it was tools.trace

@beders If you don't mind using alter-var-root to change clojure.tools.trace/tracer you can make it use tap>

seanc@DESKTOP-30ICA76:~/clojure$ clj -Sdeps '{:deps {org.clojure/tools.trace {:mvn/version "RELEASE"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.trace :refer :all])
nil
user=> (alter-var-root #'clojure.tools.trace/tracer (constantly (fn [name value] (tap> [:trace name value]))))
#object[user$eval409$fn__410 0x6411d3c8 "user$eval409$fn__410@6411d3c8"]
user=> (add-tap prn)
nil
user=> (trace (* 2 3))
6
user=> [:trace nil "6"]
(add-tap (fn [[k n v]] (when (= :trace k) (println "MY TRACE" n v))))
nil
user=> (trace (* 2 3))
6
[:trace nil "6"]
user=> MY TRACE nil 6

just fired up my repl to mess with it. Thanks a lot @seancorfield

Now I know this, I might use tools.trace more since I use tap> very heavily 🙂

anyone know how to use the dotrace macro from tools.trace?

i'd expect

(defn foo [x] (inc x))
(dotrace [foo] (foo 3))

is this a vestige of pre 1.3? > ;; As of Clojure 1.3, vars need to be explicitly marked as ^:dynamic in order for > ;; them to be dynamically rebindable:

I think that's the issue with tracer as well.

Time to open a JIRA or http://ask.clojure.org issue I think!

looks like alter-varying trace-fn-call gives me more control over the output

@beders That won't help if you just want to trace an expression I think.

true dat. I'm instrumenting whole namespaces, so I should be good. Thanks for the hint. trace unfortunately also assumes I want the to pr-str the value

What would people call the namespace prefix? Like if I have namespace com.foo.bar.core what would you call com.foo.bar, in Java it be called the package, but in Clojure?

I would call it the "stem" but I don't know whether that's common vernacular.

"parent" might be a reasonable designation?

(although namespaces are not really hierarchical)

I thought about calling it the ns-group?

meh, I guess it doesn't matter, I'll need a doc-string to explain anyways, don't think there's an intuitive name

it is tricky because in clojure they have no relationship, com.foo.bar.core is not "in" or a sub part of com.foo.bar

but because of the way namespaces are mapped to files and directories, on the filesystem there is a relationship

ns-group is definitely no good, because it implies a relationship that doesn't exist, it really isn't analogous to classes and packages in java

Hi all. Does anyone here have some experience with chesire lib? Does someone have issues withUTF-8 while using parse-string ?

@ramonp.rios should work

Something really bizarre is happening 😅 . I got the json into a string with the right utf-8 encode and after parse using parse-string, the strings at the map are not utf-8

Maybe you can make a repro

Yes, working on it. Thanks

We found the issue. Actually it was a Yada misconfiguration 😅

I tried to repo in a separate project but it turns out it was working

I thought that was chesire on the beginning because we checked the issue right after the data pass througt the parse-string

So i went check with Yada and it was missing configuration to charset UTF-8

“Semantic qualifier” or just “qualifier” maybe? I know it sort of overloads “fully qualified” keywords, but I think they serve a similar semantic purpose.

There was a trick with var (`#'`) and ring to not reload ns with handler each time after change code anywhere. Something like this

(def handler
  (-> (bidi/make-handler route/route #'resources)
      (logs/ring-debug)
      (wrap-keyword-params {:parse-namespaces? true})
      (wrap-json-body {:keywords? true :bigdecimals? true})
      (wrap-params)
      (wrap-restful-format)
      (add-headers)
      (logs/wrap-exceptions)))

ok never mind, I had to add #' deeper in the code

hi, anyone still using overtone (or any clojure based music live coding)? it's been years... i wonder if someone still manage to run it on windows 10... i tried setting it up with supercollider the other day but no luck...

got the solution... https://github.com/overtone/overtone/issues/467#issuecomment-759545131

Hi, i have a question, why i get clojure.lang.APersistentVector$SubVector type instead of PersistentVector in the following code? when I run the java code I get a PersistentVector :

IPersistentVector subvec = RT.subvec(
                PersistentVector.create(1, 2, 3, 4, 5),
                0,
                3
        );
        IPersistentVector vector = LazilyPersistentVector.create(subvec);

Because vec doesn't coerce things that are already vectors into a different type

Probably as a performance optimization, since something that returns true for vector?, which a SubVector instance does, already is a vector.

Look at the (short) source code for the Clojure function vec to see the conditions it tests there, and the fast path where it returns the existing vector if it already is one.

I'm looking at this

(defn vec
  "Creates a new vector containing the contents of coll. Java arrays
  will be aliased and should not be modified."
  {:added  "1.0"
   :static true}
  ([coll]
   (if (vector? coll)
     (if (instance? clojure.lang.IObj coll)
       (with-meta coll nil)
       (clojure.lang.LazilyPersistentVector/create coll))
     (clojure.lang.LazilyPersistentVector/create coll))))

In your Clojure expression (type (vec (subvec (vector 1 2 3 4 5) 0 3))), I believe that the code path executed within vec is the one that leads to returning (with-meta coll nil)

you can force into a new instance by replacing vec with into []

ok I get it now thank you @andy.fingerhut @bronsa: )

vec guarantees that you get a vector in the return. it makes no assertions about whether that is the same or different vector from the input value.

Hey all, do you have any recommendations for something that can quickly bootstrap a *nix server for Clojure/Script apps? I'm looking to host a bunch of different apps, each of which should have its own server, but I don't want to go through the pain of setting one up from scratch each time. I'm thinking something like Docker? Or even a script? Any recommendations will be much appreciated 🙂

Docker would be good

makes it platform agnostic

nah

Can run on a bare linux server, managed kubernetes, Google Cloud Run, anywhere really. And gives your the isolation you want

sure, and then hire 8-10 guys to run all that

and maybe a manager or two for them

if you build uberjars, java -jar whatever.jar is a great way to deploy and run stuff

I would even start off with a single repo and building a single uber jar and just run java -cp uberjar.jar clojure.main -m some.clojure.namespace to launch the different servers

@hiredman Respectfully, I think perhaps you haven’t seen what modern managed container offerings can do for you.

I’m not recommending @andyfry01 stands up and manages a kubernetes cluster.

it is also trivially simple to go from an uberjar to a docker container should you choose to

like, saying "use docker" doesn't actually answer questions about how to build and run apps, it just moves those questions into a container

OP’s question was “any recommendations for something that can quickly bootstrap a *nix server for Clojure/Script apps? I’m looking to host a bunch of different apps, each of which should have its own server, but I don’t want to go through the pain of setting one up from scratch each time.”

I am aware of managed container offerings, and have spent time (I won't call it wasted) getting them working. they do a lot of stuff, but to pretend they don't drag along a lot of complexity you can likely avoid if you are just starting is to be willfully blind

perhaps I misread

I took "server" to mean "service"

not literally devoting a machine vm or otherwise to each

:man-shrugging:

I guess what I'm looking for is something where I can define a single "standard" server configuration, and then use that to configure multiple separate servers

For simple use cases I would say, throw the jar into a container and deploy to google cloud run with minimum instance count of 1.

All of these servers will be running different apps, but they'll share a lot of stuff in common (like mongodb for instance), so I just want something to get the basics configured right away

gcloud run deploy --image http://gcr.io/cloudrun/hello --min-instances=1 helloservice

I think most people use something like ansible or chef for that kind of thing, which isn't really on topic for #clojure

replace http://gcr.io/cloudrun/hello with your container image and your have what you need.

without the 8-10 + 2 team @hiredman suggests.

Haha, not yet anyway 😛

Thanks guys, appreciate it!

🙂 Although @hiredman is right, this is off topic for #clojure

Hi all, I'm trying to find a way to rerun a unit test from within my repl using [clojure.tools.namespace.repl/refresh. Running something like (do (refresh) (test-var (var my-ns/my-test))) yields a correct result every time I update and rerun the test, but if I try to put that code behind a function (e.g. (defn r [] (refresh) (test-var (var my-ns/my-test)))), it only binds the test the first time it is run. Subsequent calls to (r) yield the original result. Anyone know what's going on there?

(yet another reason not to use "refresh"-based workflows perhaps?)

If the alternative is :reload-all then I'll stick with refresh. I'd actually like to get to the bottom of what's going on, because I don't see what would cause the two examples to behave differently

@seancorfield how do you do in case of namespace refactoring so. I dont feel confortable with refresh ( lot of special cases) but dont know to do in another way.

@caumond Can you elaborate on what you mean by "namespace refactoring"?

Use resolve instead of the var special form

Your function is holding on to an interned var that has been removed by refresh, you need to re-reolve it everytime

I have cases when I want to re organize name space. So copy paste function and recompile function is easy. But I always have doubts, did I update all functions using the older ones.

@caumond Using consistent aliases for namespaces helps a lot there (and linters like clj-kondo can help "enforce" that) so a function reference is always foo/bar everywhere (except uses in its own ns). You can also unmap the old function from the old ns by way of clean up. Generally, if I'm moving a public function from one ns to another, it tends to have a bunch of small private functions that it is implemented by, so as a "shortcut" I will sometimes remove-ns and then load-file on the old ns to clean things up a bit more -- for me that's ctrl-; r ctrl-; f -- and I always eval each top-level form as I change it anyway, and if I'm pasting multiple functions into a new ns I'll save and reload that ns: ctrl-s ctrl-; f. Moving functions around is one of the few cases where I will save/load rather than just eval'ing forms (and leaving files "dirty" -- unsaved).

I do have require .. :reload-all bound to a hot key (`ctrl-; R`) but almost never need to use that. And I don't use any sort of ns "refresh" ever in my workflow.

I ll try this. Seems reasonably simple and efficient.

(let [root-logger (.getLogger (LogManager/getLogManager) "")]
    (.setLevel root-logger Level/ALL)
    (doseq [handler (.getHandlers root-logger)]
      (.setLevel root-logger Level/ALL)))

(l/debug "foo")

thx @hiredman. Just saw this message. That worked a treat

As I understand this it is actually JUL issue, not Clojure but still I don’t know why this is happening.

Because

(let [root-logger (.getLogger (LogManager/getLogManager) "")]
    (.setLevel root-logger Level/ALL)
    (doseq [handler (.getHandlers root-logger)]
      (.setLevel root-logger Level/ALL))

    (.fine root-logger "fine"))

I would try calling .getLoggerNames on the log manager, and then iterating over each logger name and getting the logger for that name, then each handler for that logger, then setting the level on each handler

I realize this doesn’t actually answer the question, but if I might suggest a lateral solution: I gave up on Java’s baroque logging system long ago. Now I use https://github.com/ptaoussanis/timbre

I just went through something like this with log4j2 which is not going to match jul, but it took a lot of iterating over a lot of things to set the level everywhere, and what you have to do is fairly sensitive to the logging config file if you have one

Thanks @paul.legato but I did timbre -> clojure.tools.logging. Long story why but this is right thing 🙂

@hiredman what is strange it works for INFO and up, but not below this level

and I do iteration on handlers, all examples show only this

but for sure I miss something

The main thing is to try to get to the place where you have absolutely the minimum possible compilable change between each "eval" -- or turned around, that you always "eval" as often as you possibly can while making changes. Tightening that loop is what's key for a good REPL-based workflow.

what does l/debug do?

“nothing”

(l/debug "foo")
=> nil

ok, I mean, what is the definition of l/debug

(let [root-logger (.getLogger (LogManager/getLogManager) "")]
    (.setLevel root-logger Level/ALL)
    (doseq [handler (.getHandlers root-logger)]
      (.setLevel root-logger Level/ALL))

    (.fine root-logger "fine"))

so for sure it is the issue about JUL itself, not clojure.tools.logging

(do
  (doseq [ln (enumeration-seq (.getLoggerNames (java.util.logging.LogManager/getLogManager)))
          :let [logger (.getLogger (java.util.logging.LogManager/getLogManager) ln)]
          handler (.getHandlers logger)]
    (.setLevel logger java.util.logging.Level/ALL)
    (.setLevel handler java.util.logging.Level/ALL))
  (.fine
   (.getLogger (java.util.logging.LogManager/getLogManager) "")
   "fine"))

what I see locally is there is a logger called "" and a logger called "global"

don't ask me why some levels and not other get logged or whatever

thank you. I am still not sure why it works, but I will figure out this soon. I mean I am trying very similar things now and it doesn’t work. Interesting.

another thing is logger creation is stateful (I am sort of conflating log4j2 with jul here), so when you create a new logger (which tools.logging does whenever you execute a logging statement in a namespace that has had a logger created for it yet) it is initialized in someway (I don't entirely understand) to match the setting of the root logger, but then later if you change the settings of the root logger, it won't change the settings of that namespace logger

so in order to correctly configure everything you have to be careful of the order things are run in

Yes. I succeed to do that for small changes. More challenging for big ones. But im on the way. Thx

sh** I had a big “typo”

(let [root-logger (.getLogger (LogManager/getLogManager) "")]
    (.setLevel root-logger Level/ALL)

    (doseq [handler (.getHandlers root-logger)]
      (.setLevel handler Level/ALL))

    (.fine root-logger "fine"))

You save my day

❤️

is there a way to hack an already existing protocol to set :extend-via-metadata to true ?

record scratch hold up. what are you really trying to achieve?

(The hidden assumption here is that you don't own this protocol, otherwise you would simply edit your source file.)

Nothing that's a good idea, just had a thought to use maps as core.async channels (the maps have inner channels and metadata). I suppose I could flip this and just extend IMeta to channels (which I know also isn't a good idea, this is more of a thought experiment)

when I've needed to associate channels with extra information, I have either used channels as a lookup key in a map {ch {...info about ch}} (when the chans are all known to a particular controller) or flow some info that includes the channel {:some :event :in ch} (which you can send... over a channel)

the best way to associate a channel with extra data is to just enrich the data coming out of it with that data: e.g. wrap the channel (defn wrap [ch data] (async/go [data (async/<! ch)]))

channels are not values so you'll need to extend the identity metadata stuff, which will cause you call kinds of problems if you are using timeouts (similar with sticking channels in maps or sets)

I would have a hard time reading code that used map+channel hybrids, though it is technically possible to make non-channels participate in channel protocols

wrapping timeouts is very nice because you can get a nice bit of data back (this is a timeout for this connection or whatever) instead of just nil when the timeout expires

gotcha yeah all that makes sense, thanks! I think the wrapping will work well

I want

wrap is cool, but checkout withNack http://cml.cs.uchicago.edu/pages/cml.html