Clojurians Log v2

Leiningen question: how does the main function work, once you run the app? Does it start at main and go through and evaluate all the code in that file, or does it only run the code you call from within the main function?

I answered that on Zulip -- hope that helped? (and this sounds like a good question for the #beginners channel here on Slack).

But I thought one of them might be relative-to or something 🙂

I had only worked with lein and that would be a bit more involved. Plus it would add another rather large dependency.

I think I saw last time Clojure XML parser/generator which claims that it's faster than data.xml. Does anybody remember the name of it?

Probably this: https://github.com/nilern/eximia

Exactly. Thanks!

Can someone tell me what's going on with this problem I'm having?

(for [x [1 2 3]]
  (println "X" x))

; yields
; X 1
; X 2
; X 3
; (nil nil nil)

(do (println "DO")
    (for [x [1 2 3]]
      (println "X" x))
    (println "DONE"))

; yields
; DO
; DONE
; nil

for is lazy. You need something to realize it for side effects

use doseq instead of for if you want it just for side-effects

ah, side-effects!

thx

@alexmiller @hiredman Re: Clojure AOT problems with GraalVM native-image, A similar problem was reported with depstar and :aot true which has similar behavior to lein's :aot :all. https://github.com/oracle/graal/issues/1613#issuecomment-850325231

If we assume a structural editor for Clojure is desirable, what potential barriers would the design need to account for? What awesome things could the design account for?

Do you mean like paredit / parinfer? But more specific to Clojure?

Paredit or Parinfer are sources of inspiration! I’m referring to a sort of graphical/textual editor for manipulating and navigating code that prevents syntactic errors.

Something that would make it more approachable. The things I’m thinking about may apply to Lisp generally, but I’d prefer to start my explorations with Clojure

Suppose you need to add a map literal somewhere in the code, like {:a 1}. When you're typing it in a text editor - no matter how advanced the editor is, it's still text, so you will end up having an intermediate step of {:a|} where | is your text cursor (the } was inserted by the editor itself - it's a very common functionality). Something like clj-kondo will likely highlight that map literal as being incorrect. But how an editor can prevent such an error instead of just highlighting it?

Ah so are you basically saying that it would need to account for intermediate steps on the way to correct code?

I've often thought it might be fun to have a way of editing lisp with a VR helmet and something akin to an xbox kinect field sensor

Waving my arms about to do what lispy/paredit call slurping and barfing, etc

@rob370 I simply don't see any other way around it right now. So the editor will only let you save the file when there are no syntax errors (at least, that's how I understand your initial idea). So between | and {:a 1|} there will be no saves, no ability to evaluate code. But... is it really helpful to justify a new editor by itself? What if you need to enter a big nested data structure, like {:a {:b {:c ...}}} , but you also want to push your work into your Git branch somewhere in the process of entering it to save the progress?

Don't get me wrong - I'm all up for new tools. I'm just really unsure if an editor with more restrictions will be helpful. IMO it would be much more useful to have handy tools that help you make sense of the current state of your code, your development environment, and the relevant running processes. For example, a functionality to compare the code that has been evaluated most recently with the current code. And maybe it exists somewhere, but I haven't seen it.

Hmm only saving when there are no syntax errors is an option. Alternatively, one where it makes it visually clear what the gaps are and what can go in them.

> For example, a functionality to compare the code that has been evaluated most recently with the current code. Yes! This would be great.

> one where it makes it visually clear what the gaps are and what can go in them But isn't that exactly what linters like clj-kondo do?

@qmstuart Feel free to steal the idea. ;)

> For example, a functionality to compare the code that has been evaluated most recently with the current code. Yes! Would show me if function definitions and values are out of date

Clj-kondo means there’s something to build on! I just have a general open question about how to visually communicate this stuff as well as possible and I think standard text editors are gonna be limited there

Also worth noting that syntax errors specifically are IMO vastly outbalanced in terms of human-hours by any other kind of errors. A bad map literal - you already get the exact line and column number with a clear description of what's wrong. Seconds. A function call with a wrong arity - tougher because often can only be handled in run-time, but still pretty easy. Seconds to minutes, if you trigger the error. Some other run-time error - the time can vary from a minute to a month or more, easily. Some strange AOT error - from hours to months as well.

> how to visually communicate this stuff as well as possible OK, this is intriguing. Right now we have text decorations (red underlining and similar), editor decoration (scroll bar decoration, line highlighting), separate panels (list of errors in the whole project, list of errors in the current file). How could this be improved, assuming we can get past the plain text somehow?

The error messages are really hard to parse for beginners so we get stuck on them for much longer than you might. I’m not sure if a structural editor would be useful for a l33t coder but I have little doubt that something that could prevent things that literally make no sense given the structure of the code or prompt you to input the right amount of arguments would be helpful to beginners

Re ways to make structure and rules more visually apparent, a (possibly) silly idea; https://clojurians.slack.com/archives/C03RZGPG3/p1622147058166500

> For example, a functionality to compare the code that has been evaluated most recently with the current code. > Yes! Would show me if function definitions and values are out of date (edited) Doesn't cider do this currently? If a defn is evaluated it has a green highlight for me in the gutter on the left and that goes away if I change the code, until I load the buffer.

I've often thought that it would good to have a query language for my codebase (kinda like https://github.com/borkdude/grasp maybe??) integrated into an editor - so maybe you could query for all the functions called by a particular stack trace, see them all on one view and edit them together, even though they may be distributed across multiple namespaces possibly even in multiple git repos ...

but I'm not sure that's the sort of thing you're after, is it... I think if you'll be fighting an uphill battle trying to replace the immediate interface that someone uses to work with code ... the text based thing has worked very well for a long time, and visualisation graphical editing environments have repeatedly appeared and vanished without a trace ... although https://www.unisonweb.org/ looks like an interesting approach that may lend itself to changing that in the very long term??

I agree with @l0st3d about text based editing. A different approach would be to let users see your visualization of the code live in a browser while they are using their favorite text editor. This would not show all intermediate steps though, just when the user saved their file (you could have a program that sent the changes to the browser upon filechange).

Hello! How to manage such a thing, monitoring, health checks?

(def thread
    (Thread. (fn []
               (loop []
                 (println "Do some work!")
                 (recur)))))

  (.start thread)

@rende11 That is actually a fantastic question that I do not know the correct approach to

Some potential ideas -

use an executor service and a thread factory to make threads that have names that are relevant

(Executors/newSingleThreadExecutor
  (-> (BasicThreadFactory$Builder.)
      (.namingPattern "chat-subsystem-%s")
      (.build)))

or just use future if you want simpler code

you will almost always want some way to tell that thread to stop

closing over an atom that holds a boolean is one such way. if using an executor, it has shutdown methods

with a named thread you can get decent monitoring with JFR

or whatever other JVM monitoring tool

(.setName thread "queue consumer")

you can change the name from inside the thread

^^

it's a little dirty but in a case like this where you're spinning a single thread, probably a good hack

health checks you can inspect the executor, or have the tasks work on an atom when they crash

Then jconsole or jprofiler, or jstack.

Also, log from the thread and spelunk.

https://gist.github.com/bowbahdoe/ce961e6c7449c1ffe9877082465b1f47 this is the closest i've done - from a personal thing a year-ish ago

naming a thread isn't going to help with monitoring -- you want something connected to the semantics of your domain

(Spelling on splunk?)

if your background threads are poll workers, exposing metrics on # of messages consumed -> prometheus or some metric system

maintaining a reference to whatever stateful thing is potentially useful too

^that is a better approach

Also, what Ghadi said and throw domain specifics in a map in an atom that you expose to whatever you use to monitor or debug prod issues.

then set an alert on cloudwatch or wtvr for if the rate drops

:upvote:

one thing i've done in a past job is pass through a "listener" I called whenever I did a key action in the consumer

(twas in java, but same diff)

so like

(defn consumer-loop [listener]
  (let [msg           (poll-for-msg)
        _             (listener/started-processing-msg {:msg-id (:id msg)}]
    (try 
      (do 
        (insert-into-db (f msg))
        (listener/successfully-inserted-msg {:msg-id (:id msg)}))
      (catch ExpectedException e
        (listener/failed-to-inser {:msg-id (:id msg)}))))

so thats one approach to get your domain level info

Anyone know of, or feel like making, a site that will auto-reformat (re-indent) a field full of Clojure code? My use case is teaching beginners using no-installation tools that are just missing this crucial feature (currently Replit). If they could copy/paste/reformat/copy/paste via a site that just does reformatting, we'd have a workable solution, even if it's a little cumbersome.

Does the code need to be executed/executable from within the site?

Must the code be Clojure or is Clojurescript fine also?

I'm just thinking about reformating, not execution here. Execution could be fabulous too, but the goal here is just to have a formatting tool that could be used in conjunction with another solution for execution, like http://Replit.com. Clojurescript would be fine also, since the formatting is the same. I'm hoping for something like the standard reformatting capability in many IDEs like Cursive or Calva, etc.

are there any data structures and algorithm books that use clojure?

Not exactly what you ask, but there is this: http://www.sicpdistilled.com/

Oh that’s brilliant thank you @didibus

👋 anyone using open telemetry with clojure?

thank you @didibus

I did a little proof of concept work with it at my last job. I used this library: https://github.com/tendant/clj-telemetry

Ended up writing this macro (along with tracer which was an atom and this dynamic var

(def ^:dynamic *trace-parent* nil)

Ah and the java agent to capture traces by default?

Nothing by default; anything I wanted traced, I'd wrap in the with-trace macro.

Not sure it was the best approach, but it worked well enough for the PoC.

@james.r.carr IIRC the java agent only knows how to open traces for certain libraries

if your web server isn't on the handled list, that may not work for you

yeah it seems to capture most however.

at cognitect we built a trace library for a client. Wasn't OTel compliant, we emitted things to Honeycomb

yeah maybe we should just give the native DD agent a try

but I think we are likely going to need to wrap things regardless.

for example, by default we see jetty calls, but pedestal and the like are missing

you'll need some sort of integration somewhere

we had a convenient pedestal interceptor that took a set of key paths (into the context) and automatically traced those into the active span

We use the Java libs for opentracing and micrometer. Both are great. I think opentracing is moving to open telemetry? I don’t think we can switch because all the tooling hasn’t caught up yet (or something like that anyway).

yeah it's like the xkcd standards comic

endless churn in this space

none of the libraries we were using participated in open*anything, so making a tight 300LOC tracing library seemed like a low risk thing

spans are maps, that's pretty much it

I'd like to print in repl /save to file some of the keys from the list of maps I've made, see posted put in this slack thread. The idea is to align the keys under each other so that I can easily compare the values, like in a table. How can I achieve this? Eg with some sort of tabulation.

{:BUY {:lastPrice 0.5 :volume 1 }   :SELL {:lastPrice 1.95 :volume 46}  :p2margin 17.35      :ticker "TSLA" }
{:BUY {:lastPrice 0.9 :volume 100 } :SELL {:lastPrice 1.95 :volume 460} :p2margin 18.2342420 :ticker "TSLA" }   
etc ... 

({:BUY {:change 0.08000001
       :lastPrice 0.5
       :volume 1
       :etc :bunchotherkeys}
 :SELL {:lastPrice 1.95
        :safetyMargin 18.360813190331367
        :volume 46
        :etc :otherkeys}
 :p2margin 17.35
 :s-premium 1.27
 :ticker "TSLA"}
{:BUY {:change 0.08000001
       :lastPrice 0.9
       :volume 100
       :etc :bunchotherkeys}
 :SELL {:lastPrice 2.0
        :safetyMargin 18.360813190331367
        :volume 460
        :etc :otherkeys}
 :p2margin 18.2342420
 :s-premium 1.27
 :ticker "TSLA"})

Would this help? https://clojuredocs.org/clojure.pprint/print-table

cool. but only partially, because I'd need to reconstruct the maps. I'm looking for a way to extract only some keys from 2 level deep and then print a table