Clojurians Log v2

Is it safe to have a watcher send a message to the agent it is watching ?

It is as safe as recursion, you need a base case to avoid an infinite loop

What people try to do with that sort of think is often not safe though

The action your watcher sends will get interleaved with other actions being concurrently sent

So you can easily make a race condition

It's my worry. I have a bunch of futures that all send to the agent to append to a list. When the watcher detects that the list is longer than a certain size, I want it to process that list and reset the agent to the empty list

If I am not mistaken, the messages received by the agent are handled sequentially, so there shouldn't be any race condition in this scenario, right? Can there be loss of data though?

Would a better approach be to just drop the part of the vector contained in the agent that I used rather than resetting the agent?

The iteration function looks useful. Here's my attempt to use it fetch repositories with installed github app: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/collections.clj#L434-L449

(let [my-installation-token "generate-jwt-and-get-installation-access-token-from-that"
          api (fn fetch-page [page-url]
                (log/debug "Fetch all installation repositories - page: " page-url)
                (let [response (http/get page-url {:accept "application/vnd.github.v3+json"
                                                   :headers {"Authorization" (str "Bearer " my-installation-token)}
                                                   :as :json
                                                   :query-params {:per_page "100"}})]
                  (log/debug "Fetch all installation repositories - page FINISHED: " page-url)
                  response))
          api-reducible (iteration api
                                   :kf #(get-in % [:links :next :href])
                                   :vf #(-> % :body :repositories)
                                   :initk "<https://api.github.com/installation/repositories>")]
      ;; flatten the pages
      (into [] cat api-reducible))

I was wondering about the best approach for logging in a library I am building. I already saw this great post: https://lambdaisland.com/blog/2020-06-12-logging-in-clojure-making-sense-of-the-mess For libraries, my thoughts are that clojure.tools.logging would be the best approach, as consumers of the library could change the clojure.tools.logging.factory flag to decide if the library should log to slf4j, apache commons, log4j, log4j2 or java.util.logging, so I won’t force them to pull some bridge-libraries into their deps. What are your thoughts on this?

it's one of those things were I personally prefer just relying on raw logback

+ tools logging

sure it's an xml file to deal with, but at least there's no indirection and it's quite straightforward

Yeah, I like clojure tools logging too

simple to use

the only reason to use a third party lib would be if you want compat clj/cljs I would guess

but I am not expert on cljs so I don't know the details

Good point, but it won’t be targeted at cljs

Regarding logback: I wouldn’t ship it with the library, as it is a logging backend, I believe

Thanks!

I get an illegal reflective access warning when setting access on the private address field in java.nio.Buffer (java 11). Usually it goes away with type hinting but here, no matter what I hint, it doesn't.

(def ^:no-doc ^java.lang.reflect.Field -field-address

  (doto (.getDeclaredField Buffer
                           "address")
    (.setAccessible true)))

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by helins.binf.native$fn__1426 to field java.nio.Buffer.address

You are doing explicit reflection (`.getDeclaredField`) and getting a warning accordingly

When you’re proxying a class with clojure proxy is it possible to typehint protected methods on this?

I’ve wrapped (let [^MyProxyClass this this ret (.protectedMethod this) ,,,) over it and it still warns

Has anybody ever used something like redis as a substrate for a core.async channel? I'd like to be able to use the same semantics as channels across multiple processes / machines

backpressure is a hard thing to implement over the network

Is it forbidden in Java now? From what I read, people still do that and nobody seem to complain. I thought I might be doing something wrong or maybe for some reasons it could be somehow Clojure related (didn't quite follow how Clojure blends with modules).

People tend to try supressing warnings without trying to understand those first :) I don't have the authoritative source, but it's quite clear that newer JDKs want to enforce the Java privacy mechanism, forbidding things like setAccessible. Surely other reflection APIs will remain perfectly fine to use

yes... I was afraid there'd be some theoretical reason why not!

thanks 🙂

tcp includes backpressure, but it can be non-obvious, gets weird when you aren't communicating directly, and lots of apis do a bad job of exposing it

https://github.com/hiredman/roundabout/blob/master/src/com/manigfeald/roundabout.cljc is an example of how you can communicate back pressure

Yes but since accessing private fields is so common in plenty of big Java libs... Are they really going to fully remove picking at private fields?!

this makes some intuitive sense, but I’d love to understand why more formally – do you know of any good reads on the topic?

I'm not really qualified to do an assessment on that, but personally it SGTM. Privacy is there for a reason and can be solved in alternative ways (e.g. forking a dependency)

A server using async connector can theoretically handle a million connections, not read from them and slowly process them, stalling clients that are waiting for results, which is backpressure.

Has anyone tried to implement a sandboxed Clojure interpreter? I’ve looked at SCI and it’s a nice interpreter, but I’m afraid of two things: • infinitely long execution (I can solve this by timing the execution and calling Thread/stop on the executing thread from a controller thread) • infinite memory allocation (I haven’t found a way to prevent this)

Java security is built into the jvm and can do a lot of things. there is clojail as well, which puts some additional clojure restrictions over that

I have used clojail to run snippets from clojuredocs to verify specs in speculative, using this updated fork: https://github.com/borkdude/clojail (been a while)

can also sandbox at an outer layer using something like Firecracker, the VM technology that AWS uses for Lambdas

depending on the use-case

you can also run a dedicated JVM for the sandboxes and set -Xmx maybe?

...what is the use-case?

supporting user-submitted scripts in a multitenant runtime (a la Jenkins extensions)? implementing something like an interactive tutorial site (a la http://repl.it or 4clojure)? ?

basically in a mini-game I’d like to have players submit their own routines in clojure which would operate in a very closed world

unfortunately it’s very easy to nuke the server by saying (last (range 10000000000))

@roklenarcic this is something I have thought about for a while in sci and I haven't been able to come up with a satisfactory solution - it should be solved at a deeper level, like you say, e.g. use a timeout with Thread#stop or even an lower level

The best approach is usually run those as standalone processes. For Clojure this might mean a non-trivial overhead of course, especially in terms of memory usage and also the execution time (maybe GraalVM could help with this) it depends on how much resources do you want to give them and the latency you can tolerate. Here's a related question: https://stackoverflow.com/questions/6599985/is-there-a-way-to-put-maximum-memory-usage-limit-on-each-thread-in-java#:~:text=To%20place%20a%20limit%20on,memory%20each%20thread%20is%20using.

If you go that road, maybe setting it up as a CGI-ish solution works. I bet they have solved similar problems there.

https://github.com/eccentric-j/clj-cgi-example

just randomly, would it be possible to use clojure's asm to create classes that defeat what clojail is attempting to do?

@dpsutton I guess you can try that out in 4clojure

It would be surprising if clojail could not be compromised in some way. I don’t think it was submitted to any kind of sustained attack effort by experts before

This is why I like offering such solutions in browsers. If people mess up, it's their problem ;)

If I were to make a Peer-to-peer application in Clojure, is the state-of-the-art the java interop with pastry?

jGroups perhaps? http://www.jgroups.org - was pretty pleasant to use via the interop