Clojurians Log v2

Hello, I use immutant with ring. I want to write something like "tail -f" with websockets. I pass an InputStream to send! but it doesn't do what I want. Does anyone have an idea of how to do this ?

What does tail -f do?

googles furiously

It means tail -f command will wait for new strings in the file and show these strings dynamically. This command useful for observing log files .

For example try, tail -f /var/log/messages.

alright... that might help get the ball rolling on that question

Hmmm... you will probably have to send discrete packets through websockets... rather than a stream, but my confidence is only 55% on that statement

> discrete packets Can you explain what this means ?

send one drop at a time instead of a river

wait I will read more about inputstream, the stream part is confusing

I don't know immutant's websocket abstraction specifically, but the APIs I've seen expect to take objects that they then write to the socket

in theory, just writing an inputstream to a socket should be possible with <http://clojure.java.io/copy|clojure.java.io/copy>

oh neat.. yeah hopefully you get it resolved, 😃

ah thanks let me try this

and I'd expect send! to be something more abstract, that tries to serialize the objects and do the writing for you - it's higher level than what you probably want if it's anything like the send! from sente for example

wait you mean like do (io/copy inputstream channel) instead of (send! inputstream channel) ?

this really depends on what "channel" even is

sorry, I don't know immutant, but I've done this with other libs

it's a org.projectodd.wunderboss.web.async.Channel

https://github.com/projectodd/wunderboss/blob/master/web/src/main/java/org/projectodd/wunderboss/web/async/Channel.java

yeah, that's not an OutputStream, so you can't use io/copy

it wants to take an Object, which it then wraps up and puts on the socket

okay so I think the easiest solution for me is to do what @sova said

so like I open a file, and then how do I know that it changed? is there like a callback to see if it changed?

or should I just do a loop and check if it changed?

you can use a WatchService, I assume some clojure library is available to wrap it if you aren't comfortable using it directly

I will look this up, Thanks 🙏

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/FileSystem.html#newWatchService()

I just straced this On linux "tail -f" uses the select syscall.

I mean read should be blocking right? Until data is written?

Oh or that's only in a fifo

Yes I think watchservice would work @noisesmith thanks

Is there a good comparison regarding modern IDE support of Clojure/ClojureScript ? Any default preferable options for commercial development?

thanks

There’s Cursive for IntelliJ: https://plugins.jetbrains.com/plugin/8090-cursive

Not specifically created to be a comparison, but this may help you understand the basic differences. http://practicalli.github.io/clojure/clojure-editors/ They all provide excellent support for Clojure and are free for commercial and non-commercial work (although sponsorship to the project maintainers is always appreciated) IntelliJ and Cursive requires a license. It also requires an annual fee for commercial work. Recommendation is to use the editor you prefer, to reduce the learning curve.

Is there a more idiomatic way of writing if x x y?

(or x y) ?

I must have had a weak cup of coffee this morning 😛

Thanks

Anyone beginned clojure without any knowledge of java/javascript? What did you do to use clojure interop well?

I mainly use reagent, so no need for much JavaScript interop unless you really want to. For more complex UI, then there is re-frame There are excellent courses on reagent and re-frame here: https://www.jacekschae.com/ To use npm packages, many use http://shadow-cljs.org/

is there something special I have to do in a macro to inject meta data into the macro expansion? for example, I would like this macro to define a dynamic variable whose name is given by public-name .

(defmacro defn-memoized
  [[public-name internal-name] docstring & body]
  (assert (string? docstring))
  `(let []
     (defn ~internal-name ~@body)
     (def ^:dynamic ~public-name ~docstring (memoize ~internal-name))
     )) 

this is supposed to define a function which is memoized, but which I can rebind using binding to limit the dynamic extend of the memoization

but at the binding-site, clojure complains the variable is not dynamic. not sure if this is a loading order problem, or a problem with my macro not injecting meta data correctly

In a macro the metadata is being applied at the wrong time - instead use with-meta to apply the metadata in the expansion

@alexmiller can you give me an example? I looked at the doc for with-meta but it's not clear to me how the macro interaction is working with metadata. It would be nice if there were an example either in the defmacro docs or the with-meta docs. I can create such an example, once I understand it.

I.e. do I need with-meta to be called a macro expansion time, or do I want the expanded code to contain with-meta ?

The latter

you'll want something like (with-meta ~public-name {:dynamic true}) here

that doesn't work, at least as I understand your suggestion.

(defmacro defn-memoized
  [[public-name internal-name] docstring & body]
  (assert (string? docstring))
  `(let []
     (defn ~internal-name ~@body)
     (def (with-meta  ~public-name {:dynamic true}) ~docstring (memoize ~internal-name))
     ))

sorry, probably need to evaluate the outer expr ~(with-meta public-name {:dynamic true})

seems the compiler is much happier with that.

(defmacro defn-memoized
  [[public-name internal-name] docstring & body]
  (assert (string? docstring))
  `(let []
     (defn ~internal-name ~@body)
     (def ~(with-meta public-name {:dynamic true}) ~docstring (memoize ~internal-name))
     ))

What’s the Clojure equivalent for https://github.com/mperham/sidekiq or https://github.com/contribsys/faktory for background processing? There’s a 4 year old https://www.reddit.com/r/Clojure/comments/48ur9m/distributed_workers_with_retry_backoff_looking/ with some comments, looks like https://github.com/ptaoussanis/carmine is the only one under active development.

I use <https://github.com/lerouxrgd/celtuce> as my Redis client

Can’t really tell if it supports background processing looking at the README, does it?

I've never found a convincing one based on Redis. Or on anything. I'm not aware of any Clojure dep that would come with a web-ui and a deployment cookbook. Because Clojure (and really, the JVM) has solid multi threading support, lots of the things you would defer out of process in a GIL-VM like Ruby's or Python's can be done in-process instead. (future (sendmail ...)) In-process retry/backoff you'll have to deal with yourself, I'm pretty sure I've seen a nice library for this recently but I can't recall the name .. This being said, there are many good reasons to distribute tasks on several processes. For such cases, I use RabbitMQ (with http://clojurerabbitmq.info/articles/guides.html). It provides the retry feature, and better (ACK, NACK) semantics than Sidekiq, imo - if a worker disappears without terminating message processing, message will be made available again, no need to requeue on exceptions. Setting up RabbitMQ for a Sidekiq-like queue processing is probably the simplest RMQ use case - don't let the documentation confuse you 🙂 The backoff thing is however https://www.brianstorti.com/rabbitmq-exponential-backoff/. Apache Kafka seems to be the rock solid, all powerful solution for this. It might be worth starting with Kafka right away and spare yourself some of the Rabbit nonsense. I have yet to try it. I've chosen RabbitMQ because it looked simpler to operate and develop with and my needs are modest (no HA, no high throughput).

@zor, Thanks for the detailed response. I’m merely looking for a background processing lib. Using RabbitMQ for this could be something I’d explore if there’s no reliable library to handle background jobs. Will also read upon (future (,,,)) to see if that suits my need. The use-case is indeed to send emails as notifications upon an update.

There is also https://clojure.org/reference/agents, which comes with a pre-made Thread pool, clearer semantics, and more pre-made useful things for you to build on.

Thanks, will read up on it too.

And ... finally the "Message Queues" bits of https://www.clojure-toolbox.com/ 🙂 Notably https://github.com/Factual/durable-queue seems a nice in-between agents and rabbit. My gut feeling is that you can get stuff done with Agents today, and move to something more robust if you need to.

Noted this as well. https://www.clojure-toolbox.com/ is awesome! Thanks again! 😄

Also, is there any relevant channel here that you’re aware of for discussion of system designs by any chance?

What is the limitation of carmine as a messages queue I might ask? I'm looking for something similar myself. I also found clj-faktory (https://github.com/apa512/clj-faktory) but carmine is under more active development. I know it doesn't have a web ui but I can do without one.

for system designs I think there is #architecture

potentially dumb question, I need to pretty-print an edn map to a file, including any meta-data, basically doing what (binding [*print-meta* true] (pr my-cool-map)) would do

(spit "foo.edn" (with-out-str (binding [*print-meta* true] (pr my-cool-map)))) is one way

there are more cumbersome ways if you need better performance (open a stream to a file and bind out to that)

Performance isn’t an issue, but when I open the file it’s not pretty printed

well use, pprint then

pprint doesn’t seem to abide by the *print-meta* binding

perhaps I need to look into creating a custom writer

interesting. there is actually an old ticket for this https://clojure.atlassian.net/browse/CLJ-1445

the attached patch there seems to work

if you'd like to vote for this: https://ask.clojure.org/index.php/3315/pprint-prints-some-metadata-when-print-meta-bound-true-but-not

can't say I've ever noticed this before

Oh nice, thanks for finding that, voted! Also, maybe I’m just missing it, but where can I find the attached clj-1445-workaround-v1.clj file?

it's on the CLJ ticket

ah, derp, cheers Alex 🙂

One limitation of the Carmine's Message-Queue is that it doesn't seem easy to interop with non-Clojure workers - something that would be doable with RabbitMQ or Kafka. No CLJS support either, so even if you're interop needs are with Node.js processes, you seem to be out of luck. clj-faktory - as part of a faktory system, is inherently polyglot, so this solves that. And you get the nice faktory UI and scheduler, and commercial support options. I wouldn't be worried about active or inactive development. It's rather common in Clojure to have stuff that works and is in a finished state 🙂

I'm seeing this when I turn on (set! *warn-on-reflection* true) : Reflection warning, foo/bar/policy.clj:0:0 - call to java.lang.IllegalArgumentException ctor can't be resolved. Yet, nowhere in my codebase do I use IllegalArgumentException. Would it be a 3rd party dependency issue?

It could be something that foo.bar.policy is requiring, but normally reflection reports the correct filename. Maybe a macro expansion (but, again, I'd expect the filename/line number to be a bit more meaningful).

It's a puzzler

I dropped an actual patch on the clj ticket too to actually fix this

I guess: comment out everything in policy.clj except the ns form and if you still get the warning, start commenting out required nses?

Worth a shot, will give it a whirl 🙂

Useful stuff, thank you! Gonna check out clj-faktory more to see if it implements all the stuff I need from faktory

Hello! I have problem. Can run frontend but can't run backend cause "project/server.clj couldnt locate on classpath"

You are using a shadow-cljs template? I know someone recommended you ask in #beginners, but I just wanted to point out that there is also a #shadow-cljs channel where a larger fraction of people might be familiar with that template.

thank you

Sorry, I misled you again 🙂

but you should be getting closer

im here 😉