Clojurians Log v2

I'm having trouble finding the source of disparate run times in subsequent group-by calls. First call is slow. But then if you evaluate the exact same thing it's fast. Is there some kind of caching in group-by? group-by uses transient collections. Is there internal caching in transient/persistent! ?

example code here: https://pastebin.com/PszjtJcB

(time …) is not a good benchmark; Use criterium (library), or if you really care/need performance, something like YourKit is essential (it’s a paid product, it’s worth it IMO, I’ve used it)

It's due to laziness. If you add (doall tuples) before the first time measurement, then the times should be similar

Try switching tuples to: (def tuples (mapv vector kmers (next kmers))) … to see if it makes a difference.

fwiw, your example seems like a reasonable usage of time (to me).

thankyou. thats it

I agree, I didn’t notice the big discrepancy at first (which is very obviously caused by the laziness of map as you pointed out)

Has there been any attempts to do MSA with clojure? I’d like to know if there’s any good sample projects out there in github or else. So far I’ve found one book ‘microservices-with-clojure’ but couldn’t find more, definitely yet to be a mainstream way of developing with clojure. But I think the event driven nature of the architecture suits with immutability and pure function concepts in clojure (or with other fp languages). I’d like to know more the community’s perspectives on this and current situation.

Ya, this would have made more sense, but I think the two are just called arg-list and the context tells you which one is being refered too.

What's MSA?

I meant for MircoService Architecture

A system where services (typically having its own db) communicate with events on pub-sub basis

Lots of people do Micro-services with Clojure, you can have a look at Compojure-API, Yada, Liberator, Pedestal or Reitit, in no particular order, those are the most popular options for writing RESTfull HTTP API micro-services.

Then for Pub/Sub people use Kafka, RabbitMQ, AWS SNS/SQS, AWS Kinesis, etc. All those have great client support from Clojure through their Java library and a lot of them have plenty of Clojure wrappers for as well if you prefer.

Seems like I was quick to judge from just finding one book. I stumbled upon this clip and it seems great. https://www.youtube.com/watch?v=PWuuh48r2tc

There's more too, I think Redis Pub/Sub and ActiveMQ are popular as well

Ya, in fact there's probably too many options all with equal popularity, so it can be difficult choosing which one to go with since there's no clear winner.

There's also some good support for GraphQL, which for some use case is nicer to use over REST

So those libraries support clojure environment, yes? I mean it will be pretty sad if one has to call java apis in clojure code. I hope the environment revolving msa-clojure is at least actively being discussed.

Sure, I’ve used Lacinia once with clojure, and it was also great.

All the rest ones I mentioned are Clojure first. The Pub/Sub ones are Java based. But you shouldn't find it sad, Clojure is designed to leverage Java where there is no need to improve over it, it explicitly doesn't re-implement what Java does well already, and things like stateful clients to remote services is done very well by Java, the Clojure interop syntax makes the usage a breeze and even more convenient and natural than it be in Java itself.

I'm sure you'll find plenty of Clojure wrappers over them by many people, but personally I'd just use them directly with Clojure interop, they all generally just revolve around:

(doto (SomeClient. config)
  (.sendMessage message))

Hum... Actually the AWS ones have native Clojure clients maintained by the company that employs the Clojure core team itself: https://github.com/cognitect-labs/aws-api

is there a tutorial about how a macro (defined by defmacro) can extract meta data from its list of arguments?

I’d like to invoke a macro as follows: `(my-macro a b ^{some-meta-data-1-as-hash} c d e ^{some-meta-data-2-as-hash} e f)` and within the macro definition figure out that two hashes were given, the first associated with c (which might be any syntactical form, and the second hash was associated with e which again might be any syntactical form.

hmm actually its worse than that. The syntax is

(my-macro ([a b c] ….)
          (^{some-data-as-hash} [a b [c d]] …)
          ...)

@jimka.issy

user=> (defmacro foo [a] (list 'do (meta a)))
#'user/foo
user=> (foo ^:foo [])
{:foo true}
user=> (foo ^{:foo :bar} [])
{:foo :bar}

couple of things are fuzzy.. meta gets the meta data associated with its given value right? When if I pass the same value twice to the macro, how does meta know which is which?

e.g. (mymacro ^meta-data-1 42 ^meta-data-2 42)

you can't do (foo ^:foo 1) since metadata cannot be applied to a number

interesting. that means I really don’t understand. I thought meta data was applied to the list (foo 1) with some magic indication that it applies to the 2nd element.

There is hardly any magic in Clojure. meta just looks up metadata on objects that support it (notably collections and symbols, all implementing clojure.lang.IObj)

When you write (defn foo [^String x]) what you are doing is you are writing a list with symbols. The symbol x has metadata {:tag String}. And this data is processed by the defn macro (or rather the thing below it, the fn* special form)

so if I make a macro call with the same symbol twice, can I have different meta data on each occurance of the symbol? (mymacro ^meta-data-1 a ^meta-data-2 a)

yes

ok, that means the meta data is not assoicated with the symbol, because a is eq to a

or am I even more confused that I imagined?

the metadata is associated with a specific symbol. symbols are not interned in Clojure, because they do not have context-free meaning, like keywords or strings

@jimka.issy so if I’m understanding correctly, you’re trying to associate meta data to a binding form, and have the macro behave more or less like (defn …)

Is that about right?

See this example:

user=> (defmacro foo [a] (list 'quote (map (juxt identity meta) &form)))
#'user/foo
user=> (foo ^String x)
([foo nil] [x {:tag String}])

are you asking what i’d like to do after I extract the meta data?

sorry, I don’t understand your question

Nevermind 🙂

I’m writing a macro which is very similar to fn. fn allows the user to specify several sequences, the first element of each is a lambda-list (an arglist in clojure-speak). I want to optionally associate meta data with some of these lambda-lists

See example: https://clojurians.slack.com/archives/C03S1KBA2/p1619687803233400

For writing a macro that’s similar to defn/fn see this: https://github.com/lilactown/helix/blob/master/src/helix/core.clj#L116 (it has been useful to me to look at when writing something similar)

@borkdude that’s a neat example 🙂 The short version (if I’m correct):

(defmacro foo-short [a]
 `'~(map (juxt identity meta) &form))

Took me 5 min to figure out, a small mind bender 🤯 😄

That triple combo ``'~`

I have to admit that I really don’t understand. but my code works. I have written a macro named dsfn which expands to destructuring-fn as follows:

(dsfn ([x]  'aaa)
      (^{x Boolean}
         [x y] 'bbb)
      ([x y] 'ccc))

(destructuring-fn
  ([[x] {}] 'aaa)
  ([[x y] {x Boolean}] 'bbb)
  ([[x y] {}] 'ccc))

(defmacro dsfn [& forms]
  (letfn [(process [form]
            (let [meta-data (meta (first form))]
              (cons [(first form) (if (nil? meta-data) {} meta-data) ] (rest form))))]
    `(destructuring-fn
      ~@(map process forms))))

What does destructuring-fn look like?

Is that another macro?

yes it is another macro.

https://gitlab.lrde.epita.fr/jnewton/clojure-rte/-/blob/master/src/clojure_rte/rte_case.clj#L249

I hope that link is public. tell me if it is not

Yes, public.

That is neat to try to match on the structure/shape of the data.

What part of dsfn is unclear? It seems to do what it says.

hmm. perhaps it is clear, and I’m just self deprecating.

That looks like a neat project; Is it part of research or just for fun?

How am I suppose to distinguish beween vector-ish and list-ish ? For example, fn distinguishes its semantics based on whether an argument looks like [a b c] or ([a b c] …) ? I had been using the function list? but apparently given (cons 1 (list 2 3 4)) the list? predicate returns false.

my dsfn macro above calls (cons ….) which produces a non-list 😞

looks like in the code of fn that it calls vector? rather than calling list? to distinguish the two cases.

You probably want seq? instead of list? , which is specifically for IPersistentLists

thanks. yes it is both fun and research.

I’m happy to have feedback.

it’s part of a presentation at the European Lisp Symposium in two weeks.

take a look: https://gitlab.lrde.epita.fr/jnewton/clojure-rte/-/blob/master/doc/dsc.md

always use seq? is probably a good rule of thumb

Nice, any hope this can work in Clojure and ClojureScript?

I have no experience with clojure script.

the theory behind it is that it should work in a large range of dynamic reflective languages

a student of mine wants to investigate javascript

what do functions like int? and integer? and number? double? do in clojurescript?

@jimka.issy perhaps you want to create a clojure-rte channel here in Slack to discuss this?

All of the above functions exist in CLJS; there are some gotchas around numbers, but those are all caused by the differences between JS and the JVM;

Things like this actually run ok in CLJS:

(+ nil 1)
=> 1 

From here: https://clojurescript.org/about/differences • Numbers • ClojureScript currently only supports integer and floating point literals that map to JavaScript primitives ▪︎ Ratio, BigDecimal, and BigInteger literals are currently not supported ◦ Equality on numbers works like JavaScript, not Clojure: `(= 0.0 0) ⇒ true`

Related to something in #beginners : Has there been any discussion around allowing (Class/staticMethod foo bar baz) if staticMethod is a Java method receiving a variable number of things at the end, instead of forcing Clojure users to create an array?

I guess this is more difficult to support in Clojure because of the dynamic typing? (currently you can look at the type of array to choose the right method)

yes, fogus has been working on this

it's complicated of course, but going to take a run at it for 1.11 probably

ooh really, wow

go @fogus!

I guess there is an ambiguity when you have varargs of arrays, but this is pretty rare... it's a difficult thing to get right I can imagine

I didn't know the last one would work as well, but it does (clojure 1.11.0, babashka current).

user=> (defn foo [& {:keys [a b]}] [a b])
#'user/foo
user=> (foo {:a 1 :b 2})
[1 2]
user=> (foo :a 1 :b 2)
[1 2]
user=> ((partial foo :a 1) :b 2)
[1 2]
user=> ((partial foo :a 1) {:b 2})
[1 2]

https://ask.clojure.org/index.php/4224/java-method-calls-cannot-omit-varargs for anyone who wants to vote on it (such an old request, jira goes back to 2010!)

are you adding up votes from ask + the old jira or how does this work?

it wasn't even jira - that was filed back when we were using Assembla

primarily using ask, but we look at both

jira votes did not fully survive the transition into cloud jira (lost any votes from users that weren't migrated)

voted, thank you

did not really have a big impact - the highest rated things were still the highest rated things more or less

it has enough votes that it is already on the radar of course, but doesn't hurt

I am using a similar thing to your ask for some of the issues in my projects, but using github discussions. It makes elaborating on ideas easier while keeping the issue itself more crisp.

Why does defmulti go out of its way to prevent itself from being re-evaluated more than once?

because you would lose all the methods I think

can't it update itself if it already exists?

what problem are we trying to solve?

(similar to how vars are updated when you call def twice)

Hum... I can see how someone might have thought that, but I think it would have been less annoying to manage that problem than try to force re-evaluate a defmulti.

@alexmiller It might be related to a discussion in #clj-kondo where someone suggests that clj-kondo should advice using a var as the dispatch function (to which I didn't agree, but this would solve their REPL problem)

I've just seen a lot of people get burned by that and use tricks to avoid it, like passing it a var to the dispatch-fn, or adding a (def multi nil) above it.

this is one of those things where repl use has tradeoffs with non-repl use

someone should say what they're trying to do out loud. It's buried in between the lines

I thought @didibus was referring to what @ddouglass said in #clj-kondo but that might not be the case and maybe it was purely co-incidental

I was wondering if there was any downside to use those trick, since defmulti goes out of its way to not be re-evaluable.

as someone who has watched rich and alex work for a little while now:

well, it's slower

didibus is asking about "why does defmulti go out of its way not to be re-evaluable"

I think what they mean is "I want to alter or iterate on the dispatch function at the REPL"

the stuff about "preventing re-eval" is removed from this, and a presumption ^

Ya, for the Var. I guess adding (def multi nil) above it shouldn't be slower though right?

no, not sure if that impacts ability of tools to analyze

changing the dispatch function could change its arity, which would also break expectations on all the extensions

In this case I wanted to know why it prevents itself from being re-evaled, I already know the workarounds at the REPL. Was trying to reason what are the pros/cons of this:

(def multi nil)
(defmulti multi (fn[..] ...))

keep in mind the extensions could be in any file

I feel it must have been written very early. Because require doesn't force reload by default, and most people at the REPL control what they reload. So I don't see why it would cause issues nowadays, but also I can see how someone thought that it shouldn't.

what is the "it" in "I don't see why it would cause issues"?

Ya, it's a bit of a tradeoff I guess. What is more annoying, reloading the defmulti by accident and having to reload all the other defmethod for it or the other way around.

it = defmulti

I think you mean, it = making defmulti reloadable

you may not even know what the defmethods are to reload them

Hello! I'm trying to run a compojure project with ring-jetty-adapter , and tried to add wrap-reload of ring.middleware.reload so I could run lein run only once while developing, but I'm receiving an exception trying to run:

Caused by: java.io.FileNotFoundException: Could not locate ring/middleware/reload__init.class, ring/middleware/reload.clj or ring/middleware/reload.cljc on classpath.
	at clojure.lang.RT.load(RT.java:466)
	at clojure.lang.RT.load(RT.java:428)
	at clojure.core$load$fn__6824.invoke(core.clj:6126)
	at clojure.core$load.invokeStatic(core.clj:6125)
	at clojure.core$load.doInvoke(core.clj:6109)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5908)
	at clojure.core$load_one.invoke(core.clj:5903)
	at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
	at clojure.core$load_lib.invokeStatic(core.clj:5947)
	at clojure.core$load_lib.doInvoke(core.clj:5928)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$load_libs.invokeStatic(core.clj:5985)
	at clojure.core$load_libs.doInvoke(core.clj:5969)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$require.invokeStatic(core.clj:6007)
	at clojure.core$require.doInvoke(core.clj:6007)
	at clojure.lang.RestFn.invoke(RestFn.java:482)
	at fiis_api.service$eval2080$loading__6706__auto____2081.invoke(service.clj:1)
	at fiis_api.service$eval2080.invokeStatic(service.clj:1)
	at fiis_api.service$eval2080.invoke(service.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:7176)
	at clojure.lang.Compiler.eval(Compiler.java:7165)
	at clojure.lang.Compiler.load(Compiler.java:7635)
	... 96 more

The dependency?

It's an ergonomics issue, if you've defined te wrong dispatch function and want to fix it at the REPL. And it seems maybe the behavior of defmulti is only for the REPL ergonomics as well. In trying to prevent the attached methods from being lost by accidentally re-evaluating the defmulti. So I was wondering if that was all there is to it, or there was like performance, or memory or some other reasons that's not just a REPL ergonomic thing.

I added [ring "1.9.3"] to project.clj .-.

And if you bypass it because you personally prefer the other way of working at the REPL, would you risk something doing so

if you are changing the dispatch function, it is not automatically clear whether the previously extended things are compatible with the new dispatch function

Well, most of the time when you change the dispatch function it's because it was wrong to begin with. So generally either the defmethod already didn't work and you're trying to fix them, or you know you plan to just change your defmulti to work differently and will go update the defmethods too.

Ya, I see the tradeoff, maybe I feel in practice if you're changing a dispatch-fn, you're probably the owner of the defmulti and your dispatch-fn wasn't working or doing what you want. And if you've got a defmulti from a library, it won't get re-evaluated anyways because require itself doesn't reload lib-spec.

But.. I need to think about that second one more. Like if I vended a library where I expect users can extend the defmulti and my code did:

(def multi nil)
(defmulti multi (fn...))

high voter turnout requires high impact issues that fire people's imaginations, not just for Clojure issues 🙂

Hi all: The San Francisco-based Bay Area Clojure meetup is looking for speakers for a May 11 (usu. 8pm PT) zoom event. #clojure-bay-area if you are interested in sharing something or know someone who might fit that description. Past zoom events here: https://www.youtube.com/channel/UCmtrRPVjcEMZrzTfAY3fd0Q Folks from remote locations works as well. Thank you!

Currently hammock-timing this one. 🙂

it will be interesting to see how much work will be needed to make it work in clojurescript

There are quite a few places where I depend on the way symbols work and also on the JVM.

:/ - (javadoc (.getBytes "hello")) opens a google search for "allinurl:[B.html"

maybe arrays should be special cased?

what’s the best/fastest way to compare two byte-arrays?

for equality?

https://docs.oracle.com/javase/7/docs/api/java/util/Arrays.html#equals(byte[],%20byte[])

this works but allocates two vectors 😕 (compare (vec (.getBytes "bar")) (vec (.getBytes "bar")))

java.util.Arrays has some methods for sorting and equality of byte arrays

how do you compare opaque byte arrays?

just the raw byte values (i.e. ignoring any semantic meaning)?

yes

https://stackoverflow.com/questions/18120706/arrays-compareto-for-byte-arrays-in-java

they just need to be compared for a db index

depending on your exact goals, you could certainly write a tight loop over the array for this purpose (in Java for best performance) or there are probably some numerical methods if you convert the byte arrays as numbers

Yeah I don’t think that’s built in anywhere

that I know of

thank you!

thanks alex!

Does anyone know about the general approach behind Roam's multiplayer feature? Wondering if there's any libraries out there that can help me implement something similar 🙂

Not sure what roam is using, but many multiplayer features are implemented on top of CRDTs or Operational Transforms(OTs). Here’s a good video that covers many of the concepts, https://youtu.be/x7drE24geUw

The guy from the video has a crdt library called automerge, https://github.com/automerge/automerge

Thank you! 🙏