Clojurians Log v2

Cool problem! Your solution seems fairly compact, and as long as you're not looking for automatic clause combining (stuff like collapsing (< x 20) (< x 10) ) I'm not sure going all the way to rules engines is worth it. That said, if you're already using resolve and symbols, you could consider just using clojure's homoiconicity for your clauses. I'm thinking about the case where you might need to access more than one piece of data for a clause (like comparing (< :x :y) for x and y in the input value). What about an alternative representation like [accessor-map comparison]? E.g.

(def clauses '[[{:price price} (< price 2000)]
               [{:offer offer :ceiling ceiling} (= offer ceiling)}])

Very interesting suggestion, thanks!

would it make sense for packages to include the version in their namespace? would it not be more in accord with the idea of immutability and robustness? why should my project break if some implicit dependencies lose consistency over a simple update?

i would understand if this limitation is due to the legacy way of doing things in production

Rich Hickey didn't recommend keeping immutability of code -- he recommended that updates to code should not break the API provided by previous versions, i.e. you can add new functions or other kinds of Vars, or extend their capabilities to require less in terms of inputs, or provide more as return values, but not vice versa. https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md

If you want to break the current API, then you should consider creating a new name for the library and its namespaces, so it is a new thing, not a broken version of the old thing.

@andy.fingerhut are there any ecosystems that consider the version to be part of the namespace?

@andy.fingerhut does it make sense to have such a thing?

One way to provide a new library that is similar to an old one is to put a number in its name, e.g. the first version was mygreatlib, and when you want to break the API of mygreatlib, you instead publish mygreatlib2. I believe the talk I linked above mentioned a few cases of Windows APIs where this was done, but it was pretty rare.

@andy.fingerhut but say different packages dependencies are pulling different versions of the "same" library, would there not be a conflict?

Depending upon what mygreatlib and mygreatlib2 do, they might conflict with each other, e.g. they both open a network socket listening on the same TCP port, or might write to a config file with the same name, etc., but in many cases there would be ways to prevent such conflicts. I could imagine some library functionality where it doesn't make sense for them both to be loaded at the same time in the same program, and some where it would make sense.

If they have different names like mygreatlib and mygreatlib2, most package management systems will treat them as different libraries that can both be depended upon. Whether they can both be used from the same program is a separate question. Most package management systems I am aware of that have library names and version numbers do not let you depend on both mygreatlib version 1.1 and mygreatlib version 1.2 at the same time -- you must pick one.

Slightly off topic but immutability in code might be one of those things that needs a lot more of the language built up around it, like how you wouldn't be doing as much functional/immutable programming in Java. Interesting experiment on this very idea is Unison lang https://www.unisonweb.org/

hmm, but isn't it bad practice to hide such stateful actions in the behaviour of a namespace? i suspect that in any case the namespace shouldn't open files or ports implicitly without knowledge of the caller

what you described sounds like a bug to me, it does not justify the expectations of the package manager

come to think of it... how do linux package managers treat this?

@eagonmeng unison seems like a hardcore implementation of what i had in mind

If the library you are using is intended to help serve web requests, then it is supposed to listen on TCP sockets. In that particular case it is likely that merely loading the library doesn't open up sockets -- you would have to make an explicit call, and in most cases you could probably provide an optional, or required, parameter for the TCP port number to listen on, so even then two differently named libraries that do that might not conflict with each other.

Linux package managers and Clojure Leiningen/deps.edn/boot are doing related, but still quite different things.

There are Linux packages that conflict with each other, even if they have completely different names. Sometimes this is documented, sometimes not.

@eagonmeng i am not sure if the namespace/version issue is addressed there

Unison's pretty early on but as all code is immutable and more importantly, all names are transient, you get that extra layer of indirection that symbols/vars give in Clojure. See https://www.unisonweb.org/docs/codebase-organization#upgrading-libraries-to-the-latest-version

so if different versions of the "same" lib are present, does the importer include the version explicitly where the import is declared?

how does a namespace know which version it is importing?

Is it correct to say that in transit, if a json map is produced as output then it must be produced via json-verbose and there won't be any caching

i'm trying to write a transit library for elm (just doing the decoding side for now)

Code is always referenced by hash, so since it's completely immutable, nothing is ever updated. A new "same" version of a lib needs to be explicitly updated by a "patch". Think of it exactly like Git - when someone "updates" the dev branch with code, or when the update the library version, you have to merge the changes. If those merges are like cases in git where you have strictly additive changes (like Rich talked about in non-breaking changes) then you have nothing to do. If any definitions are at all changed, you need to merge those locally. Unison has an interesting mechanism for always making such "patches" or diffs visible, so you always have the granular power to update functions you want to. In fact there's not really a concept of a library, just a bunch of functions that might depend on one another as part of a big group.

It's an interesting idea overall, but I don't think it applies to Clojure very well, as I agree with Rich in general that code doesn't need the same immutability properties. I think you get a lot of leverage out of the idea if you build an entire system around it like Unison (free distributed routines, zero compile ecosystems, no "broken" code ever), but a lot of what makes that work is the Haskell-level type system and needing to rebuild the entire thing from the ground. I'm curious how many of the ideas could be somewhat imported into Clojure though

Is there a way to define a "mutually defined" :inline implementation for a function? Something like

(defn foo
  {:inline
   (fn [code]
     (if (pred code)
       `(inline-form ~@code)
       `(foo ~@code)))}
  [& args] (,,,))

I just noticed that when constructing a string from a keyword the string is not identical to its literal form. Anyone knows why this is so?

(= "foo" "foo")                ; => true
  (identical? "foo" "foo")       ; => true
  (= (str 'foo) "foo")           ; => true
  (identical? (str 'foo) "foo")  ; => true
  (= (str :foo) ":foo")          ; => true
  (identical? (str :foo) ":foo") ; => false

Keyword's toString() instances a new string, while ":foo" is statically initialized in the context of your code

Unless you use some JVM option to force it to remove duplicate strings they won't be identical

Ah, thanks. But for a symbol it is not like that, I take it?

Beware of context

(def sym 'foo)
(def s "foo")
(identical? (str sym) s);; => false

Also, can I safely instruct others that keywords are always identical, no matter how they come to be?

they were probably only identical because you initialized (symbol "foo") and "foo" in the same context

yes

because keywords are interned

I thought strings where too, and was writing in a beginners guide that they are, then tested it some…

While I have your attention then, the reason this throws is that map lookup is based on equality?

{(str :foo) :bar
 ":foo" :baz}

Yes, but it probably throws on the hash collision even before testing for equality

duh! Thanks!

Strings are usually interned but you have no guarantee they'll be the same object if not initialized statically. Especially with dynamically created strings you wouldn't want to intern them. Keywords are always interned and guarantee identity, which is why they make excellent map keys

When in doubt I just read the source

It was this suitability for map keys that I was writing about.

Then I went on a tangent with how strings can be lookup efficient too, but I have scrapped that now, since it was a bit deeper than suites the context.

The line between special forms and macros just became blurry to me. I see that the defmacro of let has metadata saying :special-form true . What is the implication of this? let is listed as a special form here as well: https://clojure.org/reference/special_forms#let

Do we have a channel to ask about Clojure internal implementation?

For example how namespaces work on the compiler level

I've been reading clojure core but I still have questions

@kevin.van.rooijen yes, #clojure-dev is the place

Thanks!

@kevin.van.rooijen https://ask.clojure.org/ is also a good place

I think #clojure-dev is more for development of clojure itself

so this #clojure channel would probably be appropriate for asking questions about clojure internals as well

Anyone here ever extended a function via metadata with Component's lifecycle protocol?

Probably you mean implementing Component's lifecycle protocol via metadata? Yes, it works well

This just crossed my mind: since clojure.spec.alpha is a transitive dependency, anyone using it should probably declare this as an explicit dependency.

That seems wrong

You should consider it part of Clojure

Even though it’s broken into a library from an impl point of view

I believe this is an implementation specific thing - let can be thought of as a special form, but in practice it's defined as a macro in terms of the actual special form let*

Can we rely on clojure.spec.alpha being there always from now on? If not, then declaring the dependency is probably a good idea for future proofing?

Thanks! Does this make sense then? https://github.com/BetterThanTomorrow/dram/blob/28958158f9c41cc13dd814492857c2f09f371a96/drams/hello_clojure.clj#L468

Makes sense to me, but it doesn't sound like an entry-level comment. let is a special form, that's it. The fact that it's implemented as a macro is an implementation detail that regular users should not care about, let alone beginners.

I mean implementing it for a function. I want to return a function which will depend on a closed over stateful component. I ended up going the other way and implemented IFn on the record

True, true. But it might be confusing to read about it in a guide where you are instructed to examine all things and find that defmacro. I actually had let among the macros in the guide first, because that’s what I thought it was.

Tricky to strike the balance. The guide promises to be basic in both the conflicting meanings of the word (short, yet foundational).

What should the function do?

I solved it by not leading with that non-beginner material, and by not explaining it so much when I do mention it: https://github.com/BetterThanTomorrow/dram/blob/56d259ab6af50ca12aa7f3aff5ceca252b61ab52/drams/hello_clojure.clj#L468

It's the handler passed to a web server

somewhere in the handler I have an interceptor with a kafka producer

I have a pretty hard time understanding what you're trying to do. It would help to state your original problem, plus your current attempt and its perceived drawbacks

I think it looks better. But I'd still mentioned that it's a special form (right now it says "is a form"). Also, a typo at https://github.com/BetterThanTomorrow/dram/blob/56d259ab6af50ca12aa7f3aff5ceca252b61ab52/drams/hello_clojure.clj#L505

The intent was that spec is part of clojure and you don't need to depend on it explicitly

whatever migration we do for spec 2 will depend on that assumption

Thanks. It is part of a section labeled special forms, but I changed it anyway. Staring at where I have a typo without seeing it.

Now I see it! I should rest from this a bit now…

I have a web server where I want to produce incoming messages to Kafka. I wanted to experiment with interceptors a bit so I used Reitit and Sieppari. I handled the production to kafka by instantiating an interceptor which takes the producer as an argument. The interceptor is instantiated when creating the router, so the function which instances a router needs to take the producer as an argument The implementation which emerges from this is two components, one for the web server, one for the producer, where inside the web server I call (http/start-server (routes producer) options) I wanted to create an implementation where the server doesn't directly depend on the producer, only on the handler

> The interceptor is instantiated when creating the router, so the function which instances a router needs to take the producer as an argument This part might be the odd one. Creating routes shouldn't have to pull much detailed knowledge about Kafka concerns. Wouldn't it be possible to create an interceptor in the producer component definition, so that the web component simply depends on producer and pulls the already-done work of creating an interceptor?

sounds good, thanks

And then the function which instances the routes will take the interceptor as an argument?

Still seems a bit arbitrary. Why just one interceptor as an argument? what about when the API expands in the future?

Is there a way to make byte arrays comparable by value, in the context of of a unit test. For example, I’d like to write a midje prerequesite with a byte array argument, something like this:

(fact … (provided (foo (.getBytes "abc")) => nil)

(.equals (.getBytes "aaa")
         (.getBytes "aaa")) ; => false

(java.util.Arrays/equals
 (.getBytes "aaa")
 (.getBytes "aaa"))

I know

(= (vec (.getBytes "aaa"))
   (vec (.getBytes "aaa")))

My question is: how do I tell = to use java.util.Arrays/equals instead of .equals ?

you cannot "tell" =, it's not a protocol (and even if it was, you should not override it for types you don't own)

So what can I do in the context of unit tests?

write your own =

The problem is that I don’t know how to progragate my definition of = to midje

Then don't use midje =)

I was hoping that = would call to a method from a protocol that I could override in the context of the unit tests

I'm half kidding, but you can't state a "fact" using some wrapper function?

A fact, yet. But not a prerequisite

Ask in #midje

Yeah

I’ll do

The interceptor wouldn't be an argument but an extra k-v in this Component systems inject component dependencies into one another via using. See ExampleComponent in https://github.com/stuartsierra/component/blob/9f9653d1d95644e3c30beadf8c8811f86758ea23/README.md , where database is used / depended-on but never passed explicitly. Translating that defrecord pattern to vanilla defns, the result would be:

(defn start-routes [{{producer-interceptor :interceptor} :producer
                     :as this}]
  (assoc this :routes (make-routes producer-interceptor ...)))

So, adding more 'external' interceptors means simply pulling more data from the destructuring part, which avoids having to add more positional arguments.

Can anyone point me to a connection pool library like https://github.com/mperham/connection_pool in Ruby? Seems a very Clojure-y concept to have a lib that splits this concern out from eg. hikari. Or does everyone just roll their own?

Isn't HikariCP specifically a connection pool library?

That's what I've always used, and it's always Just Worked™.

Maybe I’m reading wrong … isn’t it a JDBC connection pool lib?

It is.

Oh, yes it is.

My use case is for keeping 10-20 TCP connections to Apple’s notification service going.

I didn't realize you were talking about connection pooling in a more generic sense.

So many JDBC connection pools around - perhaps it would be easier to create a JDBC-compliant interface for TCP connections. :D

A quick search does not turn up any TCP connection pooling libraries for Java...

haha, maybe. I’m just a little surprised since it’s almost difficult to avoid using the word “decomplected” to describe what I’m after 🙂

there are generic object pools like https://commons.apache.org/proper/commons-pool/, I haven’t used them so can’t say if they are any good

That Ruby library doesn't seem like a whole lot of code — maybe just port that to Clojure? 😛

Hah it might come to that.

https://github.com/redis/jedis/blob/master/src/main/java/redis/clients/jedis/JedisPool.java

example of using the generic object pool

https://github.com/redis/jedis/blob/master/src/main/java/redis/clients/jedis/util/Pool.java

good example!

can’t promise a Clojure wrapper lib but I suspect a reasonable interop example is within my abilities.

How would pooling work for "plain" TCP? Wouldn't you have to do a bunch of programming just to get the pooling library to understand your protocol? How would it know whether a connection can safely be returned and used by another thread (i.e. it's not in the middle of framing a message or waiting on a server response)?

Yeah, looking at the ruby one it’s relying on every “connection” being an object so presumably in Clojure terms each “connection” would probably also have to be an object conforming to some sort of lifecycle protocol, with the actual connection buried deep in there someplace.

something’s got to be doing the bookkeeping.

taoensso.carmine.commands has this code:

(defonce ^:private command-spec
  (if-let [edn (enc/slurp-resource "taoensso/carmine/commands.edn")]
    (try
      (enc/read-edn edn)
      (catch Exception e
        (throw (ex-info "Failed to read Carmine commands edn" {} e))))
    (throw (ex-info "Failed to find Carmine commands edn" {}))))

(require '[taoensso.encore :as enc])
(enc/slurp-resource "taoensso/carmine/commands.edn")

Of course. Despite all the thinking and staring at the above text for a good minute before posting the message, only after hitting Enter did I realize that I'm compiling Clojure code on Heroku, along with some of the libraries.

Seems to work without compilation. But frankly, I still have no idea why io/resource wouldn't work after compilation. Strange.

Aaaand now I'm getting "App boot timeout" errors without the compilation. Sigh.

Are you perhaps neglecting to deploy to Heroku the directory taoensso/carmine containing a file commands.edn?

I guess you probably are deploying such a file, if a remote REPL returns a good result for (enc/slurp-resource "taoensso/carmine/commands.edn")

Yep, it's all there.