clojure

New to Clojure? Try the #beginners channel. Official docs: https://clojure.org/ Searchable message archives: https://clojurians-log.clojureverse.org/
2021-01-23T00:40:39.131100Z

Clojure core inlines a lot of functions, but from a quick glance, most of it seems only to avoid var indirection, its not actually computing things at compile time. Since we have direct linking now, is that really necessary?

2021-01-23T00:47:43.131800Z

the inlining clojure feature isn't really about computing things at compile time, it is about exposing information to the compiler

2021-01-23T00:48:55.133100Z

so for example for unchecked-add, when that gets inlined as the static method call with the right argument types, the compiler then recognizes the static method and then instead of emitting a call to the static method it knows it can just emit an ladd instruction

2021-01-23T00:51:04.134300Z

I think constant folding what you usually call computing static expressions at compile time

caumond 2021-01-23T12:04:48.138600Z

Hi guys, I am building a DSL in my application. In some configuration use cases, some expert users will need to assemble the higher level part of our language: check the grammar, and easily modify the sentences. Do you have any advise on how to do that ? the approach and tooling. I have in mind to start to build the language, small sentences first, and when it will be done, try to search for tools to present graphically the language to end users. Do you have better way to do?

simongray 2021-01-23T13:50:49.144200Z

I have been exploring this are lately, but I do not have much concrete advice. The 17th chapter of The Joy of Clojure is about making DSLs, might be of interest to you.

simongray 2021-01-23T13:51:36.144400Z

I also compiled this list which you may find helpful: https://github.com/simongray/clojure-dsl-resources

zendevil 2021-01-23T14:01:57.145600Z

I am receiving multipart params that contain a file in my handler function in the server:

(defn upload-shot-video [req]
  (prn "uploading video")

  (prn "video is " (-> req :params))
  (prn "video is " (-> req :body))
  (<http://clojure.java.io/copy|clojure.java.io/copy> (-&gt; req :body) (<http://clojure.java.io/file|clojure.java.io/file> "./resources/public/video.mov"))


  (let [filename (str (rand-str 100) ".mov")]
    (s3/put-object
     :bucket-name "humboi-videos"
     :key filename
     :file "./resources/public/video.mov"
     :access-control-list {:grant-permission ["AllUsers" "Read"]})
    (db/add-video {:name (-&gt; req :params :name)
                   :uri (str "<https://humboi-videos.s3-us-west-1.amazonaws.com/>" filename)}))
  (r/response {:res "okay!"}))

zendevil 2021-01-23T14:03:27.146600Z

(-> req :params) prints:

{:_parts [["video" {:_data {:__collector nil, :offset 0, :name "55B5E1A0-8B6F-4E5D-AB7C-75CB163BC57D.mov", :type "video/quicktime", :size 1624035, :blobId "243B8CBE-15F0-4271-9BC4-5C9A9CA15FA3"}}] ["key" "VAL"]]}
(-> req :body) prints:
#object[java.io.ByteArrayInputStream 0x284cfb69 "java.io.ByteArrayInputStream@284cfb69"]

zendevil 2021-01-23T14:03:59.147300Z

But the file that I’m saving in my handler using

(<http://clojure.java.io/copy|clojure.java.io/copy> (-&gt; req :body) (<http://clojure.java.io/file|clojure.java.io/file> "./resources/public/video.mov")) 
is coming to be 0 bytes.

zendevil 2021-01-23T14:04:03.147600Z

How to fix this error?

zendevil 2021-01-23T14:11:43.148300Z

Whereas

(-&gt; req :multipart-params)
is
{}

kwladyka 2021-01-23T15:10:48.153100Z

@ps I had similar issue yesterday. Probably another wrapper already read from body and when you run upload-shot-video the body is already used. Try to move upload-shot-video to a different order. The best to run as first one, so the last one in -&gt; macro. Then you should be able to read from body but other wrappers will fail. In general ring wrappers suc** a little. This is the worst part of ring. I started to make my own even for JSON parse - the devil is in details, but this is another topic. So if I am right there is a .reset which you can use before copy. (.reset (:body req)) will reset position of read body to the beginning. Reminder - position is on the end, because another wrapper already read from body.

kwladyka 2021-01-23T15:12:54.154400Z

This is the code which I wrote yesterday. See .reset? Default wrappers don’t do that. Using it you can read body many times.

(defn ring-debug-request [handler]
  (fn [request]
    (if (#{nil "application/json"} (get-in request [:headers "content-type"]))
      (do (l/debug "request\n" (pr-str (update request :body slurp)))
          (.reset (:body request)))
      (l/debug "request\n" (pr-str request)))
    (handler request)))

kwladyka 2021-01-23T15:14:05.155100Z

In general there are conflicts between wrappers, because of that, so the order of wrappers really matter.

zendevil 2021-01-23T15:16:23.155600Z

@kwladyka wrapping (-> req :body) with .reset gives nil

kwladyka 2021-01-23T15:21:20.156900Z

Try

(println (slurp (:body req)))
vs
(.reset (:body req))
(println (slurp (:body req)))

zendevil 2021-01-23T15:32:54.157200Z

@kwladyka Before .reset I’m getting “”

zendevil 2021-01-23T15:33:03.157500Z

After I’m getting “{\“parts\“:[[\“video\“,{\“data\“:{\“size\“:2649685,\“blobId\“:\“7296F767-8880-4E69-8933-C386071C651C\“,\“offset\“:0,\“type\“:\“video/quicktime\“,\“name\“:\“56970ABA-F1A6-4BF9-A7AB-86AF895EDB5F.mov\“,\“collector\“:null}}],[\“key\“,\“VAL\“]]}”

zendevil 2021-01-23T15:34:26.158200Z

But after the copy operation, the saved video is just 216 bytes

zendevil 2021-01-23T15:36:54.159200Z

suggesting that something else is being copied to the file. Perhaps the map and not the file.

zendevil 2021-01-23T15:37:46.159600Z

I opened the video.mov file in textedit and got the following:

{"_parts":[["video",{"_data":{"size":2971246,"blobId":"D002459C-47C5-4403-ABC6-A2DE6A46230A","offset":0,"type":"video/quicktime","name":"DCDE604A-954F-4B49-A1F9-1BCC2C2F37BC.mov","__collector":null}}],["key","VAL"]]}

zendevil 2021-01-23T15:38:09.160100Z

Whereas I want to save the actual data in the video key to the file

kwladyka 2021-01-23T15:39:19.160800Z

Unfortunately I don’t have experience with uploading files to server. Sorry I can’t help more here.

zendevil 2021-01-23T15:44:54.161600Z

The issue isn’t solved yet

caumond 2021-01-23T17:13:42.161800Z

Thx simon, very interesting, even if I am still having a detailed look of all information available there, I don't think there are any resource to present the dsl to the end user, did I miss something?

2021-01-23T17:50:52.165900Z

The other day I was thinking I’d love a Clojure that compiles to PHP so I could use it on dirt cheap web servers, fortunately some contacts suggested using a cgi-script might be close. Looking into it, my cheap bluehost account does support cgi so it indeed may work. Has anyone tried setting that up or seen any articles on it? Currently I’m thinking it might work with Graal VM or Babashka but I’m not sure how to get the binary on there. Just SFTP up the binary?

borkdude 2021-01-23T17:56:41.167Z

@jayzawrotny I'm using a tiny PHP CGI script on a server which redirects a request from nginx to clj-kondo and the output is directly returned to the front-end. https://clj-kondo.michielborkent.nl/ It just runs on a VPS. Not sure about blue-host, never heard of it

2021-01-23T18:03:31.171900Z

Bluehost is in that category of bulk, shared hosting which tend to be significantly cheaper than VPS. The downside is there isn’t root access, or access to apache beyond .htaccess files. That’s pretty close to the setup I’m aiming for. I setup a handler so that if visiting a .clj file it would run it as a cgi script. So if I can get that targeted script to run it through a binary that understands it, that might work out.

simongray 2021-01-23T18:29:58.172Z

Not sure what you mean.

holmdunc 2021-01-23T18:40:21.174900Z

In a Clojure standalone executable script (i.e. a single file with a #!/usr/bin/env clojure shebang at the top), how can we access the path of the script itself? (like $0 in Bash). It's not in *command-line-args* (that's like $* aka $1 $2 $3 ... in Bash).

kwladyka 2021-01-23T18:41:28.175500Z

#!/usr/bin/env clojure is it something official and common? Do we have “bash” scripts in Clojure heh. I didn’t hear about that.

holmdunc 2021-01-23T18:43:23.178100Z

Of course it's possible. All languages that are good citizens of Unix support shebangs!

Mno 2021-01-23T18:44:56.179600Z

Well if I understand correctly that header would tell the shell that this file should be passed to the clojure executable, which I've never tested.. But I have done something similar with #babashka which is used for scripts due to its very fast start times. So in a way yes?

kwladyka 2021-01-23T18:45:10.180Z

But I guess it has pretty slow cold start right?

Mno 2021-01-23T18:45:39.181600Z

Normal clojure yes, babashka no (at the cost of some limitations)

R.A. Porter 2021-01-23T18:45:59.182300Z

Yeah. For that use case, I’d definitely use #babashka. In that case, you can get the run script with (System/getProperty "babashka.file")

p-himik 2021-01-23T18:46:40.183400Z

#! is a comment in Clojure. And yes, it tells your shell (if it understands shebangs) to execute the file using the specified interpreter. Just a single (println "hello") in such a file takes less than a second on my PC.

holmdunc 2021-01-23T18:46:54.183900Z

The shebang causes e.g. /usr/local/bin/clojure to receive the the path to the script as its first argument. The question is whether it gets stored in a user accessible way

kwladyka 2021-01-23T18:46:58.184Z

babushka what a name heh 🙂

p-himik 2021-01-23T18:47:09.184400Z

ba-*bash-ka :)

kwladyka 2021-01-23T18:47:24.184900Z

Thanks, I was curious

Mno 2021-01-23T18:48:08.185400Z

Ohh so that's how that works.. Thanks that's a great bit of info!

👍 1
holmdunc 2021-01-23T18:50:21.186800Z

I know about and have used Babashka. That's besides the point, my question is about how to get this info in "real" Clojure :thinking_face:

p-himik 2021-01-23T18:52:32.187Z

@duncan540 *file*?

👍 1
holmdunc 2021-01-23T18:54:05.187400Z

That's it.. thanks!!

caumond 2021-01-23T18:58:12.187900Z

I mean that I dont see any lib helping an end user to build something with my underlying dsl

holmdunc 2021-01-23T19:01:00.189500Z

This kind of standalone script can even specify dependencies directly in the shebang, that will get auto-fetched if needed (`#!/usr/bin/env clojure -Sdeps {:deps{...}}}` (just don't use any whitespace inside that map)). Obviously the ever-present caveat is slow startup time.

p-himik 2021-01-23T19:16:51.189700Z

If you use white spaces in the map, you should be able to just use single quotes around it.

p-himik 2021-01-23T19:17:21.190Z

Like #!/usr/bin/env clojure -Sdeps '{:deps {...}}}'

2021-01-23T19:18:49.190200Z

You can even make the beginning of the bash script a bit more involved, if you want, e.g.: https://github.com/jafingerhut/dotfiles/blob/master/bin/clj-simple-single-file-script

p-himik 2021-01-23T19:20:47.190800Z

M-m-magic! :D

holmdunc 2021-01-23T19:21:33.191100Z

I think quoted strings are a shell thing. Processing the shebang is a kernel/OS level thing, no?

simongray 2021-01-23T19:21:51.191600Z

Now I am really confused. How can there be a library to help a user build something with a DSL that you're making? Aren't you the one who needs to make that library??

p-himik 2021-01-23T19:21:54.191900Z

Shebangs are a shell thing. Different shells might need different things.

2021-01-23T19:22:13.192700Z

Has anyone integrated aws cognito with a clojure server for authorization and authentication?

holmdunc 2021-01-23T19:23:22.192800Z

I mean the processing of the shebang. Shebang-having executables can be executed without there being any shell involved at all, as I understand it

p-himik 2021-01-23T19:23:34.193Z

Actually, I'm absolutely wrong - please disregard my comment above.

p-himik 2021-01-23T19:24:49.193300Z

Yeah, man execve describes it. I have no clue what made me think #! was a shell related thing.

👍 1
caumond 2021-01-23T19:28:24.193600Z

For sure, but I was suspecting the existence of tools to help to do that. But its true I found nothing. Intellij idea mps is an example for instance.

caumond 2021-01-23T19:31:24.193800Z

They provide some functionalities to present a simple ide. As a developer you setup mps to tell what is your dsl. I was not searching for all that features, I just mention to answer your question.

holmdunc 2021-01-23T19:31:30.194Z

I remember seeing that wild Bash/Clojure hybrid approach Andy. If the straightforward shebang couldn't support deps then maybe that would become a hit!

2021-01-23T19:34:13.194600Z

Yeah, the one I linked to might very well be more fragile/magic than one would typically want to use.

2021-01-23T19:36:16.194800Z

as I pointed out the other day, the wrap-multipart-params middleware uses a disk store, you can use the info in the request map to find that file in the disk store, or reconfigure (or stop using) the middleware in order to handle the data yourself instead see the docs for :store here https://ring-clojure.github.io/ring/ring.middleware.multipart-params.html

borkdude 2021-01-23T20:25:27.196500Z

In clj-kondo, for analysis purposes, we want to have a list of all occurring keyword including their surface form, alias, etc. For some keywords it might also be handy to have something like :def true which indicates if they were used to define something in a registry (think spec and re-frame), so you can "navigate to definition". But the "keyword definition" sounds a bit weird, maybe. What would you call this?

2021-01-23T20:26:39.196800Z

keyword registration?

2021-01-23T20:27:21.197500Z

arguably that could also include things like watch keys used with container watchers

borkdude 2021-01-23T20:29:07.198Z

Maybe :registered-by clojure.spec.alpha works. :registered-by clojure.core/add-watch

2021-01-23T20:31:20.198700Z

the second one might not be worth implementing, but it's a tuple of keyword / container, since the same watch key can be used elsewhere

2021-01-23T20:31:39.199200Z

I don't know that watches are so ubiquitous in program logic that it's even making a feature there

borkdude 2021-01-23T20:31:54.199600Z

probably not. we are thinking spec and re-frame as the most prominent examples

2021-01-23T20:32:22.200100Z

in re-frame, can the same key be used in different ratoms?

2021-01-23T20:32:56.201Z

I know reagent lets use use multiple ratoms, but I haven't dug that deep into re-frame

phronmophobic 2021-01-23T20:35:15.202Z

re-frame has subscriptions registered to keywords. afaik, subscriptions are still only global

oliver 2021-01-23T20:36:22.202400Z

@borkdude: thanks for clj-kondo. I can't even remember what life was like without it. Or maybe don't want to.

❤️ 5
teodorlu 2021-01-23T21:04:41.204400Z

Idea, looking for criticism / feedback / questions. > Object orientation conflates the notion of "namespace" with the notion of "hierarchy". In Clojure, not so much. Namespace qualified keywords are not the same as nesting in maps. In javascript, hierarchy is often used for namespacing.

teodorlu 2021-01-24T10:17:51.256700Z

> So the symbols are like nested inside the namespace, but namespaces don't nest themselves inside others, they are flat, and not hierarchical once again. @didibus good point.

teodorlu 2021-01-24T10:19:44.257Z

Very good example. "separation of concern between namespacing and hierarchies prevent reinventing namespacing in terms of hierarchies, or ad-hoc conventions for namespacing." (my interpretation) https://clojurians.slack.com/archives/C03S1KBA2/p1611466587246700?thread_ts=1611435881.204400&amp;cid=C03S1KBA2

Andy Wootton 2021-01-24T19:49:43.258900Z

Thanks to all contributors to this fascinating thread. I have started to learn Clojure for an idea about moving away from trees towards graphs, so I've been a little nervous about namespaces appearing to be a hierarchy. I've found this very helpful. I think I understand now that a syntactic hierarchy doesn't have to imply a semantic hierarchy, though I'm not sure I can trust myself to always remember that 'in the heart of battle'.

💯 1
❤️ 1
katox 2021-01-24T20:38:08.259300Z

@didibus that's not the point - the point was what do you do when you want to print/pprint/debug a map full of aliased definitions. Using @seancorfield's example ns:

{::m/a 1
 ::m/b 2
 ::mu/c 3
 ::mu/d 4
 ::mv/e 5
 ::mv/f 6}
=&gt;
{:ws.domain.member/a 1,
 :ws.domain.member/b 2,
 :ws.domain.member.update/c 3,
 :ws.domain.member.update/d 4,
 :ws.domain.member.validation/e 5,
 :ws.domain.member.validation/f 6}

✔️ 1
katox 2021-01-24T20:39:49.259600Z

unlike the original short syntax it is always lengthy and the lengthy namespaces are mostly noise

teodorlu 2021-01-25T14:46:46.292700Z

@katox Good point. I hit this problem previously trying to "push" qualified keywords on a frontend developer. His reply was simply, "I don't like this. It's tedious, longer to write, and takes more time to eyeball". He was right, really. Which makes me kind of curious about the Juxt guys' approach to JSON (#apex (doesn't exist yet?), #crux). @malcolmsparks called Clojure "JSONScript" on a Clojure conference in Berlin. That stuck with me.

2021-01-25T22:00:10.310300Z

@katox Ya, when you have a map with mixed namespaces, only one can be shortened:

#:ws.domain.member{:a 1,
                   :b 2,
                   :ws.domain.member.update/c 3,
                   :ws.domain.member.update/d 4,
                   :ws.domain.member.validation/e 5,
                   :ws.domain.member.validation/f 6}
Evan alias won't help, because you're serializing them, you don't want a contextual alias in that case. > unlike the original short syntax it is always lengthy and the lengthy namespaces are mostly noise I think this is on the app developer though. Like why you use a feature you don't benefit from? Its like if you made all your maps records for no good reasons, or if all your functions were multi-methods even if they only have a single defmethod.

2021-01-25T22:03:14.310700Z

Well, alias do help in the usage of these maps in the code, since you can alias each of these to a shorter alias, but if you care about the payload size, or the verbosity of it when say logged or something like that, alias won't help

2021-01-25T22:06:18.310900Z

I guess in theory, Clojure could come up with a data representation for it, maybe it will in the future. Actually, having a #:[] might be useful. Or something like that:

#:ws.domain.member {:a 1,
                    :b 2,
                    #:ws.domain.member.update [:c 3, d 4]
                    #:ws.domain.member.validation [:e 5, :f 6]}
This for example. The idea is that #: defines a repeated namespace but at the same level. The reader could be made to recognize this.

katox 2021-01-25T22:06:34.311100Z

@didibus yes, that's what we did - we shortened all namespaces in maps to minimum, turned off short version map printing and replaced every single ::kw and ::aliased/kw with :short/kw. And we don't have to fix all tooling all of a sudden.

2021-01-25T22:08:36.311500Z

Ya, I've actually just stop using namespaces in my actual data, unless there's a need for it, like I need to protect from name conflict. But they are useful for spec and things like that.

katox 2021-01-25T22:12:28.311900Z

I have no use for spec either. I don't do too many macros and for data malli is a much better fit because of coercions and swagger. But that's OT. Having consistent keys is a plus so I like namespaced keys pretty much everywhere.

2021-01-25T22:14:32.312100Z

It depends very much on your use case. That's why I think people need to understand it as a feature only. Keywords can have namespaces, when they do, it doesn't mean anything special for Clojure. So its really just, if you in your app need a namespace for your keys, well you're in luck, you don't need to build your own internal namespacing scheme and all, Clojure keywords can have namespaces directly.

katox 2021-01-25T22:14:52.312300Z

Aliased kws remind me very much the land of xml schemas. I doubt there is an easy solution. So to date I'd like to stop seeing aliased kws ... like completely.

2021-01-26T02:32:12.312700Z

very literally they are the same thing: namespaces on tokens

2021-01-26T02:32:56.312900Z

they are only going to be more pervasive now that clojure spec uses them

teodorlu 2021-01-23T21:05:45.204500Z

I've been thinking about Rich's The Language of the System for a long time. I'm still not sure what he meant. But I feel like I'm getting closer. https://www.youtube.com/watch?v=ROor6_NGIWU

2021-01-23T21:07:45.204800Z

that feels a bit off to me, but I'm not sure why yet

1
teodorlu 2021-01-23T21:08:09.205100Z

@noisesmith I'd be curious to hear why it feels off. Apologies for not having a more precise question.

teodorlu 2021-01-23T21:09:31.205300Z

I've often felt a weird, unsettling feeling when I encounter a library API that's a hierarchy / nested list. Examples include #malli and #hiccup. Both allow you to put arbitrary nesting in your input data. But if you allow arbitrary nesting in your input, how do you provide extensibility?

2021-01-23T21:11:08.205600Z

in clojure a namespace as an object, is a first class scope you can reflect on, namespaced symbols are prefixed so they can be specific to one of those first class scopes for lookup, namespaced keywords can be correlated to namespaces for convenience but the namespace is an opaque token (there's no heirarchy of namespaces, as scopes or as prefixes, the relations are purely textual)

Mno 2021-01-23T21:11:22.205800Z

There's still a bit of hierarchy in namespaces..right? foo.bar and foo.baz are sibling namespace (common parent foo), and foo.bar.bing is would be a grandchild.. Right?

2021-01-23T21:11:34.206Z

they are not siblings in any meaningful way

2021-01-23T21:11:58.206200Z

the coincidence / alignment helps human readers, but has no meaning in terms of lookup or binding

2021-01-23T21:13:05.206400Z

you could just use foobar and foobaz as names, without the dots, without changing the meaning of your program

2021-01-23T21:13:32.206600Z

(unless you are using gen-class, in that case you need at least two segments)

2021-01-23T21:13:55.206800Z

or you could make them foo.bar and baz.quux

Mno 2021-01-23T21:13:55.207Z

Ah ok. So there's no functional hierarchy, it's just there mostly because of folders and some Java inheritance?

2021-01-23T21:14:06.207200Z

the folders are just for lookup

2021-01-23T21:14:12.207400Z

there's no effect on inheritence

2021-01-23T21:14:51.207600Z

(except for the weird rules for classes with a single segment, solved by having at least two segments if using gen-class)

Mno 2021-01-23T21:14:55.207800Z

(I meant because of the gen class thing, not becuase of type inheritance)

2021-01-23T21:14:58.208Z

OK

2021-01-23T21:15:20.208200Z

yeah, there's no semantic relation, it's just lookup coordinates

Mno 2021-01-23T21:15:21.208400Z

Haha cool thanks for the info, very interesting

2021-01-23T21:15:28.208600Z

but this seems pretty far off from the initial question

😄 2
teodorlu 2021-01-23T21:15:51.209Z

I'd say it's tangential!

teodorlu 2021-01-23T21:15:52.209200Z

> in clojure a namespace as an object, is a first class scope you can reflect on, namespaced symbols are prefixed so they can be specific to one of those first class scopes for lookup, namespaced keywords can be correlated to namespaces for convenience but the namespace is an opaque token (there's no heirarchy of namespaces, as scopes or as prefixes, the relations are purely textual) @noisesmith not sure if I understand precisely what you mean. I guess the vagueness of my non-question might be to blame. I agree with all you say in terms of Clojure semantics. I guess a more question-like formulation is: > is there a tangible description of the principle behind providing namespace qualified keywords? Not sure that helps 🙂

2021-01-23T21:16:33.209400Z

OK - what I was attempting to clarify there (and didn't quite achieve yet), is that a namespace means at least two things in clojure

1
2021-01-23T21:16:44.209700Z

1. it's a literal object you can reflect on and resolve symbols in 2. it's the part of a symbol that controls which namespace it is looked up in 3. it's the part of a keyword that plays an analogous but different role as the namespace of a symbol

👍 1
1
1
teodorlu 2021-01-23T21:18:39.210100Z

> There's still a bit of hierarchy in namespaces..right? foo.bar and foo.baz are sibling namespace (common parent foo), and foo.bar.bing is would be a grandchild.. Right? @hobosarefriends Right! I'd say it's a bit like a hierarchy. I like to think of my keys as organized in terms of a hirerarchy, that helps me think about it. But still, there's this stragely non-hierarchical part. I mean, why use namespace qualified keys when we could just nest? Why {:person/name "mno"} over {:person {:name "mno"}} ? I don't have a good answer for that. But I'm looking for one.

2021-01-23T21:18:53.210300Z

'clojure.core/+ is resolved by the evaluator to the token under + in ns clojure.core

2021-01-23T21:19:00.210500Z

that's a namespacing operation

teodorlu 2021-01-23T21:19:26.210700Z

@noisesmith good point about actually looking at the primitive operations you have on a namespaced keyword / namespaced symbol vs for example nested maps ✔️

2021-01-23T21:19:27.210900Z

@teodorlu that "sibling" relationship is only recognized by humans and has no effect on the compiler

✔️ 2
teodorlu 2021-01-23T21:20:16.211300Z

hmmm. So, @noisesmith, would you say that the function of a namespace is to map an identifier to a value?

2021-01-23T21:20:29.211500Z

so it's closer to case 3 in my enumeration above - it's a token that helps humans categorize things, all the compiler needs is that two uses of it use the same token, which always stands for itself

✔️ 1
2021-01-23T21:21:48.212Z

well - that's why I made the enumeration above: the function of (1) namespace is to hold a map from symbols to vars holding values (along with other conveniences like shortands created by refer / :as)

2021-01-23T21:22:07.212200Z

(2) is loosely related to that, (3) is loosely related to (2)

teodorlu 2021-01-23T21:24:07.212500Z

mmmm, okay. I apologise for bad problem formulation here. Good enumeration: > 1. it's a literal object you can reflect on and resolve symbols in > 2. it's the part of a symbol that controls which namespace it is looked up in > 3. it's the part of a keyword that plays an analogous but different role as the namespace of a symbol I'm more interested in our mental notion of a namespace than the literal clojure object that you can reflect on, ask for public vars, etc. So I think I'm most interested in your (3).

2021-01-23T21:24:44.212700Z

you go from "thing that can hold other things, we know which thing it is partly based on which thing holds it" (two (1) namespaces can hold the same symbol with unrelated values), to "thing that tells you where to look a symbol up, and what to look up, in one token" (a (2) namespace is a helper object to using values from (1) namespaces), to finally "thing that is qualified by a prefix to prevent collisions, but only stands for itself"

👍 1
1
2021-01-23T21:25:02.212900Z

in that case - it's a prefix, start and end of story

2021-01-23T21:25:55.213100Z

for human convenience we can pretend they have a hierarchy, and expect :a.a/b and :a.b/b to have some relation

seancorfield 2021-01-23T21:26:10.213300Z

I feel like Stuart Halloway gave a good example on the namespaced keyword: :github/id and :twitter/id in a "person" hash map.

👍 1
1
2021-01-23T21:27:03.213700Z

yeah, coming back to pragmatics - in js that would likely be {:person {:id ...} :twitter {:id ...}} which gets closer to the initial question I think

✔️ 1
teodorlu 2021-01-23T21:27:31.214Z

So why do we prefer the namespaced keys?

2021-01-23T21:27:36.214200Z

but it would just as often be {twitter_id: ... :person_id ...} - that contains no less info

1
2021-01-23T21:27:53.214400Z

in fact, repeated prefixes on symbols are a sign you want a namespace

👍 1
2021-01-23T21:28:03.214600Z

(in the abstract sense, not the clojure impl sense)

✔️ 1
2021-01-23T21:28:34.214800Z

namespaced keys make operations simpler, and data literals easier to read and work with

teodorlu 2021-01-23T21:28:37.215Z

I like the namespaced keys better in a sense, but I can't really say why. Perhaps it's because names exist in a domain, and we can make the domain explicit. And that "name spaces" / "domains of names" is a different notion than "contains".

seancorfield 2021-01-23T21:28:58.215300Z

Because :foo_bar_baz doesn't tell you whether it is a Baz in a FooBar context or a BarBaz in a Foo context

💯 1
2021-01-23T21:29:21.215600Z

I bet if I had more coffee I could articulate the relationship between namespaced keys vs. nested maps and normalization of data

teodorlu 2021-01-23T21:30:20.215900Z

> I bet if I had more coffee I could articulate the relationship between namespaced keys vs. nested maps and normalization of data I'd happily ship you a few liters of coffee if you could solve this once and for all!

2021-01-23T21:30:27.216100Z

haha

teodorlu 2021-01-23T21:31:35.216400Z

> Because :foo_bar_baz doesn't tell you whether it is a Baz in a FooBar context or a BarBaz in a Foo context @seancorfield that's a very good point. Crisp separation between space/context/domain and name.

2021-01-23T21:33:26.216900Z

{:foo {:id ...} :bar {:id ...}} has a reification of "ownership" into the tree of the data - all you need to do is select a key to get all :foo related things, while {:foo/id ... :bar/id ...} isn't reifying ownership, it's telling a human reader either where the item came from, or who cares that it is there

1
seancorfield 2021-01-23T21:33:27.217100Z

And it's also clear (to me, at least) that there's no "containing" relationship here - no nesting.

✔️ 1
seancorfield 2021-01-23T21:34:36.217600Z

(my slow responses are due to typing on my phone)

✔️ 1
teodorlu 2021-01-23T21:34:44.217800Z

I'm curious about how you've been selecting your namespace qualified keys for jdbc.next. I feel like naïvely using ::key often gives me way more nesting than I really wanted; "Good: crisp vocabulary with little nesting, bad: throwing deeply nested keywords around".

teodorlu 2021-01-23T21:36:47.218100Z

I also sense that the Clojure core team has some very good taste in designing namespaces. For instance, there's very little nesting in clojure.core. Perhaps there's something Rich just isn't telling us, saving it for the next hit once conferences open up again 😅

teodorlu 2021-01-23T21:38:07.218400Z

> {:foo {:id ...} :bar {:id ...}} > has a reification of "ownership" into the tree of the data - all you > need to do is select a key to get all :foo related things, while  {:foo/id ... :bar/id ...} isn't reifying ownership, it's telling a human reader either where the item came from, or who cares that it is there (edited) @noisesmith would you say that "reification of "ownership" into the tree of the data" is bad? If so, why?

2021-01-23T21:38:44.218700Z

I would say it's something that OO code does even when it makes code worse

1
seancorfield 2021-01-23T21:38:51.218900Z

Only :next.jdbc/count comes to mind @teodorlu

👍 1
2021-01-23T21:39:21.219300Z

if :foo and :bar were two things that were meaningful to combine in operations, I'd rather they be sub maps that I could easily select and work on

👍 1
2021-01-23T21:40:04.219500Z

if they are actually segregating related data, then I'd rather they use a namespacing prefix and merge into the top level single item

👍 1
teodorlu 2021-01-23T21:40:32.219700Z

> Only :next.jdbc/count comes to mind @teodorlu Observation: that's a single namespace (next.jdbc) and a set of names; a vocabulary under that namespace.

seancorfield 2021-01-23T21:41:19.219900Z

Yes

2021-01-23T21:41:47.220100Z

for example - if there's no use case for combing twitter/id and user/id outside the fact that they belong to one entity communicating with the API, then there's no reason to leave any sub-maps, having both as keys in one map is an advantage

1
teodorlu 2021-01-23T21:41:48.220300Z

@noisesmith got an example of when you'd choose "combination into sub maps" and when you'd choose "namespacing prefix and merge into the top level single item"?

2021-01-23T21:42:21.220500Z

but if what I had was user/id and friend/id - those are parts of two peer objects that I could interchange and combine differently

1
teodorlu 2021-01-23T21:42:24.220700Z

would the default be namespaced keywords, and combination only if it provides benefits?

2021-01-23T21:42:39.221100Z

and it makes more sense in that case to have {:user ... :friend ...} - as each sub map contains the same data

👍 1
2021-01-23T21:44:04.221300Z

@teodorlu I guess my question is "what operations are useful", once I know that I know whether to namespace or to nest. nesting is good for modular things that are meaningful to rearrange

👍 1
💯 1
✔️ 1
teodorlu 2021-01-23T21:44:19.221500Z

> but if what I had was user/id and friend/id - those are parts of two peer objects that I could interchange and combine differently Yeah, that feels a bit weird to me. I'd reach towards {:me {:id ...} :friend {:id ...}} rather than using namespaced maps. But I can't really explain why.

2021-01-23T21:44:59.222Z

right - that's what I was trying to say, if they are meaningfully interchangible, then it's useful to leave them segregated

👍 1
teodorlu 2021-01-23T21:46:01.222300Z

> @teodorlu > I guess my question is "what operations are useful", once I know that I > know whether to namespace or to nest. nesting is good for modular > things that are meaningful to rearrange In type land, I would say that you should keep the same shape of your data when you can, so that you can write a single function that works on all the different instances. "typeable".

seancorfield 2021-01-23T21:46:07.222500Z

Nesting makes sense to me when the submap can stand alone as an entity.

👍 1
💯 1
✔️ 1
teodorlu 2021-01-23T21:46:38.222700Z

Very good point about analyzing "use maps vs use namespaced keywords" by seeing how one would design a good API of operations :thumbsup:

2021-01-23T21:46:40.222900Z

@teodorlu I think a big mistake in by-the-book OO code is the reflexive treatment of everything as a model or domain entity - some things are just useful groupings that simplify code

1
2021-01-23T21:48:03.223400Z

an OO programmer would look at your {:user/id ... :twitter/id ...} and ask "what is this entity called, and what does it represent"

👍 1
teodorlu 2021-01-23T21:48:10.223600Z

@noisesmith mmmm, yeah. And that's the cases where you would like to limit nesting. Because a single level of (possibly namespaced) keys is a very simple model. Merge, etc.

teodorlu 2021-01-23T21:48:38.223900Z

> an OO programmer would look at your {:user/id ... :twitter/id ...} and ask "what is this entity called, and what does it represent" And we would say "don't name it, we need it just here". Gotcha.

2021-01-23T21:48:50.224100Z

did you mean "simple model"? - or I guess, what do you mean by "single model"?

teodorlu 2021-01-23T21:49:15.224500Z

Simple. Thanks ✔️

seancorfield 2021-01-23T21:49:15.224700Z

OK, at the big machine now 🙂 So the other area where namespaced keywords come into next.jdbc is in result sets: the qualifier matches the table where the column came from so if you join two tables, you get a single collection of data that you've assembled, and the qualifiers indicate the table which also ensures you can mix and match data from multiple tables without any name conflicts (e.g., :person/id and :address/id when joining the person table to the address table).

👀 1
1
teodorlu 2021-01-23T21:50:54.225100Z

mmm. So the domain design you work you do, in putting names (columns) into namespaces (tables), can be reused in Clojure and SQL, rather than having two different models that you have to keep in sync. If that's what you meant?

seancorfield 2021-01-23T21:51:02.225300Z

For person/address, you may well want :address as a full submap in your person hash map -- but that's about whether you want to treat address as a distinct entity, rather than just for (say) a mailing label (where you want :person/first-name :person/last-name :address/street :address/state :address/zip)

💯 2
1
teodorlu 2021-01-23T21:53:39.225800Z

> address as a distinct entity, rather than just for (say) a mailing label (where you want :person/first-name :person/last-name :address/street :address/state :address/zip) I really like that formulation. It looks similar to what @noisesmith said about "convenicence of grouping" and "collecting stuff together to provide shared operations over it".

seancorfield 2021-01-23T21:55:43.226100Z

I think if the early examples of Clojure code had just used qualified keywords out of the gate instead of simple keywords, we wouldn't even be having this conversation.

1
teodorlu 2021-01-23T21:56:06.226400Z

That leaves me curious about whether you can know in advance whether you are going to treat address as a distinct entity, and when you just want a set of properties. Do you decide on one or the other? Or do you create functions assemble-address and splice-address or similar?

seancorfield 2021-01-23T21:56:34.226600Z

We've just gotten used to simple keywords and now we're starting to look at qualified keywords as new/strange/different -- or having some inherent "meaning" in and of themselves.

1
2021-01-23T21:56:50.226800Z

that would usually be the boilerplate layer between application code and the db, but next.jdbc can cut a bunch of it out

👍 1
teodorlu 2021-01-23T21:56:54.227Z

> I think if the early examples of Clojure code had just used qualified keywords out of the gate instead of simple keywords, we wouldn't even be having this conversation. @seancorfield are good open codebases in the wild written like this that you would recommend reading for education purposes?

seancorfield 2021-01-23T21:57:23.227400Z

With SQL, I could just select * from address where id = ? and get an address "entity", if I want it separately.

seancorfield 2021-01-23T21:58:14.227600Z

@teodorlu No, there are very few applications out there in OSS land -- and libraries almost always take an inherently different style/approach.

✔️ 1
teodorlu 2021-01-23T21:59:18.227900Z

Gotcha, thanks.

seancorfield 2021-01-23T21:59:47.228100Z

In our production codebase, we're using qualified keywords a lot more now than when we started a decade ago (see my comment above about if Clojure had used qualified namespaces from the get-go in examples).

👍 1
teodorlu 2021-01-23T22:00:05.228300Z

I remember having a look at #cljdoc, being surprised about the taste that I perceived to be put into that. But I don't remember seeing anything about namespace qualified symbols in particular.

seancorfield 2021-01-23T22:00:42.228700Z

And we pick a qualifier that signifies how "unique" the scope of data needs to be. :wsbilling/* for example meaning it's associated with the World Singles "billing" subsystem.

✔️ 1
seancorfield 2021-01-23T22:01:26.229Z

But a lot of the time, data is just internal to one subsystem and we just use the DB table name as a qualifier (in our modern code -- legacy code tends to use simple keywords).

👍 1
✔️ 1
teodorlu 2021-01-23T22:01:39.229200Z

And then you can document the semantics of :wsbilling/thing and provide a single source of truth for that?

seancorfield 2021-01-23T22:01:50.229400Z

Via Spec, yes.

teodorlu 2021-01-23T22:02:11.229800Z

Ah, of course.

teodorlu 2021-01-23T22:03:03.230Z

I'm curious about how much ... "prose", "written description about the intention of this attribute" that follows each attribute's spec documentation, then.

teodorlu 2021-01-23T22:03:25.230200Z

peeking at the Spec guide

seancorfield 2021-01-23T22:03:35.230400Z

We have a bunch of web-app-related middleware and logic that tends to use :ws.web as a qualifier (so we're not terribly consistent about dotted or non-dotted). But it's just enough to partition the data if it all ends up in one map.

👍 1
👀 1
seancorfield 2021-01-23T22:04:07.230800Z

We don't use much "prose". We just try to use expressive names.

👍 1
seancorfield 2021-01-23T22:04:54.231100Z

I'm big on docstrings, my colleague not so much. But we discuss naming issues in PR reviews, and as long as we can follow the intent of each other's code via names, that's good.

👍 1
teodorlu 2021-01-23T22:05:32.231300Z

That makes sense. I guess a large problem is solved in just being consitent in the use of :wsbilling/thing, so that you can grep around and know that it's the same thing in different places.

seancorfield 2021-01-23T22:05:50.231600Z

There's really very little comments in our code either.

1
seancorfield 2021-01-23T22:06:18.231900Z

When I write OSS, I'm much more expansive about that: I'm writing code -- and docstrings and comments -- for a lot of readers out there.

teodorlu 2021-01-23T22:06:40.232100Z

mmm. Higher fanout / read-to-write ratio?

seancorfield 2021-01-23T22:08:21.232300Z

Libraries are also more general so there's a lot less "context" than application code which always has your business domain as context.

👍 1
seancorfield 2021-01-23T22:12:13.232600Z

I believe Clojure 1.11 will give us a lightweight way to create namespace aliases so that should help encourage qualified keywords that no longer have to be tied to namespaces in people's minds (they don't have to be tied to namespaces today but it's a bit clunky to create an alias that has no code namespace behind it).

❤️ 1
seancorfield 2021-01-23T22:13:05.232800Z

I think that will help make qualified keywords more appealing since folks can set up some alias easily and then use ::foo/bar in their code instead of :foo-is-a-long-name/bar

❤️ 1
teodorlu 2021-01-23T22:24:21.233200Z

I would love for that to happen. It feels like a hack to have to construct the foo-is-a-long-name.clj file just to be able to alias the namespace.

teodorlu 2021-01-23T22:28:47.233900Z

Thanks for a wonderful discussion, @noisesmith and @seancorfield. Really helpful to hear your insights. I'm signing off, talk to you later! 👋

teodorlu 2021-01-23T22:29:38.234100Z

bah, I didn't mean to post to channel. Fat fingers, sorry for the spam.

seancorfield 2021-01-23T22:37:51.234300Z

@teodorlu You don't need a source file, you can do (alias 'foo (create-ns 'foo-is-a-long-name))

💡 1
seancorfield 2021-01-23T22:38:20.234500Z

From our codebase at work:

(alias 'm  (create-ns 'ws.domain.member))
(alias 'mu (create-ns 'ws.domain.member.update))
(alias 'mv (create-ns 'ws.domain.member.validation))
🙂

❤️ 1
seancorfield 2021-01-23T22:38:45.234700Z

But that is the "hack" that you need a namespace in memory for the alias right now.

👍 1
katox 2021-01-23T22:50:38.235300Z

What do you do about printing aliased ns in maps?

katox 2021-01-23T22:52:23.235500Z

In code maps with aliased kws look tidy when printed for debugging do you just sift through that noise?

borkdude 2021-01-23T23:24:57.236700Z

Is it possible to get a named match out of a regex somehow with core.match?

(clojure.core.match/match ["foo"] [#"f(?&lt;a&gt;.)o"] a)'
or do I use re-matches for this inside the pattern?