Clojurians Log v2

What would be nice is also to support :keys for sequential destructuring everywhere (not just on the vararg rest)

(let [{:keys [a b]} [:a 1 :b 2]] [a b])
;;> [1 2]

Then it would make sense that an attempt at associative destructuring also attempts to coerce the seq into a map

I'm imagining any attempt at an associative destructuring of a sequence would be like calling (apply hash-map [:a 1 :b 2]) on it first and then destructuring it as a map.

it is supported everywhere - you just need (let [[& {:keys [a b]}] [:a 1 :b 2]] [a b])

could the former be sugar for the latter? dunno, haven't thought about that

Well it would make everything consistent with:

(let [{:keys [a b]}
      (list :a 1 :b 2)]
 [a b])
;;> [1 2]

Does this no longer works in 1.11?

(let [{:keys [a b]}
      (list :a 1 :b 2)]
 [a b])
;;> [1 2]

vectors though are both associative and sequential so gets tricky

user=> (let [{a 0} [:hi :there]] a)
:hi

Hum... That's true. Never thought about that, and I guess this kind of makes sense why it works for sequences and not for vectors. Though in practice seems not as useful. But I never thought of doing [& {a :a}] and now that I know, it's not a big deal, and allows both form if.you ever want to destructure the vector by its index or instead as a key/value pair map.

I mean, that's not new, it's always worked liked that :)

well, since 1.2

Haha ya, but for some reason I just never really thought about it lol. I think I assumed & was like only for function varargs, but it makes sense since I think function args are just destructured as if they were a vector.

Can anyone recommend any books on designing Erlang systems? I’ve built a large clojure “microservices” system in Kafka+Datomic that is starting to feel very Erlang-y in terms of actors and message passing, so now I’d like to understand how Erlang and Beam handle those kinds of problems

Designing for Scalability with Erlang/OTP is a good one

it's full of little insights useful in non erlang contexts. I quite enjoyed reading it

Can anyone suggest a comfortable lexer or tokenizer to parse sql like syntax?

I will add a sample

table public."this""is stupid": INSERT: id[integer]:5 sometext[text]:'tere2' somedate[timestamp with time zone]:'2021-03-19 12:13:07.609859+02'

instaparse?

This is output from postgres test_decoding plugin and i'm trying to avoid reinventing a lot if wheels when parsing this.

It has funny oddities "" meaning an escaped double quote

Instaparse looks good, will check it out

Uhh https://calcite.apache.org/?

thanks!

found some instaparse grammar here: https://github.com/untangled-web/sql-datomic/blob/master/resources/sql-92.instaparse.bnf

Calcite probably won't do as the syntax is just similar to sql semantics, not sql itself

oh I forgot that one: https://www.erlang-in-anger.com/ Fred Hebert is a good writter and the bits about flow control are excellent

and it's free...

but it's way more erlang centric, I guess with that one you could cherry pick parts to read

otplike could be of interest https://github.com/suprematic/otplike

I wonder if there's a json plugin for the wal/logical decoding

wal2json? https://wiki.postgresql.org/wiki/Logical_Decoding_Plugins

i have a function that caches the return value (with an atom). I would like to skip the cache during development so I don't have to (reset! cache {}) when I make changes. is there a preferred idiom for this kind of thing?

I would think just have a *cache-results?* var that gets set to true for prod?

I’m not super experienced yet but have seen that idiom in a few different projects

Dynamic vars are slow and the thread locality can bite you

You could put the cache or a config flag in an Integrant (or Component or...) system but it might be a bit heavyweight

Also the Reloaded workflow with e.g. Integrant can reset the cache along with everything else

I can not rely on being able to install plugins on source postgres database

Source can be hosted postgres like rds, i can not install anything there.

+1 instaparse

ah, thought that wal2json was present on rds (https://aws.amazon.com/blogs/database/stream-changes-from-amazon-rds-for-postgresql-using-amazon-kinesis-data-streams-and-aws-lambda/), but yeah +1 instaparse

> It is obvious that due to JVM limitations otplike cannot replace Erlang/OTP and otplike will `NEVER` be seen as Erlang/OTP alternative.

other than the threading model - which might get patched before wet bulb temperatures kill us all - i wonder what else is a limitation

I have a library function where I have two things that have a property that is required to be numeric, and I return the absolute value between them, I'm guessing I should avoid Math/abs and copy something like https://github.com/clojure/math.numeric-tower/blob/e8387dfac1757701e842a65c52b2b5596d1c9607/src/main/clojure/clojure/math/numeric_tower.clj#L104 so that it works with bigdec, etc. This question kind of answers itself, but it feels a bit wrong still to bring in that function just to get an absolute value in one spot

@jjttjj Would it be a reasonable restriction on your code to only support long/double? Or do you need to support big integer/big decimal?

(I suspect there’s a lot of “numeric” code out there that doesn’t work properly with the “big” variants in all cases)

Technically it's extensible in a way that it might be need to support the big variants, but I suppose it's probably unlikely they'd be needed so maybe I should add the restriction

Per process gc/stack/heap, so isolation is quite extreme and allows some patterns hard to replicate on the jvm

It's also full of tricks internally to allow this share nothing approach without being wastful when operating on the same host. Also message passing works across the network natively... etc etc. I really like erlang (used it professionally) but I think it might not be a good idea to try to replicate this kind of stuff on the jvm

At least not now

So anything else new in 1.11 appart from keyword args accepting a map as well?

spec2......? 🙏

Personally I'm hoping for any ergonomic improvements to qualified keyword usage (https://clojure.atlassian.net/browse/CLJ-2123?filter=10021)

@didibus That’s the only feature in 1.11 Alpha 1.

But there are lots of things in JIRA marked for possible inclusion in 1.11 I believe. Including, possibly, map-vals and map-keys? I’ve seen those mentioned a few times.

btw, alpha1 does simplify function specs for kwarg functions, you can now just spec them as a map I guess. or was this just as easy before? can't remember.

we still have a pending enhancement to extend keys*

Looks like 31 vetted tickets marked for consideration in 1.11… including tap-> (yay!), java.util.functions interface support, #clojure/var tagged literal, lighter-weight aliasing on keywords (to help with qualified Spec names), DCL for prepl (yay!)… those are my “favorites”…

you should not put any belief that those are meaningful

consider things marked 1.11 as "top of mind" for one reason or another (some are really things possibly for 1.11, some are "important", some just had been in late consideration in the past and have been dragged along)

I don’t think that list includes https://clojure.atlassian.net/browse/CLJ-1959 which has been mentioned a couple of times? map-keys/`map-vals`

like I said that list likely bears little relation to what will end up in 1.11

I have no expectations that all of those 31 would make 1.11. I do hope tap-> makes it in though.

feature-ish wise, we've done design and some impl work on new thing related to keyword aliasing, don't really want to get into details, would rather do that when it's ready

and have done more looking at java function interop, no conclusion yet on where that will end up

I would guess 1.11 is 6-12 months out still at this point?

¯\(ツ)/¯

Haha… You’re a lot of help! 😆

we haven't decided what's in it so ... hard to estimate

Fair enough. I was a bit surprised to get an alpha so soon. Esp. with a single feature in it.

that is probably the thing next most farthest along

feedback!

For now I’m using (doto tap>) anywhere I would otherwise want tap-> so I guess that’s not really “important”.

generally, we prefer to do bigger things first (alphas), more bug/small things next (betas), then clean up

so while tap-> is likely, you may not see it till later

DCL for socket REPL and prepl would mean I could clean up my dot-clojure deps.edn file, although I take care of that in my dev.clj file now. I’m more excited about tools.build TBH than 1.11 overall 🙂

what's DCL?

I have a patch for that but on first review it raised more questions and got a bit bigger

but yes, of interest for sure

https://clojure.atlassian.net/browse/CLJ-2540

DynamicClassLoader @dpsutton

A normal REPL has that context, but socket REPL and prepl don’t.

Which means you can’t use add-libs in them unless you do a little DCL dance at startup…

Hence: https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L178

ah fair. i stay away from that kind of stuff and just restart my repl. in practice i don't add deps very often so the invitation to classloader issues is a problem without a large payoff for its complexity

@alexmiller Yeah, that comment from Stu makes perfect sense. Not sure what to suggest. I took that approach in my dev.clj file — setting up the DCL before I start Socket/Rebel/Reveal etc: https://github.com/seancorfield/dot-clojure/blob/develop/dev.clj#L182-L188

@dpsutton My HoneySQL REPL has been running since Jan 31st. I just restarted my work REPL today to switch from JDK 15 / Clojure 1.10.3 to JDK 16 / Clojure 1.11.0 Alpha 1 but it had been running since Feb 26th.

instaparse working like a champ, thanks 🙂

That comment from Stu? On https://clojure.atlassian.net/browse/CLJ-2540 or somewhere else?

Recently I’ve been hurting for a map-kv. (map (fn [[k v]] (…))) works fine, but it’d be nice to be able to use a #() with %1 == k, %2 == v instead

Yes, on that issue.

I don’t like #(..) with multiple arguments — I’ll use (fn [..] ..) as soon as there are multiple arguments. I almost never map over a hash map — I use reduce-kv for that.

Hum, seems a bit frivolous compared to just (map + (keys foo))

But why not

Java.util.functions support that would be really great

What is DCL?

DynamicClassLoader. Needed for add-libs.

oh I thought Dead Code ... but what does the L stand for? ;)

I always have a case where I started a repo and forgot some lib

I think it be an awesome feature for exploration and experimentation

I can't really think of anything major I would want from 1.11, honestly pretty content with most of Clojure 1.10. But I didn't know I wanted keyword args to work on maps before and now I really want it 😂 Definitely interested in spec2, function interface interop, and I'm intrigued by tools.build though I also don't feel I'm not getting what I need from current build tooling. Other then that I'm more into the upcoming JVM changes, project loom would be big, native image to support more code, value objects (the Jep for it even mentions Rich Hickey haha), etc. I think the overarching themes the survey often asks for, better error messages, faster startup, better performance, well I'd never say no to that. But can't really think of much beyond that in the language itself, the ergonomics, like it's all pretty damn good already 😂 Oh... I do hope one day to see a form of HEREDOC support.

Not happening

:)

I no longer wish for these since I got comfortable with transducers and came across cgrand's xforms. They could benefit libraries though

I want more runtime introspection, e.g. metrics/watchers for stuff like core.async, multimethods, namespaces

I'll keep dreaming 😁

A great place to describe a problem you are trying to solve and see if others have it too is on https://ask.clojure.org

If many people have the same problem, then the core team becomes interested :)

Actually, having more transducers in core would be nice.

I see why keyword alias and function interface are being looked at for 1.11 then haha. https://ask.clojure.org/index.php/questions?sort=votes

On that topic, I have found it harder since the move to ask.clojure to vote on things, because feature requests and bug fixes are hidden in the noise of generic questions.

(map + (keys foo)) doesn't return a map; I would appreciate a (map-keys + foo) that produced a new map with the keys updated

@didibus what's HEREDOC support, you mean multiline strings?

Just sexprs and pr-str can get you pretty far ;)

Multiline string that doesn't need to be escaped.

One of the problems with heredoc as typically implemented is that this is currently legal Clojure:

dev=> (def foo ["""
 #_=> this is a heredoc
 #_=> """])
#'dev/foo
dev=> foo
["" "\nthis is a heredoc\n" ""]

#text DELIMITER
Here I can now write whatever I want including with " : \ \ / / (foo)
All kind of error character.
DELIMITER

🙂

That would be horrible syntax in Clojure tho’…

When I last thought of it, I thought a new tag reader would be best:

#text end
Whatever multiline string here.
end

Reader tags still require legal Clojure forms though.

Sorry, not a tagged literal, like an additional reader form

More like #()

Which lets you pick the delimiter the reader will use on the next form for strings.

reader dispatch

Ya, so you tell the reader the following string is delimited by this symbol.

I would suggest you try actually implementing it in a fork of Clojure and see what problems you run into. I think you’ll find it far harder than it looks (and I still think it would be horrible syntax).

e.g.:

#+
Yolo!
+

How about

#__ 
Multi
Line
Comments!
__#

He’s suggesting something like #" sym unconstrained arbitrary text sym I think.

(presumably with required newlines after the first sym and before the second one to make it a bit more tractable?)

#+EOT
yolo! "string!"
EOT

Well I don't want it for comments 😂, it's basically for embedding other languages into Clojure or for templating.

I think that is doable

Ohhh I see

This is quite easy to hack into https://github.com/borkdude/edamame or tools.reader in the dispatch table

Ya, the user picks the delimiter, it's a bit like how as-> has you pick the name of the symbol to bind. That way the user can choose a delimeter they know their string won't contain and so they won't need to escape.

perhaps the hardest part of this would be updating all the tooling that needs to know how to move over forms

lots of code in tooling would need to be updated and it might be quite tricky

this is always the case with adding new dispatches to the reader table

Most tooling relies on tools.reader though

CIDER certainly does not and that's about 40% marketshare

But ya, it is one downsides. It also secretly open the door to things much closer to reader macros. Because tagged literal can now accept arbitrary text and parse and interpret it.

I bet I could hack the above idea in edamame within an hour or so, but since it will never be in Clojure, I won't even bother

With what else I've seen you do, I'm sure you could haha.

i would be surprised in Cursive used tools reader for its structural movements and such

and obviously calva/conjure/Chlorine and others don't use tools reader as well

Cursive probably uses something more along the lines of rewrite-clj since editors / linters etc need something more precise than s-expressions

Well if people think it be a nice to have, go upvote: https://ask.clojure.org/index.php/8520/text-block-literal-string-literal-unsescaped-string-literal 😜

calva uses clojure-lsp which uses rewrite-clj / clj-kondo (which is also based on rewrite-clj)

and rewrite-clj uses tools.reader but not in the way you typically do

I think that’s one of the few things on ask that I’ve downvoted 🙂

It was.you!! :rolling_on_the_floor_laughing:

I've had two times where I really wish Clojure had it. For scripting, it is quite nice if you need to generate some file, you can template things easily with this, without needing a seperate file, which when you have a script you don't really want resources along with it and all that. And for embedding JavaScript inside Hiccup for simple server side rendered websites.

Early on Clojure considered XML literals I think I read somewhere once

does it use that kind of stuff for movements? not analysis but for moving to the next form; for determining what "the last form" is when sending the last form

define "it"?

Part of what drew me to Scala was XML literals. After I used them in a project, I felt they were a terrible idea! 🙂

@dpsutton if you want to offer any kind of refactoring, you have to know locations and preserve whitespace, exactly what rewrite-clj is for, but I assume Cursive has its own variant of this, possibly written in Java or part of some proprietary IntelliJ API

i mean, if you put your caret at a paren and say "send this form to the repl", the determination of what the last form is is certainly in a js implementation of paredit or parinfer right?

and this multiline string allowing arbitrarily unmatched delimiters would wreak havoc on that stuff

its not impossible, but i'm just thinking of all the paredits and parinfers that would have to be updated to handle this

paredits also handle ;; with arbitrary text?

and yes i'm not talking about analysis which will likely use tools.reader

yeah that's fair

I like your twist to it borkdude.

#+SYM
Here's the big string.
SYM

#my/xml #+END
<xml>
  <foo bar="wtv">
<\xml>
END

yep

and docstrings without escaped strings as well.

True:

(defn foo
  #+DOC
  This function does bla
  Example:
   (foo "value" 1)
   ;;> "value1"
  DOC
[a b]
...)

Me too :)

@seancorfield Could you elaborate a bit on why you felt that way? What do you think about JSX that's rampant in the frontend world right now?

I hate that too 🙂

What about ERB like templates?

Scala is already fairly punctuation heavy so allowing arbitrary blocks of text full of < and > just makes for very distracting-looking code.

I’ll be honest, I find large blocks of SQL embedded as strings in Clojure to be distracting too. Although we have plenty of that at work. Where we’re not using HoneySQL, that is.

This is why Scala chose [] for generics ;)

Hmm, right, I can see that. A cognitive load creep. :)

Templates of any stripe are fine outside the code.

Scala was trying to learn from C++’s mistake with < > for templates because of the contextual lexing issue of >>.

Well I guess if a code base started using this for all strings. But I expect it be for rare instances. Like a quick on-click JS function inside Hiccup, or in a script where you need to create some config file from and want to quickly template it without setting up resources, and all. And for doc-strings it actually looks much cleaner to me. Seeing all the escape characters in them is distracting.

how do literate programming tools solve this - they must have a solution for this. do they start with foo.template.clj or so?

I don't know, any Rubyist or Perlers? Do heredoc end up polluting things?

(it will likely be unsurprising to hear that I’m no fan of literate programming either — for similar reasons)

Oh, is that the idea of map-keys ? I see I see. Though reduce-kv is pretty damn close, but ya I can see it being a bit more convenient. Then I would actually like a map-kv that returned a map.

https://clojure.atlassian.net/browse/CLJ-1959

Then why do I see you face in the contributors here: https://github.com/gdeer81/marginalia/tree/master/test/marginalia/test

=)

https://gist.github.com/seancorfield/6e8dd10799e9cc7527da5510c739e52f — using a gist via deps.edn to get map-keys / map-vals

Well it seems heredoc is quite controversial. My idea was one day when I'm bored I'd make an Emacs unescape plugin instead. Where it visually unescapes things inside Clojure strings, and automatically escapes them as I type/paste (but visually won't show the escapes. Maybe this already exists actually...

it seems marginalia just uses comments and strings, no special stuff

@borkdude Because I wanted to give it a fair whip — and used it to build an entire project at work. This came up on ClojureVerse recently, BTW.

https://clojureverse.org/t/what-is-your-favourite-way-to-document-your-programs/7353

cool. I have made some diagrams with http://draw.io which works on all platforms. it's not as as good as omnigraffle but it works and it's free

they have a desktop app too. I'm off now, g'nite.

I see. Ya, I don't know... I expect every Clojure dev to have their own "my-extra-utils" library, and this seems it could go there, amongst a bunch of other conveniences. I don't know which one are used so often that it makes sense to promote them to core or not. I wouldn't mind anyways. Though the name instinctively made it feel to me like it's mapping over the keys, not that it would become a non-lazy type preserving function.