Clojurians Log v2

i wonder how to make a "MiXeD CaSe" , upper : for odd, lower : for even?

Is there a way to get reflection / boxed math warnings excluding dependencies? Putting the globals in project.clj includes dependencies, which is a lot of noise. set!ing them in the main ns seems to be too late and I get no warnings

are you working at the repl or running from the command line?

at the repl, I can usually just set! in the main namespace and reload the buffer

at the command line, I would probably just do something like:

clj <my run args...> 2>&1 | grep 'Reflection warning' | grep "<filename I care about>" > warnings.txt

@adrianimanuel take a look at cycle and think of it used on the functions you want to apply to elements of your string

Like so:

user=> (require '[clojure.string :as str])
nil
user=> (str/join (map (fn [f c] (f (str c))) (cycle [str/upper-case str/lower-case]) "mixed case!"))
"MiXeD CaSe!"
;; or:
user=> (str/join (map (fn [f c] (f c)) (cycle [#(Character/toUpperCase %) #(Character/toLowerCase %)]) "mixed case!"))
"MiXeD CaSe!"
user=>

thank you so much, I'll digest this code... unfamiliar with cycle yet

now I just learn that we can use map like that as well 😲

clojure.core is huge -- even after over a decade of using Clojure, I'm still finding new functions (or new ways to use old functions).

I'm doing the grep way right now. I was hoping for something more... uh, refined?

afaik, warnings from warn on reflection can only be read from stderr, so any option will have to read from stderr. you could do something like https://github.com/clojure/clojure/blob/cbb3fdf787a00d3c1443794b97ed7fe4bef8e888/test/clojure/test_helper.clj#L123 if you wanted to programmatically gather warnings. It's possible that there's a tool or IDE that already does that, but I just common tools like grep ¯\(ツ)/¯

FWIW, I put (set! *warn-on-reflection* true) in every source namespace, after the ns form as a matter of course these days.

(I often put it in test namespaces too)

Hey there, total Clojure noob here! I'm reading a book called "7 concurrency models in 7 weeks" where Clojures concurrency model is one example. Now there's one thing which isn't really explained that well and where I didn't really find a quick explanation about what it exactly does under the hood (probably I just don't know where to look at). The book states that: > As its name suggests, ensure ensures that the value of the ref it returns won’t be changed by another transaction. It’s worth comparing this solution to our earlier lock-based solutions. Not only is it significantly simpler, but because it’s lockless, it’s impossible for it to deadlock. Now I don't quite understand yet how this is achieved

you can imagine (ensure r) as being the same as (alter! r identity)

I wouldn't focus so much on ensure, it is optimization basically, if it didn't exist there other ways to avoid, uh, I guess it is read skew

@ambass.joel I always found that for explaining specific Clojure concepts like concurrency around refs/agents the old https://www.amazon.com/Clojure-Programming-Practical-Lisp-World/dp/1449394701 is hard to beat; while dated in certain aspects, I found the book sections about refs/agents quite excellent and they are still pretty much 100% valid (no significant changes in Clojure in those areas as far as I know)

Is there an alternative to merge that will ignore nils. I got (merge-with #(or %1 %2) {:a 1} {:a nil}) ;=> {:a 1} but wondering if there is something more idiomatic

(into to (remove (comp nil? val)) from)?

@ambass.joel another piece of practical advice around concurrency in Clojure is also “stick to (atom …) for vast majority of cases”; way simpler and usually there’s a way to solve your problem with just atoms;

Thanks for your answeres! I have to admit that I'm (not yet) ready to look deeper into clojure as a language itself so I'm more interested on a conceptional level for now 🙂 So what I'm basically trying to understand is how ensure guarantees that other transactions don't change the ref ? Maybe there's also some missunderstanding of the guarantes ensure brings.

ensure doesn't really do that

Jup that's also mentioned and I think it makes a lot of sense for 99% of the cases.

I guess, thanks

the transaction does it, ensure just makes sure the given ref is part of the transaction

clojure does mvcc

meaning there are multiple versions of each ref, associated with each concurrent transaction, and one of those versions gets committed, and other transactions retry

alter! or ref-set! in a transaction mark a ref as part of that transaction, just reading the value of a ref doesn't

ensure is a way to read the value and mark it as part of the transaction

Got it! That makes a lot of sense.

Yes, that’s correct. In 8 years of Clojure, I don’t think I’ve used (ensure …) even once 🙂 And I’ve written some decently optimized/concurrent code with it; I can relate to the curiosity just for learning’s sake though.

I think the part that I was missing is that the other transactions would retry. Kinda stupid question but how does it determine if it needs to retry? I guess it needs to compare it's local state of the ref with the now global state or something like that? When does that happen? (Probably still some misunderstanding and/or mixing of thoughts on my side 🙂 ) EDIT: I guess it's the dosync function that does that right?

the dosync causes the code in its dynamic extent to run in a transaction

I believe basically each ref has a version number, and when a ref is modified for the first time in a give transaction, the ref and its version number are recorded in the transaction, and then when it is time to commit the versions numbers are compared to the current versions, and if the ref has been modified since, then the transaction is retried

https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LockingTransaction.java

Ah ok so similiar to Postgres transaction ids in that sense. But does the dosync block actually syncronize (I think it doesn't right?) so how is it guaranteed that there are no race conditions?

I guess reading the value of a ref does make it part of the transaction in a limited sense, in that it causes the transaction the cache the value read and return in for subsequent reads, so all reads in a transaction get the same value

it is called a LockingTransaction because it is built on locks

the only mutable things are the refs, and you can only mutate the refs in transactions, which are real transactions (ACI but no D)

So where does the locking/synchronization happen? Is the whole transaction synchronized or just when we ensure or alter the refs ?

none of those

the locking happens when a transaction commits, which is after the body has run

(technically refs all have a readwritelock, and the read lock may be held briefly while reading)

So it gets the global state of all the ref s and compares them to their local state (with a semaphore around the checking of those refs)

no

😄

no globals

it just checks the state of the refs involved in the transaction

against what?

against themselves

if they haven't been modified since the body of transaction ran, the transaction can be committed, otherwise it needs to be retried

Yes that I understand but how does it know that they have been modified? I guess they have their local cached copy of them (with a given transaction version number or something) but what does it use to see if somebody else have modified them?

the ref has a version number, not the transaction

Yes sorry that's what I mean with that 🙂

so the first time a ref is touched in a transaction, it will store in the current transaction "my version is 5", and then at the commit point, if the ref says "my version is 6", then you need a retry

if it says "my version is 5" at the commit point, you can commit and increment the version on that ref

Ok, that's what I basically meant with comparing the global state of the ref (current version number) with the local one (the one it had at the start of the transaction)

this is all based on my fuzzy recollections, at a certain point the answer to "but how does it do that?" is "it does the stuff in the code there" which is what I linked to

Yeah no that's the level I wanted to get answers to. If I had a bit more knowledge and time to invest I would have tried to get that from the source but I'm really thankful for your time and explanation! It certainly helped a lot to understand the concept and important details of it better 🙂

if I have a lib which works normally (via shadow-cljs) https://www.npmjs.com/package/jsonwebtoken I would like to do a jwt sign.. with rs256 in clojurescript, my code is:

(def creds {:client_x509_cert_url "<https://www.googleapis.com/robot/v1/metadata/x509/54354354@t.iam.gserviceaccount.com>",
            :token_uri "<https://oauth2.googleapis.com/token>",
            :type "service_account",
            :private_key_id "58f854354354354330b0fd3",
            :auth_provider_x509_cert_url "<https://www.googleapis.com/oauth2/v1/certs>",
            :client_email "<mailto:agapi@ga-balbla.com|agapi@ga-balbla.com>",
            :client_id "11765353534543507207",
            :project_id "ga-v4-auth-test",
            :private_key "-----BEGIN PRIVATE KEY-----
               MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCeHLXyO+bOEWyq
               bYNKVGMclwrMFq8IdKrUjdDg+1PT9OK4d3k2Xv7jpprjI2dSNpuLm26JIvcE0hF2
               smk4mOXKzrf79v+lPLM+5GKv6YBW7TlZfJBzIxtPMLhiZBMKNUaYZSQsldJiTISM
               ...............................................................
               dKmIUpzZ4Y1FnRVx6FCENN0JlzOaGa2PhGUZfLisXVwkWyXinyuElyaUGXo6BI64
               fazS98F6ZtU8GhElnKO43usa5bcQareDtan9N1m1S8g4Tr3x7daPbhiVAoGAU+AS
               eIBqm7zMxM3RE57+GH59o2SvVD94QpDn9GbHUs2IuKE4NTrrdp79YBphRVpyb6R8
               jeJfTr8FM3zLzUeBgRxuY9Jus6wpM5gXYzw+v8NCjwb9dsD0BSkLmRPXe1L6pwAr
               0c3lQkcB1sDwIxAIE1noM/6Ex+UdESLGHi0HXoECgYByffIreMTVaUAWhMWh0dBX
               /2eVMxOibNGtoiCWn6f3d8zKFJb4ut7NCC5fZHzUccGPyJ8hVFi1BWvJjmIIYM0M
               ZzeXhzkqttFA9uT5irrRD071tKjdF/hXkJ510RdjSVgOd7z/YmzPxR2mpOS2LGH0
               Mld1BOLpH/p+DkED5p3+WA==
               -----END PRIVATE KEY-----
               ",
            :auth_uri "<https://accounts.google.com/o/oauth2/auth>"})

and when I run:

(jwt/sign (.stringify js/JSON (clj->js {:foo "bar"})) (-> creds :private_key) (clj->js {:algorithm "RS256"}))

I got error: `Execution error (TypeError) at (<cljs repl>:1). Cannot read property ‘2’ of null`

Any idea what is the problem?

eg. this works fine: (jwt/sign (clj->js {:foo "bar"}) "sssssss")

Are you you want to be leaking a private key?

No, as you see I modified

ah yes,

Just I would like to show the structure of original creds

I don’t know what is the problem.. I can’t find out/ see. Any tip welcome! :)

Sorry! that was a false question.. I don’t know why didn’t run the first.. but now works: (jwt/sign (clj->js {:foo “bar”}) (:private_key creds) (clj->js (:algorithm “RS256"))) maybe a typo

:thumbsup:

I have recently 4clojure really good, even after I have written quite some clojure code https://www.4clojure.com/ . Question: is there such well-organized piece-by-piece-puzzle project for frontend stuff like html & css?

hey, I started using cider within a clj project. When I go to a newline between two forms, the second form is not correctly indented (I didn't have this problem in my cljs projects). Who's controlling that indentation?

for example, if I have:

(defn something [param]
  (let [data (:key param)] | (prn data)))

(defn something [param]
  (let [data (:key param)]
(prn data)))

so it works on my machine. but here's how i can diagnose things. i copied that, put my cursor where the pipe was and hit enter and it worked. then i undid that, and hit C-h c [enter] which asks emacs to tell me what its calling when i hit enter. that said newline. So i'm gonna read the documentation for newline with C-h f newline which has the following documentation > If ‘electric-indent-mode’ is enabled, this indents the final new line that it adds, and reindents the preceding line. To just insert a newline, use M-x electric-indent-just-newline. So now it seems it depends upon electric-indent-mode which is the thread i would look into now

C-c v electric-indent-mode tells me this is on for me

a bit of hint for perhaps some arcane commands here: C-h starts asking for help. C-h c is for "character", what happens when i press whatever. in this case it was the enter key. C-h f gets documentation about a function (`f`). I asked for documentation for what happens when i run the function triggered by the key i looked up. And then i checked out a variable with C-h v on electric-indent-mode and it told me some documentation about it and that it was enabled

but from all that, i would call the function (elecric-indent-mode) and then see if that fixes it

thanks a lot for the details, @dpsutton! Now, when I hit a newline I'm actually calling lispy-newline-and-indent-plain, which in my case forwards to newline-and-indent , which, besides being advised to continue a comment, in general should behave like TAB. But also, TAB, which calls indent-for-tab-command, which should ultimately call indent-relative. But even indent-relative is just sending some tabs, not aligning it the way I would expect. Also, it still works in another cljs file, which made me think that it's an issue with how my project is configured.

as far as i know none of this should really involve the project itself. there is a small bit that could ask nrepl for how to indent things since vars can have indention settings but i don't think that's what's going on here. all of this behavior doesn't take into account your project at all

Not sure if there is anything like http://4clojure.org for frontend Clojure stuff, but you may get something from this set of tutorials by practicalli

also, strangely, it seems to be indenting properly other parts, and just break in this particular case

https://practicalli.github.io/clojure-webapps/

Maybe that might help?

There's a set of tutorials and videos about constructing a simple web-based banking system.

It's not complete as far as I can tell, but it may help to get you started

This is also a good source of information about using figwheel-main

https://figwheel.org/tutorial.html

hah: the error seems to be in: clojure-indent-function as I get clojure-indent-function: Wrong type argument: number-or-marker-p, nil in my messages buffer. The cause might be that I started using emacs 28, and this broke something

are you using paredit?

ah. well you have some fun digging to do. and it sounds like you have some decent experience with emacs. so hard to tailor advice with the vast range of experience that might be in here: first day using emacs vs maintainer of a popular emacs library just new to Clojure or CIDER

right, I'll try to poke around, and in case probably just confirm that I don't have this problem in emacs27, and maybe open an issue. But thanks for the help! 🙂

how should I structure this?

(defn get-current-branch
  []
  (let [output (->> (sh "git" "branch")
                    :out)]
    (if-let [branch (re-find #"\* (\w+)" output)]
      (second branch)
      (second (re-find #"\* \(HEAD detached at (\w+)\)" output))
      )))

I'd probably use a single regex with that extra string part being optional.

(let [[_ _ branch] (re-find #"\* (\(HEAD detached at )?(\w+)" output)] branch) -- something like that?

Although \w+ will not match - and I often have dashes in my branch names.

dev=> (let [output "* master"] (let [[_ _ branch] (re-find #"\* (\(HEAD detached at )?(\w+)" output)] branch))
"master"
dev=> (let [output "* (HEAD detached at master)"] (let [[_ _ branch] (re-find #"\* (\(HEAD detached at )?(\w+)" output)] branch))
"master"
dev=> (let [output "* WS-12677-refactor-hubble-model"] (let [[_ _ branch] (re-find #"\* (\(HEAD detached at )?(\w+)" output)] branch))
"WS"
dev=> 

isn't he matching a SHA?

oh, and I didn’t think of destructuring instead of using secon

Not from git branch.

@borkdude I am matching a branch or detached head, in which case i need to get the hash

i can expand the class to include dashes

depending on how much you know about how git works and how much you like parsing git commands, there's also https://www.eclipse.org/jgit/

@grazfather Note that with the optional detached head stuff, you need to return the third item, not the second.

Right, I usually do it like this nowadays: [A-Za-z_-]

you can have dots in git branches, too 🤯

About jgit: even the clojure tools CLI is stepping away from that now by shelling out to git, so you're in good company :P

how come #{1 2 3 4} evaluates to #{1 4 3 2} but, #{1 2 3 4 5} evaluates to #{1 4 3 2 5} ?

is that to avoid dependencies or is parsing git commands actually easier?

avoid dependencies, fix a list of issues

#tools-deps for more info

I think git itself has options to emit output that is easier to parse for some use cases

Sets are unordered.

Hash maps are also unordered.

@smith.adriane Here's the list: https://clojurians.slack.com/archives/C6QH853H8/p1614809707124400

If you walk along a set or a hash map (via printing or any sequence operation), you should pretty much assume you'll get the elements in a random order.

Oh, okay, how come sets aren't ordered?

because sets have no order

nor do maps

(despite other languages' implementation choices)

meaning, sets in mathematics have no order.

> Two sets are equal if and only if they have precisely the same elements. > the order in which the elements of a set are listed in roster notation is irrelevant, so {6, 11} is the same set as {11, 6}, and {2, 6, 4}, {4, 2, 6}, {4, 6, 2}, {6, 2, 4} or {6, 4, 2} all represent the same set https://en.wikipedia.org/wiki/Set_(mathematics)#Roster_notation

There are implementations in Clojure of sets and maps that do have various kinds of ordering guarantees, e.g. sorted by key/element given a comparator function (`sorted-set`, sorted-map built into Clojure), order by insertion, etc. if you really want one, but you often don't.

Some of those variants are linked from the Clojure cheat sheet, which can be useful for other reasons, too: http://jafingerhut.github.io/cheatsheet/clojuredocs/cheatsheet-tiptip-cdocs-summary.html

But what is the point of sets in math then?

Those others aren't quite as "first class" as the default unordered sets, e.g. if a library for reading a JSON file, XML, etc. can return a Clojure map, it depends upon who developed that library whether they thought to return one of these variants of sets/maps that preserve read order, or not.

To represent which of some domain of values has some property, versus which do not.

e.g. this set of symbols appears in this program, and any not in the set, do not appear in the program.

This set of records appears in this table of a relational database, any others do not.

As far as a map goes, consider a map used to describe properties of a person. Do you have brown eyes first, and black hair second, or vice versa? Answer: You have both simultaneously.

Yeah, that's also how I Iooked at maps, they remind me a bit of js objects.

Some others who want to read, process, and write out modified JSON files that are similar to the input file might ask exactly the same kind of question about maps, if maps were used to represent the JSON data read, i.e. why don't they preserve read order by default? You can implement that in Clojure, but you might have to ensure that you use a data structure other than an unordered map to represent the JSON data you read, or else the write order could be completely different.

Should someone care about the order of keys in a JSON map? I have heard that some applications might actually do that, which other might say is a misuse of JSON. I don't know if there are any official answers to that question, or which applications might treat order as important there.

But, I haven't yet experienced needing json objects to stay in order though. I guess it would be an issue if you decided to iterate over the keys, and expect specific pairs at some point, but you shouldn't do that, since that leads to easily breakable code.

So, think of a mathematical set as only the keys of a JSON map. There are many use cases for representing membership in some class of values, where order doesn't matter, and shouldn't.

can you use a set to map a vector of maps thus filtering the maps by key to match the set?

I'm not sure I understand your question. In both mathematics and Clojure, the values that can be placed into a set can themselves be other sets, or maps, or numbers, etc. If that doesn't answer your question, please try again, maybe with an example.

In Clojure, the keys of a map can be sets, or other maps, or numbers, and many other kinds of values, which can sometimes be useful. I don't think JavaScript has such generality in its JS Objects as far as what keys it can have, but I'm no JavaScript expert.

You sure can't do that with objects in JavaScript

@andersendigital not really sure but maybe you mean this?

(let [vector-of-maps [{:a 1, :b 2} {:b 5, :c 1} {:a 1, :b 2, :c 3, :d 4}]
      a-set          #{:a :c}]
  (map #(select-keys % a-set) vector-of-maps))

=>  [{:a 1} {:c 1} {:c 3, :a 1}]

@pavlos Yeah, that looks like what I was thinking.

Given this task: Write a function, `mapset`, that works like `map` except the return value is a set. Would you say this code is a correct answer?: ´´´clojure (defn mapset   [function collection]   (set (reduce (fn [final-collection item]                  (println final-collection)                  (into final-collection [(function item)]))                []                collection))) ´´´

Seems like a good start. A couple notes: • map is lazy while this implementation is eager • the initial value for reduce is provided a vector. Is there another initial value that might simplify things? • You're using into to add a single item to final-collection . Is there another function that might be more appropriate?

Hm, is there an idiomatic way to run something before and after, but return the middle? something like with-open? Basically I am writing a script that will run a command on a certain git commit, but I want to checkout the commit where I started, so it would be like

(let [current (get-commit)
      _ (checkout commit)
       out (do-something)
      _ (checkout current)]
     out)

try try

(let [current (get-commit)]
  (try
    (checkout commit)
    (do-something)
    (finally
      (checkout current)))) 

what are some general purpose debugging strategies that folks use to find errors in lazy sequences? normally i'd inspect the stack trace to see the line at which the error occurred, but if the seq is lazy, it only tells me the line at which the error was realized (i think).

i also tried running through the code path and replacing all calls to map with mapv, thinking that would realize the error and help me hone in on it

i've been warned about using concat because it can introduce the same kind of subtle, lazy bugs.

Ah, nice 🙂 my next question was how to make sure i checkout current if there’s an issue

I have yet to learn what it means to be lazy in clojure.

you can probably ignore that for now

Oh okay, cool 😄

here's a quick overview if you're interested though, http://clojure-doc.org/articles/language/laziness.html

Taking everything you said except for laziness into account, how does this feel?

(defn mapset
  [function collection]
  (set (reduce (fn [final-collection item]
                 (conj final-collection (function item)))
               [(first collection)]
               (rest collection))))

looking better. I wouldn't add the first item to the initial value since that will give you incorrect* results if you passed mapset an empty collection

The hint about the initial value was more about the fact that your goal is to return a set so there's probably a better initial value than an empty vector

The official answer from the JSON spec is that objects are unordered

In development and testing I tend to force the realization of a sequence so any bug that is only triggered later on in the sequence will be caught early. That isn't a magic-bullet of course, the tests aren't going to cover every possibility that you care about. I would say that what you're encountering is pretty fundamental to lazy sequences. However your stack trace should still be pointing out whatever code is called to generate the next element of the sequence (and that's the only thing that deserves to be called an "error in a lazy sequence"). Many times I have found that to be enough (understanding the local context surrounding an exception being thrown when generating more elements of a sequence).