Clojurians Log v2

ah, i think its :integratedSecurity

@qmstuart You can pass any properties that the JDBC driver understands -- so you won't find them in the docs specifically, but there is an explanation of passing additional properties to the driver as part of the db-spec hash map...

Clojurians, what are we really doing here when we " { :keys [ minimum maximum ] } " as a parameter inside that function? I became lost here

That is destructuring a map into bindings by specifying the names of the keys in the map. So you function expects a parameter that is a map that contains the keys minimum and maximum After the destructuring you can just use the names minimum and maximum rather than having to do something like (:minimum your-map) e.g. Say you have a map

{:foo 5 :bar 6}

(defn foo [{:keys [foo bar]}]
   (prn foo bar))

(foo {:foo 5 :bar 6})
5 6 
=> nil

(defn foo [my-map]
   (prn (my-map :foo) (my-map :bar)))

HMMM, thank you @qmstuart! And when we use ":as" after it? Are we binding the keys to next element? e. g. Acc in the next image?

:as allows you to also bind the entire map as well, in addition to the key names specified

That is so you can still reference the entire map. So in that method your map argument is called acc, and its keys are current-bag and bags. The method also has a second parameter called stream. Im assuming this is a method to pass to reduce.

Also worth noting: if you have :or to set a default value, it binds the symbol to that value but the :as symbol is bound to the original data.

(let [{:keys [a b] :as data :or {a 1 b 2}} {:a 42 :c "Sea"}] (prn a b data))
42 2 {:a 42, :c "Sea"}

So b is bound to 2 -- the default -- but data still doesn't contain b.

Quick (?) question: what idioms should I be using instead if I'm planning to use map inside another map function

@zackteo For some problems, that might well be the right solution. Can you provide some context?

Give me a moment 🙂

So I current have a list of sheets like ( "Sheet1" "Sheet2" ) and a list of lists of java objects (( org.apache.poi.ss.util.CellRangeAddress [A1:C1] org.apache.poi.ss.util.CellRangeAddress [A2:B3] org.apache.poi.ss.util.CellRangeAddress [C2:C3] )( org.apache.poi.ss.util.CellRangeAddress [A1:C1] )) and I want to transform it into a format like so ...

{[0 0 "Sheet1"] {:lrow 0, :lcol 2},
 [1 0 "Sheet1"] {:lrow 2, :lcol 1},
 [1 2 "Sheet1"] {:lrow 2, :lcol 2},
 [0 0 "Sheet2"] {:lrow 2, :lcol 2}}

And currently I have

#(hash-map (vector (.getFirstRow %)
                   (.getFirstColumn %))
           (hash-map
                   :lrow (.getLastRow %)
                   :lcol (.getLastColumn %)))

{[0 0] {:lrow 0, :lcol 2},
 [1 0] {:lrow 2, :lcol 1},
 [1 2] {:lrow 2, :lcol 2}}

Sorry about the mess. I'm trying to figure out how to best approach the problem

Can you explain what the actual problem is that you're trying to solve?

Am trying to add merged cells into https://github.com/zero-one-group/fxl/blob/develop/src/zero_one/fxl/read_xlsx.clj#L107 by adding :lrow and :lcol into :coord when it is a merged-cell . Which should seem easy to do. But the only/best access point to find out about merged-cells seems to be using getMergedRegions on the sheet https://poi.apache.org/apidocs/dev/org/apache/poi/ss/usermodel/Sheet.html#getMergedRegions--

I think you're struggling because you can't explain the problem very well...

So I was thinking the way to approach it is by having a list of merged-regions in the let block of throwable-read-xlsx! https://github.com/zero-one-group/fxl/blob/develop/src/zero_one/fxl/read_xlsx.clj#L116 which I then can feed into poi-cell->fxl-cell to check if the current cell (it is mapping through) is a merged cell and adding the right keys accordingly

You're too deep in the weeds of the implementation.

Take a few steps back, away from the code, and think about the data at a higher level.

hmmmm

I guess I this might still be implementation level but broadly my problem seems to be a 2 step one 1. Get a hash-map which I can use to identify which cells are merged-cells. (which also contains the extra parameters I want, to make a cell a merged-cell) 2. Update the list of cells based on that hash-map

That's still implementation. You haven't actually described the problem yet.

"i have a workbook with two spreadsheets in them. i would like to ..."

Right now I have no idea what you mean by "merged-cells" and what "extra parameters" means, even in that context.

Hmmmmm :thinking_face: I wondering if this is too high level. I am adding merged-cell reading capabilities to an excel read/write library (that wraps apache poi) Merged-cells are a combination of multiple cells but only display the value of the top leftmost one

A merged cell is a range/region of cells. So a cell typically only has 1 pair of coordinates. I will need the other pair to give me the region

Like cell A1 (:row 0 :col 0) to merged-cell A1:B2 (:row 0 :col 0 :lrow 1 :lcol 1)

Maybe I was trying to jump steps by considering multiple sheets, without successfully handling a single sheet first

Hi, Who knows the best way to invoke HoneySQL from java？ or some suggestions?

The same way you invoke any Clojure from Java... https://clojure.org/reference/java_interop#_calling_clojure_from_java

Hi guys, need some opinion if possible. Not 100% related to clojure. I want to do an excel-like grid at front end, but evaluate it (the formulas) at backend. I'm thinking of casting the formulas into variable assignments like so: `A.1 = 30 + 20` , concatenating them and sending them to the backend when the user clicks a button. So, the request would contain something like this (Assuming a 2x2 grid):

A.1 = nil;
A.2 = 10;
B.1 = A.2;
B.2 = B.1 + 50;

@dumrat Can't you send a map of cells to their values instead? So something like {:A.1 nil, :A.2 10, :B.1 :A.2, :B.2 [:B.1 + 50]}?

@rextruong Yes I can. Still I'd have to parse the input because the users expect excel-like formulas. But I don't think this would solve #2 and #3 issues if I understand correctly.

To handle (2) and (3), I suggest building a topology graph to identify the leaves (e.g. A2 = 10) vs the internal nodes (e.g. B1 = A2) so you can start evaluating them in the correct order.

You can also detect loop while building topology graph iirc.

@rextruong Thanks. Do you have any suggested reading for building a topology graph? Not familiar with the concept.

This may help https://www.geeksforgeeks.org/topological-sorting/

Great, will check it out. Thanks!

Hello there. Could you please help with understanding of the async lib ans subscriptions? Suppose I want to send https req to website every second and subscribe to it's result. And do some action each time as I get response. Suppose it's just printing the response's body. So having the following code does not produce desirable effect:

(ns async.core
  (:require [clojure.core.async
             :as async
             :refer [chan <!! <! >! >!! timeout pub sub unsub unsub-all go go-loop]]
            [clj-http.client
             :as h]))

(def updates-channel (chan))

(def publishing
  (pub updates-channel :status))

(defn req [url]
  (h/get url {:async? true}
         (fn [res] 
           (go (>! updates-channel res)))
         (fn [_] nil)))

(defn listen!
  [publisher topic]
  (let [l (chan 1)]
    (sub publisher topic l)
    (go-loop []
      (when-let [{:keys [body]} (<! l)]
        (println (str "got incoming message: " "\n" body))
        (recur)))))

(listen! publishing :200)

(go-loop [seconds 1]
  (req "<https://motherfuckingwebsite.com/>")
  (&lt;! (timeout 1000))
  (recur (inc seconds)))

Hi folks, is there a fn to pop a key from a map, similar to dissoc but should also return the value of the key provided. e.g.

(pop-key {:a 1 :b 2 :c 3} :a)
;;=>[{:b 2 :c 3} 1]  

nothing builtin

(juxt dissoc get)

thanks

Hello, I am trying to find a way to write an acceptance test for my app. I want to verify it prints to the terminal the correct strings, but the last thing that the app needs to do is to exit. So I tried capturing it with with-out-str and leaving the program using System/exit 0, but it seems the program shuts down before any output is produced or captured. I reproduced a simpler version in repl, like so:

(defn print-many []
  (prn "here")
  (prn "there"))
=> #'user/print-many

(let [output (with-out-str (do (print-many)
                               (System/exit 0)))]
  (= output "here"))


Process finished with exit code 0

I think you are capturing the output of (System/exit 0) since that is the last form of your do.

that sounds correct

but does it mean that if I want to exit the program there is no way for me to capture what it prints?

The way I've handled that in the past is to split out the code that puts the string together into a separate function and unittest that function. So that would mean changing print-many to return a string and not call prn directly. Just a thought. I try to keep (defn -main ...) very tiny, for just this reason.

well, for one you're not flushing the output stream actually nevermind, prn does autoflush

invoke (flush) before exiting

and invoke System/exit after the =

you can't do any computation after the program has quit -- System/exit needs to be the last expression evaluated

I guess the problem is that I use System/exit to close the program

and I want to run a test on a whole program

your program code should not invoke System/exit anywhere but in the top level

the above is just a simplified version of how my app behaves

if you need to return from a place deep in your stack, consider using an exception

e.g in your app instead of having (if my-condition (continue-running) (System/exit 0)) consider (if my-condition (continue-running) (throw ..))

but the idea is that it prints something, and then when it’s done processing, quits

then at the very top level you can catch the exception and invoke System/exit

and in your tests you can capture the exception and return nil, so that you can capture the output

thank you!

in general, things like (System/exit) and (shutdown-agents) are world-ending kinds of calls and should be factored to the very outside of an application either by wrapping or via swappable callback so that you can test stuff without actually ending the world

if I want to make this to a back-end of a website (https://gist.github.com/RoelofWobben/98066daa4042ad9e7e5db0b0956c00c3) am I right I only need ring and cheshire so no framework

the bare minimum with ring is a web server backend (which only shows up directly in one line of code), ring core, and a router

the web server does the actual work, ring has an adaptor to that which makes it more clojurey, and the router decides which function to call for each request

the default back end is jetty, the default router is compojure

@roelof the ring wiki has an even simpler example, just ring and backend and no router - it's a "misbehaving" server with no routes, it just uses the same function for every request ignoring the route https://github.com/ring-clojure/ring/wiki/Getting-Started

I know I read that page several times and did some experiments

that is why I think things like luminus is a overkill for this back-end

looking at your code again, it looks like you are only interested in doing one thing - parsing a request URL to calculate a result - the issue you'd come up with is your function will get called with a lot of paths that won't make sense to that code

eg. browsers will ask for favicon.ico

with a router, you can have separate 404 response (or even a meaningful one), otherwise your function gets complex as it tries to handle all these unexpected strings

(or just blatantly fails for all unexpected input)

oke

so maybe ring and computure ?

with compojure, yeah

oke

first study those two again and try to make it work

@roelof this is a minimal compojure router, you can plug in any function that takes a ring request map and returns something ring knows how to use as a response https://github.com/weavejester/compojure-example/blob/master/src/compojure/example/routes.clj

oke, I have not decided if the front-end will be hiccup or maybe something like reframe

with reframe your back end router will be managing requests from the cljs code for data used in rendering the page, with hiccup your back end router ends up calling code that generates the page, the router will look similar to what's above in both cases

but I will like i said study ringa nd compujure

using hiccup is easier for a first draft

before Im going to make my own code work

or even skipping hiccup and just returning some html string

for example

in the end I hope I can build a front-end like this : https://codepen.io/rwobben/full/xxVepxG

but then without the movement

See how long that will costs me . maybe weeks or maybe months but that does not matter

i can run lein uberjar just fine, but when do java -classpath ... target/snapshot.jar i get the error /tmp/form-init1109030110411510610.clj ( no such file or directory )

wassup wid dat ?

i use the correct classpath

the uberjar task makes two jars

you are running the wrong one

java -jar uberjar works fine

(this is because it treats making a jar for your app as a subtask for making the fat jar, and doesn't delete that pre-req)

oh ?

ok, i need to do lein jar ?

no, lein uberjar creates two jars, one of which is created by using lein jar

I'm just saying that when you get an error like that, the usual cause is running the regular jar instead of the uberjar

"correct classpath" with an uberjar is just one thing- the jar itself

it contains all other things that it needs

i want to use the jar because it is smaller

but it's not smaller than all the separate classpath items you need to gather to make it usable

the file is smaller, the system loaded into ram will be the same i imagine

the convention is to use uberjar for application deployment, and regular jars for libraries

otherwise you need to deploy and run a build tool as part of app deployment, which is not a good idea

The "skinny" jar is smaller because it only contains the files from your project. You want to use the "uber" jar that contains all the transitive dependencies as well.

i usually use cider

cider is another thing you don't want to use when deploying an app

if you don't deploy, and everything is just cider usage with your editor, you don't need jar or uberjar at all

but still i wonder why a trasient file in /tmp is required

that's an implementation detail of lein, it creates a bootstrap to load up your project

yes, tmux, emacs, cider is my stack

ah well.. i was just wondering

For development, that is fine. But for deploying, you (likely) just want an uberjar.

OK - it would be foolish to deploy an app for others to use that is run that way, but if you aren't doing that you can ignore jar / uberjar entirely

this is code for my private web server, deployment from me to others would be source only

OK - running cider and emacs as a container for your app in a production environment is considered a problem. It can work but you can save yourself a lot of problems by just not doing that.

the uber is 27M, that is a lot of man hours of coding

lately i've been using the uberjar more often

not my coding, other persons, it's really amazing

sure, and now i'm looking at GraalVM, more candy to tempt me

is it ok to run "lein snapshot.uberjar &", backgrounded is what i call it

one last question For a new project can I the best use leiningen or switch to the new kid clojure clj ?

For a new project I'd use clojure CLI over lein. But be aware that you may need other tools (or aliases) to build and run. See https://github.com/clojure/tools.deps.alpha/wiki/Tools

@codeperfect If you have already created an uberjar that has a main function, then java -jar youruberjar.jar should work.

switching between the two is relatively easy, and most tutorials (especially with ring) specify lein commands for building / running, I'd recommend lein for a beginner

(unless you are following a tutorial or lesson that uses clj, then use that of course)

Hey all, I'm having a tough time wrapping my head about how to deploy a Clojure app for production. I'm hoping to explain what I'm trying to do and get pointed in the right direction! So I've got a simple ring-based HTTP server which connects to a database and serves up an index.html file for requests to "/". When I'm working on it locally, I run it with the clj tool, but now the time is coming to put it online. Coming from a Javascript background, the way I'd do that is: - install NGINX on a web server - start up a Node process running my backend server, this runs on port 3000. - the NGINX server reverse proxies requests on port 80 to port 3000, and bam, you're on the internet. This is a Clojure app though, and Clojure compiles to Java, so I need something that can build a JAR, which when invoked will start the server on port 3000. It looks like Boot and/or Leinigen will do this for you out of the box. I'm open to refactoring to use one of those tools, but I'd rather stay away from them if I can so I can stay current with language tooling. When it comes to clj though, it's unclear how you go about this. I've tried: - https://github.com/EwenG/badigeon , and - https://github.com/tonsky/uberdeps But it isn't working out great so far. Do I even need to build a JAR in the first place? Can I just run the app on the production server the same way I do locally?

@andyfry01 try depstar for creating uberjars with clj, works wonders

You don't have to build a jar though, you can start the app with clojure -M -m myapp.main as well

right from the git repo

Right, I figure I can fall back on this if I can't get it to work. I assume the downside is that the app won't be as performant and/or use more memory? Are there other downsides to that approach?

I've got an example of building an uberjar here: https://github.com/babashka/pod-babashka-buddy/blob/8ac874b44e3700666ce77aa94fee374a6cfae7df/deps.edn#L6

@andyfry01 there is no real difference if your source is loaded from an uberjar or from disk, but people also AOT their code for faster startup during deploys

this can be independently done from an uberjar though. uberjars are just easy for copying things around

but you often see the combination of both. none of this is mandatory

so if your old AOT-ed uberjar is still running and you can stop it + swap the uberjars real quickly and then start the other one, you can have less downtime

Ahhh, ok. This is great info.

this is actually one of the questions in the Clojure survey: how do you start your production app?

Looks like I'm headed over there next 👀

https://www.surveymonkey.com/r/clojure2021

question 20

Thank you! This is exactly what I needed. Sounds like my approach is going to change over to installing a process runner which can invoke my app directly with clojure on startup and reboot, and maybe upgrading to building JAR files if/when I need the advantages you mentioned above.

You've saved me hours of googling :man-bowing:

There shouldn’t be any hit to performance. Clojure -M is just a script that builds a class path. So largely saving you the trouble

> Clojure compiles to Java clojure emits bytecode that runs on the jvm

the argument against running the server on prod the way you do locally is that clj is a build tool, and it's more reliable to have a standalone thing to run (a fat jar with deps) as opposed to a program that will fetch your deps and run your code

it removes a layer of complexity / source of errors

last I checked / tried, depstar was the best way to make an uberjar with clj https://github.com/seancorfield/depstar

bonus, the author is very active here and responsive to errors or questions

@andyfry01 I'm going to emphatically disagree with the others about the benefit of using an uberjar - the main advantage as I see it is that you have a single artifact to recreate a state of your code, which eliminates many sources of uncertainty (eg. - I've had prod errors caused by the programmer having a "customized" version of a lib locally). when you have an uberjar you can directly find this sort of discrepancy (as of a specific release artifact), while using the build tool in prod means investigating the caches and other server state

Also it separates concerns - if you use a build tool on prod, you now have tooling config / proxys, auth to private repos etc. that leak into multiple environment configs. Ideally you only need these things in the local dev machine and the CI / build server.

(to elaborate more about the specific example: if you are using an uberjar, you can verify the code that runs based on what's present in the jar, to do the same debugging when using a build tool in prod, you need to somehow guarantee that your editor / dev environment sees the exact same cache of packages, and sources all artifacts from the same places that the server did - maven shenanigans are rare, but between local dev mucking and private maven server admin they do happen)

@andyfry01 if you get stuck with depstar ask for help in the #depstar channel -- happy to explain/walk you through everything there.

what is a transducer ?

it's a transform like "map" or "filter" removed from the concrete detail of a collection as input

they are mainly an optimization, but they also mean you can eg. filter a channel without turning it into a collection first

if you run (map f (map g (map h coll))) there are two lazy seqs created that are only needed in order to provide data to another immediate step. clojure doesn't optimize that away, but transducers allow building the same pipeline without that overhead.

so you can use (sequence (comp (map h) (map g) (map f)) coll) to get the same result as the nested map calls, but it's cheaper on resources

as a beginner, you can ignore transducers for a while

mmm... thanks again, and yes i don't see any need for using transducers at this time

@noisesmith I don't disagree with you. I'm just curious how "(eg. - I've had prod errors caused by the programmer having a "customized" version of a lib locally)" is fixed by an uberjar vs running using a dependency tool in an environment that should be resilient to dev-local changes.

Btw, I also use an uberjar even for GraalVM native-image builds, using depstar now. I can also just pass the classpath using $(clojure -Spath):classes but I hit a problem on Windows (too long classpath, or some other gnarly problem) which was fixed by using an uberjar.

I still need lein for uberjars in other projects due to some errors I'm getting with clj (some OS-specific .tar.gz error message)

because my coworkers periodically, locally install changes to libraries into a cache without changing version numbers, so they are unable to recreate or understand the bug

or worse, they blame the wrong thing and introduce bad changes via shotgun debugging

right

but this problem still manifests with an uberjar I guess?

considering that the uberjar is also built using the same deps that would be have been resolved in the prod env if you used a build tool there

no - because you open the source file that's in the uberjar when you go to source

but it sounded like they lein install some deps?

@dpsutton for example, yes

but any resultant uberjar from their machine would be similarly tainted right?

which is borkdude's point. build machine needs to be clean. which is largely the same problem as the prod machine needs to be clean

But that's not the only issue, there's also cases where a lib does AOT, so acording to project.clj you are using one artifact, but in the deployed artifact you use another. This can fool an editor that hooks into your build config, it doesn't fool editor+uberjar.

but my point was about dev replication

true. i don't have a strong opinion in this fight and lean towards uberjar as default

@borkdude it;s like with C code, somebody will copy and modify a system header file and include it with a sneaky #include "stupidly-modified-system-header-file.h" , which uses a locally modified "standard" file, a real bummer

it's a pragmatic concern - the expensive thing is developer time

uberjars eliminate many factors that I've seen waste days of dev time

I think one of the key points is around how to debug what happens when there is an issue. with the uberjar, it's much easier to inspect the uberjar artifact to debug what happened. if it's just a command that was run using the state on the prod machine, it's a different story. if there's an issue, you can redeploy, but how to figure out the root cause so it doesn't happen again?

well, maybe a "dependency tool" will stop that in clojure

when you use an uberjar with a lib that's already AOT-ed... that doesn't really help the fact that that other lib was already AOT-ed. I had a problem with a lib that was AOT-ed and transitively included AOT-ed sources of cheshire, while we needed a newer version. This was a major headache, but not helped by using an uberjar.

but it's really all based on filenames

eh]

when I want to know what's being used, I do: (slurp (io/resource "foo/bar.clj")) or (slurp (io/resource "foo/bar_init_.class")) which is how I managed to find the AOT issue

this works regardless of uberjar or not

@borkdude it's a pain, but when I use an uberjar, I can see via the files in the jar that the wrong thing was included - not using an uberjar slows finding issues like that

there's also cases where you deploy, it corrupts some state in the database or some other persistent storage, but you don't find out about it until a week later. if you have the actually deployed artifacts, then it makes debugging, reproducing, and fixing those types of issues much easier.

I guess I could have saved a lot of words and said "it reduces many layers of complexity when diagnosing prod issues"

ok, fixed by an uberjar, ok... i see some greater stgability now

stability, sorry

it can also cause issues, e.g include a conflicting resource. I don't see an uberjar as a cure-all, although they may cure some things. (slurp (io/resource ...)) is the cure for "what is being used".

right, but looking at the jar I can see the two resources (the .class vs. the .clj) and that they come from different artifacts

also, I tend to use io/resource to get coords, and open that coord in my editor rather than using slurp, but that's a trivial difference

@codeperfect An uberjar is not a cure-all for this. An uberjar can be built from patched files. "Reproducible environment" is the proper answer here.

(io/copy (io/resource ...) "/tmp/foo.clj") is also something I sometimes do ;)

my editor opens jars so I don't need to do that

Wondering about this

mine does too, but read-only

I can reproduce it if you want

yeah, it sounds like we found relatively equivalent process- I often use io/resource by itself to verify where the ns comes from (which is often more interesting than the precise contents at a first glance)

@borkdude heh, as usual you reduce the dataset to the essential content ;0

🙂

wait

(ins)user=> (io/resource "clojure/set.clj")
#object[java.net.URL 0x545607f2 "jar:file:/home/justin/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar!/clojure/set.clj"]

Is it an issue with a native lib?

oh there it is

my wifi really sucks, i appologize for the lag

yeah, it took me a while to figure out that our version of cheshire was using the equivalent of:

user=> (io/resource "clojure/set__init.class")
#object[java.net.URL 0x3e134896 "jar:file:/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar!/clojure/set__init.class"]

so next time I'll be looking for that darned thing first

I think so yes.

it is with the SVM dependency from GraalVM

I've got a repro here:

{:deps {borkdude/sci {:mvn/version "0.2.0"}
        cheshire/cheshire {:mvn/version "5.10.0"}
        borkdude/clj-reflector-graal-java11-fix {:mvn/version "0.0.1-graalvm-20.3.0"}}
 :aliases {:uberjar
           {:replace-deps ; tool usage is new in 2.x
            {seancorfield/depstar {:mvn/version "2.0.165"}}
            :ns-default hf.depstar
            :exec-fn uberjar
            :exec-args {:jar plsci.jar
                        :compile-ns [plsci.core]
                        :aliases [:native]}}
           :native {:jvm-opts ["-Dclojure.compiler.direct-linking=true"]
                    :extra-deps {org.clojure/clojure {:mvn/version "1.10.2-rc2"}}}}}

$ clojure -X:native:uberjar
[main] INFO hf.depstar.uberjar - Compiling plsci.core ...
[main] INFO hf.depstar.uberjar - Building uber jar: plsci.jar
{:warning "ignoring unknown file type", :path "/Users/borkdude/.m2/repository/org/graalvm/nativeimage/svm-hosted-native-darwin-amd64/20.3.0/svm-hosted-native-darwin-amd64-20.3.0.tar.gz"}
{:warning "ignoring unknown file type", :path "/Users/borkdude/.m2/repository/org/graalvm/nativeimage/svm-hosted-native-linux-amd64/20.3.0/svm-hosted-native-linux-amd64-20.3.0.tar.gz"}
{:warning "ignoring unknown file type", :path "/Users/borkdude/.m2/repository/org/graalvm/nativeimage/svm-hosted-native-windows-amd64/20.3.0/svm-hosted-native-windows-amd64-20.3.0.tar.gz"}
{:warning "ignoring unknown file type", :path "/Users/borkdude/.m2/repository/org/graalvm/truffle/truffle-nfi-native-darwin-amd64/20.3.0/truffle-nfi-native-darwin-amd64-20.3.0.tar.gz"}
{:warning "ignoring unknown file type", :path "/Users/borkdude/.m2/repository/org/graalvm/truffle/truffle-nfi-native-linux-aarch64/20.3.0/truffle-nfi-native-linux-aarch64-20.3.0.tar.gz"}
{:warning "ignoring unknown file type", :path "/Users/borkdude/.m2/repository/org/graalvm/truffle/truffle-nfi-native-linux-amd64/20.3.0/truffle-nfi-native-linux-amd64-20.3.0.tar.gz"}

^ @seancorfield does this look familiar?

it works with lein

Let's see if I can express this. I have a grouping of vector of maps. On each group, I'm invoking (mapv :fookey my-vector-of-maps) and it's sorta working, it's pulling out from the maps and constructing a vector of the values I'm interested in. However, in certain groups, the value for the key is nil, thus I end up with [nil] on occasions. Any suggestions on how to avoid constructing the vector (or returning an empty vector []) if the value of the key is nil?

it would be nice to have a tool that replicated this logic https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L427 - finding the location of first of class / cljc / clj to be found

Perhaps this can be added to the output of *loading-verbosely*:

user=> (binding [clojure.core/*loading-verbosely* true] (require '[clojure.set] :reload-all))
(clojure.core/load "/clojure/set")
nil

you can wrap it in filter identity or use (into [] (mapcat ...) coll) to skip maps with no results

yeah, it would be great if loading-verbosely was more specific about where it got that thing it was loading

yes, filter is your friend

currently that's decided at the innermost layer (in Rt/load), which makes it harder to instrument like that

well, a small function that just tries to find the first non-nil resource using this logic is quickly written

i eat millions of nils with my filter

you can hook that up with the output from loading-verbosely to print the exact locations

the fact that mapcat can return 0 or more results for each input is under-utilized, it can be an elegant replacement for a separate filter step

right, for something like this there's no harm in calculating it twice

Not sure if I've described that correctly, a moment, let me whip up a data shape...

user=> (mapcat (fn [m] (let [[fst :as result] (:k m)] (when fst [result]))) [{:k [nil]} {:k [:a :b :c]} {:k [:d :e :f]}])
([:a :b :c] [:d :e :f])

or

user=> (remove #{[nil]} (map :k [{:k [nil]} {:k [:a :b :c]} {:k [:d :e :f]}]))
([:a :b :c] [:d :e :f])

Hi, sorry it took some time...

(def my-coll-of-maps [{:k :a
                       :x nil
                       :y "456"}
                      {:k :b
                       :x "123"
                       :y "456"}
                      {:k :b
                       :x "789"
                       :y "456"}
                      {:k :c
                       :x nil
                       :y "456"}])

(for [[_ my-maps] (group-by :k my-coll-of-maps)]
  (let [x-vec (mapv :x my-maps)]
    x-vec))

[nil]
["123" "789"]
[nil]

that's without the surrounding [] of course

my examples above both remove [nil] - the second one is better actually

Ideally I would like to end up with either [] or nothing, just one entry, ie., ["123" "789"]

oh, so you do want mapcat

(mapcat :x my-maps)

trying 🙂

I end up with

()
(\1 \2 \3 \7 \8 \9)
()

yeah, I missed that for

yeah, you probably just want (remove #{[nil]} (for ...))

or turn the let into :let in the for, then add a :when that excludes [nil]

the remove works, but interested in the second option too

googling

(for [[_ my-maps] (group-by :k my-coll-of-maps)
      :let [x-vec (mapv :x my-maps)]
      :when (not= [nil] x-vec)]
  x-vec)

that works too

thank you @noisesmith that was very helpful 🙂

and educational too!

Hi there, I’m confused by an error message that I keep getting. I have function that ends with:

(remove empty? ....) 

({:CONNECT [:PLAT]} {:CONNECT [:CHART, :TRENDS]})

I can take the straight output and do: (reduce (fn [one two] (merge-with union one two)) '({:CONNECT [:PLAT]} {:CONNECT [:CHART, :TRENDS]})) and get the output I want.

But when I try to put it straight into the larger function I’m building up (reduce (fn …) (remove empty?….) I get an error > Execution error (ClassCastException) at prodops.core/eval31234$fn (form-init5220355977580653078.clj:1705). > ; class [Ljava.lang.Object; cannot be cast to class clojure.lang.IPersistentCollection ([Ljava.lang.Object; is in module java.base of loader ‘bootstrap’; clojure.lang.IPersistentCollection is in unnamed module of loader ‘app’)

Can you share more context for how you’re putting it into the larger function? Also, merge-with can apply to more than two maps at once, so this is a bit shorter:

(apply (partial merge-with union)
       '({:CONNECT [:PLAT]} {:CONNECT [:CHART, :TRENDS]}))