Clojurians Log v2

I would start by writing a for expression that destructs the input, and generates paths and values, then just reduce with assoc-in

Deconstructs

(reduce
 (fn [m v]
   (assoc-in m (pop v) (peek v)))
 {}
 (for [[a-or-b m] {:AAA {:prop1 ["A"] :prop2 ["a"]}
                   :BBB {:prop1 ["B"] :prop2 ["b"]}}
       [p1-or-p2 m] m]
   [p1-or-p2 a-or-b m]))

@grzegorz.rynkowski In your output, are you sure you want the nested structure to be a vector, or should it be a map?

ah, sorry, misread the request output

That was a question for him, actually. My bad

for vectors the same, just more annoying to construct, but always going through for to tear things apart

hi how can i know if something is set up for lazy evaluation (versus total realization) ? i assume some functions are lazy while others are not... is there a good resource I can use to find out which is which?

phrased differently, if i have a large dataset, how can i ensure i'm consuming one input at a time from the dataset, as opposed to lining it up for total eval

The only thing lazy is lazy is lazy seqs, anything not lazy seqs in -> lazy seq out is not lazy

And lazy seqs may not be strictly lazy, they may sometimes chunk up operations to improve efficiency

I run clj -X hello/run reprot error :Execution error (FileNotFoundException) at clojure.run.exec/requiring-resolve' (exec.clj:31). Could not locate hello__init.class, hello.clj or hello.cljc on classpath. I'm going to follow the code here https://clojure.org/guides/deps_and_cli

did you create src/hello.clj as stated on this page?

yes

im os is win10

looks like you running clj -X hello/run from src directory but you should run it from hello-world

clj command expects to have deps.edn in the same directory where it is invoked

yes. you'are right. good job. thank you very much!

np)

I have another question. I Include a local jar on disk. and I use (:require [libs/pdftools :as pdf]), when I press C-c C-k compile file report error:

require expects a namespace not a file

I don’t know what is inside of pdftools to suggest what to put into require.

pdftools is a local disk jar file

I want to use a local jar file from disk to my project,

sure, but the content of that jar should be either compiled java classes or clj source files. depending of type of the content you should use either :require to “require” clojure namespace or :import to load compiled java classes

in your last screenshot you have wrong argument for second :require statement (btw. this is an antipattern in organizing clojure namespaces, normally you should have single :require in one ns declaration)

could you show what is inside of that jar? jar -tf path/to/file.jar

This is my create a process pdf file tools in java

ok, so there are some java .class files then you should add into your ns something like this

(ns 
  ...
  (:import (org.pansome.pdf.common PdfUtils)
           (org.pansome.pdf.common FontUtils)
           ...))

https://clojure.org/reference/java_interop this guide might be handy

is this right?

in deps.edn file add depend file

yes

run error 😭

(org.pansome.pdf.common PdfUtils) note the difference - in my form there is no dot between common and PdfUtils

Thanks, It's run.

clj -X hello/run is run. but in Emacs C-c C-k is error.

emacs? looks like screenshots are taken in intellijidea

@delaguardo Emacs 27 with tabs and LSP activated makes it look a lot more like Intellij...

@imxingquan there is an #emacs channel for help. You might want to say what command you are running to help understand what you are doing in Emacs. The key binding could be doing anything...

Generally you need to evaluate things in order, so ensure the namespace has been evaluated in Emacs with Cider, which makes the functions from the required libraries available to the rest of the code.

How can I import dependencies from mvn. report error like this

I'm trying to use this thing: https://clojars.org/cljsjs/rangy-textrange Steps I've taken: - added cljsjs/rangy-textrange {:mvn/version "1.3.0-1"} to deps.edn under :deps - added [cljsjs.rangy-textrange :as rangy] to my ui.cljs under :require - restarted my compiler and confirmed that rangy gets installed:

shadow-cljs - starting via "clojure"
Downloading: cljsjs/rangy-textrange/1.3.0-1/rangy-textrange-1.3.0-1.pom from <https://repo.clojars.org/>
Downloading: org/clojure/clojure/maven-metadata.xml from <https://repo1.maven.org/maven2/>
Downloading: org/clojure/clojure/maven-metadata.xml from <https://repo.clojars.org/>
Downloading: cljsjs/rangy-core/1.3.0-1/rangy-core-1.3.0-1.pom from <https://repo.clojars.org/>
Downloading: cljsjs/rangy-core/1.3.0-1/rangy-core-1.3.0-1.jar from <https://repo.clojars.org/>
Downloading: cljsjs/rangy-textrange/1.3.0-1/rangy-textrange-1.3.0-1.jar from <https://repo.clojars.org/>

The required namespace "cljsjs.rangy-textrange" is not available, it was required by "sheluchin/ui.cljs".

According to https://github.com/cljsjs/packages/wiki/Using-Packages#a-quick-javascript-interoperability-refresher, you shouldn't use :as rangy . Simply require cljsjs.rangy-textrang , then use js/some-fn-from-rangy to call into it.

I'm not 100% sure your problem is related to this, because I don't get the same error. However, when I do as the link suggest, everything seems to work fine. EDIT: Nevermind, I didn't catch you were using shadow-cljs 🙂

shadow doesn't work with cljsjs packages, but with the native npm package itself

https://shadow-cljs.github.io/docs/UsersGuide.html#cljsjs

This may also be relevant: https://shadow-cljs.github.io/docs/UsersGuide.html#_using_npm_packages

Thanks all.

I’m trying to process a 2mb xml with xml-seq parsed with clojure.data.xml/parse with 1Gb heap — I’m getting

Syntax error (OutOfMemoryError) ,,,
Java heap space

(->> (xml-seq response)
       (filter #(some? (seq (:content %))))
       (keep (fn [e]
               (when-let [id (get-in e [:attrs :id])]
                 [id e])))
       (into {})))

from the OOM heap dump:

allocation profile with async profiler:

you could refactor ->> to transducer equivalent to avoid creation of intermediate collections

(into {}
      (comp (filter #(some? (seq (:content %))))
            (keep (fn [e]
               (when-let [id (get-in e [:attrs :id])]
                 [id e]))))
      (xml-seq response))

i was gonna say the filter and keep can be fit into one keep, right?

on top of transducer

the semantic is different, not sure if it is possible to replace with single operation

thanks, trying this out — but what it the actual issue here? is this somewhat expected?

(into {} (comp (keep (fn [e] (if (some? (seq (:content e))) (when-let [id (get-in e [:attrs :id])] [id e]))))) (xml-seq response))

every seq op is essentially a new chain of seq cells over the input (this is mitigated somewhat by chunking)

if that holds transducer is not needed.

transducers remove all intermediate seq cells and produce only the output collection, which is particularly good if you are going to hold the entire output in memory

favorite lib for assertions / testing ?

this definitely seems like a case where transducers would reduce memory footprint

2mb is not that big probably something else applies here

the in-memory object representation of that 2 mb is probably a lot bigger (and then there are multiple seq wrappers around each node)

other aspects: 1) some? is not needed 2) (get-in e [:content :attrs :id]) instead of (get-in e [:attrs :id]), right?

well seems a good opportunity to learn transducers.. in the meanwhile, with clj-memory-meter, the retained size of the whole xml is:

(mm/measure (xml/parse-str (slurp "/Users/bani/tmp/loadgroup.xml")))
=> "3.8 MiB"

for testing data structures https://github.com/nubank/matcher-combinators is really nice

is that fully realized or lazy?

I guess that's realized

(mm/measure (doall (xml-seq (xml/parse-str (slurp "/Users/bani/tmp/loadgroup.xml")))))
=> "26.6 MiB"

well, might be something else going on then

neat

(seems so, i failed to create a minimal example yet..)

tools like yourkit can take heap snapshots and diff heap snapshots which I find to be pretty instructive in understanding leaks

they also have a "path to root" which can tell you why something is being held

I’m finishing my read of the Brave Clojure and would like to know where to go from here. I started a small project putting the things I learned to work and would like to know more about the following topics: • Server-side clojure, or how to make a rest api with it • Deploying clojure, specially on Heroku. Any help or recommended readings would be nice, thanks!

I found this documentation helpful to deploy a very simple app to heroku. https://devcenter.heroku.com/articles/getting-started-with-clojure?singlepage=true

https://practical.li/clojure-webapps/ may have some interesting pieces for you. Also take a look at metosin/reitit as a data oriented alternative to compojure for routing

https://pragprog.com/titles/dswdcloj3/web-development-with-clojure-third-edition/