Clojurians Log v2

Question around NS keywords. I was under the impression that #:foo {:bar "baz"} and {:foo {:bar "baz"} were equivalent. But if I do (assume ns is :foo) (::bar some-map) with the first one, I get "baz" but if I try it with the second one, I get nil. Are they not equivalent? Or is there another way to be able to get the value of :bar without being explicit about the NS?

the equivalent of #:foo {:bar "baz"} is {::bar "baz"} aka {:foo/bar "baz"}

hmm, ok. I think I see why my code isn't working. It's unfortunate

thanks you!

@berkeleytrue namespaced keywords don’t all have to have the same qualifier in a hash map:

dev=> #:foo {:bar "baz" :quux/id 42}
{:foo/bar "baz", :quux/id 42}

The #:foo prefix provides a “default” namespace to use on any keys that don’t already have one.

oh that's interesting

https://clojure.org/guides/weird_characters#_and_namespace_map_syntax is a good read for this.

and https://clojure.org/guides/destructuring#_namespaced_keywords

It's that access of those inner properties. I was hoping to easily get properties off a map using the name space. (in some-ns) (::foo wide-flat-map) so that I can dynamically create a map later on (merge wide-flat-map {:some-ns (generate-data)}

Still learning so I'm mostly playing around

(-> wide-flat-map :foo :bar) or (get-in wide-flat-map [:foo :bar]) if you have nesting.

The names of keys and their depth in the hash map are orthogonal.

But qualified keys are idiomatic and I try to encourage folks to use them.

(and I’d advise against using :: unless you know what you’re doing since ::foo means something different in each ns)

(ns a.b.c)

::foo ;=> :a.b.c/foo

(ns x.y)

::foo ;=> :x.y/foo

That was kinda the point. I'm in the same ns but the generation is at runtime. I was hoping avoid doing :some-ns :foo because if I change the namespace name then I have to change all those calls. If ::foo worked then it wouldn't be a problem cause the compiler would put the right ns.

How would the namespace of the code end up tied to the keys in your nested map?

brain fried question... how can I set all values in a map (on all keys) to the same value?

Here's one way...

(zipmap (keys <my map>) (repeat <my val>))

Neat!

If we get map-vals in Clojure 1.11 (it’s been mentioned a few times but not confirmed), you could do (map-vals (constantly <my val>) <my map>)

Currently available via a gist as a :git/url dep: https://gist.github.com/seancorfield/6e8dd10799e9cc7527da5510c739e52f

Go vote for it here: https://ask.clojure.org/index.php/1926/adding-functions-map-vals-and-map-keys

I was exporting (not sure if that is the right word) a map with the key as the namespace and a function to produce the state {(keyword (namespace ::r)) some-fn}. Pretty hacky and I'm not happy with this setup.

Yeah, that sounds well dodgy to me…

I guess my basic question is “why?” — what semantics does the namespace name specify in this hash map?

It is just a way to prevent name clashes on a large flat map. I just had a thought of just using some arbitrary ns key instead of trying to get the namespace of the file {::state some-fn} and accessing would be (get-in wfm [::state :foo]) .

In learning clojure, what is the ONE thing that if you practice regularly, you will become a great clojure programmer?

Better to use semantically meaningful qualified names, in my opinion. :person/first-name, :address/street, etc.

Make them as unique as the context requires — more global data should probably use something like reverse-domain-name + entity style qualifiers, less global data can use a simple symbol for the qualifier.

If you work with JDBC via next.jdbc, you’ll find yourself with column names that are qualified by the table name (so you can do a JOIN and get distinct, meaningful names).

I would reject the premise of the question. The difference between good and great isn't a handful of big things, it's 10,000 little things. Bonus answer from the FAQ posted not too long ago: https://gist.github.com/reborg/dc8b0c96c397a56668905e2767fd697f#how-can-i-achieve-programming-mastery

Hmm. Let me rephrase then: If you had only 20 minutes to dedicate to clojure programming every day, what would you learn to have the highest impact?

It kinda depends on what you're hoping to get out of it. • Learn general programming concepts to apply in other languages? • Learn programming to automate workflows? • Generally learn more about how computer programs work? • Programmatic art/music? • Build a website? • For fun • something else?

After becoming comfortable with the core library, the highest impact set of concepts for me was becoming effective with clojure style polymorphism. defmultis, protocols, records, etc. Reading library code is often a good way to see effective usages of these features.

20min per day? I think going through the 4clojure exercises can be a good start, becoming good at the core functions to be able to manipulate data-structures is pretty important.

Once you got that down, and you have recursion all figured out, and you also see how you don't always need to use loop/recur, but can often just compose core functions instead, map/reduce/range/partition/juxt/etc. I would spend some time on different topics. Like go through all Collections and their functions and play with those at the REPL to understand them well. Then learn about sequence and laze-sequences and all of their functions, play with that at the REPL until you understand it well. Then I'd spend another few sessions on learning namespaces, Vars, what all you can do with those all their features and all that. Then I'd learn about the bootstrapping, how a Clojure program starts, where it loads its dependencies from, where it loads its module from. Then I'd try to write a full command line program, like a command line TODO app. As part of that, each time you struggle with something, I'd take a side-track to fully master that part, so say you struggle with managing the TODO lists state, I'd go learn about state management in Clojure, etc.

There's a lot to cover. You could also go here: https://clojuredocs.org/quickref and spend one 20min each day on each sections in here. Do one on arithmetics, then one on comparison, then one on bitwise ops, then one on cast, etc. top to bottom, order doesn't matter. And then I'd also go here: https://clojure.org/reference/reader and spend a 20min on each of those sections too (you might need more than 20min for some of these). Do one on the reader, then on repl and main, then on evaluation, etc. Until you grasp each one. After all that, you should start to feel comfortable doing full on projects, and that'd be my next move from this point on.

I wrote a code that is suddenly extremely slow. (:require [clojure.data.xml :as data-xml]             `[clojure.xml :as xml]`             `[http://clojure.java.io :as io]`             `[clojure.zip :as zip]`             `[clojure.pprint :refer [pprint]]`             `[fipp.edn :refer [pprint] :rename {pprint fipp}])` (defn iter-zip [zipper]     `(->> zipper`          `(iterate zip/next)`          `(take-while (complement zip/end?))))` (with-open [r (<http://clojure.java.io/input-stream|clojure.java.io/input-stream> "data/data_202106160959.xml")]     `(with-open [w (http://clojure.java.io/writer "data/goods.edn")]`       `(-> r`           `xml/parse ;; about 15sec`           `zip/xml-zip ;; another 15sec`           `iter-zip ;; plus 60 sec`           `;(filter loc-category?)`           `(fipp {:writer w}))))` It takes minutes for a 15Mb input file. What I'm doing wrong?

@alexander.moskvichev what is the zipper doing?

I think the iter-zip function is creating lots and lots of objects

Even so, 15 sec to parse 15 mb xml seems a lot, no?

@borkdude This is my first attempt to use clojure.zip Zipper suppose to be lazy, but I can use it wrong. In between reading and writing will be the filtering and transforming stage, I planned to use clojure.zip for navigating through XML.

@alexander.moskvichev It seems you are printing all intermediate zipper objects

you should at least call zip/node on it, to get back xml

I don't know what you're trying to do really

Start with a small example

@qmstuart I think so, but 15 sec is ok for now.. clojure.xml/parse not lazy, in that file roughly 20k items each with 10-20 properties. Maybe it's a normal time for one thread.

I used clojure.data.xml/parse and it is lazy.

@zengxh That right. But I use clojure.xml, not clojure.data.xml. Sorry for the mess in (:require...)

Perhaps for large xml file clojure.data.xml is better. It mentions this as one of its features "lazy - should allow parsing and emitting of large XML documents"

Hello fellow clojurists Is there an equivalent of Leiningen's :java-source-paths for clj deps.edn ?

@zengxh.. Shure, I even 'require it already. 🙂 But I think my main problem somewhere else

deps.edn on its own does no java compilation for you

there are various tools for deps that do this, they have their own config

ok, just found a guide using 'badigeon' https://gamlor.info/posts-output/2019-10-24-compile-java-with-clojure-deps/en/

I have a 90MB xml file 2.2 mil lines I replace fipp with just count and clojure.xml with clojure.data.xml Test briefly with time and it's inside a virtualbox vm. It takes 1.7sec 2.0sec 17.3sec up to parse xml-zip iter-zip respectively and the final count is 4.6mil

coming soon ...

The zipper object is the whole tree (xml doc) and your position so printing it might be expensive.

Speaking of RDD: how do you guys eval a form in the REPL that contains symbols defined outside of that form? E.g. I'd like to eval the (super-hard-calc x) form in the body of the let in this contrived example:

(def super-hard-calc inc)

(let [x 1]
  (super-hard-calc x))