Clojurians Log v2

the futures are not checking for an empty @work and they are not calling add-dosync, just referencing it. see while for a way to run something until a condition becomes false. to call add-dosync just wrap your references in parentheses: (add-dosync)

why do I get here a classcastexception on the let part :

(def work (ref (apply list (range 1e5))))


; And the sum will be a ref as well:
(def sum4 (ref 0))

; Write a function which, in a dosync transaction, removes the first number in 
; work and adds it to sum. Then, in two futures, call that function over and 
; over again until there's no work left. Verify that @sum is 4999950000. 
; Experiment with different combinations of alter and commute–if both are 
; correct, is one faster? Does using deref instead of ensure change the result?

(defn add-dosync [work sum4]
  (dosync
  (alter (ensure sum4) + (first @work))
  (ref-set work (next @work))))


(let [worker1 (future (while @work (add-dosync work sum4)))
      worker2 (future (while @work (add-dosync work sum4)))]
@worker1 ;wait for worker1
@worker2 ;wait for worker2
@sum4) ;return the result of their work

(deref sum4)

@marcus.akre you can always pick up where Tim Baldridge left off here https://github.com/halgari/com.tbaldridge.hermod

@roelof (ensure sum4) returns the current value of sum4 , which is a long. alter takes a ref, not a long

oke, then I have to think how to solve that

as far as I know I have to use ensure somewhere

you should use ensure on the ref you're reading the value of in the dosync body, which is not sum4

sorry, I do not

but still if I change the code to this :

(defn add-dosync [work-ref sum-ref]
  (dosync
  (alter sum-ref + (first @work-ref))
  (ref-set work-ref (next @work-ref)))

I get the classCastException

also, I recommend using alter too to update work: (alter work next)

also changed

are you sure you have reevaluated add-dosync after the change?

yep

I re-evaluted everything after a change

one additional problem may be that (first @work-ref) may be null. maybe the easiest way around that would be to check for that condition inside the dosync body

oke, you mean if the work collection is empty ?

that's even though you're checking in the future bodies, but there's a possible race condition there; you shouldn't rely on that

the exact error message is :

; Execution error (ClassCastException) at ground-up.chapter6/add-dosync$fn (form-init16078161836259666359.clj:80).
; class java.lang.Long cannot be cast to class clojure.lang.Ref (java.lang.Long is in module java.base of loader 'bootstrap'; clojure.lang.Ref is in unnamed module of loader 'app')

that check is OK to make the future finish when it's done, but you are not guaranteed that work won't be completed between the check and the execution of add-dosync

oke, so you are thinking I have to check for if work is empty also in the do-sync ?

yes, but that doesn't look like it could cause the particular error you're getting; you should get a nullpointerexception instead

can you paste the whole program again?

oke, and sentence 80 is alter sum-ref + (first @work-ref))

of course

; Write a function which, in a dosync transaction, removes the first number in 
; work and adds it to sum. Then, in two futures, call that function over and 
; over again until there's no work left. Verify that @sum is 4999950000. 
; Experiment with different combinations of alter and commute–if both are 
; correct, is one faster? Does using deref instead of ensure change the result?

(defn add-dosync [work-ref sum-ref]
  (dosync
  (alter sum-ref + (first @work-ref))
  (alter work-ref (next @work-ref))))

(let [worker1 (future (while @work (add-dosync work @sum4)))
      worker2 (future (while @work (add-dosync work @sum4)))]
@worker1 ;wait for worker1
@worker2 ;wait for worker2
@sum4) ;return the result of their work

(deref sum4)

remove the @ before sum4 in the future bodies

same error

that's causing the same problem you were getting before by calling (alter (ensure sum4) ,,,)

Deleting that did not solve the eror message

have you reevaluated the whole program?

yep, as I said I always do that

(alter work-ref (next @work-ref)) should be just (alter work-ref next)

now I see this error :

; Execution error (ClassCastException) at ground-up.chapter6/add-dosync$fn (form-init16078161836259666359.clj:81).
; class clojure.lang.PersistentList cannot be cast to class clojure.lang.IFn (clojure.lang.PersistentList and clojure.lang.IFn are in unnamed module of loader 'app')

yes, see my latest comment

yep

alter takes a ref and a function, not a value

now a null pointer exception as you said earlier

so you can check for a null (first @work-ref) and just exit in that case

yep, finally after some many days

thanks

need a break

changed it to :

(defn add-dosync [work-ref sum-ref]
  (when (seq @work-ref)
    (dosync
     (alter sum-ref + (first @work-ref))
     (alter work-ref next))))

but still the null pointer exception

you should definitely check inside the dosync

probably using ensure instead of deref/`@`(but give both a try to verify that ensure is needed)

oke

I tried this :

(defn add-dosync [work-ref sum-ref]
  (dosync
   (while @work-ref (alter sum-ref + (first @work-ref)))
   (while @work-ref (alter work-ref next))))

but it seems this is running forever

finallly working

(defn add-dosync [work-ref sum-ref]
  (dosync
   (when @work-ref (alter sum-ref + (first @work-ref)))
   (when @work-ref (alter work-ref next))))

and alter or commute makes no difference. the same output

hmm, but both the wrong answer 😞

@roelof Please try the following

(defn add-dosync [work-ref sum-ref]
  (dosync
    (when-let [work (seq (ensure work-ref))]
      (alter sum-ref + (first work))
      (ref-set work-ref (next work)))))

it should be 4999950000 and both gives 4999984077

Not sure if this question was discussed before thought (can't read all those pages above), but this is also not a good case for using refs, maybe only as an exercise. Because this algorithm of summing a list does not benefit from parallelization at all (all entries are processed one by one sequentially anyway).

I know and this is discuused before

but when I try your code it seems to run forever

Ok, let me fire up the repl 🙂

I still do not see a answer

Would you please paste the code?

(def work (ref (apply list (range 1e5))))


; And the sum will be a ref as well:
(def sum4 (ref 0))

; Write a function which, in a dosync transaction, removes the first number in 
; work and adds it to sum. Then, in two futures, call that function over and 
; over again until there's no work left. Verify that @sum is 4999950000. 
; Experiment with different combinations of alter and commute–if both are 
; correct, is one faster? Does using deref instead of ensure change the result?

(defn add-dosync [work-ref sum-ref]
  (dosync
    (when-let [work (seq (ensure work-ref))]
      (alter sum-ref + (first work))
      (ref-set work-ref (next work)))))


(let [worker1 (future (while @work (add-dosync work sum4)))
      worker2 (future (while @work (add-dosync work sum4)))]
@worker1 ;wait for worker1
@worker2 ;wait for worker2
@sum4) ;return the result of their work

@roelof from what I see, it's not running forever, it's just being REALLY slow 🙂

oke, I will try again and wait for the answer

and see if it's the right one

👌

does ensure makes is this slow

No idea, first time using refs myself. I'm not an expert, I was just curious 🙂

me too first time use it. The "Clojure from the ground up" was telling me to use a ref

Hi, I’ve been trying the fox-goose-corn kata and worked out this solution - https://paste.ofcode.org/mpzmsutwdEK3GczkzjZ6Tx I guess there are still a few things wrong…

Given a configuration, calc-path tries to generate the possible options using for and evaluate them recursively. If a child configuration is invalid or already explored, returns [] . If the child configuration has all the items on the right bank, returns the entire path. Because I’m using for, I had to apply a few quick fixes to make it work, for example I use remove empty? at line 80 and 86. Another one is at line 115. The path is inside a series of lists ((((path)))) so I had to find a way to extract it.

We'll crack it, no worries 🙂

yep , here still no answer 🙂

Overall, ref seems like a very heavy-weight tool, that's why it's rarely used, and people use atoms instead.

yep

now long did I take in min

I think im now at 2 - 3 min

I think around that, more or less.

I can time it.

no needed

then you have a faster computer then I

here still no answer

Yeah, 2:30 on my macbook, but I ran it as uberjar.

aha, and I did it on repl

did it again and it's now at 26.000

so 26 thousand

Anyway, I think it’s not a real evaluation of speed of how refs work. Here the workload is so that two CPU threads are constantly fighting and re-doing each-other’s work. I would expect that on a single-core CPU the result will be calculated much faster :)

yep I know

When you do this with reduce it takes almost no time

and I have a 4 core machine so a lot of fighting

84 thousand

and one with the right answer

in repl not on vs code

I wonder if there’s a better alternative to for to reach the same goal and avoid these those confusing lines

a lot learned and a lot to learn to really understand closures, ref and so on

I found this a very hard chapter

thanks a lot with your patience

Hello! Can someone help me debug this error? Here is my code

(defn get-books-with-extra-data
  [url]
  (let [cached (cache/fetch url)]
    (if (not (nil? cached))
      cached
      (do
        (timbre/info "Fetching book: " url)
        (http/get url {} (fn [{:keys [status headers body error]}]
                           (->>
                             (map :body)
                             (map #(hash-map :book-genres (get-book-genres %) :book-cover (get-book-cover %)))
                             ;(cache/store url)
                             )))))))

(defn books-with-extra-data
  [books]
  (->> books
       (map #(merge %1 (book/get-books-with-extra-data (string/trim (get-in %1 [:book :link])))))))

(pst)
clojure.lang.ExceptionInfo: null
                                      main.clj:442 clojure.main/repl[fn]
                                      main.clj:458 clojure.main/repl[fn]
                                      main.clj:458 clojure.main/repl
                                      main.clj:368 clojure.main/repl
                                  RestFn.java:1523 clojure.lang.RestFn.invoke
                         interruptible_eval.clj:79 nrepl.middleware.interruptible-eval/evaluate
                         interruptible_eval.clj:55 nrepl.middleware.interruptible-eval/evaluate
                        interruptible_eval.clj:142 nrepl.middleware.interruptible-eval/interruptible-eval[fn]
                                       AFn.java:22 clojure.lang.AFn.run
                                   session.clj:171 nrepl.middleware.session/session-exec[fn]
                                   session.clj:170 nrepl.middleware.session/session-exec[fn]
                                       AFn.java:22 clojure.lang.AFn.run
                                   Thread.java:748 java.lang.Thread.run
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: org.httpkit.client$deadlock_guard$reify__17177
                                       RT.java:557 clojure.lang.RT.seqFrom
                                       RT.java:537 clojure.lang.RT.seq
                            APersistentMap.java:40 clojure.lang.APersistentMap.cons
                                       RT.java:677 clojure.lang.RT.conj
                                       core.clj:85 clojure.core/conj
                                     core.clj:3049 clojure.core/merge[fn]
                                      core.clj:944 clojure.core/reduce1
                                      core.clj:934 clojure.core/reduce1
                                     core.clj:3048 clojure.core/merge
                                     core.clj:3041 clojure.core/merge
                                   RestFn.java:421 clojure.lang.RestFn.invoke
                                       stats.clj:2 goodstats.consumer.stats/eval17623[fn]
                                     core.clj:2755 clojure.core/map[fn]
                                   LazySeq.java:42 clojure.lang.LazySeq.sval
                                   LazySeq.java:51 clojure.lang.LazySeq.seq
                                       RT.java:535 clojure.lang.RT.seq
                                      core.clj:137 clojure.core/seq
                                 core_print.clj:53 clojure.core/print-sequential
                                core_print.clj:174 clojure.core/fn
                                core_print.clj:174 clojure.core/fn
                                  MultiFn.java:234 clojure.lang.MultiFn.invoke
                                      print.clj:21 nrepl.middleware.print/pr-on
                                      print.clj:17 nrepl.middleware.print/pr-on
                                     print.clj:148 nrepl.middleware.print/send-nonstreamed[fn]
                                     print.clj:147 nrepl.middleware.print/send-nonstreamed[fn]
                                     core.clj:2742 clojure.core/map[fn]
                                  protocols.clj:49 clojure.core.protocols/iter-reduce
                                  protocols.clj:75 clojure.core.protocols/fn
                                  protocols.clj:75 clojure.core.protocols/fn
                                  protocols.clj:13 clojure.core.protocols/fn[fn]
                                     core.clj:6884 clojure.core/transduce
                                     core.clj:6870 clojure.core/transduce
                                     print.clj:156 nrepl.middleware.print/send-nonstreamed
                                     print.clj:138 nrepl.middleware.print/send-nonstreamed
                                     print.clj:174 nrepl.middleware.print/printing-transport[fn]
                                     caught.clj:58 nrepl.middleware.caught/caught-transport[fn]
                        interruptible_eval.clj:114 nrepl.middleware.interruptible-eval/evaluate[fn]
=> nil

I have no idea how to proceed from here because I'm using a callback function to process the request so I think I'm correct to assume that the response has been received when the function is called (which is the only problem I could think of). Previously I was using clj-http and it was working fine, but I switched to http-kit for better performance and most of the requests work correctly but some fail with this error

try commute instead of alter, see how fast/correct it is?

Can anyne tell me how I can make it work that the repl is next to my code instead of the same part so I do not have to switch tabs

I will if I have the layout of vs code right

I believe http/get returns a promise, which you must deref somewhere to get the actual returned value

the Don't know how to create ISeq from: org.httpkit.client$deadlock_guard$reify__17177 error refers to the reified deadlock-guard promise, which you haven't dereffed

But that's when you don't give it a callback function right? When I try to deref the body parameter of the callback I get

java.lang.ClassCastException: java.lang.String cannot be cast to java.util.concurrent.Future

both with @body and deref body

if I read the httpkit code and docstring correctly, it always returns a promise but depending on whether you gave a callback it will resolve to either the response as-is or pass the response to your callback and resolve the return value

I think I know the problem. Since this is asynchronous, I think it's returning immediately to the caller code, which does some processing with the expected response causing it to fail

https://github.com/http-kit/http-kit/blob/master/src/org/httpkit/client.clj#L244

on some wierd way repl is now a tab instead of a seperate window

hmm

after lunch I have to figure out how to make this work

(defn add-dosync [work-ref sum-ref]
  (dosync
    (when-let [work (seq (commute work-ref))]
      (alter sum-ref + (first work))
      (ref-set work-ref (next work)))))

at some way I have to give commute another argument

You should use commute instead of alter

yep

it seems to work faster

wait for the answer

Cool. Good to know. I am not yet proficient enough to know when commute will do, and when alter is required to get a correct result.

According to the book alter/ensure if the order is important and commute when the order is not important

so as a example 1 + 2 + 3 the order is not important because also 1 + 3 + 2 - 6 also 3 + 1 + 2 = 6

Ok, makes sense

but on this the order is important (1/2)/3 in not the same as 1/ (2/3)

that 0.16 or 0,666 if im not mistaken

I guess if you’re asking about VSCode and Calva, better to ask in #calva

thanks

yep

this :

(defn add-dosync [work-ref sum-ref]
  (dosync
    (when-let [work (seq (ensure work-ref))]
      (alter sum-ref + (first work))
      (ref-set work-ref (next work)))))

(defn add-dosync [work-ref sum-ref]
  (dosync
    (when-let [work (seq (ensure work-ref))]
      (commute sum-ref + (first work))
      (ref-set work-ref (next work)))))

give the same right outcome

@euccastro

Did I do it right now ?

when I try this :

(parse-string (slurp "2008.json"))

Is the chapter7.clj source file in a directory like src/some-name-space/chapter7.clj in a Leiningen project, or something like that? If so, the current directory of the running JVM process is likely to be the top level directory of that project when you run the code.

If you print out the value of this expression, which should return a string, it should show you the current directory of the running JVM process, or at least what it was when the JVM process began: (System/getProperty "user.dir")

you could use (commute work-ref next) too

yep, it was the root dir of the projecy

"/home/roelof/ground-up/ground-up"

oke

also you thanks for the patience with me

np 🙂

Seeking advice on selection of full stack suits web app of ToDo: I'd like to develop an app of todo. It should support web browser as interface, and eventually maybe native mobile app. I hope the user experience should be better than typical web browser experience, For example, I'd like to support extensive auto-completion of user input, allow context menu by clicking the right mouse button, and have timer etc. For data storage, I guess a SQL database would be enough, as the storage would be mostly in table and sequential, Datomic would be an overkill. It should also support user authentication. Would you kindly advise me which full stack suits I should take? I've studied using ring/hiccup/compojure, rum, jdbc.next, datomic for database web app. I have not tried Coast framework, Fulcro, Luminus, Pedestal, etc. With so many seemingly good choices, I feel anxious. This is a hobby/prototype effort, for MVP (Minimal Viable Product). Thanks a lot for your help!

Wonder why the output is not sorted on the numbers

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              [(fips (fips-code county))
               (:driving_under_influence county)]))
       (into {})))

In the last step, you put the data into a Clojure map (into {}). Clojure maps are not sorted by keys.

There is a sorted map variant of Clojure maps, which are sorted by keys. If that is something you want to try, you could change the last line to (into (sorted-map)) (there is no literal representation of sorted maps that is like {} for the default (unsorted) maps).

Nor ordered, see https://github.com/clj-commons/ordered

still not ordered

{"AZ, Maricopa" 26235,
 "CA, Los Angeles" 45056,
 "CA, Orange" 17572,
 "CA, Riverside" 10814,
 "CA, Sacramento" 8589,
 "CA, San Bernardino" 13983,
 "CA, San Diego" 18562,
 "NV, Clark" 10443,
 "TX, Harris" 10432,
 "WA, King" 11439}

` (defn most-duis   "Given a JSON filename of UCR crime data for a particular year, finds the   counties with the most DUIs."   [file]   (->> file        load-json        (sort-by :driving_under_influence)        (take-last 10)        (map (fn [county]               [(fips (fips-code county))                (:driving_under_influence county)]))        (into (sorted-map)))) `

sorted-map order keys by hash function, not alphabeticaly this is wrong ) my bad

@roelof The result you showed last above is sorted alphabetically by the keys. What do you want it sorted by?

Oh, I see now you said earlier "on the numbers". One recommendation: write a little print function to print the data in the order you want to see it in, but let it be stored in a default Clojure map, which is not sorted in any particular order (well, it is sorted on a hash function of the keys, but that is unlikely to be an order you are interested in knowing).

Example:

(def d1 {"AZ, Maricopa" 26235,
         "CA, Los Angeles" 45056,
         "CA, Orange" 17572,
         "CA, Riverside" 10814,
         "CA, Sacramento" 8589,
         "CA, San Bernardino" 13983,
         "CA, San Diego" 18562,
         "NV, Clark" 10443,
         "TX, Harris" 10432,
         "WA, King" 11439})

user=> (pprint (sort-by val d1))
(["CA, Sacramento" 8589]
 ["TX, Harris" 10432]
 ["NV, Clark" 10443]
 ["CA, Riverside" 10814]
 ["WA, King" 11439]
 ["CA, San Bernardino" 13983]
 ["CA, Orange" 17572]
 ["CA, San Diego" 18562]
 ["AZ, Maricopa" 26235]
 ["CA, Los Angeles" 45056])

I know that last print result does not look like a map. That is because it is a sequence of map entries, which is what you get when you take a map and use it as a parameter to a function that expects a sequence, like sort-by does in the expression above. If all you care about is seeing it in the desired order, that shouldn't be a problem.

I have a pattern type of question for designing APIs. I come from a Java background where there are a lot of pure and impure functions mixed together, especially when reading and writing from a database. I've read Clojure guidance that says to write as much of the logic in pure functions as possible, and then use some sort of wiring layer to mix together these pure functions. It would seem that once you get past simple CRUD APIs, the coordination functions would become both very large and could potentially have a lot of duplication. Can anyone point me to either examples or tutorials that show how this works as the codebase grows?

Old lisp expert here. New to Clojure. Writing a Swank client in ClojureScript (replacement for Emacs SLIME, for the browser). Once that’s done, I’ll likely add a cider client as well. This is a retirement project for me. I don’t have the mental energy I once did, so the Clojure startup transient is harder than it would have been 10 years ago, but I’ll get there. Clojure is, after all, just another lisp with some added syntax and a new standard library to learn (he said, not really knowing what he’s saying). https://github.com/billstclair/wilfred Not much here yet. If somebody has ideas for improvement of the basic project, I’d love the feedback. I’m just copying what I found on the web at this point. I think it will be good enough to get started, however.

If you don't need constant time lookup, you might want to consider representing your data as a list of maps instead of a map:

[{:county "AZ, Maricopa" :dui 26235}
 {:county "CA, Los Angeles" :dui 45056}
 ...]

@andy.fingerhut thanks, solved it

then I need to convert my data to it because its from a external source

or do you mean I can change something here to make this output :

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              [(fips (fips-code county))
               (:driving_under_influence county)]))
       (into{})
       (sort-by val)))

yup, I'm talking about this:

(fn [county]
              [(fips (fips-code county))
               (:driving_under_influence county)])

oke, then I have to think how to make those changes

that's incorrect - ensure is only needed for values you are not altering but only reading

I'm trying to grasp clojure here - playing with stumps of code. How do I loop through a list of maps and rewrite the maps (in other languages filter can be used for this):

(def mylist
  ({:a 1 :b 2 :c 3}
   {:a 1 :b 5 :c 6}
   {:a 7 :b 8 :c 12}
   {:a 10 :b 11 :c 12}))

(map (fn [m] (assoc m :b (m c)) mylist)

with a colon before the c

Thanks. Trying that out however, Calva repl (visual studio code) just gives me:

println mylist #prints out mylist in repl window
(map (fn [m] (assoc m :b (m :c)) mylist)) #just prints out: #function[clojure.core/map/fn--5847]

so println mylist works.. map just fails.. I'm doing ctrl+shift+p and evaluate top form

It appears that it evaluated just the sub-expression map, which I would expect to return a result like you showed.

Parens are wrong, mylist should be in the last form, not inside the fn definition.

(map #(assoc % :b (% :c)) mylist)

this worked:

println (map (fn [m] (assoc m :b (m :c))) mylist)

It's like you ran map without a collection, which returns a transducer function.

Now I have something to play with.. trying to get the hang of this 🙂

FYI the println at the beginning of those lines is probably not even being evaluated at all. You should be able to leave that out.

Unless the REPL you are using is quite different than ones I have used before somehow ...

then I misunderstood the book

thanks for pointing this to me

hmm, I take a break this idea does not work

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              [{:county (fips (fips-code county)), :population (:driving_under_influence county)}]))
       (into {})))

Got it working

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              [{:county (fips (fips-code county))}, {:population (:driving_under_influence county)} ]))
       (into {})))

hopefully i did it right

you can simplify it further

can I ?

also this still creates a map

Try this out:

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              {:county     (fips (fips-code county)),
               :population (:driving_under_influence county)}))))

that gives this :

({:county "CA, Sacramento", :population 8589}
 {:county "TX, Harris", :population 10432}
 {:county "NV, Clark", :population 10443}
 {:county "CA, Riverside", :population 10814}
 {:county "WA, King", :population 11439}
 {:county "CA, San Bernardino", :population 13983}
 {:county "CA, Orange", :population 17572}
 {:county "CA, San Diego", :population 18562}
 {:county "AZ, Maricopa", :population 26235}
 {:county "CA, Los Angeles", :population 45056})

where my old code gave this :

{{:county "AZ, Maricopa"} {:population 26235},
 {:county "CA, Los Angeles"} {:population 45056},
 {:county "CA, San Diego"} {:population 18562},
 {:county "CA, Sacramento"} {:population 8589},
 {:county "CA, Riverside"} {:population 10814},
 {:county "CA, Orange"} {:population 17572},
 {:county "CA, San Bernardino"} {:population 13983},
 {:county "NV, Clark"} {:population 10443},
 {:county "TX, Harris"} {:population 10432},
 {:county "WA, King"} {:population 11439}}

yup, notice the parens instead of the curly brackets in the beginning/end?

yep

then tomorrow look if I can make it work to calculate the prevalance so this number divided by the population

as I see it now I have to make the calculation two times

hmm, this eems to do it almost

you're right.. now it outputs it without the prinln @andy.fingerhut - odd it didn't at first

I would be very surprised if it actually required that -- the P in REPL is for Print, so after reading and evaluating your expression, it should always Print the return value (in addition to printing anything that the expression itself causes to print, e.g. a function call with calls to functions print and/or println inside of it).

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              {:county     (fips (fips-code county)),
               :total_uid (:driving_under_influence county)
               :prevalance ( / (:driving_under_influence county) (:county_population county))
               }))))

{:county "CA, Los Angeles",
  :total_uid 45056,
  :prevalance 45056/9872263}

yes that's expected: https://clojure.org/reference/data_structures#_ratio

it is the correct form but you don't see the calculated value, you can run float on top of the division to get the actual result

thanks

next "problem"

it now sorts on drunk driving (sort-by :driving_under_influence)

I think I need to make a new key prevelance to see the 10 most highest ones

how many data points are there in the file?

183 thousand

he, I can do this :

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :prevalance)
       (take-last 10)
       (map (fn [county]
              {:county     (fips (fips-code county)),
               :total_uid (:driving_under_influence county)
               :prevalance (float (/ (:driving_under_influence county) (:county_population county)))
               }))))

but I have to make a check against the fact that population can be zero

that key doesn't exist yet when you sort

you could either compute it twice or move sort-by and take-last to the end, after the map

thanks

last question

I have now this :

(defn calculate_prevalance
  [county]
  ( if (zero? (:county_population county))
         0
    (float (/ (:driving_under_influence county) (:county_population county)))))
    

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (map (fn [county]
              {:county     (fips (fips-code county)),
               :total_uid (:driving_under_influence county)
               :prevalance (calculate_prevalance county)}))
       (sort-by :prevalance)
       (take 10)))

but the highest 10 are now all countys with prevelance of 0

Should I filter these out or is there a better way

sort-by sorts the data in ascending order

so take 10 takes the 10 items with the lowest prevalence

yep, so I get the countys with the lowest prevelance

exactly

Also, you might want to turn the fn into an actual function

??

this seems not to be sorted on the highest prevelance

|       :county | :total_uid | :prevalance | |---------------+------------+-------------| |      NC, Hyde |        125 | 0.024447488 | |      CA, Inyo |        424 | 0.024500173 | |      NC, Ashe |        655 | 0.025449742 | |  CO, Costilla |         85 | 0.026001835 | |   CO, Conejos |        210 | 0.026122652 | |  CO, Cheyenne |         45 | 0.026254376 | |    TX, Kenedy |         11 | 0.028132992 | |    VA, Norton |        118 | 0.031969655 | |    MS, Tunica |        432 |  0.04056338 | | WI, Menominee |        189 | 0.040935673 |

`(defn calculate_prevalance
  [county]
  ( if (zero? (:county_population county))
         0
    (float (/ (:driving_under_influence county) (:county_population county)))))
    

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (map (fn [county]
              {:county     (fips (fips-code county)),
               :total_uid (:driving_under_influence county)
               :prevalance (calculate_prevalance county)}))
       (sort-by :prevalance)
       (take-last 10)))

it's sorted, in ascending order

you're just seeing the last 10 counties in the list

if you want to see it in descending order you need to reverse the result

yep, and I assumed that were the 10 county's with the highest prevelance

yep, and then I see the 10 with prevelance zero because there is no population

so I set the return on zero

so I think I have to filter those out. Right ?

It can be hard to completely separate out database interaction from pure logic because you typically have a lot of situations where the pure logic at some point may well rely on DB activity from earlier in the request (assuming a complex DB-backed web app). The pragmatic question is: how separate does it really need to be? You definitely want to aim for separation and composability but there are practical limits to what you can realistically achieve in general, in my opinion, without resorting to somewhat artificial code structure and monadic-looking code.

https://www.youtube.com/watch?v=yTkzNHF6rMs https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell The 2 vids may help somewhat (functional core, imperative shell). Also, personal anecdote: afaict functional architectures tend to converge into middleware pattern (ring's wrap-*, re-frame interceptors, stuff I seen in enterprises).

no need to, just add reverse in the end

the result is correct, it's just in ascending order

you have the top 10 counties, starting from the 10th

so if you reverse it you'll get the top county first

yep,, this looks good

|       :county | :total_uid | :prevalance |
|---------------+------------+-------------|
| WI, Menominee |        189 | 0.040935673 |
|    MS, Tunica |        432 |  0.04056338 |
|    VA, Norton |        118 | 0.031969655 |
|    TX, Kenedy |         11 | 0.028132992 |
|  CO, Cheyenne |         45 | 0.026254376 |
|   CO, Conejos |        210 | 0.026122652 |
|  CO, Costilla |         85 | 0.026001835 |
|      NC, Ashe |        655 | 0.025449742 |
|      CA, Inyo |        424 | 0.024500173 |
|      NC, Hyde |        125 | 0.024447488 |

Years ago, I built a generic "workflow" engine library for Clojure that separated everything into "sources", pure functions, and "sinks". The idea was that sources could (only) be queried -- the sources were passed into the pure code, so you could write mock sources for testing -- and that the pure functions would return data structures that indicated what actions would need to be taken on the sinks, which could be database writes (inserts and updates) and email systems and anything else you needed to apply changes or effects on. As soon as a later part of your pure workflow needs the generated ID from an earlier DB insert, things get complex, and it also gets a bit artificial when the DB inserts you need "at the end" have to include stuff that you got from side-effecting calls to third-party systems. It's doable, but it gets pretty convoluted. And I finally gave up on the engine after using it for a while in production code because it was just too awkward (and monadic) to be readable once the actual workflow got complex. It's still on GH as an archive https://github.com/seancorfield/engine with an explanation of why I stopped using it.

Having an imperative shell drive everything is a much more straightforward approach but it can still take quite a bit of work to completely separate out pure and impure logic -- and if you have an imperative shell, you are still going to need some amount of tests to verify that the imperative combining of the pure/impure calls really does what you want. Definitely more maintainable than just arbitrarily mixing pure and impure code together tho' -- and more maintainable than going "full-monad" IMO.

next challenge from the "Clojure from the ground up"

|       :county | :prevalance | :report-count | :population |
|---------------+-------------+---------------+-------------|
| WI, Menominee | 0.040935673 |           189 |        4617 |
|    MS, Tunica |  0.04056338 |           432 |       10650 |
|    VA, Norton | 0.031969655 |           118 |        3691 |
|    TX, Kenedy | 0.028132992 |            11 |         391 |
|  CO, Cheyenne | 0.026254376 |            45 |        1714 |
|   CO, Conejos | 0.026122652 |           210 |        8039 |
|  CO, Costilla | 0.026001835 |            85 |        3269 |
|      NC, Ashe | 0.025449742 |           655 |       25737 |
|      CA, Inyo | 0.024500173 |           424 |       17306 |
|      NC, Hyde | 0.024447488 |           125 |        5113 |

I think it's interesting that many of these functional ideas can even be applied to games, https://fabiensanglard.net/quake3/index.php

hmm, I think the last challenge is nonsens

Write a test to verify that most-prevalent is correct.

how can I make a test if im not knowing the numbers myself

Make them up. The logic should be correct no matter what the actual values of the numbers are.

correctness != truth

oke, I have to think about it

so I should make a test file with only 1 county with some duis and some population

and use that for the test

I want to do some natural language processing (summarizing) and DL (one-shot-learning). I found this: https://dragan.rocks/articles/19/Deep-Learning-in-Clojure-From-Scratch-to-GPU-0-Why-Bother and Dragan recommends to use intel-mkl. He states it's 750 mb, but the only download I can find is a whooping 22gb. Are there other libraries I should use, or is this really the way to go?

Hi everybody, maybe this is a dumb question, but, I don't feel confortable with my solution up to now. I work with a defmulti, defined in a ns, and some defmethod, provided in their respective namespace. Now, when I want to call the method, I fall in situations where de defmethod are not found, until I explicitly require the namespace containing the defmethod. I would prefer a behavior where the calling function needs not to know its implementation, as a plugin is expected to act.

It is best if you install via your package manager. Failing that, install via the Intel installer. I don't believe either of those are anywhere close to 22GB. The instructions at https://neanderthal.uncomplicate.org/articles/getting_started.html#the-native-library-used-by-neanderthals-native-engine-optional should be helpful. Which OS are you on?

Windows 10

Hi everyone! Does anybody use hickory library? https://github.com/davidsantiago/hickory