Clojurians Log v2

What about imagez? https://github.com/mikera/imagez It might be a bit heavy for this though, but it lets you get image width and height. Here's a quick example:

(defn get-image-properties [path file]
  (let [img (mimg/load-image (str path "//" file))]
    {:measuremnt {:width (mimg/width img)
                  :height (mimg/height img)}}))

(let [path "//home/stuart//Pictures"
      files (->> (.list (io/file path))
                 (filter #(str/ends-with? % ".png")))]
  {:images (zipmap (map keyword files)
                   (map (partial get-image-properties path) files))})

{:images
 {:Selection_023.png {:measuremnt {:width 842, :height 412}},
  :Selection_025.png {:measuremnt {:width 600, :height 426}},
  :Selection_020.png {:measuremnt {:width 850, :height 340}},
  :Selection_024.png {:measuremnt {:width 602, :height 426}},
  :Selection_021.png {:measuremnt {:width 1292, :height 965}},
  :Screenshot from 2021-04-12 00-22-05.png {:measuremnt {:width 857, :height 335}},
  :Selection_026.png {:measuremnt {:width 1000, :height 426}},
  :puzzle-storm-33.png {:measuremnt {:width 850, :height 340}}}}

Looks like you have a couple of dependencies set up here. What’s in your namespace?

(ns reference.core
  (:gen-class)
  (:require [clojure.string :as str]
            [<http://clojure.java.io|clojure.java.io> :as io]
            [mikera.image.core :as mimg]))

With this as a dependency in my project.clj

[net.mikera/imagez "0.12.0"]

Thanks! I’m a little confused what’s going on with (str path "//" file) and the (let [path "//home/stuart//Pictures" would you mind explaining?

Yeah I guess also just what are you putting in the path and file arguments?

File is just something like "foo.png" as that io file path call only returns the filename, not the path and file name. Imagez needs full path to file so that str is just taking folder where images is and appending the filename on. Needs the // to escape a /.

How do i model a spec def to validate the value of an optional key in a map? Assuming the :c keyword and associated value are intended to be optional:

(spec/def ::acceptable-value? 
    (spec/and int? pos-int?))
  
  (spec/def ::a? ::acceptable-value?)
  (spec/def ::b? ::acceptable-value?)
  (spec/def ::c? ::acceptable-value?)

  (spec/def ::a ::a?)
  (spec/def ::b ::b?)
  (spec/def ::c ::c?)

  (spec/def ::acceptable-map? 
    (spec/and 
     (spec/map-of keyword? ::acceptable-value?)
     (spec/keys :req-un [::a ::b ::c])))

  (def my-map {:a 1 :b 2})
  (spec/valid? ::acceptable-map? my-map) => false
  (spec/explain ::acceptable-map? my-map) => {:a 1, :b 2} - failed: (contains? % :c) spec: ...

(require '[clojure.spec.alpha :as s])
(s/def ::c pos-int?)
(s/valid? (s/keys :opt-un [::c]) {}) ;; true
(s/valid? (s/keys :opt-un [::c]) {:c 3}) ;; true
(s/valid? (s/keys :opt-un [::c]) {:c "3"})  ;; false

the secret sauce you were missing was :opt-un

the docstring of s/keys didn't have quite the information i was expecting. > Creates and returns a map validating spec. :req and :opt are both > vectors of namespaced-qualified keywords. The validator will ensure > the :req keys are present. The :opt keys serve as documentation and > may be used by the generator.

thank you

that's the secret sauce. works as expected.

tx for confirmation!

I am sorry if I should ask this on ask.Clojure or not, but what would be the potential disadvantage of using clojure.core.match, except for the potential slower speed?

two things i know of. 1) i believe it uses exception throwing as part of its backtracking. i thought i remembered reading somewhere that creating lots of exceptions can have some cost to it. 2) it's susceptible to creating code that is too large for a single method on jvms. The bad part of this is that it rears its head when you've gone down that path and add a few more cases and can be tricky because there's not much you can do

I have a vector of tuples, like so:

[["foo" 1] ["bar" 1] ["quax" 1] ["king" 2] ["queen" 3] ["pawn..." 6]]

[["foo" 1] ["bar" 1] ["quax" 1]]

(let [smallest (second (first results))]
  (filter #(= (second %) smallest) results))

Oh, this seems better

(first (partition-by second [["foo" 1] ["bar" 1] ["quax" 1] ["king" 2] ["queen" 3] ["pawn..." 6]]))
=> (["foo" 1] ["bar" 1] ["quax" 1])

sort-by, then partition-by ?

In a recent article by Rich about history of clojure, there was two examples of lazy-seq provided, the last one line-seq doesn't wrap the first result "line" into LazySeq. Is that a design choice or just a matter of style? https://gyazo.com/09011955983e53357d340672b590aa01.png

I don't understand the question

(that is the actual implementation of line-seq btw)

interleave-2 wraps first result in LazySeq, line-seq doesn't, the question is why

As in, could interleave-2 also work if its call to lazy-seq were just before the recursive call to itself?

that appears to be one way one could create a modified version of interleave-2 that would be implemented more like line-seq is.

My first impression is that for some reason that I don't yet understand, interleave-2 is "one step lazier" than line-seq.

(note that the self call is not recursive btw - macro expansion is happening here in a loop, no stack frames are harmed)

Not sure what you want to use to name those calls -- what you would normally name a recursive call -- a call to the function currently being defined?

yes why not do (cons (first s1) (cons (first s2) (lazy-seq (interleave-2 (rest....))))

I think it's to make the result more lazy

I think in the line-seq case, you have to readLine so you've effectively already forced the head, so might as well reify

@nxtk I have not tested this, but you could probably get a function that behaves nearly identically to line-seq if you modified it by moving the lazy-seq call to wrap the entire body. That function would be "one step more lazy" than current line-seq, and more like interleave-2. If it works correctly, then it seems like a minor difference perhaps for no deep reason.

"you have to readLine" am i missing something? One could delay reading the first line and wrap read in LazySeq as in the interleave-2 case?

@andy.fingerhut yes, that was my question, kinda. Looks like there is no strict reason why line-seq couldn't be more lazy.

"no deep reason" approximately meaning "if Rich found a performance issue with his use of line-seq where it was doing the first readLine method call every time you called it, and he wanted to optimize for a case where he called line-seq on many files and never realized their first lines, he would probably change line-seq"

I don't know, maybe it would work - the things I'd want to look at are termination, empty file case, and where you discover errors

"no strict reason" 🙂

(like what's the behavior if the file isn't readable)

it's probably preferable to have that exception throw on the call to line-seq rather than on the realization of the seq resulting from it

Note: My comments are ignoring backwards compatibility issues on cases like Alex is mentioning, which is also important now that line-seq has been implemented in the current way for N years.

right, failing fast sounds reasonable

thank you guys

no problem. "Why" is one of the most important and interesting questions to ask, but in many cases also the most difficult to answer accurately.

bad, bad docs, undocumented semantics =)

Sometimes I think the view of the Clojure maintainers is "says no more than it needs to say, and every word is significant. Don't tie one's hands unnecessarily by the documentation in ways that prevents implementation changes in the future." I know that this is at odds with the kinds of documentation that many developers want from a library.

You can either accept that as a decision of the Clojure maintainers that seems unlikely to ever change, or you can go to alternate resources like http://ClojureDocs.org that say more, but are not "official" but community-contributed, or a book like https://www.manning.com/books/clojure-the-essential-reference that is very deep in details of the behavior of the implementation, but again written by someone who is not a Clojure maintainer, but has spent large chunks of time analyzing the Clojure implementation.

right, most of the times reading clojure sources is enough for me, yet sometimes the thing is not obvious

I have this code:

(def rules {"mov" [has-two-arguments? first-argument-is-a-register?]
            ,,, other keys and rules elided.})

(defn validate-instruction [instr args]
  (keep (fn [f]
          (let [error (f args)]
            (when error
              (str "Invalid `" instr "` call, " error)))) (rules instr)))

Have you considered using spec? This type of validation seems like something spec would both support and be a good fit for.

I know nothing about spec, I thought it was a thing used for validating inputs to functions etc. Can it be used as a general purpose thing for doing validation?

I mean, I can just valid any old string ?

yes and yes. Not only that, it can generate data matching your spec

is the instruction a string?

Generally, I would usually separate parsing into its own step where you parse a string into data.

and then the instructions would just be data

yes, so I have a little toy language I've written. With very basic assembly language style instructions. e.g. "mov :a 5" I want to parse this as things like has 2 instructions, first instruction is a register (i.e. it starts with a : ) But I could also have "mov %1 %2" if this mov was declared in a macro (I support macro expansion). So when validating "mov %1 %2", the rules are sorta different, since its in a macro I want to check fi the first argument starts with % as I know this will be expanded out to a register at macro-expansion stage. SO the function needs 2 inputs, the args and a bool for whether its in a macro or not.

hmmm, maybe the solution is just to write a proper EBNF or something

I'll look into spec though, I didn't realise you could use it like that

yea, although if you're ok with a simple grammar at first. parsing can be simple as line-seq and (clojure.string/split line #" ")

learning spec and something like instaparse may be a little too much yak shaving depending on your goals. Another option that might be interesting is just 1. parse using line-seq and clojure.string/split 2. create multimethods for validating and processing instructions that look like ["mov" arg1 arg2] that dispatch on first

You can also create a simple parser with java.util.Scanner .

Think I might try with spec, as the whole point of this project is just to help me learn clojure and I wanted to look into spec at some point

thanks for your advice

I haven't seen spec used for just validating strings. spec will be much more effective if it's validating a string that's already been parsed into data, but there might be a good way to use it on validating strings directly

I can have it parsed into [:mov :a 5] or [:add :a :b], [:call "foo"] etc at this point. That part already works great. WOuld spec be better used on these structures?

and i validate on these?

:thumbsup:

Currently when I find a clojure library I want to try out that doesn’t specify it’s full maven name and version I go to https://mvnrepository.com/ and look up the artifact to find that information so I can add it as a dependency in deps.edn. Is there a more idiomatic way to do this?

I'm using a piece of java-based proprietary software that allows you to extend certain of its classes. This basicially wraps a single function in a bunch of Java ceremony that allows it to fit into the rest of the system.

(ns my.ns.Add1
 (:require ...)
 (:import ...)
 (:gen-class
 :extends their.AbstractClass
 :yadda
 :yadda
 :yadda
)

(defn -doit [x] (pack (inc (unpack x))))

(defmacro def-doer [func arg-description])

(def-doer Add1 inc [long long]) => class my.ns.Add1
(def-doer IsPrime prime? [boolean long]) => class my.ns.IsPrime
;; etc

Any idea why I’m seeing this error? Everything else executes well

Clojars? Most libraries link directly to their clojars page. You can also use the git repo in deps.edn

I can recreate this error with mal-formed edn:

search=> (clojure.edn/read-string "{:a}")
Execution error at metabase.api.search/eval125364 (REPL:2235).
Map literal must contain an even number of forms

So the problem is with the EDN?

if you read the stacktrace by evalling *e you would know for sure. But seems like a good guess. You're reading edn data structures and getting an error that there's a map literal with an odd number of forms. the form you are evaluating has only a single well-formed map in it

How would I read this to know for sure?

the top of that stacktrace is the function which through the error

Util isn't super helpful, but the next line is EdnReader MapReader, which is pretty descriptive. The function that reads edn has a function that reads maps and it threw an error that maps must contain an even number of forms

Is there a way I could find where the map has an uneven number of forms?

and a few lines below that is read_eval_print, which is run in a loop and is your REPL

it's not in your app

its in the file "/Users'roberthaisfield/Dropbox/Roam-ups/..."

So how would I identify the error point in that file?

there aren't distinct syntax classes for keys and values, so the reader can't tell if one is missing, it can only tell that it got to the end of the map and read an uneven number of things

if you know, for example, that all the keys should be keywords and none of the values are, you can create an input stream from the file and read one character (skipping past the initial {) and then read forms from the input stream checking that the first one, and then everyone after that is a keyword

that of course assumes the outermost form is a map and it is that map that has the issue

my guess is you have a '...' somewhere in the map, because you printed at the repl with *print-level* and/or *print-length* bound

user=> (binding [clojure.core/*print-length* 3] (prn {:a 1 :b 2 :c 3 :d 4}))
{:a 1, :b 2, :c 3, ...}
nil
user=>

where likely you didn't even know that *print-length* was set, because your tooling sets it for you

and you just spit that printed invalid/truncated map out to a file

yeah. you can visually inspect the file. you're in something that has structural movements i'd start trying to identify sub parts that were valid edn to narrow down the tree

Okay, I think I might know what the problem was. Opening up the EDN directly in a text editor a while ago reformatted it in some way (VS Code asked to fix something) This time I just redownloaded an EDN export and didn’t open it directly (except through code) and that worked

totally forgot that deps.edn has git support, that probably would suffice for my cases. Thank you!

Thank you for helping. This is helping me understand how to find the issue better

weird. glad its sorted