Clojurians Log v2

https://clojure.org/guides/repl/enhancing_your_repl_workflow#writing-repl-friendly-programs

(POST "/emails" [] (fn [req] ...) ; emails-handler as anon fn -- do you mean (fn [req] (emails-handler req)) so that there's an indirection in there? I would just do (POST "/emails" [] #'emails-handler) for that indirection.

^ @stopachka

It's from the reitit doc but the idea is similar https://cljdoc.org/d/metosin/reitit/0.5.5/doc/advanced/dev-workflow

Hi everyone, is there a framework or a set of libraries that can resemble next.js functionality in Clojurescript world?

that's probably a bettter question for #clojurescript but I think reagent is the most popular react wrapper, and last I heard luminus came closest to that pre-bundled out of the box experience (these are the things used in the excellent "web development in clojure" book, which is fairly up to date)

luminus doesn't do cljs / reagent / re-frame out of the box, but can be convinced to add those features via flags

to start a repl with cursive or calva, it seems i need a deps.edn. I'm trying to tack on clojure to an existing maven project. is there a way to just have the deps.edn simply refer to the local pom.xml for dependencies, or do i need to copy them in to the deps.edn? More specifically I want other modules from the pom.xml accessible in the repl.

@joel380 I would sort of expect Cursive to know about pom.xml files, given that it's a Java IDE under the hood, but in general for Clojure projects, yes, you'll need a deps.edn that has the same :deps {...} content as the pom.xml file.

At least once you've done that, you can keep deps.edn as the source of truth for the project and re-sync the pom.xml <dependencies> from deps.edn via clojure -Spom.

if cursive can figure out from pom.xml i'd prefer that... i'm having trouble figuring out how to specify a pom module in the deps.edn (specifying 3rd party no issue, but a submodule doesn't seem to work)

If the maven project is a Java project (or other JVM library), I'd suggest creating a separate Clojure project and include the jar from the maven project as a dependency on the Clojure project.

Ah, yeah, you can't use BOM-style deps in Clojure -- you have to specify all the individual components directly.

@joel380 if you can start the repl some whatever way, then you can connect to it with Calva. You'll need to provide the cider-nrepl dependencies to the running app to get things like peek definition and completions going.

(for [x *command-line-args*] (print x)) If I save above two lines to pa.clj and run. $ clj pa.clj Clojure is even more fun , why it does not print anything to console ?

if I put (println **command-line-args**) above for, it prints the list of args.

for is lazy

use doseq instead

thanks @vlaaad.

I have a map in a -> thread, with multiple assoc/updates. I want to assoc/update a key using the value of another key from object being threaded , is there an idiomatic way that keeps it thread-like? Here's an illustration:

(-> {:foo 10}
    (update :foo inc)
    ;; A better way to do this while keeping it inside the threading macro
    ((fn [m] (assoc m :bar (:foo m)))) 
    )
;; => {:foo 11, :bar 11} 

@dromar56 I would maybe use as-> for this:

(as-> {:foo 10} $
  (update $ :foo inc)
  (assoc $ :bar (:foo $)))

but probably I would just use let and not try to force everything in a thread. let over thread :)

I usually use as-> inside -> :

(-> {:foo 10}
    (update :foo inc)
    (as-> $ (assoc $ :bar (:foo $))))

I think that's the "cleanest" way in my use case, thanks

I have yet another question about semantics of Java classes and interfaces. The class clojure.lang.ISeq is an interface, and the class java.lang.Number is an abstract. Does this mean it is possible to have an object with is both a sequence and a Number? If not why? More precisely, according to refl/type-reflect we see that clojure.lang.ISeq has flags #{:interface :public :abstract} and we see that java.lang.Number has flags #{:public :abstract} . I am currently ignoring the fact that these are :public, does that matter?

> Does this mean it is possible to have an object with is both a sequence and a Number yes

user=> (def x (proxy [java.lang.Number clojure.lang.ISeq] []))
#'user/x
user=> (number? x)
true
user=> (seq? x)
true

Possible surprises me, as it is definitely unusual. But apparently there are no JVM rules against it. Interesting.

definitely unusual, not completely unreasonable. but if you ever needed to do something like this, you'd make it a seqable number, not a seq and a number

(i.e. implement clojure.lang.Seqable not clojure.lang.ISeq )

Sure, like a JVM String, making a number seqable would perhaps make sense in order to get a seq of its decimal digits, or something like that.

yeah

or the number as a list of bits

or the number as a range

not that I would ever do it like this

but I've seen it done in the wild (not in clojure tho)

I was thinking that the justification might be that someone could create a number which is a finite sequence of digits or of bits.

That might be a reason someone would want to do it, but I suspect that there are very few reasons that the JVM disallows a class from implementing two JVM interfaces, or disallows extending a JVM abstract class and also implementing any number of JVM interfaces, whether doing so in any particular case makes sense to a person or not.

you can represent average as both number and sequence

e.g. “loss-less” average produced from n numbers, which is a number with a metadata about it’s source

Here is an article I found from a quick Google search, since I don't feel like reading the Java Language Spec to see what it says on which combinations are legal, and which are disallowed by the JVM: https://www.geeksforgeeks.org/two-interfaces-methods-signature-different-return-types/

the JVM disallows a class from implementing two JVM interfaces

@andy.fingerhut the only thing that the JVM disallows is extending multiple classes

you can implement as many interfaces as you want

I put in a thread to my previous message one article showing a case where trying to implement two interfaces with same method name but different return value type seems not to be allowed, so there are at least a few rules restricting some kinds of combinations, if that article is correct.

right but that doesn't have much to do with the class implementing two interfaces

it has to do with the JVM disallowing a class to implement methods with polymorphic return types

agreed that absent such kinds of corner cases, a class can implement an arbitrary number of interfaces

if you're willing to accept numbers as ranges, then it would also make sense to consider matrices and vectors as numbers. However, it seems this is not the intent of java.lang.Number . Rather a number seems to be something which implements the following members. In particular numbers have an integer value and a double value, which range does not but, a bit stream representing the bit representation does. It appears from this definition that a bignum would not qualify as a number.

{:bases #{java.lang.Object java.io.Serializable},
 :flags #{:public :abstract},
 :members
 #{{:name byteValue,
    :return-type byte,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public}}
   {:name java.lang.Number,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public}}
   {:name floatValue,
    :return-type float,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public :abstract}}
   {:name longValue,
    :return-type long,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public :abstract}}
   {:name shortValue,
    :return-type short,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public}}
   {:name serialVersionUID,
    :type long,
    :declaring-class java.lang.Number,
    :flags #{:private :static :final}}
   {:name intValue,
    :return-type int,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public :abstract}}
   {:name doubleValue,
    :return-type double,
    :declaring-class java.lang.Number,
    :parameter-types [],
    :exception-types [],
    :flags #{:public :abstract}}}}

I have some data that's 144mb in EDN. It's taking a significant amount of time to serialize/deserialize (60+ seconds as EDN). I'm trying Nippy now. Nippy appears to have serialized it just fine with nippy/freeze-to-file, but when I try (nippy/thaw-from-file) I get an error: GC overhead limit exceeded. It also churns 100% CPU for over a minute before throwing that error. I thought Nippy was faster to deserialize than EDN, so this is surprising. My assumpsions are that 144mb is not an unreasonable size and Nippy should be faster than EDN and it shouldn't throw this error. Are my assumptions wrong? If my assumptions are right, any thoughts on how to troubleshoot?

For what reason do you say that it appears from this definition that a bignum would not qualify as a number?

w.r.t a class implementing two different interfaces, is it possible to look at the members of two interfaces in search of contradictory type definitions. For example if one interface requires member foo to have type int and another interface requires a member of the same name to have type double, does that mean it is impossible to have a class which lists both interfaces in its ancestors set?

I sent to myself recently above, linking to an article that seems to say "yes" to your question: https://www.geeksforgeeks.org/two-interfaces-methods-signature-different-return-types/

For what reason do you say above: "It appears from this definition that a bignum would not qualify as a number."

@andy.fingerhut I'm just trying to understand, but it seems a bignum cannot have a member intValue of type int , as the largest int is 2^64 - 1, whereas the largest bignum is larger than that.

Those methods are allowed to return useless values in such cases.

or, e.g. the least significant N bits of the bignum value

In most cases I know of where they can lose information, they do so in a fashion that is at least semi-useful.

hmmm. useless values, I laughed when I read that. 😉 I wouldn't have guessed that to be the case

It wasn't a carefully thought out answer -- the later ones are closer to my opinion 🙂

If those methods cannot lose information, then the only type that can implement all of those methods at the same time is an 8-bit integer, because byteValue is in the list of methods.

There are at least a few syntactic rules that the JVM enforces about which classes can be sub-classes of others, and which combinations of interfaces that a class can implement, but except for those, the language itself makes no restrictions. That allows a lot of combinations that a person might never think to do, because they have little or no utility.

Trying to look at method names and determining from the method signatures alone which combinations are allowed and which are not is going on too little information.

I haven't used Nippy, so don't have any information for you specific to that library, unfortunately. Creating an issue on its Github repository, and/or looking through existing issues, might glean some useful information.

You can try increasing your JVM's max heap size with a java command line option such as -Xmx1g to see if that lets it go through for you, but if it is a bug where the library is trying to allocate an unlimited memory, it would only lead to GC overhead limit exceeded later, rather than earlier. Still, it might be a useful bit of information in your investigation.

There is a commercial tool called YourKit Java Profiler that has a 15-day free trial version, and I believe free licenses for use on open source projects, that can attach to a JVM started with a couple of extra command line options it gives you, that can analyze the set of allocated objects in various ways, e.g. by class, sorted by most memory occupied, that might help. There are free tools for doing that, too.

Thanks for the ideas. I think I found it in a closed issue from last year. https://github.com/ptaoussanis/nippy/issues/117

super duper om computer

Yeah my first guess is what andy said -- increase the jvm heap size via the command line option or in your project.clj ... but I don't know .. maybe Nippy is not paging data ... but isn't that supposed to be the job of the JVM?

Hmm. I don't think the heap size is the problem. It might be "a" problem. But it's not "the" problem. The EDN file is ~144 mb and gets processed by read-string in about a minute. Nippy is supposedly faster. Nippy serializes the EDN in about 10 seconds to about 80 mb. That all seems reasonable. It just blows up when desrializing. Maybe something specific about the type of data. I'll open an issue.

It looks like some Java developers run into this restriction when they have actual desires to have a single class implement multiple interfaces with conflicting method signatures, and find workarounds for that: https://stackoverflow.com/questions/2598009/java-method-name-collision-in-interface-implementation

That StackOverflow answer shows that the C# language has a way to disambiguate which method you are implementing, by also supplying the interface name in the definition of the method. Java does not have that.

I tried a few more things trying to make this easily reproducible and I think I've narrowed it down to something specific about the 140mb of edn data that I have. If I generate similar-looking data, I don't experience any issues. https://github.com/ptaoussanis/nippy/issues/136

Actually the question I'm trying to solve is given two java classes (which might be :abstract, :interface, or :final) decide whether the set of all objects of one class intersects with the set of all objects of the other class. For example the set of Serializables intersects the set of Comparables. However the set of Strings is disjoint from the set of Integers.

Currently my rule is that if both neither class is final, and neither is explicitly Object, then they are not disjoint.

But in light of this recent discussion, there are times when two interface classes are necessarily disjoint.

If you ignore Java interfaces, then the answer is true if one class is a sub-class of the other (either directly, or through a chain of sub-class relationship).

If you are trying to answer the question "given to java classes, is there an interface that they both implement?", that is also pretty straightforward to answer, but I doubt that is the question you are trying to answer.

yes if 1 class is a subclass of another they are not disjoint, that's evident

I should mention that in the JVM, an interface will show up in the output of the Clojure reflection API whose output you copied and pasted a sample above as having the keyword :interface there. In Java source code and documentation, there is usually a sharper distinction made between Java classes versus Java interfaces than that.

That is, for something that has the :interface keyword in it, most people familiar with Java would never call that a class. They would call it an interface.

no, the question is given two classes are they disjoint? where classes includes any non-nil value which find-class returns. for which class? returns true.

If you want to know for two interfaces, are there any two JVM classes that both implement that interface, then the only way to check that, that I know of, is to somehow iterate over all class definitions and check whether there are two of them that both implement that interface.

The JVM is dynamic enough in creation of classes and interfaces at run time, that the answer can change over time. e.g. Clojure's compiler creates new classes when you eval a defn form, and some others.

By class I mean any value for which class? returns true

There are libraries I have searched for, and found, that attempt to iterate through all classes currently defined in a running JVM, but there are many caveats in how complete they can be.

class? returns true for things that most people call Java interfaces, as well as things most people call Java classes.

classes always form a tree structure of sub-class relationships, with java.lang.Object at the root.

exactly. I am calling those classes, inkeeping with the class? function

what does java call the set of all classes union with the set of all interfaces?

I don't know if that has a name.

Every JVM object has one and only one class, determined when it is created.

Such a class is never a Java interface.

for my purposes I don't care about the distinction. But to be sure, anytime I talk to a java person they get upset about this.

I am not upset. I am trying to determine if the question you are trying to answer makes sense or not.

It might not be a well-formed question.

😉 yes you're cool headed.

but the people in my office always start talking about the differences in the two, and they've never convinced me that it is something I need to care about.

You said the question you are trying to answer is this: "Actually the question I'm trying to solve is given two java classes (which might be :abstract, :interface, or :final) decide whether the set of all objects of one class intersects with the set of all objects of the other class."

Ok. let me test a definition on you.

Suppose I pick two Java interfaces, I1, and I2, as the Java classes you want to ask and answer that question about.

So the more restricted question becomes: "given two java interfaces I1 and I2, decide whether the set of all objects of one class intersects with the set of all objects of the other class"

What do you mean by "the set of all objects of class I1"? Do you mean the set of all objects, each of which is a particular non-interface class X, does X implement class I1?

now set s1 = the set of all objects, whose class has l1 in its ancestors list union with the singleton set containing the class itself.

now set s2 = the set of all objects, whose class has l2 in its ancestors list union with the singleton set containing the class itself.

is s1 guaranteed to be disjoint from s2 ?

or is it possible that those sets have a non-empty intersection?

So here is how I might restate that question, which seems like it might have the same answer as your question. Given interface I1, are there two Java classes C1 and C2, such that C1 implements I1, and C2 implements I1?

Currently my code concludes that if l1 and l2 are interfaces, then it is not guaranteed that s1 is disjoint from s2.

no, that's not the same question, but it is indeed similar.

OK, my question was not as close to yours as I thought. Let me try again.

can I restate your questionb ack to you?

I am pretty sure now my question is different than yours, but you are welcome to go ahead.

change "are there two java classes" to "are there guaranteed to never be two java classes"

either now or at some point in the future

It is always possible to define new interfaces and new classes at run time in the JVM, barring some kind of JVM security manager restrictions or running out of memory. Definitely not the common case if you are running Clojure.

So it depends upon what you are willing to allow happening in the future.

every time I define a clojure Record I define a new java class, right?

at run time.

If you are allowing the future to define new classes, except under fairly unusual circumstances of two interfaces having very similar incompatible method signatures, it is always possible to define a new class that implements both interfaces.

so my code is currently supposing that it cannot guarantee interfaces are disjoint.

whether it makes sense to a human to productively do so is an entirely separate question

The new information from our discussion today leads me to believe that I can tighten that restriction. If two interfaces declare the same member with two different?/incompatible? types, then we know the interfaces are disjoint regardless of future events.

indeed, i'm not asking what is useful, I'm asking what is possible

my program does not judge the usefulness of your application

So continuing in my habit of distinguishing classes vs. interfaces, you cannot guarantee interfaces are disjoint. Similarly, if you allow the future to define new sub-classes C2 of a given class C1, you cannot guarantee that an interface I1 and a class C1 are disjoint, unless C1 is final (in which case sub-classes of C1 cannot be created).

agree. however, Object IS FINAL

It isn't clear to me whether knowing this disjointness property of two Java classes/interfaces is, but I imagine you have ideas on that.

I'm not sure if that's a one-off or whether there are other final classes we can inherit from.

The class java.lang.Object is final?

correct me if I'm wrong

If it were, then no other classes could be created that are sub-classes of it, and the class hierarchy would consist of only that class java.lang.Object

unless I am completely misremembering the meaning of final as it applies to Java classes...

I think you're right. it is :public but not :final and not :abstract. I was remembering wrong

clojure-rte.rte-core> (:flags (refl/type-reflect Object))
#{:public}
clojure-rte.rte-core> 

no worries. If it is any consolation, I was quite surprised to learn that the Clojure and Java reflection APIs treat interfaces as classes with an extra :interface attribute.

(year ago, when I first encountered that fact)

I may have to revisit my code to see what I'm doing with classes which are not :final, and not :abstract, and not :interface

Is it easy to explain why you want to prove whether two classes are disjoint?

let me try in one paragraph.

Given how common it is for Clojure functions to care more about the interfaces that an argument or return value implements, rather than its Java class (often called the concrete class in discussion you and I have had in the past couple of days with others), as soon as you associate a Clojure parameter or return value as "should implement this interface", it becomes possibly-overlapping with almost every other object that implements another interface.

I am having a project using Clojure but there is Java in it as well. How do I compile the java with clojure tools?

Imagine you have a regular expression, not of characters but of types. e.g., (:* Number) means a sequence of 0 or more numbers.

And actually, my earlier statement about two classes being related as either one is a subclass of the other, or not, if they are not final, then a future sub-class of both of them could implement a common interface, so they cannot be guaranteed disjoint either

(:* (:cat Number String)) is a sequence of alternating Number followed by String. 0 or more of them

So the only things you could ever guarantee disjoint are two final classes that don't implement a common interface I1, or maybe a handful of other cases.

@timok Are you using Leiningen or Clojure CLI? If the latter, the answer is: compile the Java manually, put the classes on your classpath. Or put the Java in its own project and build a lib JAR you can depend on from Clojure.

stopping talking now, and reading instead, sorry .... 🙂

Leiningen has a facility to compile Java code and incorporate it into your Clojure project.

mvn compile then?

I think any two final classes are disjoint. There is never an object (as I understand) which is an instance of two different final classes. correct me if I'm wrong.

(but it shells out to javac so it's not exactly magic)

Correct, it is clojure cli tools

but if those two final classes both implement interface I1, do you consider them disjoint, or not?

I mean clojure cli tools create a pom anyway so I use maven correct?

they are disjoint. The set of objects of class-a is disjoint from the set of objects of class-b, even if the classes share an interface.

if class-a and class-b are both final

and class-a != class-b

note that String = java.lang.String

ok

sure

back to the discussion

(:* (:cat Number (:or String (:not Long))))

represents the set of squences of Number followed by either a String or a Number which is not a Long.

etc etc etc.

These can be combined infinitum.

So Clojure spec has similar regex-of-objects-satisfying-predicates kind of capabilities, so this looks familiar in that sense

they so-called regular-type-expression can be represented as a deterministic finite automaton, whose transitions are disjoint types.

It is only trying to do dynamic checks so far, no kinds of static analysis that I am aware of, although a few people have investigated in that direction.

OK, I see where the disjointness guarantee can come in there

if any state in the finite automaton has two transitions labeled with types which fail to be disjoint, then the automaton fails to be deterministic

If you want to avoid nondeterministic finite automata

if the automaton is determistic then we can answer lots of questions which we cannot answer if it is non-deterministic.

For example, we can ask is there a sequence which is recognized by two different automata

I mean, every non-deterministic finite automata can be transformed into a deterministic one, but since there is a potential exponential size blowup in the number of states, that fact is often not a practically useful one.

also given a sequence, we can ask "does it match the regular-type-expression" and we can perform the computation in LINEAR time WITHOUT backtracking

correction: every finite automaton over a finite alphabet.

the set of types is infinite.

sure. I often forget there are people studying infinite alphabets, as they seem a bit fairy-tale in practical uses, but I suppose in this case their use seems a bit more realistic.

the set of clojure objects is an infinite set. so if you wanted to make a finite automaton to recognize arbitrary sequences you'd need infintite alphabet and consequently infinitely many transitions.

my research handles a special case of infinite alphabets,

I know regular expressions can be transformed into a nondeterminstic finite automata (with epsilon transitions) with a linear number of states and transitions, in the size of the regular expression (at least for finite alphabets), but I thought that sometimes the deterministic finite automata was exponential in the size of the regular expression?

alphabets which can be partitioned into recursively-enumerable sets. (edited) where the membership function is always deciable.

So I have imposed a type-calculus over the Java type system, written in clojure.

a student of mine is writing the same thing in Scala

my PhD thesis was implementing this in Common Lisp, which already was equipped with such a type system.

so I didn't have to implment the type system.

I hope that sheds some light on my wierd questions all the time...

I found a reference to this Wikipedia article which might back up my belief that there are regular expressions whose minimal DFAs have exponential size blowup: https://en.wikipedia.org/wiki/Regular_expression#Expressive_power_and_compactness

It does, and Rich Hickey when giving an early talk on Clojure spec mentioned the future possibility of comparing specs to each other as a function's API changed over time, to determine subset relationships between Clojure specs.

To my knowledge, no one has implemented such a thing yet.

If you are curious for that quote in context, search for "solved problem" in this talk transcript: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureSpec.md

questions of inclusion (subsetness) and disjointness can always be answered with RTE (regular type expressions) provided the questions can be answered on the leaf level types.

do you know Rich?

I knew that there were algorithms for doing this over finite alphabets, but had not considered whether infinite alphabets, or Java interfaces vs. classes, changed how practical such decision algorithms might be.

Not personally, no. We have had the occasional on-line and in-person interaction, but probably only a few hours over 10 years.

i'm sure there are issues/problems/errors in my theory and implementation. But I'd like to get it into a state where I can release it or publish it.

So does the inclusion problem remain solvable if you don't know the disjointness properties of pairs of types?

one important property (at least I think it is important) of my type system is that it is extensible. The hope is to create a type which represents the set of all objects which match a given spec

but I don't yet know how to do that

Or maybe a more practically useful question for software developers: suppose you consider a collection of classes and interfaces that are defined right now, and leave out of consideration possible future classes and interfaces that might be created. That seems potentially more practically useful to answer such inclusion questions for RTEs, if the answer for most pairs of interfaces is "can't be proved disjoint" (when arbitrary future classes are allowed)

Good question indeed. I believe the answer is yes. Why, given two types A and B, if you don't know their subtype relation nor their disjoint relation, then you can partition (union A B) into {(and (not A) B) (and A (not B)) (and A B)}. but any of those three might be empty, we just don't know whether it is.

the cli tools have no opinion or restriction on how you use java

There are a whole bunch of Java classes that aren't final, and imagining that some one some day might create a sub-class that implements some interface somewhere, isn't a practical future possibility.

such a partition causes the automaton to be larger and potentially have transitions which will never be taken. For example, if A is a subset of B, then (and A (not B)) is always false.

you could use an ant project and import a jar made via deps.edn for example

but the fact that deps.end can make a pom (and uses the maven ecosystem regardless) does make using maven convenient

And if people knew that a system answered certain questions for the current set of classes and interfaces, and you gave them an example of how the answer might change if certain classes were defined later, they could either re-run the algorithms later as the software changed with new class/interface definitions, or at least use their judgement about the software to determine how likely such future classes would be.

you make a good point. a practical issue I'd like to avoid is that someone loads my code, then loads his application using it. that user application invisibly builds an automaton which assumes interface-A and interface-B are disjoint (because no class currently inherits from both).

then the user loads a third library which declares a class which inherits from both. That means the automaton is now invalid. and there is nothing to trigger it to be re-computed.

Right, hence my phrase about informing them about the assumptions on which the answers are based, and what could change them in the future.

Many software developers can understand and handle such qualifications, especially with examples.

I am just wondering if your type system, and how common it is for Clojure functions not to rely on particular Java classes, but on Java interfaces, will commonly lead your RTEs to never be deterministic, or only rarely.

its a good point. i'm open about it. but cautious because i'm not sure the gain is worth the risk.

There are a lot of commonly used Clojure functions that take something that implements clojure.lang.ISeq, and returns something implementing that, for example.

but make no promises about which classs

in my system they are always deterministic, but may contain unsatisfyable transitions.

any state in the automaton has some number of transitions leading to that number of next states.

if your system says "clojure.lang.ISeq is an :interface, therefore is not provably disjoint from something implementing another thing that is an :interface, therefore I can't say much useful about it", then I suspect most Clojure functions will lead to that situation.

the transitions are labeled with disjoint types (some of which might be empty types).

I compile the transitions into a data structure sort of a decision network which is guaranteed to never make the same type test twice.

so given an object at runtime I ask whether it is Comparable, and then possibly whether it is serializable, then possibly whether it is a Number etc.

I've got to go do some other work for a bit, earn my pay and all, but been interesting chatting. I would be interested to see what kinds of results you end up with.

and depending on the answers to those questions transfer control to the appropriate next state.

Thanks for the chat. Can you suggest the best way for me to present my code to the public. There are no conferences anymore

especially if it results in code I could run that compares two specs (or a useful subset of them) to see if one is a subset of the other.

If you had something that could be used to answer that kind of question about Clojure specs, I suspect the Clojure conj conference might be interested, although it is more developer-focused and less theoretical. I do not know which of the academic conferences might be interested, but suspect there would be some -- perhaps. you know better than I. I thought several conferences were still having on-line presentations?

Only a subset, I know, but still some.

many thanks for your valuable feedback, especially finding that weird bug with ldiff

i'm going to eat supper now

ok thanks to you both! Have it working.

Is there any sanctioned way to make new reader macros in clojure?

(Ideally without having to fork the language anyway)

@leif do https://clojure.org/reference/reader#tagged_literals work for you?

intentionally, no

for data, you can use tagged literals

https://clojure.org/guides/faq#reader_macros

@teodorlu It looks like tagged literals can hold values, but not expressions (or macro references), yes?

Basically, I'm trying to do this in clojure: https://arxiv.org/abs/2010.12695

It seems like tagged literals (understandably) cannot expand like macros, is this correct?

(Basically I'm trying to do this in clojure: https://arxiv.org/abs/2010.12695 )

that's correct

I mean they could theoretically hold code (which is data) and do something with that but I'm not sure you'd be happy with where you'd end up.

Okay, another option then is a regular (textual) macro, whose invocations can be turned into source positions and let the IDE hide them.

That 'might' be doable with attached metadata....maybe...I'll have to think about that.

Actually, ya, that would work...is there any way to 'tag' arbitrary expressions in clojure?

Like you can with functions.

Maybe something like (func ^meta-info args)?

Its okay if func has to be a macro and can even specify that it requires ^meta-info

Basically I'm just looking for something an IDE can hook into.

You can attach metadata to any var: https://clojure.org/reference/metadata

@teodorlu In this case though, I'm looking to attach the metadata to the variable's use, rather than just its definition.

I see. Not sure how you'd achieve that.

Alright, thanks.

I'll look into this more.

I do admit I am sort of stretching the language here, although I would like to avoid contorting the language too much... 😉

if func is a macro, you don't metadata there - you can just pass whatever and the macro can do whatever it wants with it

Good luck!

@alexmiller good point.

I think I'm starting to see how I might go about doing this.

Thanks. 🙂

hello, can anyone explain to me why regexes as keys get duplicated in a map? i.e:

myapp.core> (assoc (assoc cmds #"cake" "hello") #"cake" "hello")
=> {#"cake" "hello", #"cake" "hello"}
myapp.core> (assoc (assoc cmds "cake" "hello") "cake" "hello")
=> {"cake" "hello"}

should I avoid using regexes as keys if I want them to be unique within the map?

regexes don't compare for equality

one workaround is to use the regex string instead (which has pros and cons)

sooo, I should avoid them for this use?

well they won't work well as keys in a map or values in a set for sure

this is one of very few things in Clojure that don't have equality as value (function instances being another)

mmmh when I try:

(= #'message #'message)
=> true

that isn't a regex

do you mean 2 same functions with same name, arguments and body don't compare?

(= #"message" #"message")

that isn't a function

#' is var quote

it means give me the var with this name

I see

thank you

correct

function equality is undecidable

they are sort of functions, because you can invoke vars as functions, similar to how you can call keywords or collections as functions

but they are not function objects

and to say regexes don't compare for equality is not entirely accurate, you can call = on them, just like you can call = on function objects

but they have identity equality, not value equality

they compare with identity is what I meant

basically you are comparing pointers

I'll try a few things on the REPL to play with regexes equality

but I'll remove them as keys of the map and find a better way

thanks for the info, this is a nice community

there's a very simple logic to it. if they are the exact same object they are equal, else they are not equal

every usage of a regex literal creates a different regex object, and different regex objects will never be equal regardless of the value of the regex

#"..." being a regex literal

it is a kind of interesting connection between regexes and functions, because they can both define something that produces the same result in more than one way, and we tend to think of equivalence for them as based on their results

Is there any good way to get source locations out of read?

(And if not, is there a better way to read a file and get source locations?)

if you use a clojure.lang.LineNumberingPushbackReader with read it will add position metadata to forms it reads (certain forms cannot have metadata attached)

check out (source source) clojure.repl/source

clojure.tools.reader (a clojure reader written in clojure) is a thing that also exists

https://github.com/clojure/tools.reader

This is an image of me starting a REPL session in Emacs/Cider, loading a namespace, letting it sit around for ~30 minutes, then calling (def data (read-string (slurp "resources/some-data.edn"))) some-data.edn is only 140mb. Can anyone explain why my heap jumps by almost 2gb when slurping/reading a 140mb edn file?

Okay. Do any of these also work in clojurescript, or are they exclusive to clojure?

tools.reader works in both clojure and clojurescript

@bronsa cool, thanks

this is the equivalent to LineNumberingPushbackReader http://clojure.github.io/tools.reader/#clojure.tools.reader.reader-types/indexing-push-back-reader

that makes more immediate sense to me than regex equality

Cool. Although these only seem to give line numbers and not also column numbers, is that correct?

as 2 same regexes casted to String should be comparable on their String representation

but I'm probably missing something

it gives both

Ah, okay, thanks.

the data is attached to the object as metadata (if possible)

but you can also ask the reader object manually for line/col/file info before/after reading

using those protocol functions http://clojure.github.io/tools.reader/#clojure.tools.reader.reader-types/IndexingReader

well

regex equality is not the same as a regex's representation equality

AHHH, okay, thanks.

(Sorry, still a bit new to the clojure docs. Thanks again.)

well obviously, it needed 2 gb of objects to read that file

whether it's 2 gb of data is a separate question - you might have a lot of garbage that can be collected (or you might not). hard to say without knowing what the data is

or invoking gc at this point and seeing what the used heap is afterwards

Two regexes could have the same string representation ?

no, if they have the same representation then they are the same regex

but two regex with different representation could be equal

so equality of representation is not equality of regex

Ah. I see. I assumed 140mb of edn would take up ~140mb of memory. I'm gathering now, that's not the case. nil might get encoded as edn to a certain number of bytes but take up a different number in memory. Same thing with datetimes, etc... And in the process of coercing, a bunch of intermediate objects get created as well.

this is really neat! I thought there something familiar and I realized I had also seen your talk, "Movies as Programs". I think there's a lot of interesting approaches that have yet to be tried.

in general regex equality is decidable, but since most regex implementations (like java's) have extended operator support (e.g. backreferencing), the complexity of equality is at least NP-complete, and I believe possibly PSPACE

All objects have overhead of on the order of dozens of bytes

An update for future readers: this appears to be a heap size issue. Confusing, because I never saw my used heap approach max heap, and it would usually churn indefinitely withut ever throwing an OOM exception, and I was only reading into memory from a few files of < 150mb and didn't expect (incorrectly, I now presume) that process to require 4+ gigs of heap. But running my repl with -Xmx6g let me deserialize the file that was previously causing issue.

Clojure strings are JVM strings (in Clojure/Java), and tend to take about 40 bytes per string overhead, plus the storage for the characters in the string, which is often 8-bits per ASCII character if a string is all ASCII, but 16-bit per character if any of them are not ASCII