I'm looking for a clojure library that implements a disk-backed LRU cache -- any suggestions? I'm reading many millions of records from a database and need to manipulate/mangle them and I'm running out of memory in the jvm process. (option B is mucking around with jvm memory settings, but if there's a clojure-friendly caching solution available I'm guessing it might be the more scalable and potentially simpler solution.)
https://github.com/ben-manes/caffeine Very well designed library.
What kind of manipulation are we talking about?
Are you able to deterministically paginate your queries from the database? Not sure what kind of mangling you're doing (e.g. if all that data is required to be in memory/cache at once)
Not a cache directly but Chronicle Map might be helpful: https://github.com/OpenHFT/Chronicle-Map
(btw, was looking at org.clojure/cache.core
-- did I overlook an implementation in that lib that reads from/writes to disk?)
looks like it has potential, will look through the readme a bit. thanks!
core.cache
is all in-memory.
rats, thought so. thanks for confirming.
Cloned your vs code / clover github - love it! (had to make the code font smaller though 😀)
Haha... I did that partly for doing screen shares... And partly because I'm old and my eyes are going 👀
So many
I have a question about java interop. Do typecasts happen automatically? Am trying to understand how it is handled under the hood. Say, char-array, which calls (. clojure.lang.Numbers char_array size init-val-or-seq)
and thus maps to static public char[] char_array(int size, Object init){
in Numbers.java. One can pass a Long as the size, but the signature for this method expects an int, which > Integer/MAX_VALUE results in an overflow. Do all interop calls attempt to coerce the inputs?
(This is just a followup on a nasty bug I encountered, where I was using arbitrarily-sized byte-arrays to store temporary data, and did not know that JVM arrays were int-indexed. I am wondering if this behaviour is general with interop. Also where is the best place to put a note or warning about such things. I don't have commit access, but it might be useful for int-indexed to be referenced in the docstring at least. Not sure who to ask about this sort of thing.)
The best place to ask questions like this is at https://ask.clojure.org
in general, most Java things are int-indexed (arrays and collections) and Clojure defaults to longs for integer values
most people don't have more than max int number of things in either of those, so this difference is not obvious
the situation with arity/type matching/casting is complicated, some info at https://clojure.org/reference/java_interop, https://clojure.org/reference/reader#_literals
Thanks. I'll read more there.
What's that Clojure talk where Rich discusses semver and how any API or breaking changes should be a separate file, function, or library?
something with speculation in the title maybe
Spec-ulation
Thanks!