Clojurians Log v2
asami
Hello 👋 We’re currently evaluating asami as our client-side triple store to complement datomic on the server and we’re excited about the durable storage on IndexedDB. It seems `zuko.util` basically brings in all of ClojureScript and has a significant size penalty https://github.com/threatgrid/zuko/blob/961ccaa484b30f28b625e5598b5288cd436f9b3c/src/zuko/util.cljc#L9-L14. Here’s the top offenders from our shadow-cljs build report in our (already pretty bloated) bundle. Is that intentional and required?
When you say that it’s brought in all of ClojureScript, I take it that this is in Clojure, right?
If so, then no. I hadn’t expected that at all!
won’t :advanced compilation get rid of it anyway?
no, that’s in our ClojureScript bundle with :advanced
hmm
That’s probably because that namespace keeps a map to all the functions in cljs.core
These are made available at runtime for predicates in queries
but it’s not required for ClojureScript (where c-eval isn’t implemented), or is it?
If you do a query with (for instance): [:find ?e :where [:e :type :my-type] [?e :value ?x] [(> ?x 5)]]
Then you’ve called for cljs.core/>
The same goes for any predicate you use. It also applies to bindings like:
[:find ?label :where [:e :type :my-type] [?e :value ?x] [(str "value=" ?x)]]
So it’s a runtime requirement
We actually limit which functions can be called, but that’s done in Asami. It could be done in Zuko to limit the number of things brought into the map
does asami only need those? (def raw-lookup {'= = 'not= not= '< < '> > '<= <= '>= >=}))
I’ll do some experiments
we also suspect known-namespaces might include the same functions several times under different keys (`'cljs.core` , 'clojure.core , "cljs.core", "clojure.core")
Yes, but that won’t bring anything else in
No. Those are just the quick and easy ones.
hmm, I don’t think I understand…
I’m AFK for a bit, sorry. But I’ll come back to this soon
no worries, I’ll play around with this in the meantime
this get is down from 1.85 MB to 550 kb.
(I haven’t tested this yet, hope travis takes care of that 😼)
Wow. I had no idea that the previous code would create duplicate structures!
I’m thinking it might help to take the Asami whitelist and shift it into Zuko. I can’t see any reason why the fn-for would ever be used in a situation that would require anything that isn’t in that list
ah, asami does have a whitelist? I’m away for a bit now but can take a look later or tomorrow
we tried to be generous with it, so there are some things in there that may seem excessive
https://github.com/threatgrid/asami/blob/main/src/asami/sandbox.cljc
We’ve listed the disallowed functions, but they’re not referenced, so I can remove them
great, thanks! I’ll continue on this tomorrow. Another thing I’ll try that should reduce the bundle size is just storing the derefed vars (i.e. functions) instead the whole functions (with docstring etc) in the map. Or do you use that metadata anywhere in asami?
never
I’m actually building the map manually right now 🙂
So it never has to reference the entire contents of the namespace
curious to find out where this leads us with the bundle size
what’s the reason behind disallowing -> etc in the sandbox? Because they’re macros?
yes
I’d like to have and or or in there, but can’t
ok, maybe another interesting use case for https://github.com/borkdude/sci, could also be an opt-in thing
for example we’re already using sci in our app, so it would be free in terms of bundle size
the whole interpreter only adds ~500 kb in our case
sci also has a :allow and :deny setting or you can just set {:namespaces {'clojurecore {'-> nil}}}
@borkdude does that even affect the bundle size then? I mean can you build a slimmer version of sci this way?
@mkvlr Those settings do not affect the bundle size, but you can get a slimmer bundle size if you copy sci/impl/namespaces.cljc to your project and just delete what you don't need