sci

https://github.com/babashka/SCI - also see #babashka and #nbb
kwrooijen 2020-11-22T20:00:49.097Z

Is there a shorthand for adding an entire namespace to sci opts? I thought that was possible but I don't see it here: https://github.com/borkdude/sci/blob/1dd8b77062f328f5cd87435858cf91f165bcadd4/src/sci/core.cljc#L168-L170

borkdude 2020-11-22T20:02:40.098Z

There isn't and there hasn't been one. You can use a combination of ns-publics and sci/copy-var

kwrooijen 2020-11-22T20:04:58.098600Z

Not sure why I thought there was :thinking_face: But this will work as well, thanks

borkdude 2020-11-22T20:05:51.099100Z

I'm not too keen on automatically doing this since I always feel there has to be some curation to keep GraalVM binary size in check :)

borkdude 2020-11-22T20:06:14.099500Z

That or maybe problems with macros or other stuff. I usually build it up var by var

kwrooijen 2020-11-22T20:29:36.099800Z

It seems like multimethods are shared between the host and the SCI environment. Is this intentional?

borkdude 2020-11-22T20:30:23.100200Z

Not sure? Can you give an example?

kwrooijen 2020-11-22T20:30:34.100600Z

(defmethod ig/init-key :some/key [_ _opts] 123)

  (sci/eval-string
   "(require '[integrant.core :as ig])
    (ig/init-key :some/key {})"
   {:namespaces {'integrant.core {'init-key ig/init-key}}})
;; => 123

kwrooijen 2020-11-22T20:31:08.101200Z

I assume it works for any multimethod that you pass in

borkdude 2020-11-22T20:31:21.101500Z

well, you are sharing the multimethod which gets information from your host environment. this is the same with any other closure

kwrooijen 2020-11-22T20:32:41.102600Z

Interesting. I was expecting the state to be isolated. But if the state is stored in the mutlimethod var, then it makes sense it gets passed along

borkdude 2020-11-22T20:33:58.103100Z

it might be different for defmulti's you define inside sci

kwrooijen 2020-11-22T20:35:26.104100Z

yeah if it's created inside of sci, then it shouldn't leak out to the host

kwrooijen 2020-11-22T20:35:35.104400Z

But defmethods are modified

borkdude 2020-11-22T20:35:54.105100Z

the "global hierarchy" in sci is per ctx

borkdude 2020-11-22T20:36:13.105600Z

but if you are passing in an external multimethod impl then it has access to your host's global hierarchy

borkdude 2020-11-22T20:36:47.106400Z

I see in that code also that it closes over an atom which belongs to that multimethod only

borkdude 2020-11-22T20:37:00.106900Z

so it's effectively just a closure you're passing into sci

kwrooijen 2020-11-22T20:37:01.107Z

That's troublesome for me then. I actually want to use sci to isolate code. So I might have to think of a different strategy

borkdude 2020-11-22T20:38:09.107700Z

A different strategy would be to redefine those multi's in sci code itself

borkdude 2020-11-22T20:40:20.108300Z

or create another defmulti in your host with a local hierarchy: > By default Clojure's multimethods dispatch off of a global hierarchy map.

kwrooijen 2020-11-22T20:41:51.109200Z

I want to use the multimethod of Integrant. I'm not sure if it's possible to override that

kwrooijen 2020-11-22T20:42:08.109700Z

I'd probably have to create my own implementation

borkdude 2020-11-22T20:42:27.110100Z

hmm, maybe the global hierarchy can be re-bound to some dynamic var? I'm not sure

borkdude 2020-11-22T20:43:11.110400Z

no, it's not dynamic. https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L1725 but you could patch that :P

kwrooijen 2020-11-22T20:43:45.110800Z

haha

kwrooijen 2020-11-22T20:43:56.111200Z

Sounds like a weird feature. being able to rebind the global hierarchy

kwrooijen 2020-11-22T20:44:05.111600Z

Might as well define your own. Sadly Integrant doesn't support that

borkdude 2020-11-22T20:44:10.111800Z

it should then really be called the default hierarchy

borkdude 2020-11-22T20:44:33.112100Z

you could propose it to the core team

kwrooijen 2020-11-22T20:44:55.112500Z

I honestly don't know if I like that idea myself

borkdude 2020-11-22T20:45:08.112800Z

with-redefs might work here

borkdude 2020-11-22T20:45:22.113100Z

note that this is not thread safe

kwrooijen 2020-11-22T20:45:22.113200Z

I think I'd rather fork integrant to actually do what I want it to do

borkdude 2020-11-22T20:47:55.113900Z

maybe you could also make just a shim function that calls defmethod but registers the value with a unique prefix for that sci ctx

kwrooijen 2020-11-22T20:52:08.114100Z

The "value" being the keyword?

kwrooijen 2020-11-22T20:52:22.114300Z

Or do you mean the function name?

borkdude 2020-11-22T20:58:06.114500Z

Something like this:

(defmulti foo identity)

(defmacro reg-method [sci-ctx & body]
  `(defmethod foo ~sci-ctx [sci-ctx#]
     ~@body))

(reg-method "sci-ctx-1" (+ 1 2 3))
(reg-method "sci-ctx-2" (* 2 4))

(foo "sci-ctx-1") ;=> 6
(foo "sci-ctx-2") ;=> 8

borkdude 2020-11-22T20:58:28.114900Z

but then the body of the reg-method macro will wrap your integrant calls

borkdude 2020-11-22T20:58:58.115500Z

and you will pass in a function which wraps reg-method to sci so it seems to the sci program as if you're calling integrant normally

borkdude 2020-11-22T21:04:53.115800Z

so integrant only supports one global system, like mount basically?

2020-11-23T08:51:11.119300Z

integrant is not defining “global systems” at all with integrant it is definitely possible to init multiple configs and run systems from them

borkdude 2020-11-23T09:01:31.121Z

It doesn’t take a local hierarchy so all multimethod defs are global, which is @kevin.van.rooijens issue

dominicm 2020-11-23T09:02:46.121200Z

Component definitions are global. System definitions are local.

borkdude 2020-11-23T09:18:26.121400Z

Exactly so when you want an isolated sci environment with defs that are only valid there, you will need a custom hierarchy

borkdude 2020-11-23T09:19:53.121600Z

If this was my problem, I would probably just fork integrant and patch it up. I've done the same with other libs, but mostly for GraalVM purposes

dominicm 2020-11-23T13:58:06.121800Z

I suppose protocols have the same problem

borkdude 2020-11-23T14:01:09.122Z

multimethods support a local hierarchy, protocols don't

borkdude 2020-11-23T14:01:26.122200Z

it's just that as a library author you should also support local hierarchies

borkdude 2020-11-23T14:01:35.122400Z

should/can/expose

dominicm 2020-11-23T14:23:52.122600Z

Do multi methods even support swapping that out?

dominicm 2020-11-23T14:29:45.122800Z

Nope. I guess you could provide your own multi methods...

kwrooijen 2020-11-23T14:30:03.123Z

You can add a custom hierarchy when defining the multimethod with defmulti

☝️ 1
dominicm 2020-11-23T14:48:06.123300Z

Right. But that doesn't really help here because sci would still inherit that hierarchy rather than the one you'd like it to

borkdude 2020-11-23T14:48:59.123500Z

That would help. You could map the sci function to call into a function which uses a local hierarchy for your ctx.

borkdude 2020-11-23T14:50:08.123700Z

Imagine if you could bind the integrant hierarchy to a custom one using dynamic var, then it would become trivial

dominicm 2020-11-23T14:50:21.123900Z

Right, but I don't think multi methods support that.

borkdude 2020-11-23T14:52:53.124100Z

of course they do:

borkdude 2020-11-23T14:53:08.124300Z

(def ^:dynamic *h* (make-hierarchy))

(defmulti f identity :hierarchy #'*h*)

(defmethod f :default [_] "default")

(prn (f 1)) ;;=> default

(prn (binding [*h* (make-hierarchy)]
       (defmethod f :default [_] "local-default")
       (f 1))) ;;=> local default

borkdude 2020-11-23T14:55:19.124600Z

you could have h point to the global hierarchy by default

borkdude 2020-11-23T14:55:37.124800Z

so it's only a small tweak to integrant to support this

dominicm 2020-11-23T15:02:13.125Z

Crazy. I'm not sure they officially support that.

dominicm 2020-11-23T15:02:18.125200Z

Interesting none the less.

borkdude 2020-11-23T15:07:24.125400Z

Yes, they do. It's well documented.

dominicm 2020-11-23T16:10:37.125600Z

Really? I think it's actually a side effect of how the implementation works. It calls the hierarchy (a map) like a function rather than using get . Vars are functions, so that works. You can't call get on a var though.

borkdude 2020-11-23T16:11:24.125800Z

Just look at the docs of defmulti .

borkdude 2020-11-23T16:11:58.126Z

> Multimethods expect the value of the hierarchy option to be supplied as > a reference type e.g. a var (i.e. via the Var-quote dispatch macro #' > or the var special form).

dominicm 2020-11-23T16:13:44.126300Z

OH. I apparently looked at the wrong thing there then. You're right, very clear 😁

borkdude 2020-11-23T16:14:37.126500Z

Full disclosure: I actually just found this yesterday :P

borkdude 2020-11-23T16:15:36.126700Z

But I suspected such a thing should be possible, given the name of global-hierarchy. I mean, why have a global one if you can't have local ones. If only one was supported, then that thing should have been private.

borkdude 2020-11-23T16:16:56.126900Z

hm, it is private

borkdude 2020-11-23T16:16:58.127100Z

oh well

borkdude 2020-11-23T16:17:18.127300Z

so a patch for integrant should be really easy

dominicm 2020-11-23T16:17:54.127500Z

Well, I think it's supposed to be local to the defmulti rather than local to the caller.

dominicm 2020-11-23T16:18:11.127700Z

This does seem like it would impact every multimethod, not just the one in integrant

borkdude 2020-11-23T16:23:29.127900Z

(def ^:dynamic *h* @#'clojure.core/global-hierarchy)

(defmulti f identity :hierarchy #'*h*)

(defmethod f :default [_] "default")

(prn (f 1))

(prn (binding [*h* (make-hierarchy)]
       (defmethod f :default [_] "local-default")
       (f 1)))
Using the above example, you can support local hierarchies in the binding only for your defmulti, so your defs are local

borkdude 2020-11-23T16:23:51.128100Z

but when you don't bind, it works like normal

kwrooijen 2020-11-22T21:05:43.116600Z

Yeah, there was an issue about adding an optional hierarchy, but that was never realized

kwrooijen 2020-11-22T21:05:51.117Z

Also Integrant development has stalled quite a bit

borkdude 2020-11-22T21:06:31.117600Z

I see, so yeah, that would be the way to get out of this, an optional hierarchy or some hierarchy you can bind via a dynamic var, defaulting to the global one.

kwrooijen 2020-11-22T21:08:44.118Z

I'll go think about what would be the best option here. Thanks for the suggestions

kwrooijen 2020-11-22T21:09:09.118300Z

Whenever I play around with SCI I always get fun problems to solve haha

dominicm 2020-11-22T21:43:03.118700Z

The definitions are global, but you can run multiple systems no problem.

borkdude 2020-11-22T21:45:53.118900Z

yeah right, you can run multiple instances of a global system

borkdude 2020-11-22T21:45:57.119100Z

but not multiple different systems?