Clojurians Log v2

@wkf: let me know which solution worked for you, so I can keep a note of it, in case others need help with the start order, thx!

took a break for dinner, but I’ll let you know when I try it :simple_smile:

@tolitius: I got it working for now, just by putting the elasticmq namespace before the sqs namespace

I say “for now” because it feels brittle

fwiw, I really miss having a way to formally express dependencies

it would be nice to be able to pass dependencies like component, maybe something like (mount/using {#’app.sqs/sqs [#’app.elasticmq/elasticmq]})

so you could add constraints to the inferred dependency graph

also, despite my criticism, mount feels better than component when developing at the repl

also, I’d like to help, if you’re open to this kind of feature

@wkf: I agree. the only case where it proves useful so far is in case of "one off" states that need a specific start order. I did not want to make it a requirement, but I agree, this is a very valid use case. it's actually not a bad idea.. to be able to specify some desired order before (mount/start) I actually welcome criticism, since it helps me think harder :simple_smile:

excellent… I have some other thoughts as well, if you’re interested

sure, keep them coming :simple_smile:

I am trying to be very careful not to complicate the API, hence I am still thinking on depends-on feature.. for about a month now

It seems like it would be quite useful to be able to wrap or transform states. for instance, I’ve got app.config/config, and in development, I’d like to add/change a few values

I could add a new state, and swap it out

but really, I just want to modify it slightly

i.e. it might make sense to add :depends-on on the actual state..

would be nice to have something like (mount/hook #’app.config/config (fn [c] (assoc c :whatever 123)))

but, hey, let’s put a pin in that, and talk about :depends-on

hooks: https://github.com/tolitius/mount/issues/16 agree

ha, sorry for not searching first :simple_smile:

all good, it's only an idea at this point

I think a central place to specify dependencies can add clarity

it’s nice to be able to look at the graph, as data

I’ve used component a bunch, and I’ve never really liked component/using

I almost exclusively use component/system-using

and at a glance, I can see how the pieces of my system fit together

I hear you.. thinking :)

there is some visual: https://github.com/tolitius/mount/issues/12#issuecomment-167150505

ah, interesting

what I didn't want to do is specify all the dependencies, since it will have to be kept in sync with a natural order.

yah

I think of it more like you’re enriching the natural dependencies

to handle indirect dependencies (ie, I depend on a rest api provided by another state, not its namespace)

but I agree. explicit dependencies to help out structure the start order are very good to have

right

one difficulty that I see with enriching is validation.. but it might be easier than I think :simple_smile:

i.e. in case a state a is "enriched" with a dependency on state b, how do we know that by starting b a bit earlier to satisfy this dependency we not hurting it (`b`) by now not yet starting states that b naturally depends on

but if we were to have access to the natural dependency graph, we could add new constraints

and derive a total order

I see the problem though

access is there, "graph" is not so much :simple_smile:

we’re left with just the linear order

yep

from clojure

I "feel" there is a solution without breaking the idea of relying on the compiler, but I have not worked hard on validation yet..

(the :depends-on idea)

whether it is on a state level or central..

yah, where to put it is just an api question, and component does both

still have to solve the problem...

if it is on a state level, but a final explicit order is still available via (graph/states-with-deps), I feel specifying :on-depends on the individual state still has value of only needing to (mount/start) without remembering (mount/something-else args) beforehand

yep, agree

so, here’s something interesting...

in my original example

the sqs component has an indirect dependency on elasticmq

in other words, I couldn’t place it on the component directly

err, state

because it doesn’t always have that dependency

("component" / "state".. same thing), yes, I am not dismissing the central meta for :depends-on, I see it being useful

gotcha, gotcha

just thinking out loud here… I’m not sure I would care if, as a first pass, the mount/using api required I (the user) specify all dependencies

maybe some kind of :after / :before can work here.. then elasticmq can be:

(defstate elasticmq :start foo 
                    :stop bar
                    :before #{#'app/sqs})

that’s interesting

usually (from my experience) introducing a public API is rarely a temp solution :simple_smile:

😉

haha

maybe an even looser constraint would work… if you use mount/using, and you specify dependencies for a state, you must specify all its dependencies

ie, once you take it out of the natural order, you need to take full responsibility

huh.. that's not a bad idea

meh, I’m wrong, that still wouldn’t work

actually, yah, it would

just working it through in real time

sorry for the spam 😉

I am not a fan if using since it feels too generic in this context

(as a word)

oh, sure

mount/start-with-deps or something

might be more clear

yea, the name will come naturally :simple_smile:

oh, that reminds me… any thoughts on separating the “configuration” from the “starting”?

seems like it might feel nice to express the ways in which you’re modifying load order/dependencies once

outside of the lifecycle

you mean something like a conf driven by boot / lein / -D ?

nothing so glamorous, something like (mount/with-states {#’blah #’bloop}) (mount/with-deps {#’hello #’sir}) (mount/start)

“persistent” changes to the way mount/start and mount/stop work

seems useful, especially when I’m using more than one start-with-* function

(an alternative would be a start function that takes different kinds of modifications… state swaps, deps, hooks, etc)

ah.. yea, that's definitely on the road map

I might sacrifice the idea of (mount/start-with) taking swaps, and have it to just take a map. this is not exactly "persistent" (i.e. before a start is called), since I like it to stay atomic, but it should achieve the flexibility:

(mount/start-with {:swap {} 
                   :only #{}
                   :except #{}
                   :with-deps {}})

something like that..

is there any specific reason you want them persistent?

if there were separate functions for setting up deps, swaps, hook

I’d like to set them

before starting the states

but if I can do it all at once

that’s probably preferable

another way I was thinking some time ago is:

(-> (with {#'app.nyse/db test-db
           #'app.nyse/publisher test-publisher}) 
    (without #{#'app/nrepl feed-listener})
    (with-args args)
    start)

but start-with taking a map looks less confusing and "easier" to visualize / share / understand

yah

could easily have both, since those functions above could just create the map

though, data > dsl

well, I'll definitely have all those functions to process the map internally anyway :simple_smile:

yea, something like a map can find usages / be provided from many places: * REPL * boot task / lein profile * config * dev.clj * etc..

it's a bit shaky to break (mount/start-with) though since people's tests depend on it.. but I don't think / hope it is not too late

yah, luckily it’s still 0.x

not too late to make changes, especially since it’ll lead to a better tool

you're right, the version would help here

hopefully this is something people read: https://github.com/tolitius/mount/blob/master/CHANGELOG.md :simple_smile:

Hi @wkf and @tolitius. I saw you mention me (and mount-lite) on GitHub.

Mount-lite actually has a 'parallel' branch now: https://github.com/aroemers/mount-lite/tree/parallel

In it are both a automatically deduced state dependency graph, and a way to influence that graph.

Maybe this can be some inspiration to Mount and above discussion?

oh, awesome

seems super useful

(just like I hope to inspire a composable API for Mount, which you also discussed I see 😄)

:simple_smile:

@arnout: the compatibility is definitely a great idea :) I saw the parallel work, it may not work for ClojureScript though

@tolitius: I wrote a parallel dependency loader in javascript

https://github.com/mojotech/entwine

would have to think a bit how to do it clojurescript

“parallel”

the idea of building a graph of off vars don't really project well to cljs since, unless bootstrap is used, have no support for Clojure ns API in :advanced mode

True

simply following @wkf idea of providing the full graph in :with-deps might solve it in use cases that need it

Heh, removing suspend and resume? Cool, I can then remove "I don't need suspending" from mount-lite's README 😉

Hmm, I think Mount is great, because of the implicit dependency "graph" it creates (whether a proper graph or sequential does not matter)

"It just works" - except for when it doesn't, in @wkf's use case

haha

it’s interesting that you say that

yea, I stumble upon these (suspend/resume) often enough to question their need. You saw it first :)

to me, the real value proposition comes form the way mount affects working at the repl

vs component

I never found it particularly tedious to express the dependency graph with data

yes, development was the essential motive

not being able to easily “play” at the repl was the real downer

But do you want to globally store such a data structure, like set-dependencies? Or do you want to specify it each time you do one of the many REPL actions (`start-with`, start-except, etc.)?

@arnout: agree with the value of implicit graph, but am option to override it does seem like a valid feature

I would tend to do something like: https://github.com/tolitius/mount/issues/48

True, hence: https://github.com/aroemers/mount-lite/blob/8e949c1dbcea7076fe3625e1b3da3fee6e2ddd7b/src/mount/graph.clj#L49-L50

(the example there)

typing from a phone.. :)

Sure, that's a very valid way to go

yea, I think your real graph in lite is quite powerful

Boasting myself here, but that graph also makes for quite a nice :up-to start/stop option. It only starts (or stops) the necessary states to start/stop the given "up-to" state.

And that :up-to option is actually also used for the "cascading stop" on a redefinition; meaning never a running state that suddenly lost its dependency. Which can still be overridden of course by the :only option.

./boasting-mode off

:) can you share a use case for :up-to?

Whenever you want to test a certain state for example. Just (start (up-to #'state/under-test)) and your good to go, as all its dependencies will have started as well, without needing any knowledge of what states that would be.

Or if you only want to stop part of your system, but some base states can keep running.

do you see it used instead of start-without / stop-except ?

And, with the automatic "cascading stop", you can safely redefine a state. It will only stop the necessary part of the app, and after a (start) everything is back up.

Yes, because those two require knowledge about the implicit state dependency sequence/graph.

Those functions are enough of course, but up-to is eh.. more convenient in that case?

I see. I tend to minimize the number of states.. so I don't have that many in a single app usually. Since I tend to create states for external resources, and usually there are not a lot of those. But I guess when apps have many, yea, it could be more convenient

thinking more about :with-deps, and the proposed changes to starts-with, and I think it’s more complicated than it needs to be

I’m thinking of a data structure like this:

(mount/start-with
  {:states
   [#'app.config/config
    #'app.server/server]
   :dependencies
   {#'app.server/server [#'app.config/config]}})

which is expanded to something like this...

{:states
  {#’app.config/config #’app.config/config
   #’app.server/server #’app.server/server}
…}

(I left out the dependencies, but they are still there)

“swaps” could be handled in the :states map

but basically… mount/start-with takes a whole “system”

and under the hood

perhaps we could generate that “system” data structure

seems like we could describe :only, :except, :with-deps

as modifications to that data structure

we could also come up with some well defined semantics for how the “natural system” could be merged with the provided system

in other words...

(mount/start-with-merge
  {:states 
    {#’app.config/config #’app.repl/config}})

would merge the states from the “natural system” with the above data

and have the effect of a swap

under the hood, there could be some refactoring to implement the start* functions in terms of this system data structure

with a 0-arity mount/start call using the “natural system”

mostly just thinking out loud

hm.. if I understood you correctly start-with-merge would mutate the running system?

what's the benefit of hiding intentions (i.e. :only, :except, ...) behind a vector / map?

sorry, there was no mutation in that case, let me try to explain more clearly

I’m thinking of a data structure that would contain an exhaustive list of states and dependencies

and then using that data structure to start/stop the states

this is mostly a question of implementation

right now, there’s only an implicit set of states and dependencies

let’s call that the system

a definition of the states in the app, and their dependencies

if we had that data structure, it would be pretty straightforward to implement the stop/start behavior as it exists now

in fact, I think start and stop could be functions of a system-spec

and then :only, :except, :with-deps, etc

would be transformations of the spec

that would yield a valid system spec

but with fewer states, or different deps, etc

looks like you are talking about a https://github.com/tolitius/yurt :simple_smile:

i.e.

dev=> (yurt/blueprint)
{:components
 {"neo.conf/config" {:status :not-started},
  "neo.db/db" {:status :not-started},
  "neo.www/neo-app" {:status :not-started},
  "neo.app/nrepl" {:status :not-started}},
 :blueprint
 {"neo.conf/config" {:order 1},
  "neo.db/db" {:order 2},
  "neo.www/neo-app" {:order 3},
  "neo.app/nrepl" {:order 4}}}

a detached system which internally (by it's type), knows how to be started / started-with / stopped

I’ll have to take a look

ha, yah

looks like it’s pretty close to what I was talking about

right.. it still does not do lifecycle fns composition, since.. well, it is just 4 days old

it was really written to mostly show that mount design does not limit developers by a single(ton) system, and many more than one system can be created, bound locally and used in the same runtime

but it can very well be used beyond that, of course

as to mount, I would like to keep it system transparent, and mostly worry about the meat of the app (that has nothing to do with mount). I see "composable lifecycle" as something that is mostly helpful in testing. I do however agree that "indirect dependencies" should be a core feature that is a lot more important to have

@wkf: In mount-lite, there are these start* and stop* functions you talk about. They take a single spec: {#'statevar {:start ... :stop ...}, ...}. This spec is being created by supplying one or more maps containing :only, :except and :substitute (and more mount-lite specific) keys to start/`stop`. Whenever you supply this to start (or stop) it is munged into the simple spec that start* expects, by starting with a complete spec, and processing the :only, :except etc options on it. This design has suited me well, by allowing me to add more options quite easily, without altering the current API. Is that what you meant?

yah, something similar to that

@tolitius: I’m not as concerned about supporting multiple systems, I’m just thinking that the “system” abstraction internally might be a decent way to handle the indirect dependencies

I cloned down the repo, I’ll take a stab at it with code, you can see what it looks like