Clojurians Log v2

I have written a WebAssembly decompiler/compiler and I would like to "spec" the WASM intermediary representation I am using. After a long while of avoiding clojure.spec, I've tried using it for that purpose but it's just too hard, too limited. So I am trying out Malli. Given how flexible it is, I have high hopes. But I am a Malli noob so here I am. My problem is actually a common one: a data schema were parts of the schema are references ("pointers") to other parts of the schema. Validation is easy but generation is hard. Spec makes that extra hard by forcing a global registry. Let's stick to this WASM example to have something concrete to work with. There exists a type section (map of type index to type) and a function section (map of function index to function where a function holds an existing type index). Generating a valid representation in pseudo-code would look like: 1. Generate a type section where type index is a nat? 2. Mutate registry so that type index becomes an enum of generated type index 3. Generate a function section , now assured that generated type index exist Solution A. I guess one way would be to do exactly that. Generating things in the right order, step by step, and keep on doing that while successively building a local registry based on what has been previously generated. I am unsure how practical that would be for a more complex example. Solution B. Another way would be to have one schema with a "local mutable registry" and declaring custom generators along the way which would mutate this "local mutable registry" based on what they generate. However I understand that local registries must be concrete maps and do not allow this. On the longer term, I am not sure this would be more pratical than Solution A and it certainly is not very functional. Any better, idiomatic way?

Note regarding Solution B: Unless I am doing it wrong, I cannot manage to use a mutable registry in a local way (ie. using it as an explicit :registry argument throws :malli.core/invalid-schema {:schema :map})

@adam678 Sounds really interesting! I can’t recall why the local registries only support maps. Would be most likely a small change it to support the Registry

one option woud be to just collect stuff to a mutable registry or an custom atom, use it via (m/schema x {:registry registry}). when everything is collected and if the schemas are serializable, you could writen them into one local (immutable) registry.

There was a similar PR recently https://github.com/metosin/malli/pull/337

I'm wondering what the "limits" of the malli value transformations should be philosophically. If I'm working with an external api that represents dates as strings, but I want to represent them as Instants in my app, is that a valid use for transformers? What about if the external api provides a nested "address" map that I want to eventually turn into an Address record type? It wouldn't necessarily be used as a two way transformation very much, I wouldn't necessarily be sending my addresses back to their api and need to encode them again as strings. Is this just a separate problem, and I should just use a function to get things into my domain records/types?

Coercing stings to proper types at least is fine use. Similar to coercing json or path or query string parameters to booleans, dates etc.

We do those sorts of things all the time, also with Schema and Spec

So it's good to use malli to get from string all the way to our full on domain objects?

sure. If it like looks complex, then move the transformation out. I haven't seen a always valid limit. string->map map->record both perfect cases.

by "move the transformation out" you mean separate it into a different transformer?

wrote a while back this generic nonsense (on spec): > Domain-specific data-macros are cool, but add some complexity due to the inversion of control. For just this reason, we have pulled out Schema-based domain coercions from some of our client projects. Use them wisely.

https://www.metosin.fi/blog/clojure-spec-as-a-runtime-transformation-engine/#data-macros

Thanks!

by moving out I meant into a separate function outside of schemas, e.g. (external->internal data) thing.

And then do you call that from a transformer still? or do you mean the data goes from

json-string-> clojure data -> malli transformers -> external->internal

@ikitommi All right, thanks, I got a sense of where to start. As a beginner I was mostly troubled by the fact that local registries can be only map-based. Since you are not sure this is intended, I took the liberty of opening an issue: https://github.com/metosin/malli/issues/389

• optimal: json-string + malli -> internal • current good practise: json-string -> EDN -> malli transformers -> internal • if the internal->external is complex. uses external data etc: json-string -> EDN -> malli transformers -> external->internal

(did a spike on deriving jackson-decoder from malli schema, but nothing production grade atm, in theory, shoud be much faster)

that makes sense, thanks again!

Hi. If I have a :dispatch multi-schema, is there an easy way to put a catch-all else clause in the matches?

@l0st3d not atm, but could, does [:or [:multi …] :default]work for you?

maybe there could be a :malli/default branch?

I think I've worked out how to write what I was trying to write in a different way (using :or, but I needed an exclusionary condition instead of :default), but it might be a useful feature to add ... I quite like the idea of :malli/default. Partly because of the parallel to multimethods. Maybe there's a clean way of overriding the keyword?

my exclusion clause in the second branch of the or is a bit big, but cos it's all data it's ok to write a function to produce the branches 😉