Clojurians Log v2

Hi all, I want to serialize/deserialize the working-memory separately so I started off by using a simple implementation for IWorkingMemorySerializer that serializes to an atom (as per Clara's docs) and deserializing from the atom by derefing it. That works perfectly fine, in that I am able to run queries and get the same results from both the original and deserialized session. However, if I add the extra step to write to fressian and read fressian to the ser/deserializer of the IWorkingMemorySerializer , all queries in the deserialized version of the session return with nil: eg. ({:?crn nil}) while the same query on the original is returning ({:?crn {:fact-type :client-representative-notification, :name "Alice", :client "Acme", :level :high}}) . I should note that I am fairly new to clara rules. Has anyone come across something like this before?

@konstantinos562, Do you have a small snippet, it would help me reproduce it locally?

Hi @ethanc do you need the rules as well?

(defrecord WorkingMemoryAtomSerializer [input output]
  d/IWorkingMemorySerializer
  (serialize-facts [_ facts]
    (reset! state (write-fressian facts)))
  (deserialize-facts [this]
    (-> @state io/input-stream read-fressian)))

with (def state (atom nil)) in the namespace

and the read and write fressian as follows

(defn write-fressian [obj]
  (let [baos (ByteArrayOutputStream.)
        wr   (frz/create-writer baos)]
    (frz/write-object wr obj)
    (.toByteArray baos)))

(defn read-fressian [b]
  (-> b frz/create-reader frz/read-object))

sorry for the long posting all

NB: I have replaced fressian with nippy/freeze and nippy/thaw and it works perfectly fine now. I am not sure what the issue was (most probably I am doing something wrong there)

@konstantinos562 not sure what frz ns is referring to

You may just not have setup good read/write handlers for your facts

Fressian’s defaults tend to be insufficient

sorry it's the requirement: [clojure.data.fressian :as frz]

or they will cause many clj data structures to become Java structures - or not the same as before

you can see Clara defines a more robust set of fressian handlers for clj to clj serialization https://github.com/cerner/clara-rules/blob/master/src/main/clojure/clara/rules/durability/fressian.clj#L231

however, there really is no compelling reason to not just use nippy if you have it set up

I wouldn’t have used Fressian to impl Clara’s session serialization if I were to do it again today. it is difficult to work with in what it provides by default and I don’t think it is any better than something like nippy or perhaps a few others. It is also not documented super well and at the “primitive” level makes some unavoidable Java collections that are awkward to convert from (and also wasting perf time)

for this example, I have not set up any handlers but the facts are fairly simple

[{:fact-type :client-representative :name "Alice" :client "Acme"}
 {:fact-type :support-request :client "Acme" :level :high}]

(this is all pluggable and could be implemented with something else on Clara’s side just as well too)

try to Fressian write/read that and see that it returns you the facts in the same structure

leaving Clara session/rules out of the test

that may give some insight

@mikerod simply using the fressian write/read does work correctly in REPL

yes, I think I will use the nippy as it seems to be something in the fressian side that I cannot track at the moment

> simply using the fressian write/read does work correctly in REPL You’ve analyzed waht is returned? is it still a clj vector with 2 clj maps etc?

thank you very much for the prompt responses and help!

@mikerod yes it returns exactly the same facts that are serialized; i.e. a vector of two clj maps

I’d have to experiment some to look close, can’t right now

I have some examples in http://www.metasimple.org/2018/02/19/clj-fressian-ext.html

where things don’t work out-of-the-box with conversions and types

Under “Default handlers”

However, didn’t mention anything with vectors/maps there

I immediately see that types at least aren’t preserved

(let [r (serde-obj [{:fact-type :client-representative :name "Alice" :client "Acme"}
                    {:fact-type :support-request :client "Acme" :level :high}])]

  (type r))
;;= java.util.Arrays$ArrayList

this isn’t immediately meaning Clara wouldn’t accept it - but it is already on shaky ground

oh I see

@mikerod thank you very much on this!

you could attempt to just add the handlers from clara.rules.durability.fressian

only issue there being you may not want to deal with the “identity preserving” parts potentially

you can see examples https://github.com/cerner/clara-rules/blob/master/src/test/clojure/clara/test_fressian.clj#L16

of how you’d do that

that makes sense

has 2 dynamically scoped bindings you add for it’s internal caching

you can see it via pform/thread-local-binding

d/node-id->node-cache doesn’t matter for working memory - so you can leave that off

d/clj-struct-holder is required I’m fairly sure - it is used to preserve object identity during (de)serialization