Clojurians Log v2

How can i uninstall lumo? i tried

npm uninstall -g lumo-cljs

which lumo
/usr/local/bin/lumo

@lepistane installing lumo-cljs to global for me starts an installation script that downloads zip and extracts it to /usr/lib/node_modules/lumo-cljs/bin and uninstalling it should remove that directory. Makes me suspect you manually copied the binary to your path?

that is possible, i installed lumo like 2 weeks ago was very hangover was just trying to get it working. is there anyway i could remove it completely from my system and then install it normally? @hlolli

yes just remove this one binary, it's safe and the only thing that needs removeing. You can also confirm that there's not lumo-cljs directory in your global node_modules directory (on my fedora it's located /usr/lib/node_modules/)

thank you to @hlolli he helped me a lot!!

I finally did it, I believe this to be the first functioning application bundled into the lumo binary that has both .jar and node_module dependencies https://s3.amazonaws.com/hlolli/lumo_quiz_game runs on Linux64Bits, of course you should never ever run a binary that someone posts on the internet, but essentially this binary packs https://github.com/hlolli/lumo-quiz-game takes ~2 seconds to boot so I'm actually not sure if this all hassle was worth it.

its size 113 mb can be explained by 53 mb database of jepordy questions I found online in json format.

@hlolli is it possible you need more caching going on?

every cljs file is cached. But when I think about it, how long time does it take to convert 53mb json into edn 😕 maybe these 2 seconds.

in practice, there's no cljs file being run, it's just js files along the _cache.json counterparts.

Maybe println before and after load can quickly answer that question? Would be good to resolve the doubt.

woah

interesting

maybe this will trigger binary cache for cljs 😄

@dominicm I give it a try

unbundled it takes 4,5 seconds, so it's bit faster bundled I guess. I think this is the bottleneck here. And if you start this terminal game, you see delays between questions, this is explicit with core.async timouts.

(def questions-db
  (let [time-before (.now js/Date)
        _ (println "Loading 53mb bundle...")
        db (-> "resources/questions.json"
               io/resource
               io/slurp
               js/JSON.parse
               (js->clj :keywordize-keys true))
        time-elapsed (- (.now js/Date) time-before)
        _ (println (str "Loading complete " time-elapsed) " ms.")]
    db))

Loading 53mb bundle...
Loading complete 4545  ms.

@hlolli I would suspect js->clj, and if it is the culprit, I would consider storing the DB as Transit instead.

Or, if your app structure is not that complicated and can afford it, just leave the DB as JSON.

@mfikes ok that's actually a good idea. But this app is very meaningless, made it only to make sure that I could bundle code with lumo that uses every type of dependency.

and json to transit sounds like a good idea for a global node module if one doesn't already exists.

@hlolli I just tried in Planck, and the overall numbers I got for your questions.json file: slurp: 244 ms js/JSON.parse: 517 ms js->clj: 1890 ms

ho! wow

yes it really makes sense to store it as transit, thanks for this!

or edn, would that matter when it's not traveling across the wire?

Yeah, or, even just leave it as JSON and interoperate with it as such in your app. Like

cljs.user=> (aget j 10)
#js {:category "EPITAPHS & TRIBUTES",
     :air_date "2004-12-31",
     :question "'\"And away we go\"'",
     :value "$400",
     :answer "Jackie Gleason",
     :round "Jeopardy!",
     :show_number "4680"}

If your app depends on it being a vector of maps, then Transit

@hlolli Edn would be slower. But transit might be able to parse your JSON string, I don't think it requires transit. Worth a shot

can't transit work out of MessagePack as well? That would be way faster

Yeah, @hlolli, @rauh is right, the Transit lib can directly read the JSON string. It produces a vector of maps in 724 ms for me. The resulting maps are keyed with strings.... wonder if there is an option to have it use keywords instead.

Yeah I realized that the keys will be strings afterwards 😞

Hi, need to stub out an env var which my lumo script expects, in order to test it. As a workaround I’m doing this, within the test ns: (def process.env.FOO "bar"). If I run the tests with: lumo -c src:test -m foo.main-test, the tests pass. But when I try to load the foo.main-test ns at the REPL, it bombs out. If I then go in and (def process.env.FOO "bar"), I can run the functions that I need to just fine. I’ve put a sample repo with just this example described above: https://github.com/dotemacs/faker How should I stub out env variables in the test ns, so that when I load it at the REPL, it doesn’t blow up?

Hmm. (mapv clojure.walk/keywordize-keys ...) is no good from a perf perspective. It does the right thing but takes 6 seconds on that data.

@dotemacs maybe this is what you're trying to achieve?

cljs.user=> (set! js/process.env.FOO "bar")
"bar"
cljs.user=> js/process.env.FOO
"bar"
cljs.user=> 

looks so obvious now 🙂 sorry for the noise and thanks @hlolli

@hlolli @mfikes You can use a custom mapbuilder with transit to parse the JSON string: https://gist.github.com/rauhs/301f59e0e2f94db4f22a4724fe50bd5f

Note line 13. If you don't have namespaces can be further sped up by writing (keyword nil s)

@rauh yes from the original it's bit over twice as fast. This looks like voodoo magic to me (in a good way).