Clojurians Log v2

I am getting an issue that when some required in javascript file changes, the final product shadow-cljs produces does not change.

I have some fork of some javascript project that I am working on then “compiling” and copying over relevant js files into my clojure project. shadow-cljs is certainly noticing that this happens and starts compiling and hot reload triggers. However the actual change in functionality does not get picked up. If I reload the page there is still no change. But if I go to the shadow-cljs dashboard and press stop then watch then the change appears.

Not sure if this is a general issue or something very specific here. What I am actually changing is a c++ file that gets compiled to js via emscripten. For this project the wasm binary is just base64 encoded and dumped into the js file. So when I am changing something the only change in the js file is that some def var wasmBinaryFile="data:application/octet-stream;base64,ABCD"; is getting altered. Just for context.

@slack1003 hard to say without knowing how you actually include those files. why not load the actual .wasm file over the ajax or so? inlining it via base64 will make it needlessly large?

I agree that it should be compiled to its own file and loaded. I am just on a fork. It is something I have thought of changing. There is a certain simplicity in just having a single js file include though

I am requiring it in via (:require #?(:cljs ["imgui-js" :as imgui]))

so its in node_modules/imgui-js? node_modules files are not hot-reloaded. to invalidate the cache it has you can touch node_modules/imgui-js/package.json which will then trigger a recompile

so I have kept it out of the node_modules right now. in my own build it just copies it into a directory on the class path. then in my shadow-cljs config I do :js-options {:resolve {"imgui-js" {:target :file, :file "src/cljc/imgui/imgui.js"}}}

do NOT use :resolve if you have stuff on the classpath anyways

not sure if that is best practice by any means but seems easiest for development

k

just use (:require ["/imgui/imgui.js" :as imgui]) https://shadow-cljs.github.io/docs/UsersGuide.html#classpath-js

i’ll get that changed then

thanks, i’ll see if that helps.

ahh, I remember there was a particular reason for why i did that! There is a separate js file I include that I have done that style: #?(:cljs ["/imgui/imgui_demo.js" :as imgui-demo]) . And it tries to do require('imgui-js') to get in that same file. Since it looked like I needed that resolve bit to make imgui_demo.js happy i just used that in my cljc file too.

can I leave the :resolve in the shadow cljs settings to keep the other include happy and move to use (:require ["/imgui/imgui.js" :as imgui]) in my cljc file then?

sorry I don't have a clue what you are doing and would need to see some actual code

otherwise there are far too many unknowns for me but it looks like you are trying to hack this together in the wrong way

using :resolve for packages you already control is almost guaranteed to be wrong

maybe… main issue is I don’t have “full” control. Just on a fork to add some functionality. and if I don’t have to change some of the build fundamentals in that project then I would like to leave them be and deal with things on the clojure side.

sorry if my explanation here is lacking. I can get back with a better summary of how things are wired together

wasm in general requires rather specific "glue" code that currently isn't very bundler friendly

It all works fine outside of the hot reload.

yeah, I'm gonna need to see some code to comment on that

Yeah I have done my own wasm tests a while back and I remember having quite a few issues with the glue code then.

k incoming essay: 🙂

So I include https://github.com/flyover/imgui-js as a git submodule.

I have a little build tooling that just runs its make file and copies the relevant compiled js files over somewhere into my class path.

Here I guess I may be going wrong and that I should register it as a node module. Not sure how to do that but I’m sure I can figure it out if thats the way to go.

Anyway, the project compiles down to two important files. imgui.js really is the thing that does everything. imgui_demo.js is several thousand lines of js that is recommended to include when developing since it both serves as documentation and a settings tweaker. Again all not really my code, though I can always change things up if necessary in the fork.

It seems like it is the interplay between those two files that causes some issues with shadow-cljs. The top of imgui_demo.js has one of those standard check all sorts of places to figure out how to require imgui.js.

// imgui_demo.js top
    (function (global, factory) {
        typeof exports === 'object' && typeof module !== 'undefined' ? factory(exports, require('imgui-js')) :
        typeof define === 'function' && define.amd ? define(['exports', 'imgui-js'], factory) :
        (global = typeof globalThis !== 'undefined' ? globalThis : global || self, factory(global.ImGui_Demo = {}, global.ImGui));
    }(this, (function (exports, ImGui) { 'use strict';
    ... );

The important thing being the require('imgui-js') it seems?

Right now I really only include them in my wrapper files.

What works with the requires is when I do the following:

(:require
      #?(:cljs ["imgui-js" :as imgui])
      ;;#?(:cljs ["/imgui/imgui.js" :as imgui]) ;; this does not work!
      #?(:cljs ["/imgui/imgui_demo.js" :as imgui-demo]))

Then everything builds fine. But alterations to the js file do not get picked up and require a stop and watch. But hardly the end of the world.

and in my shadow-cljs config include:

:js-options {:resolve {"imgui-js" {:target :file :file "src/cljc/imgui/imgui.js"}}}

Now I have tried with the following instead:

(:require
      #?(:cljs ["/imgui/imgui.js" :as imgui])
      #?(:cljs ["/imgui/imgui_demo.js" :as imgui-demo]))

Right now this results in imgui.js somehow being included twice! The imgui_demo.js file seems to get its own copy.

I can see this by looking at my shared.js which shadow-cljs is compiling.

In this setup it has the lines:

SHADOW_ENV.evalLoad("module$imgui$imgui.js", true , ......
    SHADOW_ENV.evalLoad("module$src$cljc$imgui$imgui.js", true , ......

so two copies of imgui.js which compiles fine but crashes at runtime.

I guess I could try to keep them in my node_modules instead? Maybe it will give better results. Though it feels like it should be possible to handle them being files in the class path too.

first of all forget about :resolve. as I said that this is NOT relevant here

WHY did you add it in the first place? WHY do you think require('imgui-js') is a problem?

again this is would be much much much much much easier if you just put the code into some repo so I can look at it together

Without the :resolve bit then shadow cljs is not able to build anything: X Compilation failed.

The required JS dependency "imgui-js" is not available, it was required by "imgui/imgui_demo.js".

How I have things required in is the only way that the project is building, and not throwing at runtime, and not double including imgui.js

I can try to reproduce in a simpler repo.

WHERE is imgui-js from? it is not the file you generate from the wasm right? did you modify that file in any way?

as stated before the require('imgui-js') part is in the helper files like imgui_demo.js that are also a part of that project

I asked where the file is from, not where it was included from

src/cljc/imgui/imgui.js WHERE is this from? is this also wasm generated code?

got you

that is just copied from the build output of the project. Copied unmodified like all the rest of the build artifacts I am relying on

ok, so you have two files in a directory?

yup

ok, my first suggestion would be to just modify the generated demo.js to replace require('imgui-js') with require('./imgui.js') instead

since you are generating that file anyways that should be trivial to add

yeah I can try that

the goal of this is to make everything self-contained so it doesn't rely on build config and the code expressing its intent correctly

right now require('imgui-js') is interpreted as "require the imgui-js npm library" but it is not an npm library. avoid resolve to fix this, much better to fix it in the source directly

totally. I guess you have to deal with all sorts of different js files doing some funky things when it comes to dependencies sometimes 😕

the problem really is :resolve to :file. I should probably remove that altogether since it is basically never the correct way to do anything

good to know

hey, so your suggesting on changing the require to ./imgui.js works nicely. I eliminated my use of :js-options :resolve too. just going to alter my tooling to alter the files then I can check if hot reload works

thanks a bunch for your help and time

I was going in with the idea of not altering the build artifacts and fudging things on the clojure side

hot-reload was likely broken because of the :resolve

can you send me the generated demo.js file? I'd be interested to see how that looks in full

but it looks like the way to go is to provide files with the right refs

sure

the two files where imgui_demo.js has been patched

hot-reload is probably going to be a problem. the code doesn't really allow it. but recompile and browser reload should be ok without :resolve

the caching problem should be gone though

yeah as long as the browser reload is all good then that is plenty good enough! I figured hot reload would be a problem given the important stuff happens in the wasm

would be a little easier to accomplish without the wasm inlined into the source

yeah the recompile is now working! thanks again for the help

hey all.. I am having the toughest time converting to shadow-cljs from lein, for my CIDER repl workflow. I run cider-jack-in-cljs, select shadow-cljs then shadow as the repl type,

and everything boots, but then when I try to eval a form, I see

No available JS runtime.
See <https://shadow-cljs.github.io/docs/UsersGuide.html#repl-troubleshootingnil>

my goal is to get a node repl running

so… MAYBE I am supposed to run the generated file and it will get a repl going? except this is my test target, so it probably will not…

I must be missing something obvious here, for getting up a workflow that uses node to evaluate forms without needing a browser. sorry for the fumbling question… I’ve been at CLJS a long time and build stuff can still be mystifying

node-repl is saving me… before it was failing with an error I can’t find, but I may be in business!! apologies for the flailing, I think I’m good 🙂