Clojurians Log v2

Requiring the json file is about a 10x speedup over using a macro to slurp an edn file from the resources directory and then parse the edn into a map.

Still unacceptably slow, and I think the slowness is causing Android to think the app is crashing? It only locks up for about a second when I toggle it to/from the background. The first time I send it to the background, the phone freezes for a second and then continues. The second time I send it to the background, I get an android popup saying the app keeps stopping and asks if I want to close the app.

What are you doing on app state change?

Could it be your websocket code hanging?

@ericihli btw, we’re using transit to encode the data. Vastly faster than reading the straight edn data.

I'm trying transit now. I'm trying to avoid adding filesystem reads and instead getting all the data I need from a single JS bundle. So instead of having a file, data.transit with the raw transit, I have a file data.js with the content const data = "<escaped transit>"; export default data;. Then I planned on doing a (def data (js/require "path-to-data-module"))

Sounds reasonable

Now I've hit a totally new problem that might be unrelated. Copy/pasting from beginners: Am I misunderstanding something about module resolution?

(defonce json-data (js/require "../../assets/number-to-word-tree.json"))
(defonce number-to-word-tree (js/require "../../assets/number-to-word-tree-module")) ;; This file has a .js extension. I've tried including the extension. Same result.

Error: Unable to resolve module `../../assets/number-to-word-tree-module` from `app/index.js`:

None of these files exist:
  * ../assets/number-to-word-tree-module(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
  * ../assets/number-to-word-tree-module/index(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)

const foo = <stringified-json>;
export default foo;

@ericihli FWIW this is all metro/react-native. shadow-cljs isn't involved in resolving require calls. as a guess maybe react-native doesn't allow js files in assets? dunno if it has special rules for folders

Thanks. I'll focus my attention there. I'm trying to break things down to chunks that I can verify. There's so many layers throughout this dev process, shadow, cljs, metro, babel, RN, nrepl, etc... and even just my plain-old javascript knowledge is out-of-date. The latest thing I've tried to simplify and verify is just my JS syntax and babel. test.js

const foo = "bar";
export default foo;

➜  react-native git:(master) ✗ head -c 100 ../assets/number-to-word-tree.js
const numberToWordTree = "[\"^ \",\"~i0\",[\"^ \",\"~i0\",[\"^ \",\"~i0\",[\"^ \",\"~:terminals\",[[%
➜  react-native git:(master) ✗ tail -c 100 ../assets/number-to-word-tree.js
W\",\"V\",\"AY\",\"V\",\"IH\",\"P\",\"ER\",\"AH\",\"S\"]]]]]]]]]";
export default numberToWordTree;

maybe try eslint? maybe that provides some kind of error message?

does react-native not have support for loading files lazily? bundling a huge file into the startup process doesn't seem like a good idea?

Ah. Damn.

➜  react-native git:(master) ✗ eslint ../assets/number-to-word-tree.js

/home/eihli/code/ezmonic-mobile/assets/number-to-word-tree.js
  1:1268246  error  Parsing error: Expecting Unicode escape sequence \uXXXX

✖ 1 problem (1 error, 0 warnings)

There are ways to lazily load files but it would be strictly a UX thing in this case, like how Google Sheets display an uninteractable image of a spreadsheet and then lazily loads the interactivity. Makes it feel snappier but it's just as functional as if they showed a loading spinner.

Lazily would be better, for sure. I thought this would be simpler.

This was the case. Moving the js file to the directory that everything compiles to (and addressing the unicode issue) was the fix.

Thank you everyone for the help. 10 seconds to parse 6MB EDN. 1 second to parse 6MB json. ~300ms to parse 8MB transit.

wow that’s interesting

indeed!

Might need to go the lazy read from FS route anyways. Still takes about half a second to launch the app and toggle it to/from the background. Also, it works fine the first time I open the app and send it to the background. But the second time, I get an OS popup that says "Eric's App keeps stopping" and gives me buttons for app info or close app. Logcat around that time looks like this:

01-21 09:27:26.264  4206  4229 D SoLoader: libreactnativeblob.so not found on /data/data/com.ezmonic/lib-main
01-21 09:27:26.264  4206  4229 D SoLoader: libreactnativeblob.so found on /data/app/com.ezmonic-Qr6sh3jyh-i-JC1fXszKWw==/lib/arm64
01-21 09:27:28.441  3218  3376 I clp-JNI : CLP add process info start. (channel) cfbead2 NavigationBar (server) (pkg) AppWindowToken{4dfcc04 token=Token{feed017 ActivityRecord{153dc96 u0 com.ezmonic/.MainActivity t8722}}} (action) 0
01-21 09:27:28.558  3218  3376 I clp-JNI : CLP add process info start. (channel) cfbead2 NavigationBar (server) (pkg) AppWindowToken{4dfcc04 token=Token{feed017 ActivityRecord{153dc96 u0 com.ezmonic/.MainActivity t8722}}} (action) 1
01-21 09:27:28.601  4206  4230 E AndroidRuntime: Process: com.ezmonic, PID: 4206
01-21 09:27:28.626  3218  9647 W ActivityManager: crash : com.ezmonic,0
01-21 09:27:28.627  3218  9647 W ActivityManager:   Force finishing activity com.ezmonic/.MainActivity
01-21 09:27:28.633  3218  3256 I ActivityManager: Showing crash dialog for package com.ezmonic u0
01-21 09:27:28.650  1733  4276 D AppErrorNotification: errorType : 24, process : com.ezmonic , uid : 0
01-21 09:27:28.654  3218  3721 V WindowManager: Relayout Window{6ed731b u0 com.ezmonic/com.ezmonic.MainActivity}: viewVisibility=4 req=1080x2220 {(0,0)(fillxfill) sim={adjust=resize forwardNavigation} ty=BASE_APPLICATION wanim=0x10302fe
01-21 09:27:28.659  3218 12832 I ActivityManager: Activity reported stop, but no longer stopping: ActivityRecord{153dc96 u0 com.ezmonic/.MainActivity t8722 f}

@ericihli Maybe at this point you can try to dump the thread and dump the heap and analyse it?

$ adb shell ps | grep com.ezmonic
$ adb shell run-as com.ezmonic kill -3 <pid you got from the line above>
$ adb pull <file of the thread dump>

$ adb shell am dumpheap <pid from the above> <heap dump file path>
$ adb pull <heap dump file path>

Also, when you open the app, do you let it “settle”, as in wait a bit or do you put it in the background and then try to open it straight away? I’m asking only because your app could just start going into the background and then you’re trying to bring it to foreground to use it, without it actually stopping in the first place. (Obviously, this is just a theory.)

Pretty sure it happens regardless of timing. I'll check on next install. First I've heard of the heap dump debugging stuff. Thanks. Exploring that now.

what offline database solution are ppl using for react native? anyone have success w watermelondb or realm

@alidcastano We used to use Realm but worked very poorly with Chrome devtools so now we’re on SQLite. Not very sexy but very reliable.

@vikeri watermelondb is built on top of sqlite, did you consider it at all?

Nope, I don’t think it was released when we switched. Looks interesting if you need observable data.

yea I'm considering it, just not sure how to use decorators with cljs 😅

My hunch says that if you don’t need the fancy features, go with vanilla SQLite.

yea it just has lots of optimizations, such as fast startup time, lazy data loading, etc. plus built in sync primitives but the benefit of sqlite, aside from one less dependency, is I don't have to eject from expo

Yeah that’s a pretty big one

I’d try SQLite and if it doesn’t cut it then eject and try watermelon

what do you think of datascript+async storage 😬😎

My app uses < 100 kb for storage, and I just persist my re-frame app-db to AsyncStorage as edn.