Clojurians Log v2

@stathissideris: are you familiar with https://github.com/thi-ng/geom/blob/master/geom-svg/src/core.org (for SVG) and https://github.com/thi-ng/geom/blob/master/geom-viz/src/core.org (for plotting to SVG)?

@stathissideris: MathBox is a JS library that abstracts over Three.js (which is itself an abstraction over WebGL). It’s optimized for performant, easy-to-create mathematical visualization, especially 3d animations. Here’s one of my favorite presentations that the author has created using it: http://acko.net/blog/how-to-fold-a-julia-fractal/

@eggsyntax: oh, nice thanks, I'll check it out

And here’s his article about MB2, which isn’t all that readable, but has a few nice examples of what it’s capable of: http://acko.net/blog/mathbox2/

@kephale: I'm aware of http://thi.ng in general (although I haven't used it yet) but dali has a slightly different focus: it's more about SVG, 2D layouts and an emphasis on text and "diagram" elements such as arrowheads etc

@kephale: not sure if there is space for both libs in the ecosystem, but let's see

btw, Karsten's work is super impressive

@stathissideris: mmm i’m not really trying to advertise or anything, but i just wanted to highlight that http://thi.ng can pretty much render all of its stuff to SVGs

so i wanted to bring it up in hopes that everyone will stay compatible

i’d love to be able to use dali for making academic figures in SVG though

since everything has to be hand editable for those

yeah, I found out about http://thi.ng being able to render to SVG after I started :simple_smile:

mmm, im in a similar boat, wrote brevis a few years back, then http://thi.ng came out, now i’m making brevis compatible with it

url for brevis please?

http://brevis.us is the one where the docs are, http://github.com/kephale/brevis for code

its on the simulation side of things

but in the pipeline, generally it goes, run N simulations in brevis, combine to make some sort of figures (which would be nice to do in pure Clojure->SVG via Dali; i was in the process of porting graphing to http://thi.ng), toss into a paper

but just plotting isn’t really enough for figures anyway, so if Dali is going to be better for things like annotating programmatically, then all the better

but the fact that http://thi.ng has an interface for software rendering 3D objects to SVG probably ensures that it will somehow be involved in making my SVGs in the future

oh actually @stathissideris do you have any thoughts on real-time SVG manipulation?

perhaps a silly question/thought, but @eggsyntax and @georgek have their mathbox stuff which i’m excited about for real-time plotting through a web-frontend, and currently in Brevis I’m doing realtime plotting through jfreechart (and then just writing .png’s)

Tots, the http://thin.ng stuff looks rad!

#goteam!

@kephale: We should collect them all and maintain a wiki or something!

😉

(and by we I meant the ‘royal’ we. haha)

thats true… we really should be consolidating onto a wiki

yeah, I’m not actively working on this right now and I’m sure am missing some nifty things in the stream

@kephale good point about trying to remain compatible -- that way we'll get the most synergy from the different tools we are developing. I'll add it to my list of things to look into for dali

@kephale: about realtime: react supports SVG in theory, so possibly Om and Reagent and the whole host of React libraries should be able to do it too, but I'm still a bit unclear how dali would fit into that. I'd really like to crosscompile it to cljs. should be possible in theory but I still need to investigate a bit

mmm the om reagent react stuff leads to @eggsyntax and @georgek ’s mathbox stuff

i’m thinking more about a way to do real-time SVG rendering on the JVM or some such thing

about academic illustrations: my inspiration for dali is partly LaTeX/TikZ, which is suberb for that sort of thing

ah yes, tikz is nice

you mean real time rendering in a swing window?

in whatever kind of window, but yes

right now i have jfreechart plotting time series as simulations run, then rendering a raster to PNG

which i cant really use later on, because its a raster not SVG

could re-render to SVG if need be, but if it could be the same package the whole way through, all the better

Dali uses Batik for some stuff, which has a rasterizer, so you could display the output in a window, and still output svg at the same time

ah interesting, is it survivable for real-time plotting?

don't know, performance could be a bottleneck!

@kephale: Although the cljs-mathbox lib will be completely independent of react. They do fit (fairly) nicely together, but there’s definitely no need to go down the react road to use mb.

ah ok, i’m speaking well out of my comfort zone with CLJS generally, just wanted to highlight that cljs-mathbox is building itself into that niche

i mean, even jfreechart slows down when there is too much data to plot

but it is good enough for ~1,000s of points

@kephale: the Dali "added-value" is mainly about not having to specify where everything goes (in terms of coordinates), so it's more geared towards "visual explanations" instead of plotting etc

@stathissideris: gotcha, so basically it will first be useful for generating final figures? my imagination just started running away with the idea of real-time generation of Tufte-level visualizations, with annotations and points coming up over the course of a simulation

haha that would be amazing, but I hadn't thought about realtime as much

in Tufte we trust

:simple_smile:

mmm fair enough, i’m not sure there is a great justification for real-time in all honesty

worst case, re-render

but before getting back to simulations, i’ll vote for being able to embed http://thi.ng SVGs within a Dali figure

I think the React direction could get us viable real-time: because of the virtual DOM the updates wouldn't be too expensive, and you can feed the data via a websocket

regarding embedding: you should be able to embed any SVG already, so the easy way is to render to SVG from http://thi.ng and import from dali

@stathissideris: hum, another part of Dali’s added value is hiccup style, v. http://thi.ng’s heavy usage of threading macros

i’m unclear about your point about coordinates, in the Dali examples it looks like coordinates are still being provided

sorry, the documentation is a bit out of date (I'm working on a new release now) but have a look at the second example: https://github.com/stathissideris/dali/tree/master#examples

the placement of the bars and of the numbers on top of them is achieved using a stack of stacks

(each number is "stacked" on top of each bar, the group of those two are "stacked" next to each other)

that's why the position parameter of the :rect is simply :_ (the way to say "i don't care" in dali)

i’m actually pretty sold because of the hiccup style, but fwiw http://thi.ng provides helpers that hide the need to specify locations

could you please point me to those, I'm get a bit lost every time I look at http://thi.ng's namespaces

yeah, i sympathize

in https://github.com/thi-ng/geom/blob/master/geom-viz/src/core.org, see function def for svg-plot2d-cartesian

as an example

also, since you seem to have some understanding of http://thi.ng already, how hard would it be to express something like this in it?

from what I see of Dali, i think you will win for the developer experience in making such a complicated flow chart

especially because http://thi.ng is so focused on threading macros, i’m not convinced that it has the same potential for extensibility without getting kludgey

well, extensibility in the sense that the SVG you linked is probably clearly expressible as a tree (or maybe graph) structure

but with http://thi.ng i would expect to see nested threading (possibly distributed through nested function calls)

have a look, here's the source for the image above (please ignore lines before 57, this is going to be integrated in dali itself): https://gist.github.com/stathissideris/5b55c7e805fe3a44f7b1#file-arch-clj-L57

line 107 is where it starts to get interesting

that seems quite pleasant, and it does make your point about not needing to specify coordinates more clear

well, it shows the value even more

http://thi.ng could make that SVG too, but coordinates would be getting specified manually and such

i would think that Dali (not that i’m suggesting it is desirable) could use http://thi.ng as a backend

i could say the same thing about brevis, but i probably wont do it for speed reasons

not in the near term

ok, many thanks for your insight, it's actually helping me a lot, because I was questioning the usefulness of Dali given that http://thi.ng exists. You're helping put things a bit in perspective

heh, yeah, i know that feeling

this enforces the need for a wiki I think, where we can write down what tools exist, how each covers a different area etc

but Dali is closer to what i’d actually want to use, because i know i could read it due to the hiccup style and such

just for our own benefit initially!

I have a feeling I'm going to end up writing my own mesh library at some point.

oh whoa now

ok, ping me if you decide to actually give it a go, there are bugs still, but there is a snapshot on clojars https://clojars.org/dali/versions/0.7.0-SNAPSHOT

+ examples are here: https://github.com/stathissideris/dali/blob/0.7.0/examples/src/dali/examples.clj

@meow: we should sync up about that, because i’m doing clojure meshes in my 9-5 after this current paper goes out the door

cool @stathissideris looking forward to taking it for a test drive

@kephale: yes, definitely

ok i’m hiding from slack for now, clojure datavis is far too exciting

I'm glad I found you guys :simple_smile:

@toxi welcome, we seem to have managed to invoke you with all this talk about http://thi.ng!

hehe.. actually it's complete coincident. just went through my old emails and saw I had an invite from a few months ago

btw. please do go on w/ dali! it's looking ace! :simple_smile:

@stathissideris: btw. i've just added a namespace overview diagram to the geom readme - hope that helps somewhat in your future travels. it's large project, but always looking for ways to improve nav. if you have any other ideas, please share!!! https://github.com/thi-ng/geom/blob/develop/README.md#project-overview

@toxi: nice to see you here :simple_smile:

@meow: thx! about time too. i often feel i must stay away from these channels to actually get stuff done, but it's important to stay in the loop, right? :simple_smile:

@toxi: Yeah. I tend to go in cycles. I stayed away for a while while I did OpenSCAD and then got into http://thi.ng, then got to a point where I came back here to be social again.

@toxi: http://thi.ng is like a labyrinth within a maze inside Pandora's box - could spend a lifetime lost inside it 😉

@meow: that object is a signed distance field of a simple sphere and using modulo to create recursive nesting. just a few (< 20) lines of code in total

on a serious note, I did look very briefly at geom-voxel but should look again because I don't think I grok'd it the first time

as you can tell from my twitter posts I haven't had a problem applying color to a mesh

was gonna work on some Catmull-Clark variations in a bit here

I also want to do some morphogen-style stuff but haven't gotten around to that yet

@meow: well, the SVO ns provides a naive voxel tree which you manipulate (basically a 3D grid of adaptive resolution). Once you've set your voxels (on/off), then use the surface-mesh fn in the isosurface ns to extract a standard 3D polygon mesh of the boundary surface (see Marching Cubes algorithm). Then run a Laplacian Smooth pass to get rid of the box aesthetic. You can do that either in Clojure (before export) or in MeshLab/Blender

right, I was just looking at iso/surface-mesh

basically the fact that it exists and puts me back in the mesh-land that I've grown to know and love and can then output to X3D

there is no end to what one can imagine doing with all these tools

@meow: re:colored meshes - i'm aware that adding color for just export is easy, but if I update the mesh types to support it, I also need to implement functionality for computing interpolated values when subdividing/filtering/resampling meshes and the system should not just support colors, but arbitrary attributes (e.g. UV coords, vertex weights, custom IDs etc). The other related issue with this is how to make OpenGL/WebGL serialization of these attribs as straightforward & easy-to-use as possible. I've written most of that already for a previous version of the lib, but not found the required hammock time to re-apply this to the current version

everything I'm doing now is completely in Clojure without any post processing in MeshLab or Blender

and re: voxels - a bunch of short, working examples are here: https://github.com/thi-ng/geom/blob/master/geom-voxel/src/examples.org#example-3d-simplex-noise

yeah, what I can't easily do with mesh coloring is any kind of easy inheriting of previous face colors when morphing a mesh - I can do it if the mesh isn't changing - I do that now by running the same mesh through coloring functions multiple times, but that isn't changing the number of faces in the mesh

was wondering if you thought of making face its own defrecord

for handling extended attributes, not just color

yes, a long time ago it actually was. can't remember the reason why it's not anymore though. possibly mem use. defrecords have a pretty huge overhead which becomes a real prob with multimillion face meshes

understood

the new attrib system will be per vertex or per face

yeah, I also want to play around with per-vertex coloring

specific question for you while you are here: do you have a function anywhere that will merge adjoining coplanar faces? For example, if I apply a mesh operator that results in some coplanar faces I want to eliminate the shared edges so that I now have a single polygon face.

It shouldn't be too hard for me to write this function, I just don't want to reinvent any wheels if I don't have to.

re: vertex attribs - there're lots of important edge cases to consider. e.g. what happens when collapsing an edge/merging 2 vertices w/ attribs. simply averaging them or allow user to make a per-case decision via callback etc. Also with arbitrary attribs, it's not easy/possible to simply interpolate values for new vertices (during subdivision). E.g. vertex ID attribs might be integers and need to be unique....

co-planar merging is also on the TODO list and started working on this last year, let me see if i can quickly find what i've got so far....

My POV is probably atypical in that I want total control so I want everything to be a callback so I can make use of whatever parameters I want in order to do unusual things... 😉

same here :simple_smile:

we're all control freaks when it comes to code!

you had some vertex-merging code in there, but I didn't see any edge-elimination code, at least not in the gmesh code

and I'm willing to take a hit on performance in order to have that level of control

@kephale is sort of on the other end of the spectrum in that he wants to morph meshes as participants in real-time multi-agent simulations so he needs more predictable behavior and performance is more of an issue (let me know if I got the right or not, @kephale)

whereas I'm working on creating pieces of art that can be 3D printed

it wouldn't be in the gmesh ns, looking at uncommitted stuff in geom-mesh module. I wanted this merge op as post-processing step after a CSG operation, which generates a lot of coplanar faces

@meow: thats pretty much it, these days i’m a stickler for speed (much of the current mesh code i’m working with is in C++ currently, and can’t take much slowdown)

how is the speed on the voxel stuff? i’m doing marching cubes from java at the moment for images ~1000x1000x1000

might want to look at my old http://toxiclibs.org/ project. alas none of the 100+ demos for that work online anymore (thanks to Java being killed off), but there're several physics based mesh deformation demos

@toxi: I didn't even think to look at the CSG stuff, which I should have because that's what I started using at first to wean myself of OpenSCAD, but then I quickly ran into stack overflow errors and became enamored with the ability to just do a bunch of low-level mesh operations

stackoverflows? could you please create an issue w/ example? that's news to me...

I can't find the code I used to have, but I could probably reproduce it.

@toxi: mmm toxiclibs certainly has a bunch of cool examples. for my case, the physics are derived from experiments and such, so the main question at the moment is can i get to 100% clojure via geom, stay in c++, or get partway to Clojure with some java backend (need roughly real-time speed for ~1,000-10,000 vertex meshes with each vertex getting an update)

@kephale: that's a big volume! how fast is your java impl for that? i'd reckon for this resolution I'd look at GPU solutions, for generating actual geometry out of that you'll need a ton of RAM

i stopped testing on the full volume and downsample by 0.5 for the coding, and at that scale it takes ~ a few minutes

these are 2 different cases, real-time meshes being smaller volumes, and mesh generation being for static images

(for the time being)

its the dynamic meshes that are most appealing to have in 100% clojure

I found the code in github that produced the stack overflow.

@kephale: I don't think a pure clojure approach can achieve that (sadly!) - having said that, I've built a desktop app with CLJ, JOGL & JOCL to easily update 3-5m vertices in realtime: https://www.flickr.com/photos/toxi/albums/72157630719227308 - all vertex updates are done via OpenCL. You could prob manage updating the Clojure mesh structure quite quickly, but there's no direct mapping to VBOs so you'll incur a conversion cost every frame, which in my experience is the killer

@meow: link?

https://github.com/pkobrien/cad/blob/e08ae9064fb01e10190cb78dd5236404d9d828a8/src/cad/geom.clj#L88-L109

I had taken one of your examples and was messing around with it. If you uncomment the t3 parts it should reproduce the problem I ran into.

Or it could be I was trying to do something stupid. This was the first code I wrote using http://thi.ng so I really didn't know what I was doing.

@toxi: aieee so pretty. yeah the VBO bottleneck… maybe via shaders…

@meow: thanks! will check this on Monday - have a got workshop to prepare (10am tomorrow!)

@toxi: np. Any luck finding any coplanar face merging code?

@kephale: yeah, by now you could try geometry shaders, which weren't available on OSX in '13. So OCL was only way to do this fast...

@meow: nope 😞 found something related, but the wrong co-planar case: as in 2 poly's sharing same vertices, but opposite normals.. not what we want/look for here..

@meow: again, made a note of it and will check other machine early next week. for now will have to go back to C land and finish my synth for tomorrow

@toxi: Thanks. Glad to finally have a chance to chat with you in real-time. :simple_smile:

@meow: absolutely likewise! better than 140 char limited bursts :simple_smile:

https://49.media.tumblr.com/e62da66d9d029ac71f412f9f35c324f4/tumblr_mzp4uw8vKO1qcga5ro1_500.gif

ominous

hehe.. got same accent, @stathissideris can confirm! :simple_smile:

@meow: i think part of your csg problem might be that you can't safely apply catmull-clark to a basic-mesh. always use gmesh for subdivs

ah, yeah, like I said I had little idea what I was doing when I wrote that

generally speaking, the basic-mesh is like STL, a triangle soup (https://en.wikipedia.org/wiki/Polygon_soup), whereas gmesh provides graph navigation in the mesh (winged edge structure).

I've not run into any bug issues since then

lucky then :simple_smile:

been using gmesh exclusively

ran into a clojure bug you might find interesting

http://dev.clojure.org/jira/browse/CLJ-1860

would also only recommend basic-mesh for speed reasons and when you don't need to do any structural mesh processing

had to work around it for the X3D output

all I'm doing these days in messing with the structure of the mesh 😉

and having a blast doing so

floating point issues are a pain in any language....

Alex Miller jumped on it so I'm confident it will get fixed.

would be nice if it still makes it into 1.8, though doubtful...

oki, sorry, now really have to run... laters!

#C0F0V8DT5 is the best Clojure channel on Slack - everyone hates to leave it :simple_smile:

#C0F0V8DT5 has definitely gotten pretty awesome pretty fast :simple_smile:

@toxi confirmed about the accent haha. I saw your 3d printed installation in Athens last week, it was awesome! (that's where the digital revolution exhibition is right now)

@stathissideris: ohhh! had no idea it's there already... got any pictures?

sadly no! I may go again soon, so I'll make sure to take some. Steve Vranakis did a talk in the venue, and I was pleasantly surprised to see your face on the big screen during the talk (he showed us this: https://www.youtube.com/watch?v=USrM43hZSOQ )