Clojurians Log v2

and (refresh-all) doesn't see it either. okay, I am super confused.

It looks like if I have #?(:clj (:require [clojure.math.numeric-tower :as m])) as the only required thing in the ns, it gets skipped. It's very odd. If I just use (:require [clojure.math.numeric-tower :as m]) without the reader conditional, it works as expected.

@markmandel it seems you are using tools.namespace 0.2.10 but full reader conditionals support landed in tools.namespace 0.2.11

I'll give that a shot @nberger - although it doesn't explain codox failing

But maybe? 😕

Codox uses tools.namespace, so yeah, a bit more than maybe :)

hi, I am a javascript developer and the way I build apps is to separate static files to serve straight through nginx, and proxy dynamic pages and json through to an application. I of course know nodejs best, but professionally work in a large financial institution so have learned to know a little scala (but don’t like it), clojure and a tiny bit of haskell. What I want to do is replicate my node app as close as possible in clojure (which I have started using immutant), but to test want to maximise performance to give it as fair a shot as possible, should I stick with jetty or run wildfly, is there a tool to optimise min and max memory for java etc?

i recommend you skip immutant and go straight to http-kit

Running immutant in Wildfly will be probably somewhat slower than using immutant-web directly in an uberjar-based application, since it has to conform to servlet specification.

http://www.http-kit.org/

@robert-stuttaford: why?

http-kit is less maintained nowadays

ah, i didn’t know that

Not more performant

ok. i defer to jaen’s knowledge :simple_smile:

And only thing it supports that immutant doesn't

Is a http client

That said

https://github.com/ptaoussanis/clojure-web-server-benchmarks here the couple of benchmark results. I think aleph is a good choice

I might be unaware of some of downsides of immutant compared to http-kit

You beat me to that link, heh

Yeah, I agree aleph seems like a good choice as well.

Quite curious how aleph based on undertow instead of netty would compare.

https://github.com/ptaoussanis/clojure-web-server-benchmarks/tree/master/results/60k-keepalive#xfeep20150217---throughput

I guess it’s possible to go with bare netty (which I guess 2 times faster than aleph) and use CLJ logic from it somehow

https://www.techempower.com/benchmarks/#section=data-r11&hw=peak&test=plaintext oh, according to this aleph is 4 times slower comparing to bare netty

Is anyone aware of a Clojure webserver that allow persistent Websocket/SSE/HTTP/HTTP2 connection across code reloads ?

(A la Erlang/Elixir)

That techempower test site is great!

I played with Compojure/Pedestal/Aleph and now Yada. I am not completely happy with any of those so far by the way.

Does anybody used Rapidoid? According to techempower it’s like 50% faster than netty which is really impressive

@artemyarulin: I never heard of it before. Looks impressive !

@nha: Indeed, I’m looking for any docs which describes how they were able to achieve that

I am looking for that too right now :simple_smile: Although I am not sure how representative the techEmpower becnhmarks are (they just make small plain text requests, right ?).

it’s all about tradeoffs, though, right? surely the faster ones are sacrificing some capability for speed

are the sacrifices worth the speed?

My thoughts exactly :simple_smile: I was expecting Compojure to fare worse than that also, as I though it had a thread per request ?

that’s exactly why I ask - there should be some tradeoff but I couldn’t find it

that’s the issue, the benchmarks are concerned only with speed. it’s great that we have them to compare, but it’s not the whole picture

Right. I would give a lot of performance away for convenience and predictability.

unless, of course, you’re in a game where sheer reqs/sec is a top concern for you

The one thing I have been looking for since a long time in a webserver is the ability to reboot without loosing websocket/etc. connections, I have found none so far in JavaLand.

I guess (not sure) it could be resolved with load balanced in the front - so it will maintain all the connections and serve from the different back, while your back instance it rebooting

that’s a tough one. the ‘reloaded’ workflow that we like in Clojure is set up to tear everything down and re-instantiate everything, but that’s for dev where it doesn’t matter. i think you’d need something special to prevent long-lived conn drops for production deploys

yes, as artemyarulin just said. a facade

the Erlang world has this sort of thing totally nailed

pretty much because they started there :simple_smile:

Yes, Erlang/Elixir/Phoenix look attractive because of that.

In fact I just wrote something on that : https://gist.github.com/nha/618fd914f60ccac94657

we serve a websocket app and reboots cull connections. websockets have retry built in, and we have an event-sourced model on the client with a bi-directional, fault-tolerant sync to the servers, so we don’t care if the servers go down or reconnect

So cooperation from the client.

yup

Seems like the easiest thing is either to move the concern to the architecture then. But I'm not completely happy with that.

all depends on the problem you’re trying to solve, i guess

Well I've been working on webrtc, chat stuff professionally, those things were missing a lot (was in Node.js/socket io at the time). I'd like to build a pubsub thing in Clojure now, but the availability part is not solved.

solving that is a conceptual problem, though. no amount of hijinx with code is going to prevent the fact that your processes are being restarted

you’ll need a delegation layer

websockets do have retry built-in, so assuming you can do what’s necessary to resume the websocket session on the server when the reconn comes through, you should be ok

what does the Erlang world do for down servers on the client? surely they also have a cache and sync system?

i am somewhat ignorant on this topic… i’m talking because i want to learn :simple_smile:

Not sure what they do, yes they probably have to do it anyway. But I feel it should go into that situation as little as possible.

Well I was planning to do something a la kafka, the client can request it's messages when reconnecting. Although necessary (network problems, server down), it feeld like it should be the last resort.

thing is, network is a patchy concept overall these days. mobile network switching or airplane mode, rebooting servers, AWS ELB / DNS issues, they all interrupt that connection. building your system to assume the connection will be sound will give you ulcers. therefore, if you build assuming regular disconnects, you can safely cause regular disconnects, too

losing user data sucks, which is why we invested on the client app to cache and sync transparently, retrying until server reports OK for everything

Good point. I plan to open this as a service too though, so even if I control the client I would like to avoid this. I am having a look at nginx-clojure, but it lacks a bit of docs, maybe something more heavy can do the trick too, but I would really like to have something that I can fiddle with on a single node.

I mean, that seems doable. Not exactly easy but doable.

well, good luck :simple_smile:

could you tell, what’s the bot here (and where is the repo) which keeps track of history in this community?

@razum2um: @logbot https://bitbucket.org/ul/slack-archivist http://clojurians-log.mantike.pro

Does anyone has examples how I can make template inheritance work with hiccup on a reagent project ?

@roelof: Just compose functions! https://github.com/brain-fn/essence/blob/master/src/essence/handlers.clj#L23-L74 It's not reagent, but hiccup. wrap-page in this case is similar to a base template, which takes additional content as markup parameter and wraps it in all the stuff it needs (header, etc)

oke, but if you have some 20 pages , are the handlers then not get very large

and how do I put these together ?

depends on the pages you have really. It's not really larger than html based template

just extract as much as you can into the helper functions for header, footer, menu, whatever else you would have and make a wrapper to serve as a base template (or couple of them if you need)

I try to make this work with hiccup : https://laboutique.lemonstand.com/ and I think it will take some 10 - 15 pages

@jetmind

hiccup can collapse lists

[:div (for [a [1 2 3]] [:p a])]

@jetmind any example of that sort of wrapper ? Sorry to ask so many questions. Im a beginner in clojure and in web development with clojure

(defn page-wrap [& content] (hiccup/html5 [:body content])) (defn page1 [] (page-wrap [:div [:h1 “page 1”]]) (defn page2 [] (page-wrap [:div [:h1 “page 2”]])

like that, basically. the trick is to use the & operator to soak up everything passed to page-wrap and let hiccup deal with it as a list

this is just one way to do it. once you get that it’s just Clojure data, all the possibilities open up to you :simple_smile:

@roelof: see wrap-page function in link I posted above, also look at what @robert-stuttaford told (it's the same principle really)

@robert-stuttaford: that would be a very big function if I do it like this with a lot of code in the bod. Or I misunderstood what you mean

try it out, roelof

get a feel for it and see

I see it. In my project only the body have to change so I could make a wrap page where a different body is being called.

exactly

bazinga

sorry that I did not see it first. I think I need to take more time to look more carefully at the code

I will experiment with it and hopefully get it working

100% :simple_smile: best way to get familiar with it

if you start writing macros, you’ll know you’ve gone too far 😄

@jetmind : is sente easier to learn then react ? I like to make a ecommerce site where I would like that the shopping cart is updated without a browser refresh

Those have entirely nothing in common.

@robert-stuttaford: I have not learned to write very easy macros so that would not be a problem

Sente is a library for writing websockets, more or less.

React is a library for dynamic client-side components.

@jaen: I know but as far as I know both can be used to make real-time web applications

you could use sente and react (via reagent or one of the other wrappers) very nicely together

But it solves a different problem.

Sente can talk to your server, but can't display anything

React can display whatever you want, but can't talk to your server

It's a bit like asking if you can use CSS to solve a differential equation, those are totally different things.

oke, so you need both to make a realtime app

Yes

Or neither.

For example

Instead of using sente you can use cljs-ajax or cljs-http and just do AJAX calls

That would be enough for a refreshable cart.

No need to go into websockets unless you need two-way communication

Say, you were writing a chat - yeah, sente is great.

In your case that's an overkill.

oke, I think I have a project which is more then I can chew. Many parts that I still have to learn

The same with reagent - you could use a library wrapping jQuery or goog.dom like jayq or domina and don't have to touch React.

Or choose Holpon or freactive or zelkova or any other library that does not depend on React.

It's just that React is the most popular in Clojurescript (and quite so in the wider Javascript world) and for a good reason.

And things like jQuery or goog.dom are just antiquated and don't scale well to more complex applications.

What I suggest you do

Is first write your project as a multi-page application.

Zero javascript

Just posting forms and redirecting to new URLs and so on.

That'll be pretty easy

oke, just plain hiccup

Like you would do with Rails or Sinatra or something else.

When you have that working

You can try using reagent + cljs-ajax to make parts of website interactive, like the cart.

There's nothing stopping you from using reagent on only the part of website

You don't have to go full SPA - you can just have one div reactified.

Step-by-step is the most sensible solution I think.

oke, then the next few days/ 2 weeks making my site work with all the pages I want.

Making the login work / displaying articles / working with a database (first it can be a map in memory)

Yeah, that sounds like a sensible approach

Don't overextend yourself.

that is a problem I have encountered earlier with 4clojure

Trying to make a solution in one step instead of small steps

Yep, bulding up bigger things from smaller is a thing functional programming is good at

So you should try to leverage that.

I try but old habits die not so fast

@jaen: I could start with a compojure or luminus template ?

I never used templates to be honest, I like to setup the project just how I want it, with libraries I like

But in your case

Yeah, Luminus sounds like a good choice to me.

Think somewhere between Sinatra and Padrino in Ruby world.

You get some structure, some libraries and off you go.

@roelof I’ve worked from Luminus and also from the Immutant demo project and was happy. Luminus is very flexible and well documented.

oke, first step : do some beginners tutorials in luminus

Yes, further down the road you may want to consider @danielsz system based on @stuartsierra component architecture. That also has ready-made support for immutant and a very clean configuration system.

I think that's far too early. It's just gonna confuse him.

It's better he gets comfortable with Luminus as-is.

yeah, just start with Luminus, don’t pay attention to component or system for now.

they are a little hard to understand for a beginner

Grokking how to do components wasn't all that straightforward for me, never having done any non-trivial system with dependency injection.

I had to dedicate an entire day to just component before I understood it.

it can be a set back for beginners, they should just focus on the basic tools to start with, and component is not a basic tool.

True. Luminus has started moving that way as well, though, so it might come up.

correct, I did a quick look and saw it nothing for me yet

@sventechie: You sure? Yogthos doesn't seem to like component.

but why not use the compojure template . I could then learn to add my own authentication as friend instead of boddy

buddy

Well, to me buddy is considerably simpler.

I couldn't make head or tails out of friend, so I'm not sure if it's a good choice for a beginner.

Don't jump from library to library as much. Take something simple and stick with it.

As far as I can see luminus is using friend

no, im wrong it using bouncer

Bouncer is for form validation, nothing to do with authentication.

And Luminus doesn't really force you to use anything, you can swap things as you wish.

I take a break

But for authentication I think buddy is it's first suggestion - http://www.luminusweb.net/docs/security.md#password_hashing

@jaen Re: component, not sure yogthos doesn’t seem to hate it: http://yogthos.net/posts/2015-10-01-Compojure-API.html

And I'm behind that - it's simple.

@jaen: you right. I take a break.

too long behind the screen I think

@sventechie: I'm going by this - https://www.reddit.com/r/Clojure/comments/3stw9b/managing_clojure_app_state_since_reset/

He seems to prefer mount as component seems too OO and complex, and I can get behind that actually.

I myself am eyeing https://github.com/jarohen/bounce right now, mount didn't somehow convince me - I tried it and it started components in the reverse order than I expected for reasons that weren't obvious for me.

Component, while more complex, at least is not surprising in behaviour.

Yoyo by the author of bounce was interesting, but handling monads in Clojure tends to be awkward, so I'm curious what he is planning to do with bounce to alleviate that.

@jaen Hmm, he has some good points. @danielsz systemisa bit more high-level and is being integrated with `mount` as well~.

But if I understand correctly system is just a collection of components, isn't it?

I tend to prefer write the component myself, feel like I have more control then. But system's boot task for reload is pretty cool, though I had to hack it to add regex-based exclusion.

Nice, @jaen ! If you think it can be useful to others, please feel free to push a PR.

@danielsz: but that's only boot-specific, I wouldn't know how to do it for lein.

No need.

Okay, I guess I can push a PR for that in a moment.

dziękuję

Nie ma za co ; d

hi @jaen - yes, monads in Clojure did turn out to be pretty awkward

I'm working on a readme for bounce at the moment, (on the readme branch on GH), but most of the docstrings are in place, and some tests too

Well, that depends, I implemented event sourcing event application as fold over maybes and it was pretty cool and not all that verbose, but when I tried to grok yoyo I was a bit at loss.

Component is cool and all, but it's OO-centric approach feels a bit weird.

I liked that you could just ask the dependency out of the monad without having to pass things around or write methods on records (at least that's how I understood it, maybe I'm wrong here)

mm, agreed - it's a great idea to set out your application state in the Component way, but it was that OO-ish approach that caused me to start looking around for alternative solutions - mostly just variations on a Component theme, though

yep, that's right

although it's viral - every time your callee needs to return a monadic value, you yourself need to as well

which didn't work out so well in practice

Yeah, I can imagine how it's cleaner in Haskell for example, but I'm no walking typeclassopedia at the moment so I'm taking a stop at Clojure. Monads are cool and all, but if you can be sideeffectful like in Clojure that can be an unneeded burden, I guess.

If bounce has something like that, that is being able to set up a system, and ask for dependencies without passing arguments around it could be pretty cool.

Does it resolve dependencies a la component, or will it need explicit ordering like with leaven?

so bounce is completely monad-less - which should get around a lot of the problems of yoyo

but yes, resolves dependencies like component - you give it a map of dependencies and it resolves them

(it uses Stuart Sierra's 'dependency' library under the hood, same as Component)

difference is that I'm looking into ways that you don't have to: - pass the whole system around everywhere - but keep the referential transparency related test benefits

very much in progress at the moment though, so completely subject to change! :simple_smile:

That sounds nice.

Is there an easy “transform a hash-map by mapping over the values”? Right now I’m doing (apply hash-map (mapcat #(list (first %) (dothing (second %))) my-hash-map)) which isn’t super elegant

@jarohen: Any thoughts about co-dependencies?

@jaredly: I tend to use map-vals from @weavejester's 'medley'

@jaredly: there are a number of util libraries that implement that, usually called update-vals or map-vals

https://github.com/weavejester/medley

cool

I usually use clojure.walk, didn't know about those libs.

@jaen: trying to avoid them where possible! cyclic dependencies can usually be refactored away

@jarohen: that is a fair point, but what happens if you can't?

@jaen: wrt mount, the starting and stopping of states is based on the namespace hiearchy

so the components should be started in order they’re required in

and as a side effect if a namespace is not used anywhere then its state will not be part of the lifecycle

the latest version also works with cljs now, so you can mange stuff like websocket connections with it

So if I understand correctly I have to manually make sure it is required like so?

[component.depdency]
[component.dependent]

I tried to make a component that depended on another component and couldn't seem to have them start in the right order for some reason '

well let’s say you have two states a queue and a database, and the queue state has to be started before the database state

@jaen: that's a fair point too :simple_smile: tbh, I've never come across a situation where you couldn't, so it hasn't been enough of an issue - example and/or PR gratefully received though :simple_smile:

so you might have namespaces like app.queue that has its defstate for the queue and app.db with a defstate for the db connection

if app.db requires app.queue, then the queue state will be started before the db one

I’ve switched a few apps I’m using to mount and I found I generally don’t have to think about this explicitly

the start/stop order will be dictated by the namespace hierarchy, and since namespaces that rely on a particular state will have to require namespaces that manage it then it forces the states to be started in that order

Yeah, I figured so, that's why them starting in reverse surprised me, maybe I've just misunderstood something.

Anyway, not having to lug around the system and/or write OO-ish records mount and bounce give seems like a nice thing.

yeah I think the big difference with mount is that it leverages namespace declarations, while other systems introduce an additional layer like classes

I actually like explicit system map for some reason, but I can see the appeal of that. Not having to do records is the biggest win I think.

note that mount can generate the resulting system

for me the biggest win is that I can have completely self contained component namespaces

Actually I think I will replace component by mount too in closp when I have time to do so. I think it has a better readability than component in a clojure namespace.

my main issue with having a global system is that it has to be passed all over the app

@yogthos: well, component namespaces are also self contained to be pedantic, if you use record you can close over it's fields and you don't have to worry about the implementation detail of having to carry the system about.

But then you have weird OO-ish record code.

Which is meh to read.

but the context has to be passed in from the top

Yeah, but you don't have to do that, component does that for you.

If I understand it correctly.

you generally would pass the config from the top level the way duct does for example

The problem is you either have to have methods in a record to close over the fields as to not have to get things from the system, or you have methods outside the record at the cost of having to get the system components you need from the param.

And either is awkward to be honest.

I agree :simple_smile:

one luminus question : what does this do <a href="{{servlet-context}}/about">About</a> expecially the servlet part

it’s for running on app servers like tomcat

if you’re running standalone then the context will always be empty

but if you deploy to an app server then each app has to have a unique context for routing

oke, so I can better make all my links that way when using the luminus template ?

only if you plan on running on an app server

if you’re just going to make an uberjar then you can just omit that

another case could be if you front multiple apps with apache/nginx and you don’t want to setup subdomains

maybe one time. It's not a hobby-project in my mind but maybe later on I could decide to deploy it

now time for dinner

Thanks everyone for the time and patience

@nha, @robert-stuttaford: Erlang's persistence of connections over code reloads doesn't involve a process or server going down. A client reaching a unresponsive system is the same in Erlang as it is in Clojure :simple_smile:

-grin- i suspected as much

It's closer to redefining a Var in a running system

which we can definitely do in Clojure

exactly. Both Clojure and Erlang make no guarantees about how the state may or may not play well with that new code. Erlang just has the convenience layer in it's runtime VM to say "Given a collection of new namespaces, update all the root vars accordingly."

yeah. we have to use the reloaded pattern to accomplish that, which is totally ok

ah, yeah, I see that. That's pretty cool :simple_smile:

@mlbatema: Thanks for the info ! So it means that it is totally ok for requests which are stateless, but what about websockets and such ? As I understand it you have to keep some state on the server. Does that mean that even Erlang can't do it ?

An Erlang hot-code reload can be stateful. You just need be aware of, and implement, potential code-upgrade paths to make your new code work with your old state when incompatible. However, the whole system stays alive during this reload, so there is no downtime.

for details, you can read: http://learnyousomeerlang.com/designing-a-concurrent-application#hot-code-loving

I see, like serializing/deserializing requests? And does the typical Erlang webserver helps with that ?

Ok reading your link :simple_smile:

but Sierra's reloaded pattern looks like it can do enough of that. Just put your runtime state into your defined "system" and figure how you'll want to persist that with the start/stop (serialization, dirty global state, whatever)

@nha: Webserver has nothing to do with it. All VM-level support.

Honestly, hot-code reloads can be complex and can fail. It's a feature you use because it's cheaper to write and test an upgrade path of a live application (or trust a framework to manage as much of it as it can for you) than to accept the downtime necessary to turn off v1 and power on v2.

if you keep your websocket connections in an atom (or whatever -waves hands-) outside of the stuff the reloaded workflow manages, then you can do this

yeah, mlbatema. like we were discussing earlier, we’ve allowed for the webserver to be completely down by teaching the client how to cache and sync. more work, but affords us more flexibility

yeah, that seems the simpler answer. To my knowledge, Erlang doesn't have a solution for the client when the server's completely down. Gotta write/implement all that yourself. Go make friends with CRDTs and read about the CAP theorem 😛

I just tried clj-http and it seems like a failed request in the sense of not being authorized, throws an exception, which is suprising at least. Shouldn't it just return the response from the request and let me handle errors? That's what I would expect. Am I using it wrong?

@nberger: Updating tools.namespace totally fixed that issue! Much appreciated!

Nevermind, I just found about {:throw-exceptions false}

@sveri: just keep in mind it will stop it only from throwing status code exceptions. Things like timeouts will still throw.

@jaen: ok, thank you, that is perfectly fine. Was just surprised, from a clients point of view I think, 401 or 403 are perfectly valid responses :simple_smile:

Yeah, I agree

Nice to hear @markmandel!

Hi everyone, new to Clojure and hoping to make a website for a friends nonprofit using it

welcome!