Clojurians Log v2

Good morning

Mornin'

Why do you think Clojure doesn’t have any “big framework”? (I mean a framework like Spring in Java, Symfony in PHP, Django in Python, Rails in Ruby…)` I have discussed that with a couple of other Clj devs and they think Clojure is different and doesn’t need it. But from my experience, for quick building of near-CRUD apps with some admin interface, SQL db+migrations, etc., having a quick solution might let one build these apps very fast. At this moment, even when Clojure is the highest-level and still practical language I know, I feel I am pushed to start from lower level and with less than my Python, Ruby… peers. Luminus is there, I have tried it (7 years ago) but it is not on par with 10 years old versions of Rail, Django, Symfony…

My experience, having used a few things that are somewhat similar to Rails in other tech, is that those frameworks are great for getting very standardized stuff up and running quickly and they may even get you 80% of the way there -- but that last 20% is painful. Clojure shines where you're not dealing with that "standardized stuff" and those frameworks can't really help you -- or will actually get in the way.

I agree that the initial ramp is steeper with Clojure and it feels like you're doing so much extra work to get even the basics up and running, especially if all you're really building are "CRUD apps".

Indeed, the last 20% are often 80% (my teacher on uni had once an interesting insight - software is 90% of time 90% finished). Starting without large framework helps here. At the same time I see a situation where standard things (like caching, db, forms, logging…) are tested together & using the same naming as something benefical.

Sure, but in Clojure those are "just" a handful of libraries that you can easily throw into the mix.

Forms and validation are probably the things that pain me the most, writing CRUD stuff in Clojure. But most of what I write is API-based where that's not really an issue.

(and even for form validation, I've started leaning heavily on Spec and that works well)

I know there’s Clojure style guide. And naming conventions across libraries. So pulling libraries should technically give somehow consistent experience without having a framework. But just for validation Clojure toolbox lists 17 libs.

Yes, but clj.spec/prismatic.schema is one level low and you have to add a functionality on top of that. For example Validateur http://clojurevalidations.info/articles/getting_started.html is “plug in” solution. At the same time, it’s obviously inspired by Rails and I am not sure how idiomatic the solution is.

Yeah, that's somewhat the "curse of Lisp" at play: we end up with a dozen half-arsed to somewhat decent solutions instead of just a couple of truly great solutions.

I find Spec perfectly satisfactory for form validation, with some coercion (from string to long/double/date/boolean).

That’s true. But even PHP has thousands of competing libraries. Still, they got a couple of awesome frameworks (large apps in Symfony or Laminas (former Zend Framework), midsize in Laravel or Nette, near-CRUD in CakePHP). Maybe, it’s just because the ecosystem is smaller. Ruby was lucky with 37 Signals deciding to move from PHP to Ruby and building Rails.

The rought validation functionality is there. But most validation libraries in big frameworks support automatic error retrieval with support for i18n & l10n and when you have configured your locale & translation files, it’s all automatic and you get form errors in any language with well formatted numbers, dates, currencies… for free.

Previously, I was very active in the ColdFusion community. It actually is much smaller than the Clojure community (although, when I first got started in Clojure, I suspect they were of similar size). The CFML community was very web-focused. They started building frameworks fairly early and at the peak -- around 2004 -- there were several popular competing MVC frameworks, a few competing ORMs, a couple of IoC/DI frameworks, and there were lots of "fringe" frameworks too (some of which were very ambitious -- just not popular). I contributed to all of them, I even wrote my own (ultimately, my FW/1 became one of the top two MVC frameworks in the CFML world, and DI/1 was also very popular -- names inspired by PL/1, which I learned back in the 80's).

I think that being primarily web-focused, there was a concerted effort put into web frameworks. Clojure didn't start out being web-focused although that is now the number one sector for Clojure.

Even so, web development with Clojure tends to focus on APIs rather than SSR HTML apps, with SPA frontends built in JS or cljs.

But that stuff is not "one size fits all". Where I work, i18n is a core issue for us -- we have 40+ dating sites in a dozen or so languages (including RTL) and being able to have a rich environment for managing translations is really important.

None of the frameworks do that really well (Java ResourceBundles are horrible for this, for example).

We have a completely custom system that allows granular read/write access on a per-locale level, built into our Admin Console app, as well as tracking the full history of changes to every translation, as well as providing automated reported on outdated/missing translations. Doing i18n right is hard.

We tried to use standardized i18n stuff but it was useless. Clearly the i18n systems in common use are designed by programmers and the rest of the organization just struggles along so they fit into the programmers' world.

Oh, our system also has a fully-automated "review & promote" system for content, going from staging to production, and we had a built-in system for hovering over translations in the UI of our app and displaying the underlying key (and offering authorized users the ability to edit-in-place).

The whole translation system is managed by our support and business teams -- IT is not involved with it, beyond maintenance of the "content management system" itself.

(sorry, i18n is kind of a bugbear of mine!)

In large frameworks I think it’d be done by having i18n adapter that would be some form of caching call to independent service (fed by that support & business team)

Of course, it alway have issues. When I did development at Tripomatic and at WhichAirline (now part of http://kiwi.com), we had plenty of translations. Norwegian uses a lot of words, Germans have veeery long words, so the design is always broken somehow.

Yup, we do German so I hear that. Doing an RTL language as well (Arabic) is an interesting twist too.

Compare https://arablounge.com/?locale=ar_SA and https://arablounge.com/?locale=de_DE and https://arablounge.com/

(React.js front end, Clojure backend)

I love naming modelled after PL/1. I was involved a lot in frameworks too. I was teaching Zend Framework for years, contributed to Nette framework & compared (and presented on conferences) on lots of PHP & JS frameworks back then. Maybe living around frameworks for a couple of years, got me to the point where I really miss having “one way to think about apps, one naming convention, being sure everything works toghether”. Also, it’s nice compare these mindsets behind frameworks. Zend (Lamina), Spring - everything is a design pattern Nette - usability, lots of small smart tricks CakePHP, early Rails - let all data flow in/out the database just by correct HTTP requests …

Back in 2011, I tried to port FW/1 from CFML to Clojure. Initially, I found it really hard, then I chatted with Chas Emerick about it at Clojure/conj and that got me unblocked. But once I had ported it, I realized it wasn't really idiomatic so I worked on making it more idiomatic... and then I realized that it didn't actually add anything over Ring really... https://github.com/framework-one/fw1-clj (the readme talks about why I decided this was a bad idea).

Good Morning!

Morning 👋

@jiriknesl: I think it’s also worth acknowledging that another reason for this is that these days there are many competing philosophies to web app development; that didn’t really exist back in the day. Classical web frameworks of the past all primarily focussed on CRUD apps with Server Side Rendering of HTML, so the problem they were solving was much more constrained to essentially HTTP + HTML templating + marshaling data in and out of a db with some basic form validation, maybe a bit of auth thrown in, and maybe also a way to decorate some pages with some simple client side behaviours. Now a days certain apps want to start their life with a different philosophy; e.g. if you need rich clientside interactions, you can maybe just look at an SPA framework, and forget about SSR… or maybe you need to build more of an API backend with a separate front end. Not to mention the plethora of front end frameworks etc that you may also want to integrate with, e.g. react, sass and various pre-processors etc. Or maybe you want to back it with AWS lambdas etc… I think it’s fair to say there are far fewer one size fits all web-frameworks these days, and that modern frameworks tend to focus on a particular philosophy for building an app. For example people don’t tend to choose Rails to build an SPA. So the whole ecosystem of web development has become far more fragmented and specialised anyway, which means for any framework to get significant adoption in a small community it probably needs to be a little more agnostic on the style of app you’re building; or consign itself to solving a smaller problem.

Web frameworks require agreement on certain things. You can't build an easy form library, without agreeing on a templating language. You can't build an easy form library without integrating into the http layer and pulling in csrf. You can't save the result of that form without agreeing on a database kind. Nobody in clojure wants to commit to a single one of these. Selmer vs Hiccup. Jdbc vs datomic. Ring vs pedestal. We need to figure out a way to handle these things in a way that composition is easy, rather than making it a framework. That should make it possible to gain agreement.

These all are valid points. And also these points speak exactly about unique, large or nonstandard apps. In these, Clojure really shines. But what about small & medium near-CRUD apps with some admin panel, a few custom made pages and the rest is just a set of helper pages? With Rails, Django, CakePHP, you just sit down, generate lot of functionality, plug in admin panel, import auth, i18n, l10n, caching, ORM, forms… and implement those last 2-3 pages with unique functionality. I did lots of such an apps before. Server rack configuration, small travel agency reservation system, plumber booking app, vendor & project management system for telco company. I have implemented some of them in Clojure (both Luminus and from a devstack built on top of Ring) and others in existing MVC framework in other languages. And I must say that having that fast kickstart was convenient. One spends half day adding migrations. A few hours with logging. A few hours with adding forms. …. In large projects it doesn’t matter. In these small/medium size apps you can save your customer a lot of money.

if i was starting such a thing in clj now

i would target cljs and make it a serverless wrapper tbh

a fullish framework in that area would be a killer app for clj for a bunch of reasons

currently running a typescript serverless app with ~40 endpoints, costs like 20 bucks a month to run, a couple hundred bucks for AWS Aurora, it's crazy

tho offtopic

yesterday I went to a purescript webinar and they had a session on server side purescript

a really nice DSL for doing types on your endpoints etc - not hugely unlike compojure actually in the way it looked

but

cos of how purescript is compiled it doesn't check inputs at runtime anyway

and I asked in the Q&A whether you could auto-gen documentation (e.g. swagger) from this and they were like 'oh no, you'd have to inspect the output purescript to do that'

i was like huh

they were like 'but it's better than swagger, types, static checking (etc)' to which i'm like yes, it stops ME writing bugs, but does nothing about validating inputs (like schema in clj) or generating swagger - which i want purely to hand to the front end devs as auto-gen'd docs cos I'm lazy

seems wild to me that something so powerful could be so wholly inadequate compared to, well, just data really - back to why the pragmatism of clojure tends to win in my mind

@jiriknesl I think that we can have our cake and eat it, if someone does a lot of design work to make it happen. It's easy to copy an existing tool in order to do this (well, it's low design and high effort). The real trick is making something that fits with a clojure ecosystem but is still convenient close to rails or django.

My take to do “Clojure framework” would be: Picking existing libs from Clojure toolbox and trying to: 1. make API using the same conventions 2. test they work together 3. make sure documentation exists, is accurate and also documents boundaries between these libs

It seems to me the most boring, simple thing that might work. (If the goal was closing the gap in near-CRUD small/medium apps built in MVC frameworks)

Personally I think duct is pretty close to being the right answer here, at least on one dimension of tying components together. In practice it still has some way to go; but modularising components with integrant and pulling them together works really well. Things I think duct needs are: 1. better routing (ataraxy isn’t good enough) 2. a better API story (really means de-emphasisng reliance on ring middlewares) These are things for which there are other good solutions in the clojure community… e.g. reitit/compojure-api/yada (or it’s successor). Other things would be: 3. better templates that use tools.deps 4. a template or overlay that has a better solution for including clojurescript/js workflows 5. some minor tweaks to how profiles work (though to be fair this is just a small bug-bear of mine) 6. and I guess better docs too Marketing wise I think duct should just pitch itself as a kind of module system; and there should be a separate webframework built on top of it.

Some people might also want something more like pedestal rather than ring though… not sure.

duct felt like a lot of mental overhead compared to luminus or re-frame though (maybe I'm not smart enough for it haha)

I mean, I assume that's 'cos it scales well

like @alex.lynham i'd love to see a framework that dropped nicely into serverless

there's some p good tooling in the area but I think a batteries-included thing with shadow, monads/manifold to abstract the async stuff would be awesome

cljs-lambda is p nice but it's harder to test than talking straight to the aws-sdk cos you can test as plain side-effecting functions if you don't use the macros

tbh if I had a project that i could get away with using whatever tech i wanted for a client, I'd make that

Serverless is definitely super interesting but it’s not clear to me that a framework (unless it’s more fragmented and compositional) can easily target both serverless and server usecases. Maybe it’d be pretty easy to abstract though?! Never thought about it. Hence the split in philosophies I’m on about.

oh, i'm not that bothered about targeting servers

sorry i should have been clearer

i think streaming/apache kafka type stuff is basically the only thing i wouldn't use serverless now for on a greenfield project

OH or running blockchain

yeah but other people / usecases are… and that’s why I’m saying there’s fragmentation in the field of frameworks.

ya i get you

i just mean that even in TS etc there is still a bit of a gap for a 'wow this rocks' solution in that space, and JS devs seem increasingly open to transpiled langs, so it's more about people adopting clj(s) is my thing to solve, not the One True Framework, which isn't a problem i have

wow that last comment I made was the lisp curse in a nutshell 😂

Well I think “one true framework” only happens for the kind of small bespoke apps @jiriknesl is on about… if the app actually evolves into a product or something bigger or longer lived; it will over time become more and more bespoke.

github, or basecamp even, ain’t gonna be a rails app like any other — or if they are it’s only because the rails app is a small uninteresting part of the product. The service as a whole will be very bespoke.

yep

maybe that's why i feel like i don't need the framework... the only real carcrash i've worked on in the last four years was a django app that had got out of control, and all the others have been pretty bespoke

even our current typescript thing is very bespoke apart from the serverless/cloudformation wrapper

not to say you can't make a mess with bespoke apps, but that's another story :)

I’m not really one for frameworks either — I’m happy to tie together libraries… that’s not to say I don’t want better starting points. Whenever I start a new app I typically spend a day or two, or more pulling in the libs, deployment and project infrastructure I want. Though how much time I spend on that depends very much on the scope of the app.

+1 for better starting points... if only i had some more time 😬

Well that’s what I think something like duct could help with; assuming it had sufficient buy in for pre-wrapped components

(but i really do want those starting points to let you target both serverless and server)

re the “serverless framework” how would you abstract across clouds? Another point of fragmentation.

Serverless (the confusingly named framework) already has hooks/modules for different cloud providers

although iirc their community survey showed like 80% of people were on AWS or sthg

so I'd just wrap that, bundle shadow, talk to the aws-sdk

profit

ah I hadn’t realised you were refering to this

done a few things like that pre-shadow but they were moderate pain

somewhat confusingly i meant serverless (the general technology)

i tend to capitalise the framework

but it's what we're currently using to abstract our deployments for our suite of typescript stuff

ok Serverless seems pretty GraphQL heavy which is totally reasonable for most crud apps; but what if I wanted to bake pathom in instead?

pathom?

serverless isn't graphql heavy (though obv you can run that for an endpoint)

the routing is done via your cloud provider

e.g.

API gateway for AWS

you just write the code that executes when it hits a path, so you can p much do what you want within that restriction

we treat each endpoint like a microservice and they all share the same bundle of business logic (db layer, xfm functions, interface types for validation etc)

not sure if that answers yr q tho haha

I tried using Serverless (framework) a few times but found it to be annoying. I took some time to dig into terraform and am happier with that for deploying.

we use terraform for the aws platform stuff that we don't want in sls config

mainly cos cloudformation is 💩

> we use terraform for the aws platform stuff that we don’t want in sls config Yeah I just like having everything in one place. Or is there a benefit to Serverless that I am missing?

but i struggle to think of another framework other than rails or django that could get us from zero to production in seven days in a corp env where we had to have DR, security signoff etc etc etc

what is a “DR”?

serverless gives you routing, inline permissions, incremental deploy of code changes in your functions etc - we basically use it as a packaging tool

like, it's not interesting to write idempotent deploy stuff, we wanna just write the app code

so TF preps the VPCs etc, SLS rolls out into them and makes everything routable etc (plus also works offline like an app server thanks to the sls-offline plugin) and builds our TS as well

disaster recovery

can you rebootstrap the entire platform from nothing back to live

mogge.

I used to work with a guy was part of the team that wrote the original PL/1 compiler for System/360.

ah nice, thanks you

Morn'

Interesting points being covered here, I'm getting the feeling that the biggest hump is still getting a project up, once you've done that development is a lot smoother...

yarp

it's all about whether the value add is in app code or platform which depends on how bespoke it needs to be at different levels

i guess

Help!!!!! Someone made a PR for my MQTT server/client!!!! :shocked_face_with_exploding_head:

Interesting discussion about (why we don't have) frameworks. Yeah, I agree that I'd like a simple web framework for basic CRUD apps on the Admin side, but beyond that (narrow) use case, I don't think a framework is really possible: web development is no longer the monolith it used to be and SSR CRUD apps are increasing niche these days.

yup even the basic brochureware thing that's within the wider team I'm in right now is a headless CMS that's statically-rendered ahead of time & that's in a corp/enterprise environment

a backend dev hasn't been anywhere near it

& it works just fine :man-shrugging:

Good morning

I used that compiler!

Also, increasingly apps like Notion, AirTable, Coda replace maybe 90 % of functionalities in these small apps I custom-made years ago. Having some triggers/components that could be added on top of such no-code platforms might maybe eliminate need for such a framework. But it is not ‘good enough’ in other things. With such things you loose all other things of frameworks (like logging, caching, debugging, having a code in git, feature toggles, tests). Hence, there still might be a range “sheet of paper” -> “XLS file, Google Sheets” -> “AirTable, Notion, Coda” -> “Symfony, Django, Rails app” -> “Custom made app” Currently, Clojure covers only the last one. There are exceptions (babashka, clojupyter) where you can use Clj in more low-code (scripting, not programming in the large) context. Of course, it’s nerd-only. But for midsize apps Clj only attempt is Luminus that’s definitely not good enough. It wouldn’t be competitive 5 years ago and most likely even a decade ago. We use Clojure Duct and it is really a component devstack. Nothing like Django…

Have you looked at Coast? I don't know if it is still maintained but it's meant to be fairly comprehensive as a framework (and, I believe, inspired by Rails).

Thank you. Coast goes to the right direction but it has the same issue. It has less features than CakePHP 1.0 released 14 years ago, or http://ASP.NET WebForms that’s 18 years old. Maybe something current might be using re-frame on frontend & focus only on backend part + compatibility layer (like propagation of server side validation errors to client side form).