Clojurians Log v2

Opinions on Scala?

and Haskell?

@borkdude: Quite interesting discussion here: http://discuss.purelyfunctional.tv/t/types-clojure-and-pain/585

Interesting thread, thanks

@bvulpes: well, I use whitespace more crazy than you think, here's my code editor

(hmm, I should probably change language settings in Powerpoint)

@borkdude: What's the link to that? Closest thing I've found is a handwritten version without Scala/Clojure: https://channel9.msdn.com/Blogs/Charles/Simon-Peyton-Jones-Towards-a-Programming-Language-Nirvana

https://www.youtube.com/watch?v=iSmkqocn0oQ

borkdude: and where praytell is Ada on that chart?

@bvulpes: you decide

i'd squeeze clojure and scala significantly to the left, and scala significantly down on the usefulness spectrum

@bvulpes: Interesting, your opinion is that Scala is more useless than Clojure?

Please elaborate

i've seen a lot of time wasted trying to pave over paradigmatic incompatibilities between scala libs and even differences between author styles /within an org/

@bvulpes: Have you used Scala professionally and for how long?

ofc "clearly they're doing it wrong", but otoh, "no true scotsman"

no, watched a scala org evolve through a friend who ultimately left in disgust at how the language had crippled their work.

oh wow

yeah i mean it gives me hives, i won't touch it. but he liked the team, and so there went three years of his life.

and re clojure's "usefulness" well the compiler is not particularly useful in a wide swath of applications i'd like it to be.

I'm using it at work now (also moved there because I liked the team/company). So far I must say that despite that bad noise, I quite enjoy it

right, see https://clojurians.slack.com/archives/other-languages/p1462553229000040

We use microservices, maybe that helps to prevent growing an unmaintainable mess

of course that's not a silver bullet either

I'd like to hear experiences from bigger Clojure teams vs bigger typed FP teams

The real question is, is it types that make scaling easier, or mainly other things like immutability, people in the team, documentation etc.

over my pay grade.

i generally solve medium-sized problems for small orgs on my own. last mega codebase i was in was ios and let me tell you the challenges on that project were of the human factor drop-shadow "like this other interface but different in inane and complexity-inducing ways" kind.

I’m glad @martintrojer is here — hopefully we can ask questions about your articles?

We have 30K+ production lines of Clojure and 10K+ lines of test code. We aren’t really seeing the problems cited in that Prelude article — but we do have only a small team (two full-time Clojure devs, we had a part-time third dev for a while).

We certainly don’t find the nil-punning to be as problematic as it seems some people do.

We do find we have a lot of questions around best practices for actually organizing and structuring code across namespaces — and we’re constantly refactoring and refining that organization as we learn lessons over time.

@seancorfield: do you use Schema and pre/post a lot?

I think one of the important things is to have a team that is fluent with one technology and also knows how to deal with the downsides of it, regardless the language.

We’ve tried core.typed and Schema multiple times. We did not feel the benefits were worth the costs.

Schema’s validation is only as good as your test suite so it’s only going to catch errors in structure that don’t cause your underlying code to fail the test anyway.

core.typed is better in that regard: it checks the types not the (runtime) structures.

But retroactively adding core.typed to a large codebase is an exercise in frustration due to the relatively poor type inference it does and the fact that even for a single namespace, it’s essentially an all-or-nothing affair due to the amount of "warning" noise that a partially annotated namespace provides.

I think we’d gotten around 5% coverage with core.typed before we gave up. It required too many changes to code to satisfy its analysis, or else you just slapped metadata everywhere saying "Trust me — don’t check this body!" which sort of defeats the whole purpose.

We actually started with core.typed, switched to Schema, gave up, tried core.typed again, tried Schema again, then switched back to core.typed. Then gave up again.

Re: Scala — We came to Clojure from Scala. Lots of reasons. Scala 2.7 -> 2.8 was such a painful transition we were very gunshy about future upgrades. Compiler was very slow. Toolchain was fragile and quirky. Error messages were horrible, esp. when you ran into the combined buzzsaw of implicits and builtFrom (or whatever that weird collections library signature thing was).

We had a legacy dynlang system anyway so integrating Scala was painful on many levels (calling into Scala from other JVM languages gets messy unless you’re very careful about your API).

@seancorfield: nice to hear it's working out for you :)

My experience based on many things

Having to read code to figure to argument types is pretty horrible

All the NPEs is death by a thousand cuts

But the biggest thing for me is that I'm at lest 10x faster refactoring is a Lang like Haskell

With confidence as well, that haven't messed something up

Tracking side effects in types is also a pretty big thing for code quality

Biggest teams I've been in is 8 clj devs

Total code weighing in at 40k perhaps split over many services

Steady stream of totally unnecessary bugs pushed to prod

Anyway, on a journey to find something better

Front end is pretty clear, Elm has win written all over it

Backend still looking

I agree Elm is a great choice for the front end at some point. Not sure it’s really production ready yet beyond bleeding edge adopters (I followed Elm very closely for a long time during its development — and seeing ElmConf ahead of Strange Loop makes me want to attend that more than Clojure/conj this year).

I have a love/hate relationship with Haskell. I was part of the UK university research crowd that created our own FP language in the early 80’s as part of our language research. When Haskell appeared, I truly hoped FP would go mainstream — it was so obviously better than the status quo! — but Haskell remained an academic curiosity and OOP took over everything (unfortunately).

So I’ve worked with Haskell-like languages since the early 80’s and with Haskell itself on and off for about 25 years… and, much as I love many things about Haskell, every time I actually use it to try to build a real "thing" I get so incredibly frustrated with its whole ecosystem and, often, by the attitudes of many Haskellers 😿

As for Clojure’s NPEs… I just don’t encounter them much. I know some people run into them a lot. I don’t know why some do and others don’t.

@seancorfield: one of the best thing of Clojure is the community, the people are so nice compared to other more elitist languages (my words)

@seancorfield: on the other hand, I'm jealous of the documentation and effort some other communities are putting in their artifacts

I can understand the refactoring argument. The type system means you can’t break things in so many ways that a dynlang would allow you to break things during refactoring. Although I don’t find refactoring in Clojure to be particularly problematic so it isn’t a pain point I feel strongly about.

@martintrojer: Remind me, what was your language background before Clojure? Much FP? Much dynlang, or mostly typed languages?

With 3 months of Scala, I can say that in my experience refactoring is much easier than in Clojure (2.5 years commercial experience)

Maybe some tools make it easier now (like clj-refactor), but often these tools are not mature and crash (CIDER, etc).

Running lein check and lein test in the background helps

I wonder if Cursive makes a big difference here, I don't have enough experience with that

Interesting discussion so far and I agree with a lot of the statements made here I tried haskell a few times and always gave up because of the tooling, emacs being a big part of it (dont want to start IDE flamewar here, I just dont like emacs). It seems like stack solves some of these problems, but, the last time I looked into it the emacs setup stated that it still had problems with stack (which may be figured out inbetween) Also I have a hard time figuring out parameters and shapes of maps in standard clojure. Although I found some peace in schema. But again, here I agree, a better type system, which is optional, would fit my needs more.

@borkdude: I never used cursive for refactorings in large codebases, but for my small ones it did not fail once

I refactor parameters and function names, but lately saw that it can also refactor keywords, I wonder how reliable that is

@seancorfield: before clj: c, c++, Python, c#, f#

My favorite languages were Common Lisp, then F#, then Clojure and now I don't know anymore 😉

Anyone here looked into Rust? I went through the docs and like the idea about ensuring data safety through the type system

I think theses 'favorites' were always in the context of what I could do with them at work. So I chose F# when I was doing .NET.

@sveri: I spent a fair bit of time playing with Rust last year. It’s very interesting. Not so much for the type system itself but for the borrow-tracker aspect.

If I were doing near-metal systems programming, I’d be very tempted by Rust. It’s what I had hoped Go might be. [I think Go is horrible having spent a fair bit of time playing with it the previous year!]

If F# ran on the JVM (and therefore could interop with all my legacy code), I’d probably be using that instead of Clojure. Depending on tooling etc.

@seancorfield: That was my feeling too regarding Rust. It seems like there is also a lot happening regarding game development with Rust, if anyone is into that

@seancorfield: coreClr port will open the door for f# properly

Very nearly there, and better than the JVM on pretty much every metric

So much less boxing in .net it puts JVM to shame

Also, startup not an issue

But still completely irrelevant to me if I can’t interop with my existing legacy codebase! 😸

@martintrojer: One thing I found very interesting about your Prelude article was how you were bored just working in Clojure day-in, day-out. I would assume that has also been the case in all the previous languages you’ve used?

(well, assuming perhaps you left some of them due to frustration, rather than boredom 😆 )

What is CoreCLR?

Common Lisp and Clojure had the biggest impact on my enthusiasm for programming, but yes, even Clojure can feel like work when you're doing it day in day out. It also depends on the system you're building. If you're making a CRUD app and every feature is more or less the same old stuff, it gets boring.

(and a year of playing with Clojure before that)

And I do a bunch of hobby / FOSS Clojure in my spare time too. And I help non-Clojure devs learn Clojure (I’m doing my best to convert the world, one developer at a time!).

I look at that monadic getPlayer’ function and I think of (some-> params :player get-player) or similar...

I hope we’ll be hiring at some point, maybe this year, but probably a junior dev… maybe someone who’s fairly new to Clojure but seems to "grok" FP in general...

I would also love to do some clojure as part of my job. But here it is hard to find one in my City... I have not heard of one clojure Job...

I had a job with a couple of Clojure projects, but it was a software services company and every project was only a couple of months. I was missing working in a team and building up deeper knowledge of something.

What is it called in English when a company builds software and also maintains it for other companies (as opposed to help the developer team at some other company started and then leaving)

sounds like a consultancy

there are two types of consultancies then: one that builds all your software and maintains it, and the type that can help you accelerate your software team

@ericnormand: I read the discussion on your board. You mentioned 'types are the future, but...'. I thought it was interesting. Why do you think types are the future, but still do mainly Clojure?

they're the future, not the present :simple_smile:

haha :simple_smile:

Clojure still rocks, and has a ton of great design decisions

true

and I do want to expand into other languages

You mentioned some Haskell experience. Some Clojure devs seem to go into that direction.

yeah

seems really common when people get frustrated

Haskell is a great language, up there with Clojure.

Clojure has a huge advantage with the Java ecosystem

Haskell is good, but the rhetoric is tiring.

I wish there were more options of typed languages to compare.

at the moment, saying "I want good types" seems to mean "I want Haskell".

pssst! There’s a Leiningen plugin that lets you do mixed Clojure/Frege projects

I think Rich Hickey once said in one of his talks: it's either this or all the way to Haskell

what does "this" refer to?

@ericnormand: dynamic typing?

@ericnormand: I should look it up, can't remember the exact phrase

@ericnormand: maybe it was about 'the language of the system'

@ericnormand: programming like you do with services, but in the small

the next language I design (in like 30 years at this rate :simple_smile: ) will explore types

Back when I was exploring FP language design and implementation (my unpublished PhD thesis in the early 80’s), I created a dynamic language based on the notation in W.H. Burge’s book and implemented it as a transpile to LispKit (that part was based on Peter Henderson’s book). But I used a Prolog program to analyze types across whole programs to verify behavior.

(so, yeah, I had a transpiler to Lisp(Kit) and a transpiler to Prolog for separate purposes)

coooool

Prolog could produce a sequence of all types that satisfied the program — and it was pretty easy to see when you had an illegal program because Prolog, bless its little cotton socks, would just act the two year old child and respond with No.

It was also nice that if your program was essentially "any -> any" then you’d done something wrong 😆

It was very interesting to hear Matthias Felleisen ragging on Hindley-Milner type inference at Clojure/West.

oh yeah?

what did he say?

Oh dude, you have to watch the talk! It’s awesome!

yes, it was an awesome talk

https://www.youtube.com/watch?v=XTl7Jn_kmio&index=25&list=PLZdCLR02grLq4e8-1P2JNHBKUOLFTX3kb

An hour and fifteen — it stands out like a sore thumb by virtue of being half an hour longer than any other talk at the conference! 😆 No one wanted him to stop!

"Everyone in Clojure thinks in types, but nobody is writing them down" was a sentence I remembered

Isnt that how you do it in every language that has insufficient types?

Of course. But his story went on. At least you could write them down as documentation. But then it becomes a problem that they are not checked and the documentation isn't right anymore.

Just like it always is with documentation

Right. So it would be nice if you could check it. And this is how he arrived at Typed Racket more or less, if I remember correctly.

Yes, I watched it too, no objections from my side, I just wanted to point out that this problem is not specific to clojure

Agreed

I feel like a weirdo when mentioning my programming opinions (liking lisps doesn't help), but I avoid scaling problems by avoiding writing large programs.

I believe there's lots of techniques that help you do this. (After all, using a high-level language is one.)

Not just technically, but socially. Finding another programmer with whom you can synch well with imposing constraints on your mutual work... not so easy in my experience.

To the extent that large programs are necessary (for example, accepting Tim Daly can't simply avoid a large Axiom codebase, given its 1.5 million lines of Common Lisp), it's a new game, and static types are only one of many techniques available.

@tjg What other techniques can we use in Clojure?

Guess I'll write about this in the http://discuss.functional.tv discussion, but Tim Daly frequently mentions literate programming. http://lambda-the-ultimate.org/node/3663#comment-51878

So for example, he edited a literate programing book of Clojure. http://daly.axiom-developer.org/clojure.pdf

"Beyond a certain point of complexity you can only understand a program by talking to the author about why they wrote it the way they did." I think he's right about that, types or no types

He proposes literate programming for software that must survive beyond 5 years

Are there really any large systems in production written using literate programming?

And doesn't literate programming suffer from the same problem as other documentation, that it isn't checked and therefore is incorrect a lot of the time?

BTW, I don’t personally advocate literate programming, just one of many perspectives I’d consider if I ever entered the world of Large Clojure Programs. As I understand Martin Trojer’s article, a sufficiently large Clojure system requires you to spend “more than 50% of the total line count” on maintaining its architecture.

Analyses like that are one of the reasons I simply try to avoid large systems... like I avoid hard computational complexity classes. (For example, “Small Things, Loosely Joined, Written Fast, Discarded Readily.” http://blog.cognitect.com/blog/2016/3/17/the-new-normal-protected-asset-or-disposable-inventory )

I have no idea about literate programming, but I personally devote much time to documentation: - an org-mode notebook in git, where I try to boil a domain down to a conceptual language, and constantly describe project case-studies in terms of it. - “Battle Stories” (which I first developed in a company with 15 devs, using narrative structure so people read my boring tech notes) - video walkthroughs (timestamped), where developers walk me through their systems as I ask questions