other-languages

here be heresies and things we have to use for work
Yehonathan Sharvit 2021-03-06T19:46:20.003600Z

A few thoughts and questions related to the value of “loose-coupling” between code and data: In Clojure, a `customer` namespace that manipulates a map that represents `customer` data is loosely coupled with customer data. While in Java, `Customer` class with a static methods  that manipulate a `Customer` record (or an immutable data class) is strongly coupled with Customer.  One benefit of Clojure approach is that the namespace doesn’t have to “import” the class definition for Customer On the other hand, a Java developer might argue that the contract between code that manipulate Customer data and Customer data shape is not explicit  For instance, if you want to rename a field in Customer data, how would you discover all the pieced of code that need to be updated Rich likes to say: > We should program the insides of our sytems like we progam the outsides A Java developer might object: >  Does it worth it to have code and data loosely coupled when they both live in the same program?

Yehonathan Sharvit 2021-03-09T01:49:52.028300Z

In the data-oriented programming approach (the one Rich taught us), code and data are coupled in the same sense that a frontend and a backed are coupled via an API definition. If the backend breaks the API definition, the fronted code has to be updated. But they are coupled in a loose way. Meaning that one doesn’t care about the implementation details of the other

benoit 2021-03-09T12:58:41.028500Z

> "one doesn’t care about the implementation details of the other" This is abstraction, the distinction between an interface and an implementation, and both, Clojure and Java, support it. The functions in Java and Clojure that manipulate those records and maps are coupled to the definitions of these records and maps. > One benefit of Clojure approach is that the namespace doesn’t have to “import” the class definition for Customer It is not because you don't have to import the class customer that it is not coupled. In Java, the coupling is visible because, like you say, you have to "import" the Customer class. And the functions that take a customer as parameter specify the type of the argument. In Clojure, you don't have to "import" the Customer class. This is actually a drawback of because in doing so you create a hidden coupling. Specs are one way to make this coupling visible. Coupling is a concept at the logical layer of software, not at the concrete implementation layer. For a definition of these layers, see http://www.pathsensitive.com/2018/01/the-three-levels-of-software-why-code.html Don't get me wrong. It is much better to represent records as maps rather than Java classes, but it has nothing to do with abstraction or coupling.

Yehonathan Sharvit 2021-03-09T15:36:10.029Z

@cgrand Could you rescue me here? I couldn’t find the proper words to answer @me1740’s questions 😬

Yehonathan Sharvit 2021-03-09T15:38:16.029200Z

@cgrand You are so convincing with your sophisticated vocabulary (strawman argument, discrete getter, typed generic getter, payload evolution, external state, concretions …). I wish my french was as refined as your english 😊

Yehonathan Sharvit 2021-03-09T15:40:23.029400Z

@me1740 the connection between loose coupling and representing data with maps was made by Rich Hickey on his talk https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureMadeSimple.md (Around 0:10:29).

benoit 2021-03-09T16:12:30.029900Z

I'm not seeing the connection between loose-coupling and using maps. He says that we need flexibility and loose-coupling, I think we all agree on this. Then there are some quotes. Then he talks about the fact that we should recognize that what we manipulate most of the time in our programs are records, simple data, and that we don't need fancy abstractions on top of it. I agree with this. When your program manipulate records of data then you need to reuse the same abstraction for all your records (like a Clojure map). Sometimes though, it will help you to create new abstractions to control the complexity in your programs. You have to find the right balance between reusing existing abstractions and creating a new one. You can do the same in Java by using the Map abstraction to represent your records instead of creating the Customer class but it is much more awkward to use, especially with static typing. Regarding coupling, wether you have customer.getName() or (:name customer), your function in Java and Clojure will be coupled to the interface of your customer. There is just no way around it.

cgrand 2021-03-09T16:25:34.032800Z

Couple of remarks (since @viebel summoned me): keywords namespacing is important and one keyword could be seen as 1 interface. You couple against what you use vs coupling against the whole thing.

benoit 2021-03-09T17:03:14.035200Z

Agree in this case. That has an impact on coupling. The corresponding operation in Java would be to put the subset of method(s) in an interface and have the function depends on this interface rather than the class.

Yehonathan Sharvit 2021-03-09T18:31:56.035500Z

Accessing data in a map via a keyword is fundamentally different than accessing data in an object via a method (or a member). The only information that needs to be known in order to access data via keyword is the name of the field. That’s how I understand the loose coupling. Of course there is a coupling. The question is: is the coupling loose or tight? I’d like to claim that accessing data in a map is loosely coupled with the representation of the data while accessing data in an object is tightly coupled with the representation of the data.

Yehonathan Sharvit 2021-03-09T18:33:32.035700Z

But I cannot find the proper arguments. I’m not even sure that I formulated my claim properly.

Yehonathan Sharvit 2021-03-09T18:33:56.035900Z

But I’m quite sure that when Rich talks about loose coupling, he refers to “just use maps”

benoit 2021-03-09T18:50:35.036100Z

I would suggest that you define the terms before making a claim. To make sure you say what you mean and to make sure people don't misunderstand you. Coupling is about propagation of change. Two things are coupled if changing something in one thing requires changing something else in the other thing. The change propagates from one to the other. Two things will be tightly coupled if every time you change the slightest thing, you have to change the other thing as well. And two things are loosely-coupled if you have a lot of degrees of freedom when changing one thing without impacting the other thing. A lot of software design is about finding the right level of coupling to make sure that things work together (there will be a mimimal amount of coupling) while still being able to change each thing independently (without having to change any of the other things).

Yehonathan Sharvit 2021-03-09T19:05:38.036400Z

I am not sure that your definition of loose coupling is accurate, at least that’s not what Rich Hickey means by loose coupling. Richen often claims “we should program the insides of our systems like we progam the outsides” The interfaces between two components inside a program should be the same kind of interfaces between two programs. When two programs agree on an API, each one of them is free to change the implementation providing that they don’t break the API. I’m pretty sure that this is what Rich means by loose coupling. Now, how the API between program should be formulated ? In terms of data. Not in terms of code (like DCOM etc…) According to Rich’s philosophy, we components inside a program should define their API in terms of data. Not in terms of classes and methods.

benoit 2021-03-09T19:44:53.036800Z

> When two programs agree on an API, each one of them is free to change the implementation providing that they don’t break the API. I’m pretty sure that this is what Rich means by loose coupling. That's what I also mean by loose coupling. The changes you make to your implementation do not propagate to the user of your API as long as you respect the contract of this API. By "define an API in terms of data", do you mean making sure that the functions of your API take data as input and return data as output? If so, it might make sense for software modules that deal with information records but there are APIs that do other things that simply manipulating information records. When Rich says "use maps", he means that when we want to represent records of information, we should use maps. And I totally agree with that. But it doesn't mean that we should use maps of static data everywhere. Take an API like Datomic that Rich designed himself. Would you call the database value that you pass as argument to most of its functions data? Thankfully you don't have a gigantic map containing all the data in your database. The API is a set of functions that lets you query this database value.

Yehonathan Sharvit 2021-03-10T02:10:58.037Z

I think we agree on most of the things

2021-03-06T19:54:10.004900Z

There are times when Clojure developers want to use Spec, but want the "closed world assumption", e.g. if they didn't define a key in a map explicitly, it should be an error for it to be there. Rich has argued that very often when someone does this, they later end up regretting it as they wish to extend their data model.

borkdude 2021-03-06T19:54:15.005200Z

> For instance, if you want to rename a field in Customer data, how would you discover all the pieced of code that need to be updated Namespaced keywords and something like clojure-lsp or Cursive ;)

2021-03-06T19:55:01.006300Z

There are uses for the closed world assumption, I think, e.g .when you are using experimental early code versions to learn what the data model is, iteratively, e.g. you add a few keys for a map that you know should be there, but you want running the code to tell you if any other keys are found you haven't anticipated, with an exception or similar error.

2021-03-06T19:55:38.007100Z

Later, after you believe you have done enough experiments that you have covered all of the keys that exist today, you switch the spec to allow other keys in the future, if you believe that the open world assumption is the long term way to enable growth of the data model.

2021-03-06T19:56:17.007900Z

But lots of programmers (myself included) feel better knowing they they have covered all of the existing ground of the data model, even if there is no explicit documentation for what the data model is, so experiments on large data sets are the best we know how to get there.

2021-03-06T19:57:17.009100Z

Lots of developers also like it if they have errors occur in a closed-world assumption data model forever in their code, because they believe that will let them know when the data model (and assumptions they may have built into their code based upon it) have changed, in some other piece of software that someone else has modified, without informing the world.

2021-03-06T19:58:15.010100Z

That liking of "I get errors if the data model is extended by someone else" is not particular to developers of one programming language, I don't think. It is a general feeling of "I want a loud noisy error message if the world has changed around me".

2021-03-06T20:02:45.011700Z

Java classes with a fixed set of fields, and similar constructs in many other languages with similar kinds of type systems, enable that closed world assumption by default, but in most of those language also tie the kinds of functions / methods / code that can update and read those fields to the specific class or type name. Clojure maps don't, but leave many developers with that uneasy "I might have forgotten to change all occurrences of the code that should be changed when I add a new field/key-in-my-map" when certain kinds of code changes occur.

2021-03-06T20:04:25.013Z

Probably a way that Rich Hickey might describe that is (my words here, NOT his): "many developers like that form of coupling. They have become accustomed to it so much that they rely upon it, and cannot imagine developing software without it".

Yehonathan Sharvit 2021-03-06T21:01:27.015300Z

Interesting @borkdude It means that decomplected keywords are more intrinsic to data-oriented programming that it may seem at first!

borkdude 2021-03-06T21:02:28.015500Z

I think that's one of the main ideas about spec, inspired by RDF: each attribute name indicates something unique

borkdude 2021-03-06T21:02:42.015700Z

so :customer/name always means exactly that

Yehonathan Sharvit 2021-03-06T21:04:46.015900Z

It means that the loose coupling between code and data is in fact not between code and maps but between code and map fields.

Yehonathan Sharvit 2021-03-06T21:06:24.017200Z

Could you guys think of other benefits of having a loose coupling between code and data?

Yehonathan Sharvit 2021-03-07T08:52:35.025Z

@jayzawrotny I think your example illustrates the benefits of separating code from data. Could you have gained the benefits you mentioned by storing bid data in various data classes like BidForVendors, BidForUsers …?

2021-03-07T16:08:23.025700Z

If you know all the possible contexts from the beginning, it may be possible to get some of those advantages with subclasses. But realistically, new contexts emerge which means a continuous cycle of moving methods and attributes from the base class into the subclasses. The one wall you might hit is if you want to perform a sequence of actions across contexts. So if an admin closes the bid, we may want to send closed emails to both the vendors and users. With that separate Clojure setup, that’s trivial to do since you have a single bid object hash-map that can be passed between functions across any context. With subclasses, you would need to create new instances of the next subclass you need:

2021-03-07T16:14:16.026100Z

(ns company.admin.bid.api
  (:require
   [company.db :as db]
   [company.user.bid :as user]
   [company.vendor.bid :as vendor]))

(defn close
  [bid]
  (let [closed-bid (assoc bid
                          :closed true
                          :status :closed)]
    (db/update! db/conn bid)
    (user/send-closed-bid-email closed-bid)
    (vendor/send-closed-bid-email closed-bid)))

Yehonathan Sharvit 2021-03-07T16:21:36.026300Z

I see. I think it’s called the ClassAWithThisAndThat problem

Yehonathan Sharvit 2021-03-07T16:21:52.026500Z

You need to create a class for every combination of fields!

Yehonathan Sharvit 2021-03-07T16:21:58.026700Z

That’s insane 😱!

Yehonathan Sharvit 2021-03-07T16:23:15.027100Z

What’s even more insane is that Java folks don’t seem to be aware of this insanity

borkdude 2021-03-06T21:11:03.018500Z

Yes, that's also one of the core ideas: you can have arbitrary combinations of fields, there is not such a thing as a type which is a fixed combination of fields

borkdude 2021-03-06T21:11:33.018700Z

Watch Hickey's talk titled "Effective Programs" or "Maybe Not" (I think the last one emphasizes this idea more)

Yehonathan Sharvit 2021-03-06T21:11:53.018900Z

I think I watched those talks

Yehonathan Sharvit 2021-03-06T21:12:11.019100Z

I understand the idea behind decomplected keywords

Yehonathan Sharvit 2021-03-06T21:17:37.019600Z

I always thought that Clojure rationale was: “just use maps” because maps are more appropriate than classes to represent data to be manipulated. Now, I understand that according to Clojure, “there is not such a thing as a type which is a fixed combination of fields” as you wrote. In a sense, it means that according to data-oriented programming, field keys are first-class citizens.

borkdude 2021-03-06T21:42:52.022200Z

yes, mixing attributes that travel together

borkdude 2021-03-06T21:43:18.022400Z

and this "together" is usually maps