Clojurians Log v2

I was honored to be a part of a panel discussion comparing machine learning across languages. Got some nice opportunities to plug the techascent stack, Neanderthal, Smile, and of course Clojure in general 🙂. https://twimlai.com/the-great-ml-language-un-debate/?utm_source=tw_linkedin&utm_medium=social&utm_campaign=advancing_data_sci

🆒

I’ve been looking at feature stores and feature engineering systems lately. Big tech has shared some solutions (some here http://featurestore.org/) but these sound very big and complex because they handle large amounts of data, streams etc. My use case doesn’t need something so high-throughput, but having a service that creates and curates features is still useful, and it does sound like a use case where clojure would shine. Is there actually such a project in clj that I’m not aware about?

I've successfully sold my boss on Datomic for this exact purpose (+ also doing model metrics tracking), but we're not yet at the implementation stage for it. I think Datomic (or Datahike if you prefer open source) is particularly suited to use as a feature store because of its vastly more flexible data model and more powerful query engine. You're not constrained by the Hadoop-like architecture that so many of the off-the-shelf projects seem to have, so you can build the schema and data model that actually suits your workflow. Time travel on schemas (supported by Datomic) allows you to grow your feature schema organically rather than locking you in to a single data model.

https://www.logicalclocks.com/blog/how-to-build-your-own-feature-store Go through this flowchart and look at how many of these decisions you don't have to make if you use something like Datahike or Crux.

Feature store is one side of the equation, what about feature engineering? I haven’t used Datomic so far, but last time I checked it was a bit Clojure focused right? as in there’s no client for PHP (what my company uses), or was there REST interface?

https://docs.datomic.com/cloud/analytics/analytics-concepts.html Datomic's REST API is deprecated, it looks like. If all you need is to pull the data you could use the PrestoDB connector and SQL. Getting it back in would be a little more tricky - you'd probably need to engineer something that passes between your front-end and the Datomic transactor.