datahike

https://datahike.io/, Join the conversation at https://discord.com/invite/kEBzMvb, history for this channel is available at https://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/datahike
alekcz 2020-09-23T09:55:02.000400Z

LevelDB is ready to rumble: https://github.com/alekcz/konserve-leveldb

🦜 6
timo 2020-09-23T10:04:55.000700Z

did you do it from scratch?

alekcz 2020-09-23T10:25:49.001200Z

Not entirely. I reused a lot of the code from the other konserve backends I worked on.

alekcz 2020-09-23T10:26:29.001400Z

it did take a while get the linear storage layout sorted though

alekcz 2020-09-23T10:26:31.001600Z

And RocksDB https://github.com/alekcz/konserve-rocksdb

6
timo 2020-09-23T10:27:56.001800Z

really cool. I want to get into backends soonish as well. I want to write a cassandra-konserve sometime...

alekcz 2020-09-23T10:28:52.002Z

Happy to chat about how I approach it when you're ready to get started

timo 2020-09-23T10:29:17.002200Z

will do, thanks!

2020-09-23T12:22:09.002600Z

You mean Reeeeady to REPL! πŸ™‚

πŸ˜„ 1
2020-09-23T13:51:33.003700Z

I am trying to use datahike server, and I can get to the main to run, but I can't get to create a single transaction

2020-09-23T13:52:07.004400Z

I basically stop at the coercion all the time.

2020-09-23T13:52:16.004700Z

Would you have an example ?

2020-09-23T13:58:43.005100Z

I get a java.class.Assertion error.

2020-09-23T14:01:03.005700Z

Moreover, I think the uberjar from the repository does not hold the latest dependencies and breaks.

2020-09-23T19:28:26.006100Z

So I made it work, but only with application/edn. I wondered if it would be possible to make it work with json?

timo 2020-09-23T19:36:23.007500Z

Hi @neo2551 thanks a lot for testing it!

timo 2020-09-23T19:36:44.008Z

In case it is not mentioned on the repo it is still in early stages.

2020-09-23T19:36:45.008100Z

So it is strange: the datahike server works as expected if I use application/edn, but once I communicate application/json, I have a bad entity

2020-09-23T19:36:46.008300Z

[{:age 35, :name David} {:age 20, :name Myriam}] 20-09-23 19:34:40 amd-3700x ERROR [datahike.db:1054] - Bad entity value 35 at [nil nil :age 35 nil] , value does not match schema definition. Must be conform to: (= (class %) java.lang.Long) {:error :transact/schema, :value 35, :attribute :age, :schema {:db/unique :db.unique/identity, :db/valueType :db.type/long, :db/cardinality :db.cardinality/one, :db/ident :age}}

alekcz 2020-09-26T19:41:28.006400Z

@neo2551 which client are you using to send the json reqs?

2020-09-26T20:52:10.007Z

I was using python, with the requests module.

timo 2020-09-23T19:37:06.008900Z

We will be working on JSON support very soon

2020-09-23T19:37:19.009500Z

No problem, I feel honored to help you πŸ™‚

πŸ™‚ 1
timo 2020-09-23T19:38:25.010900Z

and I think there is some kind of problem with JSON right now. I did not investigate it further but some kind of types or namespaces will not be translated correctly right now.

2020-09-23T19:38:27.011Z

I also had a problem with a jdbc backend: I had to add the dependency manually in datahike-server.core, otherwise it would no be found.

2020-09-23T19:39:22.011800Z

But I will continue to study datahike, but it seems my team are going to try to use it in a polyglot setting as well (python and R).

kkuehne 2020-09-24T09:07:03.000300Z

That's our aim for the remote server, so people from other environments may use Datahike. For me the question for JSON is about how to represent datalog in it and how to handle namespaces and keywords.

2020-09-24T13:01:10.001200Z

I would enforce β€œ:namespace/name” in keys of json.

2020-09-24T13:01:32.001900Z

I don’t really know how this would interact with Clojure though.

2020-09-24T13:01:45.002400Z

Maybe we could use transit in other languages?

2020-09-23T19:39:28.012Z

Thanks a lot for your work.

timo 2020-09-23T19:41:21.013Z

Great David. That is very good to hear. We are working a lot on improving Datahike and Datahike server is a very important project for us.

timo 2020-09-23T19:44:10.014300Z

If you can describe your problem with JDBC a bit more in detail I could investigate it some more. But we will do some more testing with JDBC as well of course.

2020-09-23T19:52:17.016900Z

I tried to use Datahike server with a jdbc backend [if you could migrate to deps.edn it would be nice as well], and the Uber jar did not want to start because Datahike-jdbc.core was not required (as it was AOT compiled I guess?).

2020-09-23T19:52:46.017700Z

I solved it by requiring Datahike-jdbc in main.

timo 2020-09-23T20:18:34.018600Z

Actually we do migrate to clj-tools with deps.edn for the CLJS-support that is in the works.

timo 2020-09-23T20:18:39.018800Z

πŸ™‚