jackdaw

https://github.com/FundingCircle/jackdaw
finchharold 2021-04-12T14:12:42.004500Z

Hey everyone

finchharold 2021-04-12T14:13:01.004900Z

I'm trying to use jackdaw along with lacinia(graphql)...

finchharold 2021-04-12T14:14:35.006200Z

in my docker-compose file I have the kafka config like this:

BOOTSTRAP_SERVERS=kafka:9092
But in the application I've kept it like this:
"bootstrap.servers" "localhost:9092"
So, now when I'm running the app it is saying : WARN [org.apache.kafka.clients.NetworkClient:725] - [AdminClient clientId=adminclient-1] Connection to node 1 (/172.20.0.4:9092) could not be established. Broker may not be available.

finchharold 2021-04-12T14:14:46.006500Z

Any help on what's going wrong?

gphilipp 2021-04-12T14:18:04.007900Z

@kishorekaranam99 There might be multiple issues. Can you try with this docker-compose https://github.com/confluentinc/cp-all-in-one/blob/6.1.1-post/cp-all-in-one/docker-compose.yml?

finchharold 2021-04-12T14:19:52.009100Z

Can I use other one? coz I kept it as per my app like for postgres I'm running it on port 5433 and all. This is mine:

version: '2'
services:
  zookeeper:
    image: debezium/zookeeper:${DEBEZIUM_VERSION}
    ports:
     - 2181:2181
     - 2888:2888
     - 3888:3888
  kafka:
    image: debezium/kafka:${DEBEZIUM_VERSION}
    ports:
     - 9092:9092
    links:
     - zookeeper
    environment:
     - ZOOKEEPER_CONNECT=zookeeper:2181
  postgres:
    image: debezium/example-postgres:${DEBEZIUM_VERSION}
    ports:
     - 5433:5432
    environment:
     - POSTGRES_USER=pguser
     - POSTGRES_PASSWORD=pgpwd
    volumes:
     - ~/backup:/data
  connect:
    image: debezium/connect:${DEBEZIUM_VERSION}
    ports:
     - 8083:8083
    links:
     - kafka
     - postgres
    environment:
     - BOOTSTRAP_SERVERS=kafka:9092
     - GROUP_ID=1
     - CONFIG_STORAGE_TOPIC=my_connect_configs
     - OFFSET_STORAGE_TOPIC=my_connect_offsets
     - STATUS_STORAGE_TOPIC=my_connect_statuses
     - INTERNAL_KEY_CONVERTER=org.apache.kafka.connect.json.JsonConverter
     - INTERNAL_VALUE_CONVERTER=org.apache.kafka.connect.json.JsonConverter

finchharold 2021-04-12T14:21:42.009600Z

Can I just copy the whole thing you've pasted above and remove the ksql and add postgres?

gphilipp 2021-04-12T14:24:08.009800Z

That should work yes

gphilipp 2021-04-12T14:24:49.010300Z

you can just docker-compose up broker

finchharold 2021-04-12T14:26:07.010900Z

I didn't get it, you mean while using the above yaml or ?

finchharold 2021-04-12T14:27:38.011400Z

I've added the broker part from the config you;ve sent and did docker-compose up broker

gphilipp 2021-04-12T14:28:46.011900Z

mkdir runapp && cd runapp
curl -O <https://raw.githubusercontent.com/confluentinc/cp-all-in-one/6.1.1-post/cp-all-in-one/docker-compose.yml>
docker-compose up broker

gphilipp 2021-04-12T14:29:46.012400Z

docker-compose ps
  Name               Command            State                    Ports
----------------------------------------------------------------------------------------
broker      /etc/confluent/docker/run   Up      0.0.0.0:9092-&gt;9092/tcp,
                                                0.0.0.0:9101-&gt;9101/tcp
zookeeper   /etc/confluent/docker/run   Up      0.0.0.0:2181-&gt;2181/tcp, 2888/tcp,
                                                3888/tcp

finchharold 2021-04-12T14:29:57.012500Z

oh ok cool

finchharold 2021-04-12T14:30:11.012800Z

it's still downloading

finchharold 2021-04-12T14:33:08.013100Z

ERROR: for zookeeper  Cannot start service zookeeper: driver failed programming external connectivity on endpoint zookeeper (0792f04ecbe54eceb3e6081c67b8da5c6d8fb1aa45506d158cdb36d3f62e70d3): Bind for 0.0.0.0:2181 failed: port is already allocated

finchharold 2021-04-12T14:33:18.013400Z

zookeeper is already running...

gphilipp 2021-04-12T14:33:28.013700Z

Kill the previous containers

finchharold 2021-04-12T14:36:59.014Z

got it

finchharold 2021-04-12T14:37:02.014200Z

thank you so much