做pcb網(wǎng)站昆明網(wǎng)絡(luò)推廣
背景:找了一圈資料,都是東講講西講講,最后我還沒搞好,最終決定參考官網(wǎng)說明。
官網(wǎng)指導(dǎo)手冊(cè)地址:Apache Kafka
需要預(yù)備的知識(shí),keytool和openssl
關(guān)于keytool的參考:keytool的使用-CSDN博客
關(guān)于openssl的參考:openssl常用命令大全_openssl命令參數(shù)大全-CSDN博客
先只看SSL安全機(jī)制方式。
Apache Kafka 允許客戶端通過 SSL 進(jìn)行連接。默認(rèn)情況下,SSL 處于禁用狀態(tài),但可以根據(jù)需要打開。
- 1為每個(gè) Kafka 代理生成 SSL 密鑰和證書
部署一個(gè)或多個(gè)支持 SSL 的代理的第一步是為集群中的每臺(tái)計(jì)算機(jī)生成密鑰和證書。您可以使用 Java 的 keytool 實(shí)用程序來(lái)完成此任務(wù)。我們最初會(huì)將密鑰生成到臨時(shí)密鑰庫(kù)中,以便稍后使用 CA 導(dǎo)出和簽名。
keytool -keystore server.keystore.jks -alias localhost -validity 700 -genkey -keyalg RSA
您需要在上面的命令中指定兩個(gè)參數(shù):
- 密鑰庫(kù):存儲(chǔ)證書的密鑰庫(kù)文件。密鑰庫(kù)文件包含證書的私鑰;因此,它需要安全保存。這里是server.keystore.jks
- 有效期:證書的有效時(shí)間,單位為天。這里是700天。
可以看到,目錄下生成了對(duì)應(yīng)文件
之后可以運(yùn)行以下命令來(lái)驗(yàn)證生成的證書的內(nèi)容:
keytool -list -v -keystore server.keystore.jks
- 2創(chuàng)建您自己的 CA
完成第一步后,群集中的每臺(tái)計(jì)算機(jī)都有一個(gè)公鑰-私鑰對(duì),以及一個(gè)用于標(biāo)識(shí)計(jì)算機(jī)的證書。但是,該證書是未簽名的,這意味著攻擊者可以創(chuàng)建此類證書來(lái)偽裝成任何計(jì)算機(jī)。
因此,通過為群集中的每臺(tái)計(jì)算機(jī)對(duì)證書進(jìn)行簽名來(lái)防止偽造證書非常重要。證書頒發(fā)機(jī)構(gòu) (CA) 負(fù)責(zé)對(duì)證書進(jìn)行簽名。CA的工作方式類似于簽發(fā)護(hù)照的政府——政府在每本護(hù)照上蓋章(簽名),使護(hù)照變得難以偽造。其他政府會(huì)驗(yàn)證印章以確保護(hù)照的真實(shí)性。同樣,CA 對(duì)證書進(jìn)行簽名,而加密技術(shù)保證簽名證書在計(jì)算上難以偽造。因此,只要 CA 是真實(shí)且受信任的頒發(fā)機(jī)構(gòu),客戶端就可以高度保證它們連接到真實(shí)的計(jì)算機(jī)。
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
生成的 CA 只是一個(gè)公鑰-私鑰對(duì)和證書,它旨在對(duì)其他證書進(jìn)行簽名。
下一步是將生成的 CA 添加到客戶端的信任庫(kù)中,以便客戶端可以信任此 CA:
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
與步驟 1 中存儲(chǔ)每臺(tái)機(jī)器自己的身份的密鑰庫(kù)不同,客戶機(jī)的信任庫(kù)存儲(chǔ)客戶機(jī)應(yīng)信任的所有證書。將證書導(dǎo)入到信任庫(kù)中還意味著信任由該證書簽名的所有證書。如上所述,信任政府 (CA) 也意味著信任它簽發(fā)的所有護(hù)照(證書)。此屬性稱為信任鏈,在大型 Kafka 集群上部署 SSL 時(shí)特別有用。您可以使用單個(gè) CA 對(duì)集群中的所有證書進(jìn)行簽名,并讓所有計(jì)算機(jī)共享信任該 CA 的同一信任庫(kù)。這樣,所有計(jì)算機(jī)都可以對(duì)所有其他計(jì)算機(jī)進(jìn)行身份驗(yàn)證。
- 3對(duì)證書進(jìn)行簽名
下一步是使用步驟 2 中生成的 CA 對(duì)步驟 1 生成的所有證書進(jìn)行簽名。首先,您需要從密鑰庫(kù)中導(dǎo)出證書:
keytool -密鑰庫(kù) client.truststore.jks -alias CARoot -import -file ca-certkeytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
然后與 CA 一起簽名:
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 700 -CAcreateserial -passin pass:{ca-password}
最后,您需要將 CA 的證書和簽名的證書都導(dǎo)入到密鑰庫(kù)中:
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-certkeytool -keystore server.keystore.jks -alias localhost -import -file cert-signed
參數(shù)的定義如下:
- 密鑰庫(kù):密鑰庫(kù)的位置
- ca-cert:CA的證書
- ca-key:CA的私鑰
- ca-password:CA的密碼
- cert-file:導(dǎo)出的服務(wù)器未簽名證書
- cert-signed:服務(wù)器的簽名證書
- 4配置 Kafka 代理
Kafka Broker 支持偵聽多個(gè)端口上的連接。我們需要在 server.properties 中配置以下屬性,該屬性必須具有一個(gè)或多個(gè)逗號(hào)分隔值:
如果未為代理間通信啟用 SSL(請(qǐng)參閱下文了解如何啟用它),則需要 PLAINTEXT 和 SSL 端口。
listeners=PLAINTEXT://localhost:9092,SSL://localhost:9092
代理端需要以下 SSL 配置
ssl.keystore.location=/home/lighthouse/server.keystore.jksssl.keystore.password=test1234ssl.key.password=test1234ssl.truststore.location=/home/lighthouse/server.truststore.jksssl.truststore.password=測(cè)試1234
注意:ssl.truststore.password 在技術(shù)上是可選的,但強(qiáng)烈建議使用。如果未設(shè)置密碼,則對(duì)信任庫(kù)的訪問仍然可用,但完整性檢查將被禁用。值得考慮的可選設(shè)置:
- ssl.client.auth=none(“required” => 需要客戶端身份驗(yàn)證,“requested” =>請(qǐng)求客戶端身份驗(yàn)證,沒有證書的客戶端仍然可以連接。不建議使用“requested”,因?yàn)樗峁┝隋e(cuò)誤的安全感,并且配置錯(cuò)誤的客戶端仍將成功連接。
- ssl.cipher.suites(可選)。密碼套件是身份驗(yàn)證、加密、MAC 和密鑰交換算法的命名組合,用于協(xié)商使用 TLS 或 SSL 網(wǎng)絡(luò)協(xié)議的網(wǎng)絡(luò)連接的安全設(shè)置。(默認(rèn)值為空列表)
- ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 (列出要從客戶端接受的 SSL 協(xié)議。請(qǐng)注意,SSL 已被棄用,取而代之的是 TLS,不建議在生產(chǎn)中使用 SSL)
- ssl.keystore.type=JKS
- ssl.truststore.type=JKS
- ssl.secure.random.implementation=SHA1PRNG
如果要為代理之間的通信啟用 SSL,請(qǐng)將以下內(nèi)容添加到 server.properties 文件(默認(rèn)為 PLAINTEXT)
security.inter.broker.protocol=SSL
- 5配置 Kafka 客戶端
SSL 僅支持新的 Kafka 生產(chǎn)者和使用者,不支持較舊的 API。對(duì)于生產(chǎn)者和使用者,SSL 的配置是相同的。
如果代理中不需要客戶機(jī)認(rèn)證,那么下面是一個(gè)最小配置示例:
security.protocol=SSL協(xié)議ssl.truststore.location=/var/private/ssl/client.truststore.jksssl.truststore.password=測(cè)試1234
注意:ssl.truststore.password 在技術(shù)上是可選的,但強(qiáng)烈建議使用。如果未設(shè)置密碼,則對(duì)信任庫(kù)的訪問仍然可用,但完整性檢查將被禁用。如果需要客戶機(jī)認(rèn)證,那么必須像步驟 1 中一樣創(chuàng)建密鑰庫(kù),并且還必須配置以下內(nèi)容:
ssl.keystore.location=/var/private/ssl/client.keystore.jksssl.keystore.password=test1234ssl.key.password=test1234
根據(jù)我們的要求和代理配置,可能還需要其他配置設(shè)置:
- ssl.provider(可選)。用于 SSL 連接的安全提供程序的名稱。缺省值是 JVM 的缺省安全提供程序。
- ssl.cipher.suites(可選)。密碼套件是身份驗(yàn)證、加密、MAC 和密鑰交換算法的命名組合,用于協(xié)商使用 TLS 或 SSL 網(wǎng)絡(luò)協(xié)議的網(wǎng)絡(luò)連接的安全設(shè)置。
- ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1。它應(yīng)該列出至少一個(gè)在代理端配置的協(xié)議
- ssl.truststore.type=JKS
- ssl.keystore.type=JKS
生產(chǎn)者和消費(fèi)者共同使用到的client-ssl.properties文件內(nèi)容如下:
使用 console-producer 和 console-consumer 的示例:
./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config ./config/client-ssl.properties
./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --consumer.config ./config/client-ssl.properties
報(bào)錯(cuò)了:
還要在用戶目錄下執(zhí)行如下命令,信任客戶端:
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.keystore.jks -alias CARoot -import -file ca-cert
如果密碼錯(cuò)了,還會(huì)報(bào)如下錯(cuò)誤:
lighthouse@VM-8-10-ubuntu:~/kafkaWithZk/kafka_2.12-2.2.1$ ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config ./config/client-ssl.properties
org.apache.kafka.common.KafkaException: Failed to construct kafka producerat org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:431)at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:299)at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:44)at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /home/lighthouse/client.truststore.jks of type JKSat org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:73)at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:146)at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:67)at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:99)at org.apache.kafka.clients.producer.KafkaProducer.newSender(KafkaProducer.java:439)at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:420)... 3 more
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /home/lighthouse/client.truststore.jks of type JKSat org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:144)at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:71)... 8 more
Caused by: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /home/lighthouse/client.truststore.jks of type JKSat org.apache.kafka.common.security.ssl.SslFactory$SecurityStore.load(SslFactory.java:357)at org.apache.kafka.common.security.ssl.SslFactory.createSSLContext(SslFactory.java:248)at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:141)... 9 more
Caused by: java.io.IOException: keystore password was incorrectat java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2092)at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:243)at java.base/java.security.KeyStore.load(KeyStore.java:1479)at org.apache.kafka.common.security.ssl.SslFactory$SecurityStore.load(SslFactory.java:354)... 11 more
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.... 15 more
然后我核對(duì)了client-ssl.properties文件中的配置(包含密碼),再次啟動(dòng)producer,會(huì)報(bào)如下錯(cuò):
lighthouse@VM-8-10-ubuntu:~/kafkaWithZk/kafka_2.12-2.2.1$ ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config ./config/client-ssl.properties
>[2024-03-19 13:42:49,783] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:49,835] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:49,937] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:50,140] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:50,543] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:51,298] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:52,203] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:53,158] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:54,264] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:55,220] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 13:42:56,376] WARN [Producer clientId=console-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
^C^Clighthouse@VM-8-10-ubuntu:~/kafkaWithZk/kafka_2.12-2.2.1$
核對(duì)了server.properties文件的密碼后,啟動(dòng)kafka還是報(bào)錯(cuò),報(bào)的錯(cuò)關(guān)鍵信息如下:
[2024-03-19 14:34:31,955] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2024-03-19 14:34:31,957] WARN SSL handshake failed (kafka.utils.CoreUtils$)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: No name matching localhost foundat java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:360)at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:303)at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:298)at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1076)at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1063)at java.base/java.security.AccessController.doPrivileged(Native Method)at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1010)at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)at org.apache.kafka.common.network.Selector.poll(Selector.java:483)at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)at kafka.server.KafkaServer.doControlledShutdown$1(KafkaServer.scala:510)at kafka.server.KafkaServer.controlledShutdown(KafkaServer.scala:563)at kafka.server.KafkaServer.$anonfun$shutdown$2(KafkaServer.scala:585)at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:86)at kafka.server.KafkaServer.shutdown(KafkaServer.scala:585)at kafka.server.KafkaServerStartable.shutdown(KafkaServerStartable.scala:48)at kafka.Kafka$$anon$1.run(Kafka.scala:72)
Caused by: java.security.cert.CertificateException: No name matching localhost foundat java.base/sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:234)at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:103)at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:461)at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:435)at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283)at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)... 24 more
[2024-03-19 14:34:31,957] ERROR [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:31,960] INFO [/config/changes-event-process-thread]: Shutting down (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2024-03-19 14:34:31,961] INFO [/config/changes-event-process-thread]: Shutdown completed (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2024-03-19 14:34:31,961] INFO [/config/changes-event-process-thread]: Stopped (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2024-03-19 14:34:31,962] INFO [SocketServer brokerId=0] Stopping socket server request processors (kafka.network.SocketServer)
[2024-03-19 14:34:31,979] INFO [SocketServer brokerId=0] Stopped socket server request processors (kafka.network.SocketServer)
[2024-03-19 14:34:31,980] INFO [data-plane Kafka Request Handler on Broker 0], shutting down (kafka.server.KafkaRequestHandlerPool)
[2024-03-19 14:34:31,988] INFO [data-plane Kafka Request Handler on Broker 0], shut down completely (kafka.server.KafkaRequestHandlerPool)
[2024-03-19 14:34:31,995] INFO [KafkaApi-0] Shutdown complete. (kafka.server.KafkaApis)
[2024-03-19 14:34:31,997] INFO [ExpirationReaper-0-topic]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,059] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
^C[2024-03-19 14:34:32,114] INFO Terminating process due to signal SIGINT (org.apache.kafka.common.utils.LoggingSignalHandler)
[2024-03-19 14:34:32,132] INFO [ExpirationReaper-0-topic]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,132] INFO [ExpirationReaper-0-topic]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,134] INFO [TransactionCoordinator id=0] Shutting down. (kafka.coordinator.transaction.TransactionCoordinator)
[2024-03-19 14:34:32,135] INFO [ProducerId Manager 0]: Shutdown complete: last producerId assigned 1000 (kafka.coordinator.transaction.ProducerIdManager)
[2024-03-19 14:34:32,136] INFO [Transaction State Manager 0]: Shutdown complete (kafka.coordinator.transaction.TransactionStateManager)
[2024-03-19 14:34:32,136] INFO [Transaction Marker Channel Manager 0]: Shutting down (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2024-03-19 14:34:32,139] INFO [Transaction Marker Channel Manager 0]: Stopped (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2024-03-19 14:34:32,140] INFO [Transaction Marker Channel Manager 0]: Shutdown completed (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2024-03-19 14:34:32,141] INFO [TransactionCoordinator id=0] Shutdown complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2024-03-19 14:34:32,141] INFO [GroupCoordinator 0]: Shutting down. (kafka.coordinator.group.GroupCoordinator)
[2024-03-19 14:34:32,144] INFO [ExpirationReaper-0-Heartbeat]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,160] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,261] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,344] INFO [ExpirationReaper-0-Heartbeat]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,344] INFO [ExpirationReaper-0-Heartbeat]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,344] INFO [ExpirationReaper-0-Rebalance]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,362] INFO [ExpirationReaper-0-Rebalance]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,362] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,363] INFO [ExpirationReaper-0-Rebalance]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,363] INFO [GroupCoordinator 0]: Shutdown complete. (kafka.coordinator.group.GroupCoordinator)
[2024-03-19 14:34:32,364] INFO [ReplicaManager broker=0] Shutting down (kafka.server.ReplicaManager)
[2024-03-19 14:34:32,364] INFO [LogDirFailureHandler]: Shutting down (kafka.server.ReplicaManager$LogDirFailureHandler)
[2024-03-19 14:34:32,366] INFO [LogDirFailureHandler]: Stopped (kafka.server.ReplicaManager$LogDirFailureHandler)
[2024-03-19 14:34:32,366] INFO [LogDirFailureHandler]: Shutdown completed (kafka.server.ReplicaManager$LogDirFailureHandler)
[2024-03-19 14:34:32,368] INFO [ReplicaFetcherManager on broker 0] shutting down (kafka.server.ReplicaFetcherManager)
[2024-03-19 14:34:32,369] INFO [ReplicaFetcherManager on broker 0] shutdown completed (kafka.server.ReplicaFetcherManager)
[2024-03-19 14:34:32,370] INFO [ReplicaAlterLogDirsManager on broker 0] shutting down (kafka.server.ReplicaAlterLogDirsManager)
[2024-03-19 14:34:32,370] INFO [ReplicaAlterLogDirsManager on broker 0] shutdown completed (kafka.server.ReplicaAlterLogDirsManager)
[2024-03-19 14:34:32,370] INFO [ExpirationReaper-0-Fetch]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,463] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,492] INFO [ExpirationReaper-0-Fetch]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,492] INFO [ExpirationReaper-0-Fetch]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,492] INFO [ExpirationReaper-0-Produce]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,564] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,666] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,674] INFO [ExpirationReaper-0-Produce]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,674] INFO [ExpirationReaper-0-Produce]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,674] INFO [ExpirationReaper-0-DeleteRecords]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,692] INFO [ExpirationReaper-0-DeleteRecords]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,692] INFO [ExpirationReaper-0-DeleteRecords]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,693] INFO [ExpirationReaper-0-ElectPreferredLeader]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,768] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,870] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-03-19 14:34:32,893] INFO [ExpirationReaper-0-ElectPreferredLeader]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,893] INFO [ExpirationReaper-0-ElectPreferredLeader]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-03-19 14:34:32,897] INFO [ReplicaManager broker=0] Shut down completely (kafka.server.ReplicaManager)
[2024-03-19 14:34:32,898] INFO Shutting down. (kafka.log.LogManager)
[2024-03-19 14:34:32,934] INFO Shutdown complete. (kafka.log.LogManager)
[2024-03-19 14:34:32,960] INFO [ZooKeeperClient] Closing. (kafka.zookeeper.ZooKeeperClient)
[2024-03-19 14:34:32,964] INFO Session: 0x100cd6124170002 closed (org.apache.zookeeper.ZooKeeper)
[2024-03-19 14:34:32,966] INFO EventThread shut down for session: 0x100cd6124170002 (org.apache.zookeeper.ClientCnxn)
[2024-03-19 14:34:32,966] INFO [ZooKeeperClient] Closed. (kafka.zookeeper.ZooKeeperClient)
[2024-03-19 14:34:32,968] INFO [ThrottledChannelReaper-Fetch]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:33,168] INFO [ThrottledChannelReaper-Fetch]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:33,168] INFO [ThrottledChannelReaper-Fetch]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:33,168] INFO [ThrottledChannelReaper-Produce]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:33,170] INFO [ThrottledChannelReaper-Produce]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:33,170] INFO [ThrottledChannelReaper-Produce]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:33,170] INFO [ThrottledChannelReaper-Request]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
^C[2024-03-19 14:34:33,740] INFO Terminating process due to signal SIGINT (org.apache.kafka.common.utils.LoggingSignalHandler)
^C[2024-03-19 14:34:33,972] INFO Terminating process due to signal SIGINT (org.apache.kafka.common.utils.LoggingSignalHandler)
[2024-03-19 14:34:34,170] INFO [ThrottledChannelReaper-Request]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:34,170] INFO [ThrottledChannelReaper-Request]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-03-19 14:34:34,172] INFO [SocketServer brokerId=0] Shutting down socket server (kafka.network.SocketServer)
^C[2024-03-19 14:34:34,204] INFO [SocketServer brokerId=0] Shutdown completed (kafka.network.SocketServer)
[2024-03-19 14:34:34,204] INFO Terminating process due to signal SIGINT (org.apache.kafka.common.utils.LoggingSignalHandler)
[2024-03-19 14:34:34,206] INFO [KafkaServer id=0] shut down completed (kafka.server.KafkaServer)
經(jīng)過一番清理ca-*,cert-*,client-*,server-*文件后,然后重新生成秘鑰證書和CA、簽名,步驟如下:
一、生成 SSL 密鑰和證書
keytool -keystore server.keystore.jks -alias localhost -validity 700 -genkey -keyalg RSA
keytool -keystore server.truststore.jks -alias localhost -validity 700 -genkey -keyalg RSA
keytool -keystore client.keystore.jks -alias localhost -validity 700 -genkey -keyalg RSA
keytool -keystore client.truststore.jks -alias localhost -validity 700 -genkey -keyalg RSA2、創(chuàng)建我自己的CA
openssl req -new -x509 -keyout ca-key -out ca-cert -days 700
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 700 -CAcreateserial -passin pass:1234563、對(duì)證書進(jìn)行簽名
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signedkeytool -keystore server.truststore.jks -alias CARoot -import -file ca-certkeytool -keystore client.keystore.jks -alias CARoot -import -file ca-certkeytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
再次啟動(dòng)zookeeper和kafka,然后執(zhí)行生產(chǎn)producer命令,發(fā)現(xiàn)還是報(bào)錯(cuò):
[2024-03-19 17:52:38,773] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2024-03-19 17:52:38,876] INFO [Controller id=0, targetBrokerId=0] Failed authentication with localhost/127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2024-03-19 17:52:38,876] ERROR [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2024-03-19 17:52:38,876] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2024-03-19 17:52:38,979] INFO [Controller id=0, targetBrokerId=0] Failed authentication with localhost/127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.netw
ork.Selector)
[2024-03-19 17:52:38,980] ERROR [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) failed authentication due to: SSL handshake failed(org.apache.kafka.clients.NetworkClient)
[2024-03-19 17:52:38,980] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2024-03-19 17:52:39,083] INFO [Controller id=0, targetBrokerId=0] Failed authentication with localhost/127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.netw
ork.Selector)
[2024-03-19 17:52:39,083] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2024-03-19 17:52:39,083] ERROR [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) failed authentication due to: SSL handshake failed(org.apache.kafka.clients.NetworkClient)