1、服务连kafka集群报错:Group coordinator xxx.xxx.xxx.xxx:9093 (id: 2147383250 rack: null) is unavailable or invalid 客户端的日志:
完整日志信息: [2021-12-10 14:10:49.244][INFO][promotionEventConsumer-0-C-1][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][[,,,]][TID: N/A] - [Consumer clientId=consumer-4, groupId=promotionEventGroup] Group coordinator xxx.xxx.xxx.xxx:9093 (id: 2147383250 rack: null) is unavailable or invalid, will attempt rediscovery
服务端的日志: /data/kafka_data/logs/server.log.9:[2021-12-10 09:15:17,785] ERROR [KafkaApi-100397] Error when handling request {group_id=ckafka-65eo4z4v#promotionEventGroup,session_timeout=10000,rebalance_timeout=300000,member_id=,protocol_type=consumer,group_protocols=[{protocol_name=range,protocol_metadata=java.nio.HeapByteBuffer[pos=0 lim=31 cap=31]}]} from xxx.xxx.xxx.xxx:13000-xxx.xxx.xxx.xxx:57854-1756241 (kafka.server.KafkaApis)
消费组超限这种情况有两个解决办法: 1.调高消费组配额 2.删除多余的消费组,避免大于限制额度
2、消费Kafka消息时报InconsistentCroupProtocolError的原因说明
具体错误是这样:InconsistentCroupProtocolError: JoinGroupResponse_v2
原因是,用户不同的消费客户端(python 和 go) 加入了同一个 group.id 导致 JoinGroup协议 不一致了
3、flink向ckafka生产消息报This message has failed its CRC checksum的解决方案
flink 向kafka写消息 报错:
2021-12-15 14:14:48,066 ERROR [kafka-producer-network-thread | dwd_dt_video_start_hi] com.tencent.datahub.blade.flink.component.platform.kafka.FlinkKafkaProducer - Error while sending record to Kafka: This message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt. org.apache.kafka.common.errors.CorruptRecordException: This message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt.
compression.type 强制设置为none后解决了
4、springboot、spring-kafka、kafka-client三者兼容性
详情参考官网:https://spring.io/projects/spring-kafka