Docker部署RocketMQ踩坑记录

搜索镜像

可以选择在官方网站 hub.docker.com 上搜索镜像;

或者在命令行界面中执行命令 docker search rocketmq 搜索镜像;

以下表格是排名靠前的镜像对比:

镜像名称

下载量Downloads

收藏数Stars

最后维护时间

说明文档

foxiswho/rocketmq

500K+

54

7个月前

有文档

rocketmqinc/rocketmq

100K+

51

2年前

无文档

没有官方的镜像,看起来 foxiswho/rocketmq 总体认可度较高,就决定使用这个镜像。

下载镜像

在命令行界面中执行以下命令:

docker pull foxiswho/rocketmq

下载完成后,可以继续在命令行界面执行 docker images 命令查看镜像是否下载成功。

关于版本的问题,有点小插曲,我以为 latest 默认是 4.8.0,但是其实结果不然,在宿主机命令行窗口执行后发现:

docker image inspect foxiswho/rocketmq:latest | findstr "VERSION"

latest 版本下载下来是 4.5.2,这一点是写Java客户端的时候才发现的。

如果使用控制台功能,可以下载另一个镜像 styletang/rocketmq-console-ng:

docker pull styletang/rocketmq-console-ng

docker-compose.yml

version: '3.5'

services:

rmqnamesrv:

image: foxiswho/rocketmq

container_name: rmqnamesrv

ports:

- 9876:9876

volumes:

- ./data/logs:/opt/logs

- ./data/store:/opt/store

command: sh mqnamesrv

networks:

rmq:

aliases:

- rmqnamesrv

rmqbroker:

image: foxiswho/rocketmq

container_name: rmqbroker

ports:

- 10909:10909

- 10911:10911

volumes:

- ./data/logs:/opt/logs

- ./data/store:/opt/store

- ./data/brokerconf/broker.conf:/etc/rocketmq/broker.conf

environment:

NAMESRV_ADDR: "rmqnamesrv:9876"

JAVA_OPTS: " -Duser.home=/opt"

JAVA_OPT_EXT: "-server -Xms128m -Xmx128m -Xmn128m"

command: sh mqbroker -c /etc/rocketmq/broker.conf

depends_on:

- rmqnamesrv

networks:

rmq:

aliases:

- rmqbroker

rmqconsole:

image: styletang/rocketmq-console-ng

container_name: rmqconsole

ports:

- 8080:8080

environment:

JAVA_OPTS: "-Drocketmq.namesrv.addr=rmqnamesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false"

depends_on:

- rmqnamesrv

networks:

rmq:

aliases:

- rmqconsole

networks:

rmq:

name: rmq

driver: bridge

参考自《基于 Docker 安装 RocketMQ》

注意本文中的文件有所改动,4.7.0 及以后 版本镜像 将 不在根据base镜像生成 server,broker 镜像,统一使用 base 镜像,两者区别只是调用的 启动文件不同。

所以, 启动容器 rmqnamesrv 对应 command: sh mqnamesrv;启动容器 rmqbroker 对应 command: sh mqbroker -c /etc/rocketmq/broker.conf;

尝试运行

此时,我们在命令行界面,运行以下命令(注意当前命令行工作目录应该是 docker-compose.yml 文件所在目录)

docker-compose up -d

比如我的命令行当前工作目录是 F:\Docker\rocketmq

但是,我执行 docker ps 时,发现只有 rmqconsole 正常启动,而 rmqnamesrv 和 rmqbroker 启动失败了。

F:\Docker\rocketmq> docker logs rmqbroker

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (Is a directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.(FileInputStream.java:138)

at java.io.FileInputStream.(FileInputStream.java:93)

at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:128)

at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (Is a directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.(FileInputStream.java:138)

at java.io.FileInputStream.(FileInputStream.java:93)

at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:128)

at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)

在本例中,Windows 宿主机的 F:\Docker\rocketmq\data\brokerconf\broker.conf 对应容器中的 /etc/rocketmq/broker.conf,但是问题是:现在前者是个文件夹,而不是文件。

broker.conf

RocketMQ Broker 需要一个配置文件,按照上面的 Compose 配置,我们需要在 ./data/brokerconf/ 目录下创建一个名为 broker.conf 的配置文件,内容如下:

# Licensed to the Apache Software Foundation (ASF) under one or more

# contributor license agreements. See the NOTICE file distributed with

# this work for additional information regarding copyright ownership.

# The ASF licenses this file to You under the Apache License, Version 2.0

# (the "License"); you may not use this file except in compliance with

# the License. You may obtain a copy of the License at

#

# http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

# 所属集群名字

brokerClusterName=DefaultCluster

# broker 名字,注意此处不同的配置文件填写的不一样,如果在 broker-a.properties 使用: broker-a,

# 在 broker-b.properties 使用: broker-b

brokerName=broker-a

# 0 表示 Master,> 0 表示 Slave

brokerId=0

# nameServer地址,分号分割

# namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876

# 启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <192.168.0.120:10909> failed

# 解决方式1 加上一句 producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP

# brokerIP1=192.168.0.253

# 在发送消息时,自动创建服务器不存在的topic,默认创建的队列数

defaultTopicQueueNums=4

# 是否允许 Broker 自动创建 Topic,建议线下开启,线上关闭 !!!这里仔细看是 false,false,false

autoCreateTopicEnable=true

# 是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭

autoCreateSubscriptionGroup=true

# Broker 对外服务的监听端口

listenPort=10911

# 删除文件时间点,默认凌晨4点

deleteWhen=04

# 文件保留时间,默认48小时

fileReservedTime=120

# commitLog 每个文件的大小默认1G

mapedFileSizeCommitLog=1073741824

# ConsumeQueue 每个文件默认存 30W 条,根据业务情况调整

mapedFileSizeConsumeQueue=300000

# destroyMapedFileIntervalForcibly=120000

# redeleteHangedFileInterval=120000

# 检测物理文件磁盘空间

diskMaxUsedSpaceRatio=88

# 存储路径

# storePathRootDir=/home/ztztdata/rocketmq-all-4.1.0-incubating/store

# commitLog 存储路径

# storePathCommitLog=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/commitlog

# 消费队列存储

# storePathConsumeQueue=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/consumequeue

# 消息索引存储路径

# storePathIndex=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/index

# checkpoint 文件存储路径

# storeCheckpoint=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/checkpoint

# abort 文件存储路径

# abortFile=/home/ztztdata/rocketmq-all-4.1.0-incubating/store/abort

# 限制的消息大小

maxMessageSize=65536

# flushCommitLogLeastPages=4

# flushConsumeQueueLeastPages=2

# flushCommitLogThoroughInterval=10000

# flushConsumeQueueThoroughInterval=60000

# Broker 的角色

# - ASYNC_MASTER 异步复制Master

# - SYNC_MASTER 同步双写Master

# - SLAVE

brokerRole=ASYNC_MASTER

# 刷盘方式

# - ASYNC_FLUSH 异步刷盘

# - SYNC_FLUSH 同步刷盘

flushDiskType=ASYNC_FLUSH

# 发消息线程池数量

# sendMessageThreadPoolNums=128

# 拉消息线程池数量

# pullMessageThreadPoolNums=128

访问控制台

接着,我在宿主机上访问 http://localhost:8080 就可以登入控制台了。

右上角有一个“切换语言”的功能,可以自由切换中文和英文;

选择某个主题时,可以向该主题发送一条消息;

发送消息时需要填写 tag、key、body:

发送成功以后,可以在消息一栏搜索到刚才发送出去的消息以及历史消息:

客户端代码

在 Maven 项目中引入依赖:

org.apache.rocketmq

rocketmq-client

4.5.2

之所以选择 4.5.2 这个版本是因为我们通过 docker exec -it rmqbroker ps -ef 命令查询到当前容器对应的 broker 版本是 4.5.2。因此,我选择了一个版本相同的rocketmq客户端。

写一个Java生产客户端:

package org.coderead.rocketmq;

import org.apache.rocketmq.client.producer.DefaultMQProducer;

import org.apache.rocketmq.client.producer.SendResult;

import org.apache.rocketmq.common.message.Message;

import org.apache.rocketmq.remoting.common.RemotingHelper;

public class SyncProducer {

public static void main(String[] args) throws Exception {

//Instantiate with a producer group name.

DefaultMQProducer producer = new

DefaultMQProducer("producer-group-1");

// Specify name server addresses.

producer.setNamesrvAddr("localhost:9876");

//Launch the instance.

producer.start();

//Create a message instance, specifying topic, tag and message body.

Message msg = new Message("broker-a" /* Topic */,

"TagA" /* Tag */,

("Hello RocketMQ.").getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */

);

//Call send message to deliver message to one of brokers.

SendResult sendResult = producer.send(msg);

System.out.printf("%s%n", sendResult);

//Shut down once the producer instance is not longer in use.

producer.shutdown();

}

}

客户端踩坑记录

★ org.apache.rocketmq.remoting.exception.RemotingTooMuchRequestException: sendDefaultImpl call timeout

Exception in thread "main" org.apache.rocketmq.remoting.exception.RemotingTooMuchRequestException: sendDefaultImpl call timeout

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:640)

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1310)

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1256)

at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:339)

at org.coderead.rocketmq.SyncProducer.main(SyncProducer.java:24)

超时问题,解决方案是给producer设置一下发送超时时间:

// 这段代码放在 producer.start() 之前

producer.setSendMsgTimeout(10000);

参考自 https://blog.csdn.net/q1335882/article/details/114842345

★ org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to <172.19.0.4:10911> failed

Exception in thread "main" org.apache.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [6433]ms, Topic: broker-a, BrokersSent: [broker-a, broker-a, broker-a]

See http://rocketmq.apache.org/docs/faq/ for further details.

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:638)

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1310)

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1256)

at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:339)

at org.coderead.rocketmq.SyncProducer.main(SyncProducer.java:25)

Caused by: org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to <172.19.0.4:10911> failed

at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:392)

at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:465)

at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:449)

at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:403)

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:831)

at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:557)

... 4 more

访问失败的问题,修改 broker.conf:

brokerIP1=10.24.99.61

这个IP填什么?在宿主机命令行输入 ipconfig,找到 以太网适配器 以太网 的 IPv4 地址,填入到 brokerIP1 中。

接着,你可以通过访问控制台的 http://localhost:8080 集群,我们可以发现地址发生了变化:

接着,我们在控制台——消息中可以查到这条消息:

消息的详细内容如下图所示:

了解更多

RocketMQ之console监控平台使用详解

RocketMQ 官方文档 简单示例

悟空单车退款时间是多久?退款需要多久?
苹果便签在哪里找