应该是我们比较常用的了吧,不管是缓存,还是消息队列,或者是快速查询的短存储,都经常用到。这篇主要是进行在K8S上进行Redis的主从部署。不讲太多了,大部分都是已经写好的配置文件,直接奉上。
首先创建Redis-PV,打算创建三个节点的主从,配置如下
redis-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-redis-sentinel-0
namespace: database
spec:
capacity:
storage: 4Gi
accessModes:
- ReadWriteMany
volumeMode: Filesystem
persistentVolumeReclaimPolicy: Recycle
storageClassName: "redis-sentinel-storage-class"
nfs:
# real share directory
path: /data/nfs/redis-cluster/0
# nfs real ip
server: nfs ip
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-redis-sentinel-1
namespace: database
spec:
capacity:
storage: 4Gi
accessModes:
- ReadWriteMany
volumeMode: Filesystem
persistentVolumeReclaimPolicy: Recycle
storageClassName: "redis-sentinel-storage-class"
nfs:
# real share directory
path: /data/nfs/redis-cluster/1
# nfs real ip
server: nfs ip
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-redis-sentinel-2
namespace: database
spec:
capacity:
storage: 4Gi
accessModes:
- ReadWriteMany
volumeMode: Filesystem
persistentVolumeReclaimPolicy: Recycle
storageClassName: "redis-sentinel-storage-class"
nfs:
# real share directory
path: /data/nfs/redis-cluster/2
# nfs real ip
server: nfs ip
依旧是用了NFS
kubectl apply -f redis-pv.yaml
这里部署Redis Config,直接上文件
redis-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: redis-sentinel-config
namespace: database
labels:
addonmanager.kubernetes.io/mode: Reconcile
data:
redis-master.conf: |
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel notice
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/
slave-serve-stale-data yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 64mb 16mb 60
hz 10
aof-rewrite-incremental-fsync yes
redis-slave.conf: |
port 6379
slaveof redis-sentinel-master-ss-0.redis-sentinel-master-ss.database.svc.cluster.local 6379
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel notice
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 64mb 16mb 60
hz 10
aof-rewrite-incremental-fsync yes
redis-sentinel.conf: |
port 26379
dir /data
sentinel monitor mymaster redis-sentinel-master-ss-0.redis-sentinel-master-ss.database.svc.cluster.local 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
Redis的相关配置。
kubectl apply -f redis-config.yaml
这里我们来创建Redis的服务,因为是要主从服务,创建方式分为Master,Slave
redis-master-service.yaml
kind: Service
apiVersion: v1
metadata:
labels:
app: redis-sentinel-master-ss
name: redis-sentinel-master-ss
namespace: database
spec:
clusterIP: None
ports:
- name: redis
port: 6379
targetPort: 6379
selector:
app: redis-sentinel-master-ss
redis-slave-service.yaml
kind: Service
apiVersion: v1
metadata:
labels:
app: redis-sentinel-slave-ss
name: redis-sentinel-slave-ss
namespace: database
spec:
clusterIP: None
ports:
- name: redis
port: 6379
targetPort: 6379
selector:
app: redis-sentinel-slave-ss
创建执行
kubectl apply -f redis-master-service.yaml -f redis-slave-service.yaml
熟悉Redis的持久化方式的就清楚rbac了,不多说了,直接上配置
redis-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: redis-sentinel
namespace: database
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: redis-sentinel
namespace: database
rules:
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: redis-sentinel
namespace: database
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: redis-sentinel
subjects:
- kind: ServiceAccount
name: redis-sentinel
namespace: database
执行
kubectl apply -f redis-rbac.yaml
关键主从主体了。
redis-sfs.yaml
kind: StatefulSet
apiVersion: apps/v1
metadata:
labels:
app: redis-sentinel-master-ss
name: redis-sentinel-master-ss
namespace: database
spec:
replicas: 1
selector:
matchLabels:
app: redis-sentinel-master-ss
serviceName: redis-sentinel-master-ss
template:
metadata:
labels:
app: redis-sentinel-master-ss
spec:
containers:
- args:
- -c
- cp /mnt/redis-master.conf /data/ ; redis-server /data/redis-master.conf
command:
- sh
image: redis
imagePullPolicy: IfNotPresent
name: redis-master
ports:
- containerPort: 6379
name: masterport
protocol: TCP
volumeMounts:
- mountPath: /mnt/
name: config-volume
readOnly: false
- mountPath: /data/
name: redis-sentinel-master-storage
readOnly: false
serviceAccountName: redis-sentinel
terminationGracePeriodSeconds: 30
volumes:
- configMap:
items:
- key: redis-master.conf
path: redis-master.conf
name: redis-sentinel-config
name: config-volume
volumeClaimTemplates:
- metadata:
name: redis-sentinel-master-storage
spec:
accessModes:
- ReadWriteMany
storageClassName: "redis-sentinel-storage-class"
resources:
requests:
storage: 4Gi
redis-slave-sfs.yaml
kind: StatefulSet
apiVersion: apps/v1
metadata:
labels:
app: redis-sentinel-slave-ss
name: redis-sentinel-slave-ss
namespace: database
spec:
replicas: 2
selector:
matchLabels:
app: redis-sentinel-slave-ss
serviceName: redis-sentinel-slave-ss
template:
metadata:
labels:
app: redis-sentinel-slave-ss
spec:
containers:
- args:
- -c
- cp /mnt/redis-slave.conf /data/ ; redis-server /data/redis-slave.conf
command:
- sh
image: redis
imagePullPolicy: IfNotPresent
name: redis-slave
ports:
- containerPort: 6379
name: slaveport
protocol: TCP
volumeMounts:
- mountPath: /mnt/
name: config-volume
readOnly: false
- mountPath: /data/
name: redis-sentinel-slave-storage
readOnly: false
serviceAccountName: redis-sentinel
terminationGracePeriodSeconds: 30
volumes:
- configMap:
items:
- key: redis-slave.conf
path: redis-slave.conf
name: redis-sentinel-config
name: config-volume
volumeClaimTemplates:
- metadata:
name: redis-sentinel-slave-storage
spec:
accessModes:
- ReadWriteMany
storageClassName: "redis-sentinel-storage-class"
resources:
requests:
storage: 4Gi
执行
kubectl apply -f redis-master-sfs.yaml -f redis-slave-sfs.yaml
验证一下主从是否成功
kubectl exec -ti redis-sentinel-slave-ss-1 -n database -- redis-cli -h redis-sentinel-master-ss-0.redis-sentinel-master-ss.database.svc.cluster.local info replication
验证信息如下
# Replication
role:master
connected_slaves:2
slave0:ip=179.20.2.29,port=6379,state=online,offset=179994198,lag=1
slave1:ip=179.20.1.16,port=6379,state=online,offset=179994501,lag=0
master_replid:0485ebc3da868283b253f584a02775cca2e140a6
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:179994501
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:178945926
repl_backlog_histlen:1048576
查看nfs是否有信息文件,完成创建