Redis+Caffeine 實(shí)現(xiàn)兩級緩存

背景

? 事情的開始是這樣的，前段時間接了個需求，給公司的商城官網(wǎng)提供一個查詢預(yù)計送達(dá)時間的接口。接口很簡單，根據(jù)請求傳的城市+倉庫+發(fā)貨時間查詢快遞的預(yù)計送達(dá)時間。因?yàn)樯坛窍聠尉蜁{(diào)用這個接口，所以對接口的性能要求還是挺高的，據(jù)老員工的說法是特別是大促的時候，訪問量還是比較大的。

? 因?yàn)閿?shù)據(jù)量不是很大，每天會全量推今天和明天的預(yù)計送達(dá)時間到MySQL，總數(shù)據(jù)量大約7k+。每次推完數(shù)據(jù)后會把數(shù)據(jù)全量寫入到redis中，做一個緩存預(yù)熱，然后設(shè)置過期時間為1天。

? 鑒于之前Redis集群出現(xiàn)過壓力過大查詢緩慢的情況，進(jìn)一步保證接口的高性能和高可用，防止redis出現(xiàn)壓力大，查詢慢，緩存雪崩，緩存穿透等問題，我們最終采用了Reids + Caffeine兩級緩存的策略。

本地緩存優(yōu)缺點(diǎn)

優(yōu)點(diǎn)：

1. 本地緩存，基于本地內(nèi)存，查詢速度是很快的。適用于：實(shí)時性要求不高，更新頻率不高等場景。（我們的數(shù)據(jù)每天凌晨更新一次，總量7k左右）
2. 查詢本地緩存與查詢遠(yuǎn)程緩存相比可以減少網(wǎng)絡(luò)的I/O，降低網(wǎng)絡(luò)上的一些消耗。（我們的redis之前出現(xiàn)過查詢緩慢的情況）

缺點(diǎn)：

1. Caffeine既然是本地緩存，在分布式環(huán)境的情況下就要考慮各個節(jié)點(diǎn)之間緩存的一致性問題，一個節(jié)點(diǎn)的本地緩存更新了，怎么可以同步到其他的節(jié)點(diǎn)。
2. Caffeine不支持持久化的存儲。
3. Caffeine使用本地內(nèi)存，需要合理設(shè)置大小，避免內(nèi)存溢出。

流程圖

代碼實(shí)現(xiàn)

MySQL表

CREATE TABLE `t_estimated_arrival_date`  (`id` int(11) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '主鍵id',`warehouse_id` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '貨倉id',`warehouse` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '發(fā)貨倉',`city` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '簽收城市',`delivery_date` date NULL DEFAULT NULL COMMENT '發(fā)貨時間',`estimated_arrival_date` date NULL DEFAULT NULL COMMENT '預(yù)計到貨日期',PRIMARY KEY (`id`) USING BTREE,UNIQUE INDEX `uk_warehouse_id_city_delivery_date`(`warehouse_id`, `city`, `delivery_date`) USING BTREE
) ENGINE = InnoDB  COMMENT = '預(yù)計到貨時間表(具體到day：T, T+1,近90天到貨時間眾數(shù))' ROW_FORMAT = Dynamic;INSERT INTO `t_estimated_arrival_date` VALUES (9, '6', '湖熟正常倉', '蘭州市', '2024-07-08', '2024-07-10');
INSERT INTO `t_estimated_arrival_date` VALUES (10, '6', '湖熟正常倉', '蘭州市', '2024-07-09', '2024-07-11');
INSERT INTO `t_estimated_arrival_date` VALUES (11, '6', '湖熟正常倉', '興安盟', '2024-07-08', '2024-07-11');
INSERT INTO `t_estimated_arrival_date` VALUES (12, '6', '湖熟正常倉', '興安盟', '2024-07-09', '2024-07-12');
INSERT INTO `t_estimated_arrival_date` VALUES (13, '6', '湖熟正常倉', '其他', '2024-07-08', '2024-07-19');
INSERT INTO `t_estimated_arrival_date` VALUES (14, '6', '湖熟正常倉', '其他', '2024-07-09', '2024-07-20');
INSERT INTO `t_estimated_arrival_date` VALUES (15, '6', '湖熟正常倉', '內(nèi)江市', '2024-07-08', '2024-07-10');
INSERT INTO `t_estimated_arrival_date` VALUES (16, '6', '湖熟正常倉', '內(nèi)江市', '2024-07-09', '2024-07-11');
INSERT INTO `t_estimated_arrival_date` VALUES (17, '6', '湖熟正常倉', '涼山彝族自治州', '2024-07-08', '2024-07-11');
INSERT INTO `t_estimated_arrival_date` VALUES (18, '6', '湖熟正常倉', '涼山彝族自治州', '2024-07-09', '2024-07-12');
INSERT INTO `t_estimated_arrival_date` VALUES (19, '6', '湖熟正常倉', '包頭市', '2024-07-08', '2024-07-11');
INSERT INTO `t_estimated_arrival_date` VALUES (20, '6', '湖熟正常倉', '包頭市', '2024-07-09', '2024-07-12');
INSERT INTO `t_estimated_arrival_date` VALUES (21, '6', '湖熟正常倉', '北京城區(qū)', '2024-07-08', '2024-07-10');
INSERT INTO `t_estimated_arrival_date` VALUES (22, '6', '湖熟正常倉', '北京城區(qū)', '2024-07-09', '2024-07-11');

pom.xm

        <dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-aop</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-redis</artifactId></dependency><!--redis連接池--><dependency><groupId>org.apache.commons</groupId><artifactId>commons-pool2</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-cache</artifactId></dependency><dependency><groupId>com.github.ben-manes.caffeine</groupId><artifactId>caffeine</artifactId><version>2.9.2</version></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>8.0.28</version></dependency><dependency><groupId>com.baomidou</groupId><artifactId>mybatis-plus-boot-starter</artifactId><version>3.3.1</version></dependency>

application.yml

server:port: 9001
spring:application:name: springboot-redisdatasource:name: demourl: jdbc:mysql://localhost:3306/test?userUnicode=true&&characterEncoding=utf8&allowMultiQueries=true&useSSL=falsedriver-class-name: com.mysql.cj.jdbc.Driverusername: password: # mybatis相關(guān)配置mybatis-plus:mapper-locations: classpath:mapper/*.xmlconfiguration:cache-enabled: trueuse-generated-keys: truedefault-executor-type: REUSEuse-actual-param-name: true# 打印日志#    log-impl: org.apache.ibatis.logging.stdout.StdOutImplredis:host: 192.168.117.73port: 6379password: root
#  redis:
#    lettuce:
#      cluster:
#        refresh:
#          adaptive: true
#          period: 10S
#      pool:
#        max-idle: 50
#        min-idle: 8
#        max-active: 100
#        max-wait: -1
#    timeout: 100000
#    cluster:
#      nodes:
#        - 192.168.117.73:6379
logging:level:com.itender.redis.mapper: debug

配置類

• RedisConfig

/*** @author yuanhewei* @date 2024/5/31 16:18* @description*/
@Configuration
public class RedisConfig {@Beanpublic RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory connectionFactory) {RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();redisTemplate.setConnectionFactory(connectionFactory);Jackson2JsonRedisSerializer<Object> serializer = new Jackson2JsonRedisSerializer<>(Object.class);ObjectMapper mapper = new ObjectMapper();mapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);mapper.activateDefaultTyping(LaissezFaireSubTypeValidator.instance, ObjectMapper.DefaultTyping.NON_FINAL, JsonTypeInfo.As.PROPERTY);serializer.setObjectMapper(mapper);// 如果不序列化在key value 使用redis客戶端工具 直連redis服務(wù)器 查看數(shù)據(jù)時 前面會有一個 \xac\xed\x00\x05t\x00\x05 字符串// StringRedisSerializer 來序列化和反序列化 String 類型 redis 的 key valueredisTemplate.setKeySerializer(new StringRedisSerializer());redisTemplate.setValueSerializer(serializer);// StringRedisSerializer 來序列化和反序列化 hash 類型 redis 的 key valueredisTemplate.setHashKeySerializer(new StringRedisSerializer());redisTemplate.setHashValueSerializer(serializer);redisTemplate.afterPropertiesSet();return redisTemplate;}
}

• CaffeineConfig

/*** @author yuanhewei* @date 2024/7/9 14:16* @description*/
@Configuration
public class CaffeineConfig {/*** Caffeine 配置類*  initialCapacity：初始緩存空間大小*  maximumSize：緩存的最大數(shù)量，設(shè)置這個值避免內(nèi)存溢出*  expireAfterWrite：指定緩存的過期時間，是最后一次寫操作的一個時間*  容量的大小要根據(jù)自己的實(shí)際應(yīng)用場景設(shè)置** @return*/@Beanpublic Cache<String, Object> caffeineCache() {return Caffeine.newBuilder()// 初始大小.initialCapacity(128)//最大數(shù)量.maximumSize(1024)//過期時間.expireAfterWrite(60, TimeUnit.SECONDS).build();}@Beanpublic CacheManager cacheManager(){CaffeineCacheManager cacheManager=new CaffeineCacheManager();cacheManager.setCaffeine(Caffeine.newBuilder().initialCapacity(128).maximumSize(1024).expireAfterWrite(60, TimeUnit.SECONDS));return cacheManager;}
}

Mapper

這里采用了Mybatis Plus

/*** @author yuanhewei* @date 2024/7/9 18:11* @description*/
@Mapper
public interface EstimatedArrivalDateMapper extends BaseMapper<EstimatedArrivalDateEntity> {} 

Service

/*** @author yuanhewei* @date 2024/7/9 14:25* @description*/
public interface DoubleCacheService {/*** 查詢一級送達(dá)時間-常規(guī)方式** @param request* @return*/EstimatedArrivalDateEntity getEstimatedArrivalDateCommon(EstimatedArrivalDateEntity request);/*** 查詢一級送達(dá)時間-注解方式** @param request* @return*/EstimatedArrivalDateEntity getEstimatedArrivalDate(EstimatedArrivalDateEntity request);
}

實(shí)現(xiàn)類

/*** @author yuanhewei* @date 2024/7/9 14:26* @description*/
@Slf4j
@Service
public class DoubleCacheServiceImpl implements DoubleCacheService {@Resourceprivate Cache<String, Object> caffeineCache;@Resourceprivate RedisTemplate<String, Object> redisTemplate;@Resourceprivate EstimatedArrivalDateMapper estimatedArrivalDateMapper;@Overridepublic EstimatedArrivalDateEntity getEstimatedArrivalDateCommon(EstimatedArrivalDateEntity request) {String key = request.getDeliveryDate() + RedisConstants.COLON + request.getWarehouseId() + RedisConstants.COLON + request.getCity();log.info("Cache key: {}", key);Object value = caffeineCache.getIfPresent(key);if (Objects.nonNull(value)) {log.info("get from caffeine");return EstimatedArrivalDateEntity.builder().estimatedArrivalDate(value.toString()).build();}value = redisTemplate.opsForValue().get(key);if (Objects.nonNull(value)) {log.info("get from redis");caffeineCache.put(key, value);return EstimatedArrivalDateEntity.builder().estimatedArrivalDate(value.toString()).build();}log.info("get from mysql");DateTime deliveryDate = DateUtil.parse(request.getDeliveryDate(), "yyyy-MM-dd");EstimatedArrivalDateEntity estimatedArrivalDateEntity = estimatedArrivalDateMapper.selectOne(new QueryWrapper<EstimatedArrivalDateEntity>().eq("delivery_date", deliveryDate).eq("warehouse_id", request.getWarehouseId()).eq("city", request.getCity()));redisTemplate.opsForValue().set(key, estimatedArrivalDateEntity.getEstimatedArrivalDate(), 120, TimeUnit.SECONDS);caffeineCache.put(key, estimatedArrivalDateEntity.getEstimatedArrivalDate());return EstimatedArrivalDateEntity.builder().estimatedArrivalDate(estimatedArrivalDateEntity.getEstimatedArrivalDate()).build();}@DoubleCache(cacheName = "estimatedArrivalDate", key = {"#request.deliveryDate", "#request.warehouseId", "#request.city"},type = DoubleCache.CacheType.FULL)@Overridepublic EstimatedArrivalDateEntity getEstimatedArrivalDate(EstimatedArrivalDateEntity request) {DateTime deliveryDate = DateUtil.parse(request.getDeliveryDate(), "yyyy-MM-dd");EstimatedArrivalDateEntity estimatedArrivalDateEntity = estimatedArrivalDateMapper.selectOne(new QueryWrapper<EstimatedArrivalDateEntity>().eq("delivery_date", deliveryDate).eq("warehouse_id", request.getWarehouseId()).eq("city", request.getCity()));return EstimatedArrivalDateEntity.builder().estimatedArrivalDate(estimatedArrivalDateEntity.getEstimatedArrivalDate()).build();}
}

這里的代碼本來是采用了常規(guī)的寫法，沒有采用自定義注解的方式，注解的方式是參考了后面那位大佬的文章，加以修改實(shí)現(xiàn)的。因?yàn)槲业腃acheKey可能存在多個屬性值的組合。

Annotitions

/*** @author yuanhewei* @date 2024/7/9 14:51* @description*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface DoubleCache {/*** 緩存名稱** @return*/String cacheName();/*** 緩存的key，支持springEL表達(dá)式** @return*/String[] key();/*** 過期時間，單位：秒** @return*/long expireTime() default 120;/*** 緩存類型** @return*/CacheType type() default CacheType.FULL;enum CacheType {/*** 存取*/FULL,/*** 只存*/PUT,/*** 刪除*/DELETE}
}

Aspect

/*** @author yuanhewei* @date 2024/7/9 14:51* @description*/
@Slf4j
@Component
@Aspect
public class DoubleCacheAspect {@Resourceprivate Cache<String, Object> caffeineCache;@Resourceprivate RedisTemplate<String, Object> redisTemplate;@Pointcut("@annotation(com.itender.redis.annotation.DoubleCache)")public void doubleCachePointcut() {}@Around("doubleCachePointcut()")public Object doAround(ProceedingJoinPoint point) throws Throwable {MethodSignature signature = (MethodSignature) point.getSignature();Method method = signature.getMethod();// 拼接解析springEl表達(dá)式的mapString[] paramNames = signature.getParameterNames();Object[] args = point.getArgs();TreeMap<String, Object> treeMap = new TreeMap<>();for (int i = 0; i < paramNames.length; i++) {treeMap.put(paramNames[i], args[i]);}DoubleCache annotation = method.getAnnotation(DoubleCache.class);String elResult = DoubleCacheUtil.arrayParse(Lists.newArrayList(annotation.key()), treeMap);String realKey = annotation.cacheName() + RedisConstants.COLON + elResult;// 強(qiáng)制更新if (annotation.type() == DoubleCache.CacheType.PUT) {Object object = point.proceed();redisTemplate.opsForValue().set(realKey, object, annotation.expireTime(), TimeUnit.SECONDS);caffeineCache.put(realKey, object);return object;}// 刪除else if (annotation.type() == DoubleCache.CacheType.DELETE) {redisTemplate.delete(realKey);caffeineCache.invalidate(realKey);return point.proceed();}// 讀寫，查詢CaffeineObject caffeineCacheObj = caffeineCache.getIfPresent(realKey);if (Objects.nonNull(caffeineCacheObj)) {log.info("get data from caffeine");return caffeineCacheObj;}// 查詢RedisObject redisCache = redisTemplate.opsForValue().get(realKey);if (Objects.nonNull(redisCache)) {log.info("get data from redis");caffeineCache.put(realKey, redisCache);return redisCache;}log.info("get data from database");Object object = point.proceed();if (Objects.nonNull(object)) {// 寫入Redislog.info("get data from database write to cache: {}", object);redisTemplate.opsForValue().set(realKey, object, annotation.expireTime(), TimeUnit.SECONDS);// 寫入CaffeinecaffeineCache.put(realKey, object);}return object;}
}

因?yàn)樽⒔馍系呐渲靡С諷pring的EL表達(dá)式。

public static String parse(String elString, SortedMap<String, Object> map) {elString = String.format("#{%s}", elString);// 創(chuàng)建表達(dá)式解析器ExpressionParser parser = new SpelExpressionParser();// 通過evaluationContext.setVariable可以在上下文中設(shè)定變量。EvaluationContext context = new StandardEvaluationContext();map.forEach(context::setVariable);// 解析表達(dá)式Expression expression = parser.parseExpression(elString, new TemplateParserContext());// 使用Expression.getValue()獲取表達(dá)式的值，這里傳入了Evaluation上下文return expression.getValue(context, String.class);}public static String arrayParse(List<String> elStrings, SortedMap<String, Object> map) {List<String> result = Lists.newArrayList();elStrings.forEach(elString -> {elString = String.format("#{%s}", elString);// 創(chuàng)建表達(dá)式解析器ExpressionParser parser = new SpelExpressionParser();// 通過evaluationContext.setVariable可以在上下文中設(shè)定變量。EvaluationContext context = new StandardEvaluationContext();map.forEach(context::setVariable);// 解析表達(dá)式Expression expression = parser.parseExpression(elString, new TemplateParserContext());// 使用Expression.getValue()獲取表達(dá)式的值，這里傳入了Evaluation上下文result.add(expression.getValue(context, String.class));});return String.join(RedisConstants.COLON, result);}

Controller

/*** @author yuanhewei* @date 2024/7/9 14:14* @description*/
@RestController
@RequestMapping("/doubleCache")
public class DoubleCacheController {@Resourceprivate DoubleCacheService doubleCacheService;@PostMapping("/common")public EstimatedArrivalDateEntity getEstimatedArrivalDateCommon(@RequestBody EstimatedArrivalDateEntity estimatedArrivalDate) {return doubleCacheService.getEstimatedArrivalDateCommon(estimatedArrivalDate);}@PostMapping("/annotation")public EstimatedArrivalDateEntity getEstimatedArrivalDate(@RequestBody EstimatedArrivalDateEntity estimatedArrivalDate) {return doubleCacheService.getEstimatedArrivalDate(estimatedArrivalDate);}
}

代碼中演示了Redis + Caffeine實(shí)現(xiàn)兩級緩存的方式，一種是傳統(tǒng)常規(guī)的方式，另一種是基于注解的方式實(shí)現(xiàn)的。具體實(shí)現(xiàn)可以根據(jù)自己項(xiàng)目中的實(shí)際場景。

最后的測試結(jié)果也是兩種方式都可以實(shí)現(xiàn)查詢先走一級緩存；一級緩存不存在查詢二級緩存，然后寫入一級緩存；二級緩存不存在，查詢MySQL然后寫入二級緩存，再寫入一級緩存的目的。測試結(jié)果就不貼出來了

總結(jié)

本文介紹Redis+Caffeine實(shí)現(xiàn)兩級緩存的方式。一種是常規(guī)的方式，一種的基于注解的方式。具體的實(shí)現(xiàn)可根據(jù)自己項(xiàng)目中的業(yè)務(wù)場景。

至于為什么要用Redis+Caffeine的方式，文章也提到了，目前我們Redis集群壓力還算挺大的，而且接口對RT的要求也是比較高的。有一點(diǎn)好的就是我們的數(shù)據(jù)是每天全量推一邊，總量也不大，實(shí)時性要求也不強(qiáng)。所以就很適合本地緩存的方式。

使用本地緩存也要注意設(shè)置容量的大小和過期時間，否則容易出現(xiàn)內(nèi)存溢出。

其實(shí)現(xiàn)實(shí)中很多的場景直接使用Redis就可以搞定的，沒必要硬要使用Caffeine。這里也只是簡單的介紹了最簡單基礎(chǔ)的實(shí)現(xiàn)方式。對于其他一些復(fù)雜的場景還要根據(jù)自己具體的業(yè)務(wù)進(jìn)行設(shè)計。我自己也是邊學(xué)邊用。如果有問題或者其他好的實(shí)現(xiàn)方式歡迎各位大佬評論，一起進(jìn)步！！！

參考

https://blog.csdn.net/weixin_45334346/article/details/136310010