记录一次gateway HandlerStrategies.withDefaults().messageReaders() 导致的内存炸裂的问题
背景
年前出现了一次内存炸裂的生产事故。导致其他请求无法请求通过。
[boundedElastic-55] [Loggers.java:314]:Scheduler worker in group main failed with an uncaught exception
[TID:] 2023-01-18 10:40:33.189 [INFO] [boundedElastic-55] [AccessTokenGatewayFilterFactory.java:223]:ReqId:[req167400963317915899035994841384],执行时间:[3]ms
[TID:] 2023-01-18 10:40:35.280 [ERROR] [boundedElastic-55] [Loggers.java:314]:Scheduler worker in group main failed with an uncaught exception
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method) ~[na:1.8.0_342]
at java.lang.Thread.start(Thread.java:719) ~[na:1.8.0_342]
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) ~[na:1.8.0_342]
at java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603) ~[na:1.8.0_342]
at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334) ~[na:1.8.0_342]
at java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:573) ~[na:1.8.0_342]
at reactor.core.scheduler.BoundedElasticScheduler.start(BoundedElasticScheduler.java:175) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.scheduler.Schedulers.newBoundedElastic(Schedulers.java:498) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.scheduler.Schedulers.newBoundedElastic(Schedulers.java:454) ~[reactor-core-3.4.1.jar!/:3.4.1]
at org.springframework.http.codec.multipart.DefaultPartHttpMessageReader.<init>(DefaultPartHttpMessageReader.java:77) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.http.codec.support.ServerDefaultCodecsImpl.extendTypedReaders(ServerDefaultCodecsImpl.java:72) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.http.codec.support.BaseDefaultCodecs.getTypedReaders(BaseDefaultCodecs.java:275) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.http.codec.support.BaseCodecConfigurer.getReaders(BaseCodecConfigurer.java:98) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.http.codec.support.DefaultServerCodecConfigurer.getReaders(DefaultServerCodecConfigurer.java:27) ~[spring-web-5.3.2.jar!/:5.3.2]
at org.springframework.web.reactive.function.server.DefaultHandlerStrategiesBuilder.build(DefaultHandlerStrategiesBuilder.java:101) ~[spring-webflux-5.3.2.jar!/:5.3.2]
at org.springframework.web.reactive.function.server.HandlerStrategies.withDefaults(HandlerStrategies.java:89) ~[spring-webflux-5.3.2.jar!/:5.3.2]
at com.oneworld.gateway.filter.SignatureCacheBodyParamsFilter.getHttpMessageReadersWithMaxInMemorySize(SignatureCacheBodyParamsFilter.java:53) ~[classes!/:1.3.0-SNAPSHOT]
at com.oneworld.gateway.filter.SignatureCacheBodyParamsFilter.filter(SignatureCacheBodyParamsFilter.java:41) ~[classes!/:1.3.0-SNAPSHOT]
at org.springframework.cloud.gateway.handler.FilteringWebHandler$GatewayFilterAdapter.filter(FilteringWebHandler.java:137) ~[spring-cloud-gateway-server-3.0.0.jar!/:3.0.0]
at org.springframework.cloud.gateway.filter.OrderedGatewayFilter.filter(OrderedGatewayFilter.java:44) ~[spring-cloud-gateway-server-3.0.0.jar!/:3.0.0]
at org.springframework.cloud.gateway.handler.FilteringWebHandler$DefaultGatewayFilterChain.lambda$filter$0(FilteringWebHandler.java:117) ~[spring-cloud-gateway-server-3.0.0.jar!/:3.0.0]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:173) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:73) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.innerNext(FluxConcatMap.java:281) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:860) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:73) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1784) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.innerNext(FluxConcatMap.java:281) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:860) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1784) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onNext(MonoFilterWhen.java:149) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2346) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onSubscribe(MonoFilterWhen.java:112) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:448) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:98) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:44) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:270) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:228) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.request(FluxDematerialize.java:127) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:235) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onSubscribe(FluxDematerialize.java:77) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:164) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:54) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:448) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:218) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:164) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:81) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onComplete(MonoPeekTerminal.java:299) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onComplete(MonoPeekTerminal.java:299) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:148) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:73) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFilter$FilterSubscriber.onNext(FluxFilter.java:113) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:99) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.innerNext(FluxConcatMap.java:281) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:860) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1784) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFilter$FilterSubscriber.onNext(FluxFilter.java:113) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onNext(FluxPeekFuseable.java:854) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:73) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFilter$FilterSubscriber.onNext(FluxFilter.java:113) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.tryEmitScalar(FluxFlatMap.java:487) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:420) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:270) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:228) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.request(FluxPeekFuseable.java:144) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.onSubscribe(FluxFlatMap.java:370) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onSubscribe(FluxPeekFuseable.java:178) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:164) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:448) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:218) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:164) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:173) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:81) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFilter$FilterSubscriber.onComplete(FluxFilter.java:166) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onComplete(FluxPeekFuseable.java:940) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:84) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2348) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2154) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2028) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Mono.subscribe(Mono.java:4046) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:81) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoNext$NextSubscriber.onComplete(MonoNext.java:102) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFilter$FilterSubscriber.onComplete(FluxFilter.java:166) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.checkTerminated(FluxFlatMap.java:845) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:607) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.drain(FluxFlatMap.java:587) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.onComplete(FluxFlatMap.java:464) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onComplete(FluxPeekFuseable.java:277) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:292) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:228) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.request(FluxPeekFuseable.java:144) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFlatMap$FlatMapMain.onSubscribe(FluxFlatMap.java:370) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onSubscribe(FluxPeekFuseable.java:178) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:164) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1784) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onComplete(FluxDefaultIfEmpty.java:107) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:142) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:142) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxFilter$FilterSubscriber.onComplete(FluxFilter.java:166) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:269) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1785) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:328) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onNext(MonoCacheTime.java:345) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:199) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:73) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.publisher.FluxSubscribeOnCallable$CallableSubscribeOnSubscription.run(FluxSubscribeOnCallable.java:251) ~[reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) [reactor-core-3.4.1.jar!/:3.4.1]
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) [reactor-core-3.4.1.jar!/:3.4.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_342]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_342]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_342]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_342]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_342]
at java.lang.Thread.run(Thread.java:750) ~[na:1.8.0_342]
根据调用栈找到最先的代码片段
目的是为了保证在gateway某次请求不限制body大小的问题,需要从网关的策略类中拿到一些reader,并且修改当前read的读取字节的大小。
本来是一个比较简单问题。
List<HttpMessageReader<?>> HTTP_MESSAGE_READERS = HandlerStrategies.withDefaults().messageReaders();
反而在gateway使用过程中,频繁发生
java.lang.OutOfMemoryError: unable to create new native thread
根据最上一条包含com【本公司代码前缀】,找到定位到这句话。
这句本身,没有问题,但是其实是一个坑,在HandlerStrategies.withDefaults()获取默认策略过程中,
会在底层调用 编解码配置类中的readers。
其实拿这个东西,问题不大,基本上都是单例模式的拿。也造成不了内存溢出。
而是在拿的过程中,即在初始化的时候一直在new东西。
这就问题比较大了
根据调用链,我们找到
BaseDefaultCodecs:
final List<HttpMessageReader<?>> getTypedReaders() {
if (!this.registerDefaults) {
return Collections.emptyList();
}
List<HttpMessageReader<?>> readers = new ArrayList<>();
addCodec(readers, new DecoderHttpMessageReader<>(new ByteArrayDecoder()));
addCodec(readers, new DecoderHttpMessageReader<>(new ByteBufferDecoder()));
addCodec(readers, new DecoderHttpMessageReader<>(new DataBufferDecoder()));
if (nettyByteBufPresent) {
addCodec(readers, new DecoderHttpMessageReader<>(new NettyByteBufDecoder()));
}
addCodec(readers, new ResourceHttpMessageReader(new ResourceDecoder()));
addCodec(readers, new DecoderHttpMessageReader<>(StringDecoder.textPlainOnly()));
if (protobufPresent) {
addCodec(readers, new DecoderHttpMessageReader<>(this.protobufDecoder != null ?
(ProtobufDecoder) this.protobufDecoder : new ProtobufDecoder()));
}
addCodec(readers, new FormHttpMessageReader());
// client vs server..
extendTypedReaders(readers);
return readers;
}
中的 extendTypedReaders方法。
class ServerDefaultCodecsImpl:
protected void extendTypedReaders(List<HttpMessageReader<?>> typedReaders) {
if (this.multipartReader != null) {
addCodec(typedReaders, this.multipartReader);
return;
}
DefaultPartHttpMessageReader partReader = new DefaultPartHttpMessageReader();
addCodec(typedReaders, partReader);
addCodec(typedReaders, new MultipartHttpMessageReader(partReader));
}
此时就会new DefaultPartHttpMessageReader();
class DefaultPartHttpMessageReader:
private Scheduler blockingOperationScheduler = Schedulers.newBoundedElastic(Schedulers.DEFAULT_BOUNDED_ELASTIC_SIZE,
Schedulers.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, IDENTIFIER, 60, true);
最后可以看到这个类对象内部维护了一个内部对象变量,这个变量就是创建一个线程。
用完之后不会释放。也没有释放的入口
以上就导致了内存溢出。
解决方案
在filter创建过程中,利用@SmartInitializingSingleton注解,初始化一个对象,这样不用每次调用都创建一个新的对象。全局维护一个
SmartInitializingSingleton 生命周期在spring bean容器初始化之后,但是在结束spring完成之前的一步小操作
1、 此时也不用担心应用在调用该类的时候造成空指针问题
2、也确保调用的方法中的实例对象都已经被初始了。
上图就是优化之后的流程