Background #
在配置 NCS55 Egress Qos 的时候出现如下的报错:
interface GigabitEthernet0/0/0/111.123
service-policy output test-001-out
!!% 'DPA' detected the 'warning' condition 'SDK - No resources for operation'
!
end
我尝试去理解是哪一种资源不够用的时候，发现了一个 NCS55 中比较少见的资源 rate-profiles, 研究了下它是如何被消耗掉.
Rate-Profiles 定义 #
关于官方的解释如下:
A maximum value of up to 64 unique queues is supported. Each unique queue-limit consumes one rate profile in the Traffic manager. Out of 64 unique queues, few are reserved for default configs and the remaining are usable.
Modular QoS Configuration Guide for Cisco NCS 5500 Series Routers
- 64/per NPU
- 5 or 6 profiles been reserved.
- unique queue-limit will consume 1 rate-profile.
如何去理解 unique queue-limit， 这里就要引入 GSR（Guaranteed service rate）.
The bandwidth or a shaper configuration is creating a GSR (Guaranteed service rate). Per default 10% of GSR will be taken as a queue-limit.
所以字面意思，我们可以将具有相同值的 queue buffer size 认为消耗 1 个 rate-profile （并不是相同的 GSR 会被算做消耗一个 rate-profile）。
记录一下一个 Buffer size 的计算方式：
Guaranteed service rate = (100Gbps – shape rate) * remaining bandwidth = 50 000 000 kbps * 1% = 500 000 kbps.
Buffer size = ( 500 000 kbps / 8 ) * 10ms / 1000 = 625 KB
policy-map MC_out
class TC_6
priority level 1
shape average percent 50
!
<snip>
!
class TC_1
bandwidth remaining percent 1
!
class class-default
bandwidth remaining percent 1
!
end-policy-map
RP/0/RP0/CPU0:CORE1-NCS5516-A#show qos interface hundredGigE 0/1/0/34 output
Level1 Class = class-default
Egressq Queue ID = 1744 (Default LP queue)
Queue Max. BW. = 101803495 kbps (default)
Queue Min. BW. = 0 kbps (default)
Inverse Weight / Weight = 60 / (1%)
Guaranteed service rate = 549450 kbps
TailDrop Threshold = 688128 bytes / 10 ms (10 ms)
Rate-Profiles 的计算 #
默认情况 queue limit 为 10ms.
用 test-001-child-out 为例子:
默认 queue-limit 为 10ms， 但是因为带宽不一样， 会导致 GSR 不一样，这样算出来的 queue-limit 的 buffer 就会不一样，在以下事例中消耗的 rate-profile 为 5 个.
| Cfg | Command output |
|---|---|
| policy-map test-child-out class test-out-cos5 shape average percent 10 priority level 1 ! class test-out-cos4 bandwidth percent 20 ! class test-out-cos3 bandwidth percent 20 ! class test-out-cos2 bandwidth percent 10 ! class test-out-cos1 bandwidth percent 5 ! class class-default bandwidth percent 25 ! end-policy-map |
RP/0/RP0/CPU0:NCS-57C3-C#show qos interface twentyFiveGigE 0/0/0/0 output Mon May 13 06:20:20.259 UTC NOTE:- Configured values are displayed within parentheses Interface TwentyFiveGigE0/0/0/0 ifh 0x328 – output policy NPU Id: 0 Total number of classes: 7 Interface Bandwidth: 25000000 kbps Policy Name: test-out SPI Id: 0x0 VOQ Base: 1024 PFC enabled: 0 Accounting Type: Layer1 (Include Layer 1 encapsulation and above) —————————————————————————— Level1 Class = class-default Queue Max. BW. = 2999936 kbps (150000000 bits/sec) Queue Min. BW. = 2999936 kbps (default) Inverse Weight / Weight = 1 / (BWR not configured) Level2 Class (HP1) = test-out-cos5 Egressq Queue ID = 1029 (HP1 queue) Queue Max. BW. = 15625 kbps (10 %) Guaranteed service rate = 15000 kbps Peak burst = 36864 bytes (default) TailDrop Threshold = 18688 bytes / 10 ms (default) < < < < < < < < WRED not configured for this class Level2 Class = test-out-cos4 Egressq Queue ID = 1028 (LP queue) Queue Max. BW. = no max (default) Queue Min. BW. = 46875 kbps (30 %) Inverse Weight / Weight = 1 / (BWR not configured) Guaranteed service rate = 45000 kbps Peak burst = 36864 bytes (default) TailDrop Threshold = 56064 bytes / 10 ms (default) < < < < < < < < WRED not configured for this class Level2 Class = test-out-cos3 Egressq Queue ID = 1027 (LP queue) Queue Max. BW. = no max (default) Queue Min. BW. = 31250 kbps (20 %) Inverse Weight / Weight = 1 / (BWR not configured) Guaranteed service rate = 30000 kbps Peak burst = 36864 bytes (default) TailDrop Threshold = 37376 bytes / 10 ms (default)< < < < < < < < WRED not configured for this class Level2 Class = test-out-cos2 Egressq Queue ID = 1026 (LP queue) Queue Max. BW. = no max (default) Queue Min. BW. = 15625 kbps (10 %) Inverse Weight / Weight = 1 / (BWR not configured) Guaranteed service rate = 15000 kbps Peak burst = 36864 bytes (default) TailDrop Threshold = 18688 bytes / 10 ms (default) < < < < < < < < WRED not configured for this class Level2 Class = test-out-cos1 Egressq Queue ID = 1025 (LP queue) Queue Max. BW. = no max (default) Queue Min. BW. = 7812 kbps (5 %) Inverse Weight / Weight = 1 / (BWR not configured) Guaranteed service rate = 7500 kbps Peak burst = 36864 bytes (default) TailDrop Threshold = 9216 bytes / 10 ms (default) < < < < < < < < WRED not configured for this class Level2 Class = class-default Egressq Queue ID = 1024 (Default LP queue) Queue Max. BW. = no max (default) Queue Min. BW. = 39062 kbps (25 %) Inverse Weight / Weight = 1 / (BWR not configured) Guaranteed service rate = 37500 kbps Peak burst = 36864 bytes (default) TailDrop Threshold = 46848 bytes / 10 ms (default) < < < < < < < < WRED not configured for this class |
使用下面命令输出芯片 programming 信息， 过滤“uint32_t rate_profile_id” 可以查看目前我们的 proflile 数量。
RP/0/RP0/CPU0:NCS#attach location 0/0/cPU0
Mon May 13 06:42:49.674 UTC
Last login: Sat Jan 27 22:10:32 2024 from 172.0.17.1
export PS1='#'
[xr-vm_node0_0_CPU0:~]$export PS1='#'
#dpa_tmrateprofile_show_client -D
上述命令等同于 show dpa objects tmrateprofile location 0/0/CPU0
另外从 7.11.2 开始新增加命令查看该 profile 的实用情况:
show controllers npu resources qos rate-class-profile instance all location all
========================================================================
QoS TM Rate Profile Information:
Note: - This is a global resource and shared across all LCs
Used( %) Free Total Reserved Highest Used
59(100%) 0 59 5 59
如何减少 Rate-Profiles 的数量 #
You can avoid exhausting rate profiles by configuring queue limits in absolute units (such as bytes, kilobytes, or megabytes) for class maps and sharing these limits with the policy maps.
根据官方的 Guide 提示，我们可以配置固定大小的 buffer size，如下这种配置情况，只会使用一个 rate-profile。
policy-map test-child-out
class test-out-cos5
priority level 1
shape average percent 20
queue-limit 18688 bytes
!
class test-out-cos4
bandwidth percent 20
queue-limit 18688 bytes
!
class test-out-cos3
bandwidth percent 20
queue-limit 18688 bytes
!
class test-out-cos2
bandwidth percent 20
queue-limit 18688 bytes
!
class test-out-cos1
bandwidth percent 5
queue-limit 18688 bytes
!
class class-default
bandwidth percent 15
queue-limit 18688 bytes
!
end-policy-map
!