Network equipment vendors, especially Cisco, are making a variety of hardware platform and thus they are having a hard time keeping uniform OS across the board. Here are a few articles that I read lately -
1. Michael Morris
http://www.networkworld.com/community/node/42294
2. Q&A with Scott Kriens (It's the operating system, stupid!)
http://www.networkworld.com/news/2008/013008-juniper-kriens.html
3. IOS vs JUNOS
http://features.techworld.com/storage/4066/ciscos-ios-vs-junipers-junos/?print
Excerpts from 1.
Back when I worked in network operations at AT&T, I was (strangely) proud that I knew all the different versions, feature sets, and varieties of Cisco IOS. It was one of the things I could use quickly when dealing with bugs and outages. However, now, all the different versions of IOS are making me sick. Especially now being a manager having to deal with all of them.
Let's take a short stroll down IOS lane...
Best I can tell there are nine varieties of IOS:
1. Regular IOS - the old warhorse running the routers.
2. IOS for 6500s and 7600s - this is different than "regular IOS" since it has specific versions and release tracks. (12.2(33)SXI is the latest strain.)
3. Modular IOS for 6500s and 7600s - same as above, but now "modular".
4. IOS for 3750 and 3550s - again, IOS-like but with a completely different release model and versions.
5. IOS XE for the ASR line - an abstracted version of IOS that runs the IOS shell in a process along with all other hardware functions.
6. IOS XR for the BFRs
7. NX-OS for the new Nexus line of switches - based on SAN-OS, but used in Ethernet switches. It actual has two different lines - one for the Nexus 7000 and another for the Nexus 5000.
8. Cisco IOS for ASAs
9. IOS for CSM, ACE, and other service blades
So, that's nine different varieties of IOS. Then, of course, you can divide each line into its various versions. In good old IOS there's the ISRs, the 7200s, and 7300s. For each hardware line there's a version line - 12.1, 12.2, 12.3, and 12.4 (or vice-versa, for each version line there is a hardware line). Inside the versions there's mainline, T-code, and special releases. There are also qualifiers like FCS, LD, GA, and GD...but, last year, those went away and now we have MD.
So, just for regular IOS, let's do some math:
4 versions * 3 hardware lines * 3 release modes * 5 qualifiers = 180 different versions
AHHHHHH!
Oh, wait, I forgot the feature sets. You know, the very clear and simple to understand different capability varieties you can order - Enterprise, Advanced Enterprise, Advanced IP Services, IP Services, Advanced Security, IP Voice, plus crypto versions. Probably a good 10 different varieties here. That brings us to 1,800 different versions....just for "regular IOS".
Thursday, October 1, 2009
Thursday, June 18, 2009
OSPF Reference Bandwidth in NX-OS (4.1(5))
To control how Open Shortest Path First (OSPF) calculates default metrics for an interface, use the auto-cost command. To assign the default reference bandwidth of 40Gb/s, use the no form of this command.
auto-cost reference-bandwidth bandwidth [Gbps | Mbps]
Defaults
40 Gb/s. The bandwidth defaults to Gb/s if you do not specify the Gpbs or Mbps keyword.
I wonder why Cisco is leaning towards 40G instead of 100G.
One would think that Cisco will sell 40G linecards to the customer and .... a few months later the same customer has to buy 100G linecards.
auto-cost reference-bandwidth bandwidth [Gbps | Mbps]
Defaults
40 Gb/s. The bandwidth defaults to Gb/s if you do not specify the Gpbs or Mbps keyword.
I wonder why Cisco is leaning towards 40G instead of 100G.
One would think that Cisco will sell 40G linecards to the customer and .... a few months later the same customer has to buy 100G linecards.
Sunday, June 7, 2009
14th is a sweet number
I think Roger will regain No.1 ranking before the end of the year.
Friday, June 5, 2009
Nexus 7000 Notes - 4 (from test lab)
"write mem" (or any write command for that matter) is gone.
"copy run star" take approx 15 seconds to write the config and idle for another 35 seconds (exactly) before returning to the command. (have to figure out what is going on during that time)
Nexus no longer supports PagP. It is either LACP or EtherChannel.
OSPF process can either be a number or a name i.e. supports alpha-numeric OSPF process ID. OSPF network statements are now configured per interface basis instead of in router configuration. Not sure if it a good thing since OSPF network statements are scattered all over the places.
"show process cpu history" is gone.
"show process cpu" shows unfamiliar processes (looks like link processes). My co-worker says Cisco fixed high CPU problem by not showing the numbers to the users - at least with a familiar command.
"copy run star" take approx 15 seconds to write the config and idle for another 35 seconds (exactly) before returning to the command. (have to figure out what is going on during that time)
Nexus no longer supports PagP. It is either LACP or EtherChannel.
OSPF process can either be a number or a name i.e. supports alpha-numeric OSPF process ID. OSPF network statements are now configured per interface basis instead of in router configuration. Not sure if it a good thing since OSPF network statements are scattered all over the places.
"show process cpu history" is gone.
"show process cpu" shows unfamiliar processes (looks like link processes). My co-worker says Cisco fixed high CPU problem by not showing the numbers to the users - at least with a familiar command.
Monday, May 25, 2009
Nexus 7000 Notes - 3 (commands - this list will grow)
(It seems like command reference for NX-OS 4.1 on CCO is not completed. All the commands with * are missing - not that they look like undocumented commands.).
show hardware fabric-utilization timestamp *
- gives peak fabric utilization time stamp. In 6500, timestamp option was not available.
show hardware capacity *
- gives a lot of useful information including the command above
show port-channel traffic
- gives utilization of each link in a port channel
show license host-id
- do not forget when activating license !!
show hardware fabric-utilization timestamp *
- gives peak fabric utilization time stamp. In 6500, timestamp option was not available.
show hardware capacity *
- gives a lot of useful information including the command above
show port-channel traffic
- gives utilization of each link in a port channel
show license host-id
- do not forget when activating license !!
Nexus 7000 Notes - 2 (Linecards)
Nexus linecards come with an integrated M series engine. The first generation for forwarding engine is referred to as M1 forwarding engine. M1 can deliver 60Mbbps of L2 and L3 IPv4 unicast forwarding. (30Mpps of IPv6) across all ports on a single linecard. 10-slot chassis with 8 M1 forwarding engines delivers up to 480 Mpps of IPv4 unicast forwarding. (Sup720 claims up to 400Mpps).
32 ports 10G card (N7K-M132XP-12)
Part number can be dissected as follows -
N7K - for Nexus 7000
M1 - forwarding engine
32 - number of ports
X - - port speed (in Roman ?)
P - ??
1 - fabric version
2 - number of fabric required for full bandwidth (w/o redundancy)
Each fabric connection is 40Gpbs (dCEF720 has 20Gbps) with maximum of 2 fabric connectors. N7K-M132XP-12 has 2 fabric connections (i.e 80Gbps bandwidth to fabric) and makes it 4:1 oversubscription for a fully populated line card. The card can be run in non-oversubscribed mode by dedicating fabric access to 1 port in each port block. First port in each port block of 4 will become dedicated port - port 1,2,9,10,17,18,24,25.
"rate-mode dedicated" interface command is used to make the first port in port group in dedicated mode - the rest of the ports in port block are disabled.
48 ports 1G card (N7K-M148GS-11 or N7K-M148GT-11)
N7K - for Nexus 7000
M1 - forwarding engine
48 - number of ports
G - - port speed (in Roman ?)
S/T - S for SFP and T for copper ?
1 - fabric version
1 - number of fabric required for full bandwidth (w/o redundancy)
48 ports linecard has 40Gbps fabric access and thus it is 1:1.2 oversubscribed.
32 ports 10G card (N7K-M132XP-12)
Part number can be dissected as follows -
N7K - for Nexus 7000
M1 - forwarding engine
32 - number of ports
X - - port speed (in Roman ?)
P - ??
1 - fabric version
2 - number of fabric required for full bandwidth (w/o redundancy)
Each fabric connection is 40Gpbs (dCEF720 has 20Gbps) with maximum of 2 fabric connectors. N7K-M132XP-12 has 2 fabric connections (i.e 80Gbps bandwidth to fabric) and makes it 4:1 oversubscription for a fully populated line card. The card can be run in non-oversubscribed mode by dedicating fabric access to 1 port in each port block. First port in each port block of 4 will become dedicated port - port 1,2,9,10,17,18,24,25.
"rate-mode dedicated" interface command is used to make the first port in port group in dedicated mode - the rest of the ports in port block are disabled.
48 ports 1G card (N7K-M148GS-11 or N7K-M148GT-11)
N7K - for Nexus 7000
M1 - forwarding engine
48 - number of ports
G - - port speed (in Roman ?)
S/T - S for SFP and T for copper ?
1 - fabric version
1 - number of fabric required for full bandwidth (w/o redundancy)
48 ports linecard has 40Gbps fabric access and thus it is 1:1.2 oversubscribed.
Nexus 7000 Notes - 1
I just got my hands on Nexus 7010 last week.
First thing that I noticed is that the box is very deep. Its back is protruding towards the aisle between data center rows.
It is 33.1" deep (as opposed to 6509's 18.2").
Nexus decouples fabric from the supervisor and the fabric is scalable (can be upgraded up to 5 fabric modules). Fabric cards are inserted from the back.
It has front-bottom to read-top air flow.
There are -
- 2 fan tray for supervisor and linecards (6 fans in each tray)
- 2 fan for fabric.
All the fans are hot-swappable.
Cisco claims first generation fabric linecard (N7K-C7010-FAB-1) can forward 46Gpbs - thus fully populated 5 fabric cards can forward up to 46 x 5 = 230Gbps.
Currently shipping supervisor 1 bandwidth is 115Gpbs/slot and the bandwidth of I/O linecard is 230Gbps.
According to Cisco maths, 7010 has maximum bandwidth of -
230Gbps / slot x 8 slots = 1840Gbps (I/O linecard)
115Gbps / slot x 2 slots = 230Gbps (sup)
(1840 + 230) x 2 (for full duplex operations) = 4.1 Tbps system.
(6500 with sup-720 is, as it name applies, a 720Gbps system.)
First thing that I noticed is that the box is very deep. Its back is protruding towards the aisle between data center rows.
It is 33.1" deep (as opposed to 6509's 18.2").
Nexus decouples fabric from the supervisor and the fabric is scalable (can be upgraded up to 5 fabric modules). Fabric cards are inserted from the back.
It has front-bottom to read-top air flow.
There are -
- 2 fan tray for supervisor and linecards (6 fans in each tray)
- 2 fan for fabric.
All the fans are hot-swappable.
Cisco claims first generation fabric linecard (N7K-C7010-FAB-1) can forward 46Gpbs - thus fully populated 5 fabric cards can forward up to 46 x 5 = 230Gbps.
Currently shipping supervisor 1 bandwidth is 115Gpbs/slot and the bandwidth of I/O linecard is 230Gbps.
According to Cisco maths, 7010 has maximum bandwidth of -
230Gbps / slot x 8 slots = 1840Gbps (I/O linecard)
115Gbps / slot x 2 slots = 230Gbps (sup)
(1840 + 230) x 2 (for full duplex operations) = 4.1 Tbps system.
(6500 with sup-720 is, as it name applies, a 720Gbps system.)
Sunday, May 10, 2009
Buffers, Queues and Thresholds
When QoS is enabled on a 65xx switch, queues are automatically allocated based on architecture of the line card.
For example -
1p3q8t for 6748 and 6724 (10/100/1000 linecard)
1p7q8t for 6704 (4 ports 10G card)
1p7q4t for 6708 (8 ports 10G card)
Queue size, numbers and architecture are different based on line card.
Here is the detail list as of 2009.
show queueing interface command will give away a lot of information about the port and the linecard.
Queue configuration is applied to a block of ports per ASIC (Rohini on 6724 and 6748). For 6724, wrr algorithm and qos-map configuration applied on one port will affect all 12 ports of the same ASIC, and for 6748 all 8 ports of the same ASIC.
"default interface" on one of the ports will reset wrr allocation of queues but not the qos-map, if it were altered from default *
For example -
1p3q8t for 6748 and 6724 (10/100/1000 linecard)
1p7q8t for 6704 (4 ports 10G card)
1p7q4t for 6708 (8 ports 10G card)
Queue size, numbers and architecture are different based on line card.
Here is the detail list as of 2009.
show queueing interface command will give away a lot of information about the port and the linecard.
Queue configuration is applied to a block of ports per ASIC (Rohini on 6724 and 6748). For 6724, wrr algorithm and qos-map configuration applied on one port will affect all 12 ports of the same ASIC, and for 6748 all 8 ports of the same ASIC.
"default interface" on one of the ports will reset wrr allocation of queues but not the qos-map, if it were altered from default *
Wednesday, March 4, 2009
Beyond STP
Several enhancements were made to the layer 2 loop prevention mechanism, the original STP (IEEE 802.1d). Examples are multiple spanning tree, rapid spanning tree.
One of the major drawbacks of spanning tree is - it prohibits the use of multiple paths in layer 2. After BPDU exchange, STP will put redundant ports in blocking mode.
Earlier solutions to disable SPT and to take advantage of multiple layer 2 paths were the use of PortChannel - FEC or GEC (Proprietary), PagP (Proprietary) and LACP (802.3ad, later 802.3AX).
Biggest limitation of PortChannel is that it can only be used between two adjacent devices.
Cisco came up with vPC (virtual PortChannel) in its Nexus platform (NX-OS) which will let multiple active layer 2 paths exist. See here for detail explanation.
vPC is Cisco solution before a standard based technology is agreed upon. That technology is in work in progress for a few years (since 2007 I think) and it has a weird sounding acronym. It is IETF's TRILL - Transparent Interconnection of Lots of Links. TRILL uses IS-IS (yes that L3 routing protocol) to manage L2MP management traffic.
L2MP - Layer 2 MultiPath
One of the major drawbacks of spanning tree is - it prohibits the use of multiple paths in layer 2. After BPDU exchange, STP will put redundant ports in blocking mode.
Earlier solutions to disable SPT and to take advantage of multiple layer 2 paths were the use of PortChannel - FEC or GEC (Proprietary), PagP (Proprietary) and LACP (802.3ad, later 802.3AX).
Biggest limitation of PortChannel is that it can only be used between two adjacent devices.
Cisco came up with vPC (virtual PortChannel) in its Nexus platform (NX-OS) which will let multiple active layer 2 paths exist. See here for detail explanation.
vPC is Cisco solution before a standard based technology is agreed upon. That technology is in work in progress for a few years (since 2007 I think) and it has a weird sounding acronym. It is IETF's TRILL - Transparent Interconnection of Lots of Links. TRILL uses IS-IS (yes that L3 routing protocol) to manage L2MP management traffic.
L2MP - Layer 2 MultiPath
Wednesday, February 11, 2009
ASICs
- "show asic-version slot x" will show the count of ASICs like Rohini (Port ASIC), Janus (Fabric interface)
Took it from here.
There is no oversubscription built in to the Rohini-Janus interconnect. The oversubscription on these cards is from janus to the fabric. Also intra Janus there is a 26Mpps limitation.
To elaborate, 6748-GETX is as you say, 1-12, 13-24, etc. The port mapping for the fiber version of 6748, and for 6724, is a bit different, it is even & odd (so eg, 1-23 odd, 2-24 even, etc) but yes it is 12 ports per rohini in all cases.
WRT oversubscription, again, the oversub is at the fabric level, not the port level as you would see on some 4500 series cards & on the 6500 series 6148-GETX etc.
Took it from here.
There is no oversubscription built in to the Rohini-Janus interconnect. The oversubscription on these cards is from janus to the fabric. Also intra Janus there is a 26Mpps limitation.
To elaborate, 6748-GETX is as you say, 1-12, 13-24, etc. The port mapping for the fiber version of 6748, and for 6724, is a bit different, it is even & odd (so eg, 1-23 odd, 2-24 even, etc) but yes it is 12 ports per rohini in all cases.
WRT oversubscription, again, the oversub is at the fabric level, not the port level as you would see on some 4500 series cards & on the 6500 series 6148-GETX etc.
Notes on 6708
- There are 2 flavors - 3C and 3CXL.
- 6704 has 16MB per port buffer, 6708 has 200MB per port buffer
- 8 x 10G ports with 40Gbps (2 x 20Gbps) to Fabric (2:1 fabric oversubscription ?)
- 64Gbps local switching (1.25:1 local oversubscription ?)
- 48Mpps switching
- "no hw-module slot x oversubscription" will turn off oversubscription, effectively shutting down port 3,4,7 and 8.
- in non-oversubscribed mode, port 3,4,7 and 8 cannot be unshut and "show interface" output will be "disabled for performance" instead of normal admin disable
- "mac-address-table synchronize" is enabled by default (and fix PFC/DFC MAC synchronization problem )
- 6704 has 16MB per port buffer, 6708 has 200MB per port buffer
- 8 x 10G ports with 40Gbps (2 x 20Gbps) to Fabric (2:1 fabric oversubscription ?)
- 64Gbps local switching (1.25:1 local oversubscription ?)
- 48Mpps switching
- "no hw-module slot x oversubscription" will turn off oversubscription, effectively shutting down port 3,4,7 and 8.
- in non-oversubscribed mode, port 3,4,7 and 8 cannot be unshut and "show interface" output will be "disabled for performance" instead of normal admin disable
- "mac-address-table synchronize" is enabled by default (and fix PFC/DFC MAC synchronization problem )
Tuesday, January 6, 2009
Gulbis thumps Djokovic in Aussie Warmup
Final score = 6-4, 6-4
Both players made too many unforced errors. Djokovic uncharacteristically committed too many double faults and he looked flat throughout both sets.
This result means that Federer will maintain world number 2 ranking and most likely will be seeded number 2 in Australian Open.
So far, Andy Murray seems to be the best among top 4.
Both players made too many unforced errors. Djokovic uncharacteristically committed too many double faults and he looked flat throughout both sets.
This result means that Federer will maintain world number 2 ranking and most likely will be seeded number 2 in Australian Open.
So far, Andy Murray seems to be the best among top 4.
Subscribe to:
Posts (Atom)
