ACI Deep Dive

• TEP address pool should not overlap with internal address space
• /16 address space is default for TEP pool

Switch discovery

• LLDP between switch and APIC
• DHCP request from switch for lo0
• ISIS between leaf and spine
• IFM = inter fabric messaging (secured with x.509 certificates) 
• VXLAN tunnels built for connectivity to all other leaf / spine switches

Useful Commands

! Show switches in fabric

#acidiag fnvread

#acidiag verifyapic

#acidiag avread

! NXOS like interface

#vsh

#vsh_lc
#show cli list

! overlay-1 is the "underlay"

#show ip interface vrf overlay-1
#show ip route vrf overlay-1

https://<apic-ip>/visore

#moquery

! query faults - uses http port 777
#icurl


#show system internal epm endpoint mac aaaa.bbbb.cccc

! Leaf command to ping (vrf aware unlike native linux)
#iping

! TCPDUMP can be used for control plane traffic only
#tcpdump -i eth0 

ELAM - data plane traffic capture

! See denied packet between EPGs
#show logging ip access-list internal packet-log deny
#show logging ip access-list cache deny

vzAny - contract for an EPG to consume everything in a VRF

! Like BGP debug
#show bgp event-history events

My Five Most Annoying IOS Features

That is Cisco IOS by the way - if you think this is anything to do with iphones I suggest you run along because Cisco have been calling it IOS since Apple was just a sapling.

I like Cisco IOS but there are just a few annoying "features" than continually annoy / baffle me. There are probably legitimate reasons for their existence but to me they just seem like little flaws that could be easily ironed out but we have all just learned to live with.

1. Context sensitive help and autocomplete do not work in configuration mode
   Being lazy by nature I always type as little as I need to which is why autocomplete is great. I can type "sh int" and I get a list of the interfaces on the router. If I am not sure if that is the command I want I can hit tab and it will show me the autocomplete entry for what I have typed so far:
   
   R1#sh int <Press TAB>
   R1#sh interfaces
   
   If there are multiple autocomplete entries for what I have typed so far tab does nothing but a question mark will show me what my options are:
   
   R1#sh in <Press TAB>
   R1#sh in <Press TAB again, slightly harder while frowning>
   R1#sh in?
   interfaces  inventory
   That's all great. Now let's go into configuration mode. We can use the "do" command to enter exec level commands when in configuration mode:
   
   R1#conf t
   Enter configuration commands, one per line.  End with CNTL/Z.
   R1(config)#do sh int
   This works, but only because "show interfaces" is the only autocomplete option for "sh int". Forgotten the command? Hard luck:
   
   R1(config)#do sh in <Press TAB>
   R1(config)#do sh in? <Press ENTER>
   LINE    <cr>
   <Bang head on keyboard - what the hell does LINE mean???>
   So you are stuck in a weird situation where, if you know the full command or the only viable autocomplete option then you can enter it, otherwise you get no help. This seems very obtuse to me - it is like IOS knows what you want but will only help you when it feels like it. Would it really be so hard to fix this?
2. You must write the full interface name in the extended ping
   Not a huge labour but a bit irksome. When you run an extended ping and specify the source interface in the shortened form IOS gets all sniffy and makes you write the whole thing. It takes me about 4 hours for me to type "gigabitethernet"
   
   Protocol [ip]:
   Target IP address: 1.2.3.4
   Repeat count [5]:
   Datagram size [100]:
   Timeout in seconds [2]:
   Extended commands [n]: y
   Source address or interface: fa0/0
   Translating "fa0/0
   % Invalid source. Must use IP address or full interface name without spaces (e.g. Serial0/1)Source address or interface: fastethernet0/0
3. Sometimes you have to add a parameter when it should really be done automatically
   Some commands only take one keyword as a parameter but IOS forces you to put it there even though there are no other options than having it - so why not put it there automatically?
   
   An example:
   
   R1(config-if)#ip nbar ?
     protocol-discovery  Enable NBAR protocol discovery
   
   R1(config-if)#ip nbar
   % Incomplete command.
   
   R1(config-if)#ip nbar ?
     protocol-discovery  Enable NBAR protocol discovery
   
   R1(config-if)#ip nbar protocol-discovery ?
     <cr>
   
   Here we have the command "ip nbar" which only takes the parameter "protocol-discovery". You can't leave it off and it is the only parameter it can take. So why not just fill it in? Like I haven't got enough to do in my busy day...
4. Going in to configuration mode and not making changes is still logged as you making changes
   I am sure there is a good reason for this one but I can't put my finger on it. If you go in to configuration mode and then just exit out again without making any changes it is logged in the log as you having made a change. Watch this:
   
   R1#sh clock
   *07:11:35.710 UTC Fri Mar 1 2002
   R1#conf t
   Enter configuration commands, one per line.  End with CNTL/Z.
   R1(config)#^Z
   R1#
   *Mar  1 07:11:38.986: %SYS-5-CONFIG_I: Configured from console by bob on console
   R1#sh log
   *Mar  1 07:11:38.986: %SYS-5-CONFIG_I: Configured from console by bob on console
   
   This is written to the log file and also added to the top of the running config. In a multi user environment this can lead to much finger pointing when things go wrong:
   dim-witted non-technical management type: Bob, it says you changed the config yesterday a few hours before that network meltdown we had.
   bob: No, I did not change anything.
   dim-witted non-technical management type: Well it says here that you were the last person to change the config
   bob: No honestly I did not change anything. I just entered configuration mode and then exited it.
   dim-witted non-technical management type: Clear your desk bob
5. Pipe include sometimes lies
   You can use the pipe command to filter the results of a command to make it easier to read. The pipe include command says only shows the lines including a certain string except sometimes it lies. Consider this:
   
   R1#sh ip route | inc .1.0
   C    192.168.11.0/24 is directly connected, FastEthernet0/0
   C    192.168.1.0/24 is directly connected, FastEthernet0/0
   
   The string here is ".1.0" which is in the second line but not in the first. I don't know why this happens - I assume it must be ignoring the trailing . for some reason. 

Cisco Default Values

HSRP

Hello = 3 seconds
Dead = 10 seconds

#interface fa0
#standby <group> timers x y

Where x is a value between <1-254> seconds
Where y is a value between <2-255> seconds

#show standby

=========================
EIGRP

Hello = 5 seconds
Dead = 15 seconds 

#interface fa0
#ip hello-interval eigrp <AS> x
#ip hold-time eigrp <AS> x

Where x is a value between <1-65535> seconds

#show ip eigrp interfaces detail fa0

=========================
OSPF

Ethernet:
Hello = 10 seconds
Dead = 40 seconds
Wait = 40 seconds
Retransmit = 5 seconds
(Dead time is automatically set to 4 x the hello interval)

Non-broadcast:
Hello = 30 seconds
Dead = 120 seconds

#interface fa0
#ip ospf hello-interval x
#ip ospf dead-interval x
#ip ospf retransmit-interval x

Where x is a value between <1-65535> seconds

=========================
BGP Route Selection Criteria

1 Weight
2 Local Preference
3 Network or Aggregate
4 Shortest AS_PATH
5 Lowest origin type
6 Lowest multi-exit discriminator (MED)
7 eBGP over iBGP
8 Lowest IGP metric
9 Multiple paths
10 External paths
11 Lowest router ID
12 Minimum cluster list
13 Lowest neighbor address

=========================

Routing Administrative Distance

Connected interface	0
Static route	1
Enhanced Interior Gateway Routing Protocol (EIGRP) summary route	5
External Border Gateway Protocol (BGP)	20
Internal EIGRP	90
IGRP	100
OSPF	110
Intermediate System-to-Intermediate System (IS-IS)	115
Routing Information Protocol (RIP)	120
Exterior Gateway Protocol (EGP)	140
On Demand Routing (ODR)	160
External EIGRP	170
Internal BGP	200
Unknown*	255

Retrieve Cisco Config with wget

On the router or switch:

#conf t
#ip http server
#ip http authentication local
#username cisco priv 15 pass cisco

Then on your PC:

wget --user cisco --password cisco http://192.168.0.1/level/15/exec/show/running-config/view/full  -O cisco-config.txt
(substitute your IP address for 192.168.0.1). 

Show Tech-support:

wget --user cisco --password cisco http://192.168.0.1/level/15/exec/show/tech-support/CR  -O show-tech.txt

PPP Multilink

username R2 password 0 cisco

! Configure a user account with the hostname of the peer and a matching password

! 

interface Multilink1

 ip address 1.1.1.1 255.255.255.252

 ppp multilink

 ppp multilink group 1

!

interface Serial0/1

 no ip address

 encapsulation ppp

 clock rate 2000000

 ppp authentication chap

 ppp multilink

 ppp multilink group 1

!

interface Serial0/2

 no ip address

 encapsulation ppp

 clock rate 2000000

 ppp authentication chap

 ppp multilink

 ppp multilink group 1

=====================

username R1 password 0 cisco

!

interface Multilink1

 ip address 1.1.1.2 255.255.255.252

 ppp multilink

 ppp multilink group 1

!

interface Serial0/1

 no ip address

 encapsulation ppp

 clock rate 2000000

 ppp authentication chap

 ppp multilink

 ppp multilink group 1

!

interface Serial0/2

 no ip address

 encapsulation ppp

 clock rate 2000000

 ppp authentication chap

 ppp multilink

 ppp multilink group 1

=====================

R1#show ppp multilink

Multilink1, bundle name is R2

  Username is R2

  Endpoint discriminator is R2

  Bundle up for 00:01:29, total bandwidth 4632, load 1/255

  Receive buffer limit 36000 bytes, frag timeout 1000 ms

    0/0 fragments/bytes in reassembly list

    0 lost fragments, 0 reordered

    0/0 discarded fragments/bytes, 0 lost received

    0x4 received sequence, 0x9 sent sequence

  Member links: 2 active, 0 inactive (max not set, min not set)

    Se0/1, since 00:01:29

    Se0/2, since 00:01:29

No inactive multilink interfaces

R2#show interface multilink1
Multilink1 is up, line protocol is up
  Hardware is multilink group interface
  Internet address is 1.1.1.2/30
  MTU 1500 bytes, BW 3088 Kbit/sec, DLY 100000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open, multilink Open
  Open: IPCP, CDPCP, loopback not set
  Keepalive set (10 sec)
  DTR is pulsed for 2 seconds on reset
  Last input 00:00:52, output never, output hang never
  Last clearing of "show interface" counters 00:36:36
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     1071 packets input, 117686 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     1078 packets output, 132324 bytes, 0 underruns
     0 output errors, 0 collisions, 5 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

#debug ppp authentication

Switch Based Access Control

• Control access to and from devices in the same VLAN using MAC address:

#conf t
#vlan access-map MAC_DENY 10
#action drop
#match mac address MAC_DENY_ACL

#mac access-list extended MAC_DENY_ACL
#permit host 0000.0000.0001 host 0000.0000.0002

#vlan filter MAC_DENY vlan-list 100

This means traffic from 0000.0000.0001 to 0000.0000.0002 will be dropped in VLAN 100

• Control access to and from devices in the same VLAN using IP address:

#conf t

#vlan access-map IP_DENY 10
#action drop
#match ip address 150

#access-list 150 permit ip host 192.168.199.1 host 192.168.199.2

#vlan filter IP_DENY vlan-list 100

This means traffic from 192.168.199.1 to  192.168.199.2 will be dropped in VLAN 100

RIP over GRE Tunnel with QoS Configuration

In this configuration I have 4 routers R1, R2, R3 and R4. R2 and R3 run External BGP. There is also a GRE tunnel running between R2 and R3 over which we run RIP. R1 and R4 also run RIP. QoS configuration is applied to the tunnel interface via a service policy which shapes the traffic based on which access list it matches. 

This configuration was made on GNS3 with 2691 routers running C2691-ADVENTERPRISEK9-M. 

Irrelevant parts of the config below have been omitted for brevity.

=~=~=~=~=~=~=~=~=~=~=~= R1=~=~=~=~=~=~=~=~=~=~=~=

hostname R1

!

! Two loopback interfaces to allow us to differentiate the traffic for the class-maps

interface Loopback0

 ip address 1.1.1.1 255.255.255.255

!

interface Loopback1

 ip address 11.11.11.11 255.255.255.255

!

interface FastEthernet0/0

 ip address 192.168.1.1 255.255.255.0

 speed 100

 full-duplex

!

interface FastEthernet0/1

 ip address 192.168.0.1 255.255.255.0

 duplex auto

 speed auto

!

! We run RIP to learn routes from R2

router rip

 version 2

 network 1.0.0.0

 network 11.0.0.0

 network 192.168.0.0

 network 192.168.1.0

 no auto-summary

!

=~=~=~=~=~=~=~=~=~=~=~= R2=~=~=~=~=~=~=~=~=~=~=~=

hostname R2

! We create two class-maps which match named access lists

class-map match-all CMAP_MATCH11

 match access-group name MATCH11

class-map match-all CMAP_MATCH1

 match access-group name MATCH1

!

! We have a policy-map which assigns 8K and 512K to each respective class-map.

! The overall method of the policy maps says, if you match ACL MATCH1 then you 

! will be allocated 8K of bandwidth, if you match ACL MATCH11 then you will get 

! 512K of bandwidth

policy-map TUNNEL

 class CMAP_MATCH1

  shape average 8000

 class CMAP_MATCH11

  shape average 512000

 class class-default

!

! 

interface Loopback0

 ip address 2.2.2.2 255.255.255.255

! We have a tunnel interface with a service policy applied

interface Tunnel0

 ip address 10.0.0.1 255.255.255.0

 tunnel source Loopback0

 tunnel destination 3.3.3.3

 service-policy output TUNNEL

!

interface FastEthernet0/0

 ip address 192.168.1.254 255.255.255.0

 speed 100

 full-duplex

!

interface FastEthernet0/1

 ip address 192.168.2.1 255.255.255.0

 speed 100

 full-duplex

! We run RIP over the LAN and tunnel interfaces only

router rip

 version 2

 passive-interface default

 no passive-interface FastEthernet0/0

 no passive-interface Tunnel0

 network 10.0.0.0

 network 192.168.1.0

 no auto-summary

! BGP to R3 to carry the tunnel

router bgp 1

 no synchronization

 bgp log-neighbor-changes

 redistribute connected

 neighbor 192.168.2.254 remote-as 2

 neighbor 192.168.2.254 next-hop-self

 no auto-summary

!

! ACLs to match the source and destination loopbacks

ip access-list extended MATCH1

 permit ip host 1.1.1.1 host 4.4.4.4

ip access-list extended MATCH11

 permit ip host 11.11.11.11 host 44.44.44.44

!

!

=~=~=~=~=~=~=~=~=~=~=~= R3 =~=~=~=~=~=~=~=~=~=~=~=

hostname R3

!

!

! Class-map, policy-map and ACLs are basically the reverse of R2

class-map match-all CMAP_MATCH44

 match access-group name MATCH44

class-map match-all CMAP_MATCH4

 match access-group name MATCH4

class-map match-all MyClass

!

!

policy-map TUNNEL

 class CMAP_MATCH4

  shape average 8000

 class CMAP_MATCH44

  shape average 512000

 class class-default

!

! 

interface Loopback0

 ip address 3.3.3.3 255.255.255.255

!

interface Tunnel0

 ip address 10.0.0.2 255.255.255.0

 tunnel source Loopback0

 tunnel destination 2.2.2.2

 service-policy output TUNNEL

!

interface FastEthernet0/0

 ip address 192.168.2.254 255.255.255.0

 speed 100

 full-duplex

!

interface FastEthernet0/1

 ip address 192.168.3.1 255.255.255.0

 speed 100

 full-duplex

!

router rip

 version 2

 passive-interface default

 no passive-interface FastEthernet0/1

 no passive-interface Loopback0

 no passive-interface Tunnel0

 network 10.0.0.0

 network 192.168.3.0

 no auto-summary

!

router bgp 2

 no synchronization

 bgp log-neighbor-changes

 redistribute connected

 neighbor 192.168.2.1 remote-as 1

 neighbor 192.168.2.1 next-hop-self

 no auto-summary

!

ip access-list extended MATCH4

 permit ip host 4.4.4.4 host 1.1.1.1

ip access-list extended MATCH44

 permit ip host 44.44.44.44 host 11.11.11.11

!

=~=~=~=~=~=~=~=~=~=~=~= R4 =~=~=~=~=~=~=~=~=~=~=~=

hostname R4

! Again, R4 is basically a mirror of R1

interface Loopback0

 ip address 4.4.4.4 255.255.255.255

!

interface Loopback1

 ip address 44.44.44.44 255.255.255.255

!

interface FastEthernet0/0

 ip address 192.168.3.254 255.255.255.0

 speed 100

 full-duplex

!

interface FastEthernet0/1

 ip address 192.168.4.1 255.255.255.0

 speed 100

 full-duplex

!

router eigrp 1

 network 0.0.0.0

 no auto-summary

!

router rip

 version 2

 network 44.0.0.0

 network 0.0.0.0

 no auto-summary

!

=~=~=~=~=~=~=~=~=~=~=~= Verification~=~=~=~=~=~=~=~=~=~=~=

A ping from R1 lo0 to R4 lo0 goes via the tunnel interface

R1#traceroute 4.4.4.4 source 1.1.1.1

Type escape sequence to abort.

Tracing the route to 4.4.4.4

  1 192.168.1.254 48 msec 24 msec 16 msec

  2 10.0.0.2 44 msec 44 msec 28 msec

  3 192.168.3.254 96 msec *  68 msec

An extended ping with a larger packet size - note the average RTT is 482ms

R1#ping 4.4.4.4 source 1.1.1.1 size 500 rep 50

Type escape sequence to abort.

Sending 50, 500-byte ICMP Echos to 4.4.4.4, timeout is 2 seconds:

Packet sent with a source address of 1.1.1.1

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Success rate is 100 percent (50/50), round-trip min/avg/max = 40/482/1008 ms

An extended ping but this time we specify the other loopbacks as source and destination so we hit the QoS policy with a higher bandwidth - note the much better average RTT of 58ms

R1#ping 44.44.44.44 so 11.11.11.11 size 500 rep 50

Type escape sequence to abort.

Sending 50, 500-byte ICMP Echos to 44.44.44.44, timeout is 2 seconds:

Packet sent with a source address of 11.11.11.11

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Success rate is 100 percent (50/50), round-trip min/avg/max = 20/58/92 ms

On R2 if we issue the policy map interface command we see the following. Note how we see delayed packets on the CMAP_MATCH1 class and none on the CMAP_MATCH11 class.

R2#sh policy-map interface

 Tunnel0

  Service-policy output: TUNNEL

    Class-map: CMAP_MATCH1 (match-all)

      361 packets, 184332 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: access-group name MATCH1

      Traffic Shaping

           Target/Average   Byte   Sustain   Excess    Interval  Increment

             Rate           Limit  bits/int  bits/int  (ms)      (bytes)

             8000/8000      2000   8000      8000      1000      1000

        Adapt  Queue     Packets   Bytes     Packets   Bytes     Shaping

        Active Depth                         Delayed   Delayed   Active

        -      0         361       175776    174       87000     no

    Class-map: CMAP_MATCH11 (match-all)

      460 packets, 239040 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: access-group name MATCH11

      Traffic Shaping

           Target/Average   Byte   Sustain   Excess    Interval  Increment

             Rate           Limit  bits/int  bits/int  (ms)      (bytes)

          5120000/5120000   32000  128000    128000    25        16000

        Adapt  Queue     Packets   Bytes     Packets   Bytes     Shaping

        Active Depth                         Delayed   Delayed   Active

        -      0         460       228000    0         0         no

    Class-map: class-default (match-any)

      139 packets, 15568 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: any