Route My World!

OSPF Routers and LSA Types

OSPF Router Types

• Different OSPF router types control the type of traffic that go in and out of OSPF areas.
• When an area becomes too big, some of the following concerns become important:
  • Freqency of SPF calculations
  • Routing tables getting bigger
  • LSDBs also getting bigger.
• A solution to an increasing network is to implement a hierarchical area structure for the OSPF network. Some advantages of multiple OPSF areas are:
  • Reduced frequency of SPF calculation
  • Smaller routing tables
  • Reduced LSU overhead
• Here are the different router types:
  • Internal router - router’s whose interfaces are in the same area. Routers in the same area have the same LSDBs.
  • Backbone router - These routers sit on the perimeter of the backbone area (area 0) so it has at least one interface connected to area 0.
  • Area Border Router (ABR) -
    • Have interfaces attached to multiple areas.
    • It contains a separate LSDB for each area.
    • Route traffic destined for or arriving from other areas.
    • Exit points for the area, meaning that routing information destined for another area can get there through the ABR of that area.
    • Can summarize routing information.
  • Autonomous System Border Router
    • Have at least one interface attached to another autonomous system, such asa RIP network.
    • Perform route redistribution - a process of importing non-OSPF information to the OSPF network and vice versa.
• A router can be more than one router type.
• For each area that a router connects, it maintains a separate LSDB. Routers in the same area will have identical LSDBs for that area.
• An LSDB is synchronized between pairs of adjacent routers. On broadcast (LAN) networks, an LSDB is synchronized between the DROTHER.

OSPF LSA Types

Each LSA is a record that holds information for the database. As a whole, all these records make up the entire topology of an OPSF network.

Type 1: Router LSA

• A Type 1 LSA, or Router LSA is, flooded by each router in an area. A type 1 LSA describes the collective states of the router’s directly connected links (interfaces).
• Each of the router’s links (interfaces) is categorized into four diffrent link types as follows:

• *A stub network is a dead-end link that has only one router attached.
• For each of these links, there is a link data field that provides 32 bits of extra information.
  • For most link types this is the IP address of the associated router interface.
  • For stub network links, this link data field contains the subnet mask.
• Type 1 LSAs also indicates OSPF cost for each link, and whether the router is an ABR or ASBR.

Type 2: Network LSA

• Generated by the DR.
• Generated for every LAN (broadcast) or or NBMA transit network. An example of a transit network is an Ethernet LAN.
• The Type 2 LSA lists all the attached routers that make up the transit network, including the subnet mask of the link.
• Type 2 LSAs never cross the area boundary
• The link-state ID for a Network LSA is the IP address of the DR’s interface that advertised it.

Type 3: Network Summary LSA

• Sent by the ABR.
• A type 3 LSA advertises routes from one area into other areas in the OSPF autonomous system.
• When type 1 LSAs reach the ABR, the information from the type 1 LSAs are sent out by the ABR to other areas in the form of type 3 summary LSAs.
• By default, OSPF does not automatically summarize groups of contiguous subnets. It also does not summarize a network to its classful boundary.
• By default, a type 3 LSA is advertised into the backbone area for every subnet defined in the originating area.
• Manual summarization should be used to alleviate problems caused by significant flooding from too many networks being advertised.
• Summary LSAs do not, by default, contain summarized routes. Therefore all subnets in an area will be advertised, unless of course the network operator configures manual  summarization.

Type 4: ASBR Summary LSA

• A type 4 summary LSA is used to announce the presence of an ASBR. Therefore a type 4 summary LSA is only used when an ASBR exists within an area.
• It identifies the ASBR and provides a route to it.
• The link-state ID is the ASBR’s router ID.
• The ASBR sends a type 1 router LSA with a bit (known as the  external bit or e-bit) that identifies itself as and ASBR. When an ABR (that is identified with a border bit or b-bit in the router LSA) receives this type 1 LSA, it builds a type 4 LSA and floods it to the backbone or area 0.

Type 5: External LSA

• Describe routes to external OSPF autonomous systems.
• These are generated by the ASBR and are flooded to the entire autonomous system.
• The link-state ID is the external network number.
• Again, because summarization does not occur by default, the network operator should consider manual route summarization at the ASBR to prevent problems with over flooding.

OSPF LSDB & Routing Table

OSPF LSDB

The command show ip ospf database allows one to view the contents of the OSPF LSDB.

Router# show ip ospf database

OSPF Router with ID(192.168.1.11) (Process ID 1)

                 Router Link States(Area 0)

 Link ID           ADV Router        Age         Seq#       Checksum Link count

 192.168.1.8       192.168.1.8       1381      0×8000010D    0xEF60   2

 192.168.1.11      192.168.1.11      1460      0×800002FE    0xEB3D   4

 192.168.1.12      192.168.1.12      2027      0×80000090    0×875D   3

 192.168.1.27      192.168.1.27      1323      0×800001D6    0×12CC   3

                 Net Link States(Area 0)

 Link ID          ADV Router        Age         Seq#       Checksum

 172.16.1.27      192.168.1.27      1323      0×8000005B    0xA8EE

 172.17.1.11      192.168.1.11      1461      0×8000005B    0×7AC

                 Type-10 Opaque Link Area Link States (Area 0)

  Link ID         ADV Router        Age         Seq#       Checksum Opaque ID

 10.0.0.0         192.168.1.11      1461      0×800002C8    0×8483     0

 10.0.0.0         192.168.1.12      2027      0×80000080    0xF858     0

 10.0.0.0         192.168.1.27      1323      0×800001BC    0×919B     0

 10.0.0.1         192.168.1.11      1461      0×8000005E    0×5B43     1

The following explains the purpose of each column:

• Link ID - Identifies the Router ID number
• ADV Router - Identifies the advertising router ID. This is the source router of the LSA
• Age - The age of the Link state. The maximum is 3600 seconds (1 hour).
• Seq# - The link state sequence number. The sequence number starts at 0×80000001 and increments by one each time it is updated. This helps detect old and duplicate LSAs.
• Checksum - Ensures the reliable receipt of the LSA
• Link Count - Shows how many links are attached.
  • Used only on Type 1 Router LSAs.
  • The link count includes all point-to-point, transit, and stub links.
  • Point-to-point serial links count as 2
  • All others count as one.

Route Types in the Routing Table

Different designations describe the route types generated by OSPF:

• O - Indicates that the route comes from within the router’s area. These routes are advertised by router LSAs and network LSAs
• O IA - The “IA” stands for inter-area. It indicates that the routes come from networks outside the router’s area (but still within the same autonomous system.) This type of route is advertised by ABRs through summary LSAs.
• O E1 - External LSA type 1. Route costs are calculated by adding the external cost to the internal cost of each link. This type is useful when multiple ASBRs are advertising external routes to the same AS - it avoids suboptimal routing.
• O E2 - External LSA type 2. The route coast never change and it is always the cost of the external route.

OSPF LSDB Overload Protection

• OSPF LSDB overload protection can protect the routers from resource (CPU and memory) drains. An example of such an instance is a misconfiguration of routers that causes a redistribution of a a large number of prefixes, in turn generating excessive amount of LSAs that are generated.
• This feature is available with Cisco IOS Software Release 12.3(7)T and later, as well as some specific earlier releases.

max-lsa maximum-number [threshold-percentage] [warning-only] [ignore-time minutes] [ignore-count count-number] [reset-time minutes]

The parameters are as follows:

Changing the Cost Metric

The general formula used to calculate OSPF metric is 100Mbps/(bandwidth in Mbps).

For example:

1. A 64 kbps link has a metric of 1562:
   • 64kbps/1000kbps = 0.064 –> 100Mbps/0.064Mbps = 1562.5
2. A T1 link gets a metric of 64
   • 100Mbps / 1.544Mbps = 64.7

• The problem with that formula is that the maximum interface it can do is 100Mbps, which will yield a metric of 1.
• For interfaces faster than 100mbps, use the auto-cost-reference-bandwidth ref-bw command.
  • The ref-bw is any range between 1 to 4,294,967 in megabits per second. The default is 100.
• Also, remember to use the bandwidth value interface configuration command to accurately depict the correct interface bandwidth, in kilobits per second

• The ip ospf cost interface-cost configuration command to override the default cost. The interface-cost is an integer from 1 to 65,535.
  • The lower the number, the better (and more preferred) link.

Resources:

1. Link State Advertisement Formats
2. IP Routing Protocols Commands - show ip ospf…
3. OSPF E2 vs E1 Routes - Chris Bryant
4. OSPF Link State Database Overload Protection

The last few days I’ve been stepping up my reading efforts in order to keep pace with the reading schedule I set at the beginning of the week. I’m quite pleased with myself for having almost finished the chapter on BGP ahead of schedule.  I began printing other documents from Cisco’s website in order to supplement what I just studied. There’s no doubt that I’ll have to go back and re-read the chapter again, hopefully before next week, because the whole topic is just loaded with juicy details. I’ll definitely have to know and master the ten or so steps for BGP path selection (in order) and know how each of those attributes work. But all in all, it’s been a fun few days. I’ve skipped watching TV altogether since Sunday night (except the time during and after dinner when my 2 year old watches Curious George and Barney.

Tonight, though, is a special night because the USC trojans are playing. So, I’m rewarding myself to a 4-hour night of pure trojan domination as they start their PAC-10 play.  The PAC-10 seems to be pretty weak this season (it has yet to be concluded since there’s more football to come in the next 8-10 weeks), but for some reason, PAC-10 teams seem to give us some problems here and there. We seem to take care of the competition when playing out-of-conference powerhouses but couldn’t take care of lowly Stanford, or Oregon State, or even UCLA the past few seasons. But this year, I feel like we can take care of business and go undefeated all the way to the title game on January of ‘09, at the Dolphin Stadium (formerly the Orange Bowl Stadium) in Miami.

So tonight I’m putting down the books, shutting down the laptop, and popping a chilled can of (root) beer ’cause in 30 minutes, it’s game time!

Update: Well I’m back from watching the game. And if you haven’t already heard, the Trojans lost the game. There goes my dream for a perfect season. And that is why laptop is on and the books are wide open - I’m going to drown all my sorrows with some BGP fun! Fight On!

I finished my first pass on redistribution topics last week. In an effort to keep the momentum going, I started on BGP today and will continue with the getting-to-know-you readings on BGP into the weekend. Because of this new strategy, I will not be taking notes until my second pass. There’s just too much materials to go through on this topic if I have to stop and take detailed notes on each major sections. This time my goal is to finish up reading BGP, followed by multicast, then IPv6 in the next 4 weeks. After that, I’ll go back and do the second pass with more intensive readings and detailed notes.

I lost too much study time in the last month and a half working on our network at my workplace. While the experience I gained is very valuable, I also can’t lose focus on my goal to get the CCNP out of the way (I’m shooting for no later than summer of 2010). Now that most of the major changes and network configurations are done, I sort of retracted back to my usual mop up duties (changing printer toners, maintaining RF scanners, software installs here and there, etc). I’m using some free time I have to ramp up my readings. At the same time, I’m also trying to keep abreast of different techniques of maintaining and monitoring my network. Yeah, it feels nice to say “my network” as I’ve almost fully taken ownership of all our five offices and their network connectivities.

Here’s this week’s scheduled readings:

Mon, September 22: Read pp. 469-480 - BGP Concepts, Autonomous System, Multihoming
Tue, September 23: Read pp. 481-492 - Path Vecctor characterisics, IBGP, EBGP
Wed, September 24: Read pp. 492-505 - Synchronization, tables, message types, as-path, next-hop
Thu, September 25: Read pp. 505-516 - Origin attr, local pref attr, community attr, MED, weight, configure.
Fri, September 26: Read pp. 516-529 - Configure:  multi-hop, next-hop, authentication, synchronization
Sat, September 27: Read pp. 529-541 - Configuration examples, verify and troubleshoot
Sun, September 28: Read pp. 541-556 - Path manipulation using route maps

I was again asked to “make something happen” to our network that I previously had no idea how to do. We have an application server in our office that several folks from home used to be able to connect remotely into using Remote Desktop connection. Since I moved all our outgoing and incoming traffic to the PIX, that has since been broken. With some direction from my manager, I was able to figure out what to do. Basically, it entails using NAT in order to map a local IP address to a globally routable address.

The basic requirements are:

• Configure static NAT on the PIX to map the outside interface to the inside host.
• Create an access list that allows RDP access

Here’s a simple diagram of my network to give you a pictorial view of the packet’s path:

Below is my configuration that “makes it happen”:

PIX Version 7.2(4)
!

access-list acl_outside extended permit tcp any host 72.x.x.x eq 3389
!
!
!
static (inside, outside) 72.x.x.x 10.100.194.33 netmask 255.255.255.255
!
!
access-group acl_outside in interface outside

Lets go over the config line by line:

1. The first line is the software version of the PIX
2. The second line is an exclamation mark
3. Then a space…. OK, I’m being not funny!

Basically, I created an access list, called “acl_outside” which allows a source IP from any hosts on the internet to access destination 72.x.x.x on TCP port 3389 (the default port used by RDP) - stuff I learned in CCNA.

Destination 72.x.x.x is mapped to a local address 10.100.194.33 using a one-to-one static mapping - stuff I also learned in CCNA.

The last line applies the access list I created above to the outside interface of the PIX - stuff I just learned recently.

And somehow, magically, I’m now able to establish RDP connection to the box in our little server room. Oh what beauty to behold! Now if anybody has a best-practice suggestion that can make my config even better, I’m all ears. As always, I’m sure there’s better ways to accomplish the same task. But for now, it makes happen.

This weekend was jam-packed with happy happenings that I barely did any studying. Started Saturday morning with my usual Fall Saturday morning routine - watch a lot of College Gameday. The Gameday crew was in town at USC and I so wanted to be there but couldn’t. A friend of mine and I vowed that next time they’re in town, we are going to be there. So hopefully the crew visits again next year.

I did about an hour of reading and review of IS-IS, then on to getting ready for my friend’s wedding (for which I was one of nine groomsmen - we had a lot of friends in college!) The wedding was awesome, except for one “minor” incident where all the groomsmen and one little miss packed a small elevator and got stuck for what seemed like an eternity. I seriously thought I was gonna die. Ok maybe not, but it was a pretty scary moment. Finally we got a hold of the management and they heroicaly got us out of that pickle. Other than that, we had a blast and acted like we used to when were in college - oh for only that moment, at least. The family and I got back home at 11:30pm and I started to watch the SC game which I recorded on DVR - the best invention by man since..ummm… man was invented.

Sunday was Church day so we went to church.

I guess I haven’t really caught up with sleep because I’m extremely tired today. But I’m hoping to get through route redistribution this week, so I’ll have to suck it up and get through the first part of my readings. There’s approximately 77 pages of detailed information to go through, minus end of chapter reviews and configuration exercises. So if my basic math skills can still be trusted, 77 pages divided by 7 days equals 11 pages/day. That works out pretty good for me because, for one, I can’t read more than few pages in one sitting without my mind going to lala land by the 7th or 8th page. Eleven pages of reading, spread out a whole day, helps me get into the pages in more detail and ensures that I give myself the best chance to retain data. It’s also my habit to supplement my readings with extra materials from Cisco docs and other online sources like wikipedia while reading my main study source. So by the end of the day, I would have read twice what I have planned on reading. My day to day schedule at work, with changing circumstances, also contribute to how much and how well I take in new information. I’m very tired today so it will be an uphill battle trying to digest new information. But then, so is it for everyone else. So I’m not complaining.

This is the schedule that I’m gonna try my hardest to follow for this week:

Mon, September 15: Read pp. 372-382 - Redistribution overview
Tue, September 16: Read pp. 383-394 - Redistributing RIP, Ridistributing OSPF
Wed, September 17: Read pp. 394-404 - Redistributing EIGRP, IS-IS, default-metric, passive-interface
Thu, September 18: Read pp. 405-416 - Controlling routing update traffic, Distribute List, Route Map.
Fri, September 19: Read pp. 416-427 - Configuring route maps, Redistribution using AD,
Sat, September 20: Read pp. 428-439 - Configuring DHCP, DHCP Server
Sun, September 21: Read pp. 440-448 - DHCp Server Options, Relay Agent, DHCP Client

Hopefully, the week following that, I can start an in-depth review and notes as well as do the configuration exercises.

No I don’t mean packet collision

This whole week, the LA sports air waves have been talking up and hyping up this weekend’s showdown between The #1 ranked USC Trojans and #5 ranked Ohio State Buckeyes. And I’m getting more and more excited as the Saturday nears. I believe ESPN College Gameday will also be at the SC campus on Saturday (can’t have enough of Chris, Lee, and Kirk). Me so wanna be there. But alas! I will be getting ready for an old college buddy’s wedding on Saturday morning. I won’t even get to see the game live.

I haven’t always been a fan of college football. I only started getting into it when my wife, who went to USC, took me to my fist ever college football game in 2001. From then on I fell in love with the whole pageantry, the tradition, and mystique that goes on with the whole event. It’s amazing how a whole stadium of 90,000 people can be roused together when they hear their school’s fight song and get them all in their feet and altogether chanting their long-standing slogans and battle cries. You can feel the tradition emanate in the stadium as you look around the alumni section with folks in their 60s, and 70s, and 80s, decked out in their SC gear toting their little granddaugters in their little SC cheerleading outits and grandsons flashing their over-sized foam victory signs.

Two things stand in my way from being present at the game: I will be at a wedding of an college roommate and a $400-$1000 price tag to get a decent seat at the LA Coliseum.

[Edit] I just realized this was my 100th post. So imma go buy me a beer and leave it in the fridge until our next house party and one of my friends finds it there and drinks it. Yeah!

It’s been over a month since I began my research and knowledge-gathering on the re-implementation of our network VPN infrastructure. It’s not yet complete but I feel very accomplished and edified with the ways things have turned out so far.

We started out with all our remote offices/sites connecting to our corporate site via a mixture of different router-to-router VPN solutions (i.e. IPsec/GRE and DMVPN). Today we have all the routers in our remote sites connected on IPsec VPNs to our corporate office on an old PIX that we recently recomissioned. VPN client requests are also hitting our new (old) PIX and authenticated by a Win2003 RADIUS server.

I guess the sense of accomplishment comes from the fact that this is the first time I’ve ever implemented such a design. Add to that fact that I received little to no help from anyone at work - minus, of course, some tips from some excellent bloggers who read this little blog-o’-mine. Studying really does pay off!

Next on my list (this project is only half complete):

• Configure dynamic routing, most likely OSPF. I think this one needs GRE to work so I will be reading up on that. Actually I’ve already read up on it so now I just need to see if I can lab it up. Or I can always test on the production routers like I’ve been doing. Real men test on production servers!
• My boss would like to have some sort of redundancy implemented so I will be working on that.
• This is just for my own use but I’d like to get MRTG up and running for bandwidth and traffic utilization monitoring. I’ve read about it before and was able to successfully install it. But I still wasn’t sure how to use it and exactly what it did. So I’d like to know more.
• I have to factor in, also, my BSCI studies. Because a big chunk of the time that I use to learn and configure our network is done during off hours (meaning during the times when I would be studying for BSCI). I’m trying to re-dedicate a good balance of time to get back on track with finishing BSCI and shooting end of October to take the test.

Ok, cool!

I’m about to smack a helpless dog from all this frustration. I’ve been trying to create an ipsec tunnel between a PIX and an Edgewater device on a remote location since yesterday and I’m not getting anywhere. Checked all my configs and checked them twice five times. Hmmmm………

Just kidding about smacking a helpless dog - for you dog-lovers out there. I meant to say a helpless cat.

I just finished another remote site in Arkansas tonight, adding it to the list of routers I have successfully configured with ipsec vpn. And each time I add another crypto map and tunnel-group entry into the PIX, the more natural it becomes. It feels nice to see that continous ping finally show “Reply…” instead of the dreaded “Request timed out”. It’s also a fist-pumping moment to see that the tracert result shows that it is now using the new tunnel instead of the old.

In addition I successfully configured RADIUS authentication for our VPN client users today at work. I can add that now to my resume of small accomplishments .

I should have finished earlier but I was also watching a local high school football game on ESPN2. Normally I don’t watch hs football but two future recruits for the USC Trojans were playing on each side. But I really wanted to see more of how well the much-hyped future Trojan quarterback, Matt Barkley, was going to perform. He happens to be the first junior to ever be awarded the Gatorade national football player of the year. They ended up winning in triple OT, despite a three interception performance from Barkley. Oh yeah, he also happens to go to Mater Dei HS, the same program where previous Heisman trophy winning Trojan QB Matt Leinart went. So yeah, add another one to the win column… Not that anyone cares.

What a trip this last few weeks have been. I have mentioned previously that I have been busy with some cool implementation projects at work. Specifically, I have been tasked to configure our PIX appliance to accept remote VPN client requests. This is a very interesting and fun project for me because I have never done any of these before. I have never even been inside a pix OS nor even seen one in my IT career. I have mentioned before that aside from the few good years where I maintained and implemented a Windows Active Directory infrastructure at my old job, most of my career was relegated to doing menial help desk support - something I’ve made a decision to change. And nine months after a made that decision, I’m finally seeing that change.

Last week I was able to finally see my work bear some fruits - in a matter of saying. I now have remote users from our company hitting our pix and able to access local resources in our corporate office (thanks to Barry of bitbucketblog.com, in part). There’s still a lot of work I need to do to clean up my configurations but seeing my implementation actually working is a big boost on my confidence.

Some of the things I need to clean up for sure is the routing. Everything so far is static (which is fine for our purposes since we don’t have a lot of routers or sites that need dynamic routing.) But it would be nice to have OSPF running later. Also, right now, the users authenticate against a local username/password on the pix appliance. Ideally, we would like them to authenticate on a Windows RADIUS server.

Despite all that, though, I already learned a ton of things. Some things I’ve never used before but now understand a little better:

• What IPsec is all about
• Configure ISAKMP parameters
• Configure IPsec parameters
• Crypto maps
• Dynamic crypto maps
• NAT
• NAT-T
• Split-tunnels
• Better understanding of IP access-lists
• Reverse Route Injection
• a few more that I probably am not remembering

Now I still can’t say that I understand them well. But at least I have a better idea of what these things are all about. And with time and experience, I can develop a more solid understanding of them. In fact, learning how to do the step by step configuration was pretty easy. The real challenge is to really understand everything behind all the commands I was typing in. And for the most part, I took particular attention to what I was asked to type in by the Cisco documentations. I’ve downloaded and printed out thousands of pages of Cisco docs to peruse to better understand what I was doing. I’ve spent late nights and weekends reading for hours on every configuration command that was asked of me. Needless to say, my brain is packed with information that I’m sure I will forget 75% of. But that’s ok. There’s absolutely no doubt in my mind that I’ve learned something valuable. In fact, I printed out the running configs on my routers and pixs and I can honestly say that I can read them in a whole new light because of my new understanding. And that my friends is pretty exciting.

Oh by the way, last Thursday, prompted by a renewed confidence in me, my boss asked me if I was up to tearing down our old router-to-router gre tunnels to our remote sites and configure a multiple router-to-pix ipsec vpn tunnels to replace the old one. Not wanting to miss out on the opportunity I immediately said, “hells yeah!” Much of the initial configuration was very similar to the client configurations so I thought I can fumble my way around it. It turns out that my boss’s confidence in me was a little bit pre-mature because I failed miserably. In fact, I think he might have gotten a little annoyed in me for being so confident that I could do it. He told me at first that if I wasn’t comfortable, that I should tell him right then and there when he asked me. I wanted to do it so bad, partly to get the “hands-on” and partly to show him initiative and that I can do it. But it proved to be a little bit over-whelming as I worked on it from 8am to 9pm almost non-stop that day only to end up breaking things. In the end my boss told me to go home and no to touch the routers any further. A little bit dejected and hit with a little dose of you-are-way-in-over-your-head reality I went home and cracked open a thick binder of documentations I printed from work and dug in through the steps and looked for what I was doing wrong.

The next day the boss ( a former CCIE, but years separated from hard core IOS hands on) was in his office with his room door shut working away at fixing some of the configs I broke. That whole day sure felt very long and uncomfortable and I knew my boss was not particularly happy because he was short with me when I ask him questions. So I just sat in my corner and used every opportunity to continue researching on what I did wrong. I was just resigned to let things be with an almost nonchalant “oh well” attitude. By the end of the day, my boss has not succeeded in getting the configuration running and the deadline to get the tunnel up was at the end of that day because the primary Internet circuit that the current tunnel is running on is about to get turned down at the end of business day. To make things worse he had to leave early that day. So, faced with frustration of the whole day, my boss turned to me again and told me to look through his configuration because he has been looking at it all day and tunnel vision (pun) has impaired his brains that he is having a hard time spotting little mistakes that he might have made but otherwise could not spot. He told me what to look for and I started looking at the configurations line by line. Much to my surprise, or non-surprise, most of the configuration he put in there were very similar to what I had initially configured. In fact they were pretty much the same ones minus a few changes (e.g. where I configured a 3des, he put in a des or where i put in an md5 hash, he substituted a sha). I even spotted an acl that he configured that I thought was not right.

And so thinking that nothing else could possibly go more insane, I cleared all his configurations - with his approval, of course (or so I interpreted something he said as approval ;)). And with the notes that I jotted down from the night before and all through the day, I rebuilt the configuration… And what do you know! A few hours of careful and meticulous reconfiguration, I finally got one tunnel up and endpoints talking to each other. In the process of him fixing my mess, he also broke the client vpn configurations I made earlier that week. But I was also able to reconfigure it back to its proper working order. I tested all the routing and ping and traceroute outputs were flying back and forth. I felt vindicated. Actually, I wanted to say out loud in a sinister tone, “vengeance is mine!” but that didn’t feel quite right. After that, configuring all the other routers were cake.

Now I can’t claim that I’m smarter than my boss or anything. Because 999 times out of 1000, he will out-configure me. He is also 100 times smarter than me.  But I can’t say that I got lucky either, because this has nothing to do with luck. It’s either configured correctly or not. Maybe it was more of him being unlucky just for that day that allowed me to out-do his work

Looking back, I don’t know what it is that I did wrong the first time around that I didn’t do this time that made it work or vice versa. The irony is that, I found what was wrong with his configuration which I worked to resolve. I guess if he hadn’t changed the configs that allowed me to see something that didn’t look right, I wouldn’t have had the werewithal to change it again for fear of breaking anything further. Change is good.

In the end, I have a pix authenticating remote vpn clients and three remote sites configured with router-to-pix tunnel up and running. And all that was done on a production network by an (almost)engineer with nearly no experience or business being on a router. In any other environment, I might not have had this opportunity. But one thing is for sure, whether the opportunity is there or not, I learned that you must always be prepared and constantly train yourself by reading, asking, testing, tinkering, labbing, etc. Because when real opportunity comes, you’d have already armed yourself with the ability to say “yes” to that opportunity, even though you might not feel entirely ready.